How Risk Assessment Fails

Fukushima Dai Ichi Nuclear plant before the meltdown
This entry is part 2 of 8 in the series Risk Assessment

Fukushima Dai Ichi Power Plant after the explosionsThe events unfold­ing at Japan’s Fukushima Dai Ichi Nuclear Power plant are a case study in ways that the risk assess­ment pro­cess can fail or be abused. In an art­icle pub­lished on Bloomberg​.com, Jason Clenfield item­izes dec­ades of fraud and fail­ures in engin­eer­ing and admin­is­tra­tion that have led to the cata­stroph­ic fail­ure of four of six react­ors at the 40-​year-​old Fukushima plant. Clenfield’s art­icle, ‘Disaster Caps Faked Reports’, goes on to cov­er sim­il­ar fail­ures in the Japanese nuc­le­ar sec­tor.

Most people believe that the more ser­i­ous the pub­lic danger, the more care­fully the risks are con­sidered in the design and exe­cu­tion of pro­jects like the Fukushima plant. Clenfield’s art­icle points to fail­ures by a num­ber of major inter­na­tion­al busi­nesses involved in the design and man­u­fac­ture of com­pon­ents for these react­ors that may have con­trib­uted to the cata­strophe play­ing out in Japan. In some cases, the cor­rect actions could have bank­rup­ted the com­pan­ies involved, so rather than risk fin­an­cial fail­ure, these fail­ures were covered up and the work­ers involved rewar­ded for their efforts. As you will see, some­times the degree of care that we have a right to expect is not the level of care that is used.

How does this relate to the fail­ure and abuse of the risk assess­ment pro­cess? Read on!

Risk Assessment Failures

Earthquake and Tsunami damage - Fukushima Dai Ichi Power PlantThe Fukushima Dai Ichi nuc­le­ar plant was con­struc­ted in the late 1960’s and early 1970’s, with Reactor #1 going on-​line in 1971. The react­ors at this facil­ity use ‘act­ive cool­ing’, requir­ing elec­tric­ally powered cool­ing pumps to run con­tinu­ously to keep the core tem­per­at­ures in the nor­mal oper­at­ing range. As you will have seen in recent news reports, the plant is loc­ated on the shore, draw­ing water dir­ectly from the Pacific Ocean.

Learn more about Boiling Water Reactors used at Fukushima.

Read IEEE Spectrum’s “24-​Hours at Fukushima”, a blow-​by-​blow account of the first 24 hours of the dis­aster.

Japan is loc­ated along one of the most act­ive fault lines in the world, with plate sub­duc­tion rates exceed­ing 90 mm/​year. Earthquakes are so com­mon­place in this area that the Japanese people con­sider Japan to be the ‘land of earth­quakes’, start­ing earth­quake safety train­ing in kinder­garten.

Japan is the county that cre­ated the word ‘tsunami’ because the effects of sub-​sea earth­quakes often include large waves that swamp the shoreline. These waves affect all coun­tries bor­der­ing the worlds oceans, but are espe­cially pre­val­ent where strong earth­quakes are fre­quent.

In this envir­on­ment it would be reas­on­able to expect that con­sid­er­a­tion of earth­quake and tsunami effects would mer­it the highest con­sid­er­a­tion when assess­ing the risks related to these haz­ards. Remembering that risk is a func­tion of sever­ity of con­sequence and prob­ab­il­ity, the risk assessed from earth­quake and tsunami should have been crit­ic­al. Loss of cool­ing can res­ult in the cata­stroph­ic over­heat­ing of the react­or core, poten­tially lead­ing to a core melt­down.

The Fukushima Dai Ichi plant was designed to with­stand 5.7 m tsunami waves, even though a 6.4 m wave had hit the shore close by 10 years before the plant went on-​line. The wave gen­er­ated by the recent earth­quake was 7 m. Although the plant was not washed away by the tsunami, the wave cre­ated anoth­er prob­lem.

Now con­sider that the react­ors require con­stant forced cool­ing using elec­tric­ally powered pumps. The backup gen­er­at­ors installed to ensure that cool­ing pumps remain oper­a­tion­al even if the mains power to the plant is lost, are installed in a base­ment sub­ject to flood­ing. When the tsunami hit the sea­wall and spilled over the top, the flood­wa­ters poured into the backup gen­er­at­or room, knock­ing out the dies­el backup gen­er­at­ors. The cool­ing sys­tem stopped. With no power to run the pumps, the react­or cores began to over­heat. Although the react­ors sur­vived the earth­quakes and the tsunami, without power to run the pumps the plant was in trouble.

Learn more about the acci­dent.

Clearly there was a fail­ure of reas­on when assess­ing the risks related the loss of cool­ing cap­ab­il­ity in these react­ors. With sys­tems that are mis­sion crit­ic­al in the way that these sys­tems are, mul­tiple levels of redund­ancy bey­ond a single backup sys­tem are often the min­im­um required.

In anoth­er plant in Japan, a sec­tion of pip­ing car­ry­ing super­heated steam from the react­or to the tur­bines rup­tured injur­ing a num­ber of work­ers. The pipe was installed when the plant was new and had nev­er been inspec­ted since install­a­tion because it was left off the safety inspec­tion check­list. This is an example of a fail­ure that res­ul­ted from blindly fol­low­ing a check­list without look­ing at the lar­ger pic­ture. There can be no doubt that someone at the plant noticed that oth­er pipe sec­tions were inspec­ted reg­u­larly, but that this par­tic­u­lar sec­tion was skipped, yet no changes in the pro­cess res­ul­ted.

Here again, the risk was not recog­nized even though it was clearly under­stood with respect to oth­er sec­tions of pipe in the same plant.

In anoth­er situ­ation at a nuc­le­ar plant in Japan, drains inside the con­tain­ment area of a react­or were not plugged at the end of the install­a­tion pro­cess. As a res­ult, a small spill of radio­act­ive water was released into the sea instead of being prop­erly con­tained and cleaned up. The risk was well under­stood, but the con­trol pro­ced­ure for this risk was not imple­men­ted.

Finally, the Kashiwazaki Kariwa plant was con­struc­ted along a major fault line. The design­ers used fig­ures for the max­im­um seis­mic accel­er­a­tion that were three times lower than the accel­er­a­tions that could be cre­ated by the fault. Regulators per­mit­ted the plant to be built even though the rel­at­ive weak­ness of the design was known.

Failure Modes

I believe that there are a num­ber of reas­ons why these kinds of fail­ures occur.

People have a dif­fi­cult time appre­ci­at­ing the mean­ing of prob­ab­il­ity. Probability is a key factor in determ­in­ing the degree of risk from any haz­ard, yet when fig­ures like ‘1 in 1000’ or ‘1 x 10-5 occur­rences per year’ are dis­cussed, it’s hard for people to truly grasp what these num­bers mean. Likewise, when more sub­ject­ive scales are used it can be dif­fi­cult to really under­stand what ‘likely’ or ‘rarely’ actu­ally mean.

Consequently, even in cases where the sever­ity may be very high, the risk related to a par­tic­u­lar haz­ard may be neg­lected because the risk is deemed to be low because the prob­ab­il­ity seems to be low.

When prob­ab­il­ity is dis­cussed in terms of time, a fig­ure like ‘1 x 10-5 occur­rences per year’ can make the chance of an occur­rence seem dis­tant, and there­fore less of a con­cern.

Most risk assess­ment approaches deal with haz­ards singly. This is done to sim­pli­fy the assess­ment pro­cess, but the prob­lem that can res­ult from this approach is the effect that mul­tiple fail­ures can cre­ate, or that cas­cad­ing fail­ures can cre­ate. In a mul­tiple fail­ure con­di­tion, sev­er­al pro­tect­ive meas­ures fail sim­ul­tan­eously from a single cause (some­times called Common Cause Failure). In this case, back-​up meas­ures may fail from the same cause, res­ult­ing in no pro­tec­tion from the haz­ard.

In a cas­cad­ing fail­ure, an ini­tial fail­ure is fol­lowed by a series of fail­ures res­ult­ing in the par­tial or com­plete loss of the pro­tect­ive meas­ures, res­ult­ing in par­tial or com­plete expos­ure to the haz­ard. Reasonably fore­see­able com­bin­a­tions of fail­ure modes in mis­sion crit­ic­al sys­tems must be con­sidered and the prob­ab­il­ity of each estim­ated.

Combination of haz­ards can res­ult in syn­ergy between the haz­ards res­ult­ing in a high­er level of sever­ity from the com­bin­a­tion than is present from any one of the haz­ards taken singly. Reasonably fore­see­able com­bin­a­tions of haz­ards and their poten­tial syn­er­gies must be iden­ti­fied and the risk estim­ated.

Oversimplification of the haz­ard iden­ti­fic­a­tion and ana­lys­is pro­cesses can res­ult in over­look­ing haz­ards or under­es­tim­at­ing the risk.

Thinking about the Fukushima Dai Ichi plant again, the com­bin­a­tion of the effects of the earth­quake on the plant, with the added impact of the tsunami wave, res­ul­ted in the loss of primary power to the plant fol­lowed by the loss of backup power from the backup gen­er­at­ors, and the sub­sequent par­tial melt­downs and explo­sions at the plant. This com­bin­a­tion of earth­quake and tsunami was well known, not some ‘unima­gin­able’ or ‘unfore­see­able’ situ­ation. When con­duct­ing risk assess­ments, all reas­on­ably fore­see­able com­bin­a­tions of haz­ards must be con­sidered.

Abuse and neglect

The risk assess­ment pro­cess is sub­ject to abuse and neg­lect. Risk assess­ment has been used by some as a means to jus­ti­fy expos­ing work­ers and the pub­lic to risks that should not have been per­mit­ted. Skewing the res­ults of the risk assess­ment, either by under­es­tim­at­ing the risk ini­tially, or by over­es­tim­at­ing the effect­ive­ness and reli­ab­il­ity of con­trol meas­ures can lead to this situ­ation. Decisions relat­ing to the ‘tol­er­ab­il­ity’ or the ‘accept­ab­il­ity’ of risks when the sever­ity of the poten­tial con­sequences are high should be approached with great cau­tion. In my opin­ion, unless you are per­son­ally will­ing to take the risk you are pro­pos­ing to accept, it can­not be con­sidered either tol­er­able or accept­able, regard­less of the leg­al lim­its that may exist.

In the case of the Japanese nuc­le­ar plants, the oper­at­ors have pub­licly admit­ted to falsi­fy­ing inspec­tion and repair records, some of which have res­ul­ted in acci­dents and fatal­it­ies.

In 1990, the US Nuclear Regulatory Commission wrote a report on the Fukushima Dai Ichi plant that pre­dicted the exact scen­ario that res­ul­ted in the cur­rent crisis. These find­ings were shared with the Japanese author­it­ies and the oper­at­ors, but no one in a pos­i­tion of author­ity took the find­ings ser­i­ously enough to do any­thing. Relatively simple and low-​cost pro­tect­ive meas­ures, like increas­ing the height of the pro­tect­ive sea wall along the coast­line and mov­ing the backup gen­er­at­ors to high ground could have pre­ven­ted a nation­al cata­strophe and the com­plete loss of the plant.

A Useful Tool

Despite these human fail­ings, I believe that risk assess­ment is an import­ant tool. Increasingly soph­ist­ic­ated tech­no­logy has rendered ‘com­mon sense’ use­less in many cases, because people do not have the expert­ise to have any com­mon sense about the haz­ards related to these tech­no­lo­gies.

Where haz­ards are well under­stood, they should be con­trolled with the simplest, most dir­ect and effect­ive meas­ures avail­able. In many cases this can be done by the people who first identi­fy the haz­ard.

Where haz­ards are not well under­stood, bring­ing in experts with the know­ledge to assess the risk and imple­ment appro­pri­ate pro­tect­ive meas­ures is the right approach.

The com­mon aspect in all of this is the iden­ti­fic­a­tion of haz­ards and the applic­a­tion of some sort of con­trol meas­ures. Risk assess­ment should not be neg­lected simply because it is some­times dif­fi­cult, or it can be done poorly, or the res­ults neg­lected or ignored. We need to improve what we do with the res­ults of these efforts, rather than neg­lect to do them at all.

In the mean time, the Japanese, and the world, have some cleanup to do.

Series NavigationISO Withdraws Machinery Risk Assessment Standards”>ISO Withdraws Machinery Risk Assessment StandardsThe Problem with Probability

Author: Doug Nix

+DougNix is Managing Director and Principal Consultant at Compliance InSight Consulting, Inc. ( in Kitchener, Ontario, and is Lead Author and Managing Editor of the Machinery Safety 101 blog.

Doug's work includes teaching machinery risk assessment techniques privately and through Conestoga College Institute of Technology and Advanced Learning in Kitchener, Ontario, as well as providing technical services and training programs to clients related to risk assessment, industrial machinery safety, safety-related control system integration and reliability, laser safety and regulatory conformity.

Follow me on

  • Tom Vardon

    The risk assess­ment, if not prop­erly facil­it­ated by a non-​stakeholder, can be skewed by the stake­hold­ers.

  • Pingback: Worse than Worst? | Machinery Safety 101()

  • Roberta Nelson Shea

    Very inter­est­ing art­icle!