The Problem with Probability

Risk Factors

When risk is analyzed, at least in the industrial sector, we usually follow a process defined in ISO 12100. This approach defines two broad parameters, severity and probability, and then further subdivides probability into sub-parameters that are helpful for analyzing machinery hazards. This post explores the difficulty with assessing the probability parameter.

If you’re interested in buying ISO 12100, here’s a link to the ISO Web site where you can get a copy. If you need to save a few dollars, you can buy the EN version from Estonian Standards. It’s technically identical and available in English as well as Estonian.

Severity

There are two key factors that need to be understood when assessing risk: Severity and Probability (or Likelihood). Sometimes the term ‘consequence’ is used instead of ‘severity’, and in the case of machinery risk assessment, they can be considered to be synonyms.  Severity seems to be fairly well understood?most people can fairly easily imagine what reaching into a spinning blade might do to the hand doing the reaching. There is a problem that arises when there is an insufficient understanding of the hazard, but that’s the subject for another post.

Probability

Probability or likelihood is used to describe the chance that an injury or a hazardous situation will occur. Probability is used when numeric data is available and probability can be calculated, while “likelihood” is used when the assessment is subjective. The probability factor is often broken down further into three sub-factors as seen in Figure 3 below [1]:

There is No Reality, only Perception…

Whether you use probability or likelihood in your assessment, there is a fundamental problem with people’s perception of these factors. People have a dif?fi?cult time appre?ci?at?ing the mean?ing of prob?a?bil?ity. Probability is a key fac?tor in deter?min?ing the degree of risk from any hazard, yet when fig?ures like “1 in 1000” or “1 x 10?5 occur?rences per year” are dis?cussed, it?s hard for peo?ple to truly grasp what these num?bers mean. When prob?a?bil?ity is dis?cussed as a rate, a fig?ure like “1 x 10?5 occur?rences per year” can make the chance of an occur?rence seem inconceivably dis?tant, and there?fore less of a concern. Likewise, when more sub?jec?tive scales are used it can be dif?fi?cult to really under?stand what “likely” or “rarely” actu?ally mean. Consequently, even in cases where the sever?ity may be very high, the risk related to a par?tic?u?lar haz?ard may be neglected if the prob?a?bil?ity is deemed low.

To see the other side, consider people’s attitude when it comes to winning a lottery. Most people will agree that “Someone will win” and the infinitesimal probability of winning is seen as significant.  The same odds given in rrelationto a negative risk might be seen as ‘infinitesimally small’, and therefore negligible.

For example, consider the decisions made by the Tokyo Electric Power Corporation (TEPCO) when they constructed the Fukushima Dai Ichi nuclear power plant. TEPCO engineers and scientists assessed the site in the 1960’s and decided that a 10 meter tsunami was a realistic possibility at the site. They decided to build the reactors, turbines and backup generators 10 meters above the surrounding sea level, then located the system critical condensers in the seaward yard of the plant at a level below 10 meters. To protect that critical equipment they built a 5.7 meter high seawall, almost 50% shorter than the predicted height for a tsunami! While I don’t know what rationale they used to support this design decision, it is clear that the plant would have taken significant damage from even a relatively mild tsunami. The 11-Mar-11 tsunami topped the highest prediction by nearly 5 meters, resulting in a Level 7 nuclear accident and decades for recovery. TEPCO executives have repeatedly stated that the conditions leading to the accident were “inconceivable”, and yet redundancy was built into the systems for just this type of event, and some planning for tsunami effects were put into the design. Clearly was neither unimaginable or inconceivable, just underestimated.

Risk Perception

So why is it that tiny odds are seen as an acceptable risk and even a reasonable likelihood in one case, and a negligible chance in the other, particularly when the ignored case is the one that will have a significant negative outcome?
According to an article in Wikipedia [2], there are three main schools of thought when it comes to understanding risk perception: psychological, sociological and interdisciplinary. In a key early paper written in 1969 by Chauncy Starr [3], it was discovered that people would accept voluntary risks 1000 times greater than involuntary risks. Later research has challenged these findings, showing the gap between voluntary and involuntary to be much narrower than Starr found.
Early psychometric research by Kahneman and Tversky, showed that people use a number of heuristics to evaluate information. These heuristics included:
  • Representativeness;
  • Availability;
  • Anchoring and Adjustment;
  • Asymmetry; and
  • Threshold effects.
This research showed that people tend to be averse to risks to gains, like the potential for loss of savings by making risky investments, while they tend to accept risk easily when it comes to potential losses, preferring the hope of losing nothing over a certain but smaller loss. This may explain why low-probability, high severity OHS risks are more often ignored, in the hope that lesser injuries will occur rather than the maximum predicted severity.

Significant results also show that better information frequently has no effect on how risks are judged. More weight is put on risks with immediate, personal results than those seen in longer time frames. Psychometric research has shown that risk perception is highly dependent on intuition, experiential thinking, and emotions. The research identified characteristics that may be condensed into three high order factors:

  1. the degree to which a risk is understood;
  2. the degree to which it evokes a feeling of dread; and
  3. the number of people exposed to the risk.

“Dread” describes a risk that elicits visceral feelings of impending catastrophe, terror and loss of control. The more a person dreads an activity, the higher its perceived risk and the more that person wants the risk reduced [4]. Fear is clearly a stronger motivator than any degree of information.

Considering the differing views of those studying risk perception, it’s no wonder that this is a challenging subject for safety practitioners!

Estimating Probability

Frequency and Duration

Some aspects of probability are not too difficult to estimate. Consider the Frequency or Duration of Exposure factor. At face value, this can be stated as “X cycles per hour” or “Y hours per week”. Depending on the hazard, there may be more complex exposure data, like that used when considering audible noise exposure. In that case, noise is often expressed as a time-weighted-average (TWH), like “83 dB(A), 8 h TWH”, meaning 83 dB(A) averaged over 8 hours.

Estimating the probability of a hazardous situation is usually not too tough either. This could be expressed as “15 minutes, once per day / shift” or “2 days, twice per year”.

Avoidance

Estimating the probability of avoiding an injury in any given hazardous situation is MUCH more difficult, since the speed of occurrence, the ability to perceive the hazard, the knowledge of the exposed person, their ability to react in the situation, the level of training that they have, the presence of complementary protective measures, and many other factors come into play. Depth of understanding of the hazard and the details of the hazardous situation by the risk assessors is critical to a sound assessment of the risk involved.

The Challenge

The challenge for safety practitioners is twofold:

  1. As practitioners, we must try to overcome our biases when conducting risk assessment work, and where we cannot overcome those biases, we must at least acknowledge them and the effects they may produce in our work; and
  2. We must try to present the risks in terms that the exposed people can understand, so that they can make a reasoned choice for their own personal safety.

I don’t suggest that this is easy, nor do I advocate “dumbing down” the information! I do believe that risk information can be presented to non-technical people in ways that they can understand the critical points.

Risk assessment techniques are becoming fundamental in all areas of design. As safety practitioners, we must be ready to conduct risk assessments using sound techniques, be aware of our biases and be patient in communicating the results of our analysis to everyone that may be affected.

References

[1] “Safety of Machinery?General Principles for Design?Risk Assessment and Risk Reduction”, ISO 12100, Figure 3, ISO, Geneva, 2010.
[2] “Risk Perception”, Wikipedia, accessed 19/20-May-2011, http://en.wikipedia.org/wiki/Risk_perception.
[3] Chancey Starr, “Social Benefits versus Technological Risks”, Science Vol. 165, No. 3899. (Sep. 19, 1969), pp. 1232?1238
[4] Paul Slovic, Baruch Fischhoff, Sarah Lichtenstein, ?Why Study Risk Perception??, Risk Analysis 2(2) (1982), pp. 83?93.

© 2011 – 2018, Compliance inSight Consulting Inc. Creative Commons Licence
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.