Scoring Severity of Injury — Hidden Probabilities

This entry is part 8 of 8 in the series Risk Assess­ment

I’ve been think­ing a lot about risk scor­ing tools and the algo­rithms that we use. One of the key ele­ments in risk is the Sever­i­ty of Injury. There are hid­den prob­a­bil­i­ties attached to the Sever­i­ty of Injury scores that are assigned that are not dis­cussed clear­ly in any of the risk assess­ment stan­dards that are com­mon­ly in use. This all start­ed when I was chal­lenged to write an analy­sis of the prob­lems with the CSA Risk Scor­ing Tool that you can find in the 2014 ver­sion of CSA Z434. That tool is deeply flawed in my opin­ion, but that is not the top­ic of this post. If you want to read my analy­sis, you can down­load the white paper and the pre­sen­ta­tion notes for my analy­sis from the Com­pli­ance inSight Pub­li­ca­tions page [1].

Scor­ing risk can be a tricky thing, espe­cial­ly in the machin­ery sec­tor. We rarely have much in the way of real-world data to use in the analy­sis, and so we are left with the opin­ions of those build­ing the machine as the basis for our eval­u­a­tion. Sever­i­ty is usu­al­ly the first risk para­me­ter to be esti­mat­ed because it’s seen as the “easy” one — if the char­ac­ter­is­tics of the haz­ard are well known. One aspect of sever­i­ty that is often missed is the prob­a­bil­i­ty of a cer­tain sever­i­ty of injury. We’re NOT talk­ing about how like­ly it is for some­one to be injured here; we’re talk­ing about the most like­ly degree of injury that will occur when the per­son inter­acts with the haz­ard. Let me illus­trate this idea anoth­er way: Let’s call Sever­i­ty “Se”, any spe­cif­ic injury “I”, and the prob­a­bil­i­ty of any spe­cif­ic injury “Ps”. We can then write a short equa­tion to describe this rela­tion­ship.

Se f (I,Ps)

Since we want there to be a pos­si­bil­i­ty of no injury, we should prob­a­bly relate these para­me­ters as a prod­uct:

Se = I x Ps

Ok, so what? What this equa­tion says is: the Sever­i­ty (Se) of any giv­en injury (I), is the prod­uct of the spe­cif­ic type of injury and the prob­a­bil­i­ty of that injury. More sim­ply yet, you could say that you should be con­sid­er­ing the most like­ly type of injury that you think will occur when a per­son inter­acts with the haz­ard. Con­sid­er this exam­ple: A work­er enters a robot­ic work cell to change the weld tips on the weld­ing gun the robot uses. This task has to be done about once every two days. The entry gate is inter­locked, and the robot was locked out before entry. The floor of the work cell has wire­ways, con­duits and pip­ing run­ning across it from the edges of the cell to the var­i­ous pieces of equip­ment inside the cell, cre­at­ing uneven foot­ing and lots of slip and trip haz­ards. The work­er miss­es his foot­ing and falls. What can you expect for Se in this case?

We know that falls on the same lev­el can lead to fatal­i­ties, about 600/year in the USA [2], but that these are most­ly in the con­struc­tion and min­ing sec­tors rather than gen­er­al man­u­fac­tur­ing. We also know that bro­ken bones are more like­ly than fatal­i­ties in falls to the same lev­el. About a mil­lion slips and falls per year result in an emer­gency room vis­it, and of these, about 5%, or 50,000, result in frac­tures. Ok, so what do we do with this infor­ma­tion? Let’s look at typ­i­cal sever­i­ty scale, this one tak­en from IEC 62061 [3].

Table 1 – Sever­i­ty (Se) clas­si­fi­ca­tion [2, Table A.1]

Con­se­quences Sever­i­ty (Se)
Irre­versible: death, los­ing an eye or arm 4
Irre­versible: bro­ken limb(s), los­ing a finger(s) 3
Reversible: requir­ing atten­tion from a med­ical prac­ti­tion­er 2
Reversible: requir­ing first aid 1

Using Table 1, we might come up with the fol­low­ing list of pos­si­ble sever­i­ties of injury. This list is not exhaus­tive, so feel free to add more.

Table 2 — Poten­tial Injury Sever­i­ties

Pos­si­ble Injury Sever­i­ty (Se)
Fall on same lev­el — Fatal­i­ty 4
Fall on same lev­el — Bro­ken wrist 3
Fall on same lev­el — Bro­ken col­lar­bone 3
Fall on same lev­el — Torn rota­tor cuff 2
Fall on same lev­el — Bruis­es 1
Fall on same lev­el — Head Injury 3
Fall on same lev­el — Head Injury 4

How do we score this using a typ­i­cal scor­ing tool? We could add each of these as line items in the risk reg­is­ter, and then assess the prob­a­bil­i­ty of each, but that will tend to cre­ate huge risk reg­is­ters with many line items at very low risks. In prac­tice, we decide on what we think is the most like­ly degree of injury BEFORE we score the risk. This results in a sin­gle line item for the haz­ard, rather than sev­en as would be the case if we scored each of these poten­tial injuries indi­vid­u­al­ly.

We need a prob­a­bil­i­ty scale to use in assess­ing the like­li­hood of injuries. At the moment, no pub­lished scor­ing tool that I know of has a scale for this, so let’s do the sim­ple thing: Prob­a­bil­i­ty (Ps) will be scored from 0–100%, with 100% being a cer­tain­ty.

Going back to the sec­ond equa­tion, what we are real­ly doing is assign­ing a prob­a­bil­i­ty to each of the sever­i­ties that we think exist, some­thing like this:

Table 3 — Poten­tial Injuries and their Prob­a­bil­i­ties

Pos­si­ble Injury (I) Sever­i­ty (Se) Prob­a­bil­i­ty (Ps)
Fall on same lev­el — Fatal­i­ty 4  0.0075%
Fall on same lev­el — Bro­ken wrist 3  5%
Fall on same lev­el — Bro­ken col­lar­bone 3  5%
Fall on same lev­el — Torn rota­tor cuff 2  5%
Fall on same lev­el — Bruis­es 1  90%
Fall on same lev­el — Head Injury 3 1%
Fall on same lev­el — Head Injury 4   0.0075%
Fall on same lev­el — Lac­er­a­tions to hands 2 90%

The per­cent­ages for fatal­i­ties and frac­tures we tak­en rough­ly from [1]. Ok, so we can look at a table like this and say that cuts and bruis­es are the most like­ly types of injury in this case. We can either decide to group them for the over­all risk score, or we can score each indi­vid­u­al­ly, result­ing in adding two sep­a­rate line items to the risk reg­is­ter. I’m going to use the oth­er para­me­ters from [2] for this exam­ple, and devel­op an exam­ple risk reg­is­ter, Table 4. In Table 4,

Se = Sever­i­ty

Pr = Prob­a­bil­i­ty of the Haz­ardous Event

Fr = Fre­quen­cy and Dura­tion of Expo­sure

Av = Pos­si­bil­i­ty to Avoid or Lim­it Harm

The algo­rithm I am using to eval­u­ate the risk is R = Se x [Pr x (Fr + Av)] [1]. Note that where I have com­bined the two poten­tial injuries into one line item (Item 1 in the reg­is­ter), I have select­ed the high­est sever­i­ty of the com­bined injuries. The less like­ly sever­i­ties, and in par­tic­u­lar the fatal­i­ties, have been ignored. You can click on  Table 4 to see a larg­er, more read­able ver­sion.

Table 4 - Example Risk Register
Table 4 — Exam­ple Risk Reg­is­ter

Note that I did not reduce the Se scores in the Final Risk Score, because I have not made changes to the slip/trip and fall haz­ards, only to the like­li­hood of the injury occur­ring. In all cas­es, we can show a sig­nif­i­cant risk reduc­tion after mit­i­ga­tion. I’m not going to get into risk eval­u­a­tion (i.e., Is the risk effec­tive­ly con­trolled?) in this par­tic­u­lar arti­cle, but the fact that you can show a sig­nif­i­cant risk reduc­tion is impor­tant. There are lots of con­sid­er­a­tions in deter­min­ing if the risk has been effec­tive­ly con­trolled.

Conclusions

Con­sid­er­a­tion of the prob­a­bil­i­ty of cer­tain kinds of injuries occur­ring must be con­sid­ered when esti­mat­ing risk. This process is large­ly undoc­u­ment­ed but nev­er­the­less occurs. When risk ana­lysts are con­sid­er­ing the sever­i­ty of injury from any giv­en haz­ard, this arti­cle gives the read­er one pos­si­ble approach than could be used to select the types of injuries most like­ly to occur before scor­ing the rest of the risk para­me­ters.

References

[1] D. Nix, ‘Eval­u­a­tion of Prob­lems and Chal­lenges in CSA Z434-14 Annex DVA Task-Based Risk Assess­ment Method­ol­o­gy’, 2015.

[2] Nation­al Floor Safe­ty Insti­tute (NFSI), ‘Quick Facts — Slips, Trips, and Falls’, 2015. [Online]. Avail­able: http://nfsi.org/nfsi-research/quick-facts/. [Accessed: 21- Jul- 2015].

[3] ‘Safe­ty of machin­ery – Func­tion­al safe­ty of safe­ty-relat­ed elec­tri­cal, elec­tron­ic and pro­gram­ma­ble elec­tron­ic con­trol sys­tems. IEC 62061.’, Inter­na­tion­al Elec­trotech­ni­cal Com­mis­sion (IEC), Gene­va, 2005.

 

Digiprove sealCopy­right secured by Digiprove © 2015
Acknowl­edge­ments: Inter­na­tion­al Elec­trotech­ni­cal Com­mis more…
Some Rights Reserved

The Problem with Probability

This entry is part 3 of 8 in the series Risk Assess­ment

Risk Factors

Severity

There are two key fac­tors that need to be under­stood when assess­ing risk: Sever­i­ty and Prob­a­bil­i­ty (or Like­li­hood). Some­times the term ‘con­se­quence’ is used instead of ‘sever­i­ty’, and in the case of machin­ery risk assess­ment, they can be con­sid­ered to be syn­onyms.  Sever­i­ty seems to be fair­ly well understood—most peo­ple can fair­ly eas­i­ly imag­ine what reach­ing into a spin­ning blade might do to the hand doing the reach­ing. There is a prob­lem that aris­es when there is an insuf­fi­cient under­stand­ing of the haz­ard, but that’s the sub­ject for anoth­er post.

Probability

Prob­a­bil­i­ty or like­li­hood is used to describe the chance that an injury or a haz­ardous sit­u­a­tion will occur. Prob­a­bil­i­ty is used when numer­ic data is avail­able and prob­a­bil­i­ty can be cal­cu­lat­ed, while like­li­hood is used when the assess­ment is sub­jec­tive. The prob­a­bil­i­ty fac­tor is often bro­ken down fur­ther into three sub-fac­tors as seen in Fig­ure 3 below [1]:

There is No Reality, only Perception…

Whether you use prob­a­bil­i­ty or like­li­hood in your assess­ment, there is a fun­da­men­tal prob­lem with people’s per­cep­tion of these fac­tors. Peo­ple have a dif­fi­cult time appre­ci­at­ing the mean­ing of prob­a­bil­ity. Prob­a­bil­i­ty is a key fac­tor in deter­min­ing the degree of risk from any haz­ard, yet when fig­ures like “1 in 1000” or “1 x 10–5 occur­rences per year” are dis­cussed, it’s hard for peo­ple to tru­ly grasp what these num­bers mean. When prob­a­bil­ity is dis­cussed as a rate, a fig­ure like “1 x 10–5 occur­rences per year” can make the chance of an occur­rence seem incon­ceiv­ably dis­tant, and there­fore less of a con­cern. Like­wise, when more sub­jec­tive scales are used it can be dif­fi­cult to real­ly under­stand what “like­ly” or “rarely” actu­ally mean. Con­se­quent­ly, even in cas­es where the sever­ity may be very high, the risk relat­ed to a par­tic­u­lar haz­ard may be neglect­ed if the prob­a­bil­ity is deemed low.

To see the oth­er side, con­sid­er people’s atti­tude when it comes to win­ning a lot­tery. Most peo­ple will agree that “Some­one will win” and the infin­i­tes­i­mal prob­a­bil­i­ty of win­ning is seen as sig­nif­i­cant.  The same odds giv­en in rela­tion­ship to a neg­a­tive risk might be seen as ‘infin­i­tes­i­mal­ly small’, and there­fore neg­li­gi­ble.

For exam­ple, con­sid­er the deci­sions made by the Tokyo Elec­tric Pow­er Cor­po­ra­tion (TEPCO) when they con­struct­ed the Fukushi­ma Dai Ichi nuclear pow­er plant. TEPCO engi­neers and sci­en­tists assessed the site in the 1960’s and decid­ed that a 10 meter tsuna­mi was a real­is­tic pos­si­bil­i­ty at the site. They decid­ed to build the reac­tors, tur­bines and back­up gen­er­a­tors 10 meters above the sur­round­ing sea lev­el, then locat­ed the sys­tem crit­i­cal con­densers in the sea­ward yard of the plant at a lev­el below 10 meters. To pro­tect that crit­i­cal equip­ment they built a 5.7 meter high sea­wall, almost 50% short­er than the pre­dict­ed height for a tsuna­mi! While I don’t know what ratio­nale they used to sup­port this design deci­sion, it is clear that the plant would have tak­en sig­nif­i­cant dam­age from even a rel­a­tive­ly mild tsuna­mi. The 11-Mar-11 tsuna­mi topped the high­est pre­dic­tion by near­ly 5 meters, result­ing in a Lev­el 7 nuclear acci­dent and decades for recov­ery. TEPCO exec­u­tives have repeat­ed­ly stat­ed that the con­di­tions lead­ing to the acci­dent were “incon­ceiv­able”, and yet redun­dan­cy was built into the sys­tems for just this type of event, and some plan­ning for tsuna­mi effects were put into the design. Clear­ly was nei­ther unimag­in­able or incon­ceiv­able, just under­es­ti­mat­ed.

Risk Perception

So why is it that tiny odds are seen as an accept­able risk and even a rea­son­able like­li­hood in one case, and a neg­li­gi­ble chance in the oth­er, par­tic­u­lar­ly when the ignored case is the one that will have a sig­nif­i­cant neg­a­tive out­come?
Accord­ing to an arti­cle in Wikipedia [2], there are three main schools of thought when it comes to under­stand­ing risk per­cep­tion: psy­cho­log­i­cal, soci­o­log­i­cal and inter­dis­ci­pli­nary. In a key ear­ly paper writ­ten in 1969 by Chauncy Starr [3], it was dis­cov­ered that peo­ple would accept vol­un­tary risks 1000 times greater than invol­un­tary risks. Lat­er research has chal­lenged these find­ings, show­ing the gap between vol­un­tary and invol­un­tary to be much nar­row­er than Starr found.
Ear­ly psy­cho­me­t­ric research by Kah­ne­man and Tver­sky, showed that peo­ple use a num­ber of heuris­tics to eval­u­ate infor­ma­tion. These heuris­tics includ­ed:
  • Rep­re­sen­ta­tive­ness;
  • Avail­abil­i­ty;
  • Anchor­ing and Adjust­ment;
  • Asym­me­try; and
  • Thresh­old effects.
This research showed that peo­ple tend to be averse to risks to gains, like the poten­tial for loss of sav­ings by mak­ing risky invest­ments, while they tend to accept risk eas­i­ly when it comes to poten­tial loss­es, pre­fer­ring the hope of los­ing noth­ing over a cer­tain but small­er loss. This may explain why low-prob­a­bil­i­ty, high sever­i­ty OHS risks are more often ignored, in the hope that less­er injuries will occur rather than the max­i­mum pre­dict­ed sever­i­ty.

Sig­nif­i­cant results also show that bet­ter infor­ma­tion fre­quent­ly has no effect on how risks are judged. More weight is put on risks with imme­di­ate, per­son­al results than those seen in longer time frames. Psy­cho­me­t­ric research has shown that risk per­cep­tion is high­ly depen­dent on intu­ition, expe­ri­en­tial think­ing, and emo­tions. The research iden­ti­fied char­ac­ter­is­tics that may be con­densed into three high order fac­tors:

  1. the degree to which a risk is under­stood;
  2. the degree to which it evokes a feel­ing of dread; and
  3. the num­ber of peo­ple exposed to the risk.

Dread” describes a risk that elic­its vis­cer­al feel­ings of impend­ing cat­a­stro­phe, ter­ror and loss of con­trol. The more a per­son dreads an activ­i­ty, the high­er its per­ceived risk and the more that per­son wants the risk reduced [4]. Fear is clear­ly a stronger moti­va­tor than any degree of infor­ma­tion.

Con­sid­er­ing the dif­fer­ing views of those study­ing risk per­cep­tion, it’s no won­der that this is a chal­leng­ing sub­ject for safe­ty prac­ti­tion­ers!

Estimating Probability

Frequency and Duration

Some aspects of prob­a­bil­i­ty are not too dif­fi­cult to esti­mate. Con­sid­er the Fre­quen­cy or Dura­tion of Expo­sure fac­tor. At face val­ue this can be stat­ed as “X cycles per hour” or “Y hours per week”. Depend­ing on the haz­ard, there may be more com­plex expo­sure data, like that used when con­sid­er­ing audi­ble noise expo­sure. In that case, noise is often expressed as a time-weight­ed-aver­age (TWH), like “83 dB(A), 8 h TWH”, mean­ing 83 dB(A) aver­aged over 8 hours.

Esti­mat­ing the prob­a­bil­i­ty of a haz­ardous sit­u­a­tion is usu­al­ly not too tough either. This could be expressed as “15 min­utes, once per day / shift” or “2 days, twice per year”.

Avoidance

Esti­mat­ing the prob­a­bil­i­ty of avoid­ing an injury in any giv­en haz­ardous sit­u­a­tion is MUCH more dif­fi­cult, since the speed of occur­rence, the abil­i­ty to per­ceive the haz­ard, the knowl­edge of the exposed per­son, their abil­i­ty to react in the sit­u­a­tion, the lev­el of train­ing that they have, the pres­ence of com­ple­men­tary pro­tec­tive mea­sures, and many oth­er fac­tors come into play. Depth of under­stand­ing of the haz­ard and the details of the haz­ardous sit­u­a­tion by the risk asses­sors is crit­i­cal to a sound assess­ment of the risk involved.

The Challenge

The chal­lenge for safe­ty prac­ti­tion­ers is twofold:

  1. As prac­ti­tion­ers, we must try to over­come our bias­es when con­duct­ing risk assess­ment work, and where we can­not over­come those bias­es, we must at least acknowl­edge them and the effects they may pro­duce in our work; and
  2. We must try to present the risks in terms that the exposed peo­ple can under­stand, so that they can make a rea­soned choice for their own per­son­al safe­ty.

I don’t sug­gest that this is easy, nor do I advo­cate “dumb­ing down” the infor­ma­tion! I do believe that risk infor­ma­tion can be pre­sent­ed to non-tech­ni­cal peo­ple in ways that they can under­stand the crit­i­cal points.

Risk assess­ment tech­niques are becom­ing fun­da­men­tal in all areas of design. As safe­ty prac­ti­tion­ers, we must be ready to con­duct risk assess­ments using sound tech­niques, be aware of our bias­es and be patient in com­mu­ni­cat­ing the results of our analy­sis to every­one that may be affect­ed.

References

[1] “Safe­ty of Machinery—General Prin­ci­ples for Design—Risk Assess­ment and Risk Reduc­tion”, ISO 12100, Fig­ure 3, ISO, Gene­va, 2010.
[2] “Risk Per­cep­tion”, Wikipedia, accessed 19/20-May-2011, http://en.wikipedia.org/wiki/Risk_perception.
[3] Chancey Starr, “Social Ben­e­fits ver­sus Tech­no­log­i­cal Risks”, Sci­ence Vol. 165, No. 3899. (Sep. 19, 1969), pp. 1232–1238
[4] Paul Slovic, Baruch Fis­chhoff, Sarah Licht­en­stein, “Why Study Risk Per­cep­tion?”, Risk Analy­sis 2(2) (1982), pp. 83–93.