Scoring Severity of Injury – Hidden Probabilities

This entry is part 8 of 8 in the series Risk Assessment

I’ve been think­ing a lot about risk scor­ing tools and the algorithms that we use. One of the key ele­ments in risk is the Severity of Injury. There are hid­den prob­ab­il­it­ies attached to the Severity of Injury scores that are assigned that are not dis­cussed clearly in any of the risk assess­ment stand­ards that are com­monly in use. This all star­ted when I was chal­lenged to write an ana­lys­is of the prob­lems with the CSA Risk Scoring Tool that you can find in the 2014 ver­sion of CSA Z434. That tool is deeply flawed in my opin­ion, but that is not the top­ic of this post. If you want to read my ana­lys­is, you can down­load the white paper and the present­a­tion notes for my ana­lys­is from the Compliance inSight Publications page [1].

Scoring risk can be a tricky thing, espe­cially in the machinery sec­tor. We rarely have much in the way of real-​world data to use in the ana­lys­is, and so we are left with the opin­ions of those build­ing the machine as the basis for our eval­u­ation. Severity is usu­ally the first risk para­met­er to be estim­ated because it’s seen as the “easy” one – if the char­ac­ter­ist­ics of the haz­ard are well known. One aspect of sever­ity that is often missed is the prob­ab­il­ity of a cer­tain sever­ity of injury. We’re NOT talk­ing about how likely it is for someone to be injured here; we’re talk­ing about the most likely degree of injury that will occur when the per­son inter­acts with the haz­ard. Let me illus­trate this idea anoth­er way: Let’s call Severity “Se”, any spe­cif­ic injury “I”, and the prob­ab­il­ity of any spe­cif­ic injury “Ps”. We can then write a short equa­tion to describe this rela­tion­ship.

Se f (I,Ps)

Since we want there to be a pos­sib­il­ity of no injury, we should prob­ably relate these para­met­ers as a product:

Se = I x Ps

Ok, so what? What this equa­tion says is: the Severity (Se) of any giv­en injury (I), is the product of the spe­cif­ic type of injury and the prob­ab­il­ity of that injury. More simply yet, you could say that you should be con­sid­er­ing the most likely type of injury that you think will occur when a per­son inter­acts with the haz­ard. Consider this example: A work­er enters a robot­ic work cell to change the weld tips on the weld­ing gun the robot uses. This task has to be done about once every two days. The entry gate is inter­locked, and the robot was locked out before entry. The floor of the work cell has wire­ways, con­duits and pip­ing run­ning across it from the edges of the cell to the vari­ous pieces of equip­ment inside the cell, cre­at­ing uneven foot­ing and lots of slip and trip haz­ards. The work­er misses his foot­ing and falls. What can you expect for Se in this case?

We know that falls on the same level can lead to fatal­it­ies, about 600/​year in the USA [2], but that these are mostly in the con­struc­tion and min­ing sec­tors rather than gen­er­al man­u­fac­tur­ing. We also know that broken bones are more likely than fatal­it­ies in falls to the same level. About a mil­lion slips and falls per year res­ult in an emer­gency room vis­it, and of these, about 5%, or 50,000, res­ult in frac­tures. Ok, so what do we do with this inform­a­tion? Let’s look at typ­ic­al sever­ity scale, this one taken from IEC 62061 [3].

Table 1 – Severity (Se) clas­si­fic­a­tion [2, Table A.1]

Consequences Severity (Se)
Irreversible: death, los­ing an eye or arm 4
Irreversible: broken limb(s), los­ing a finger(s) 3
Reversible: requir­ing atten­tion from a med­ic­al prac­ti­tion­er 2
Reversible: requir­ing first aid 1

Using Table 1, we might come up with the fol­low­ing list of pos­sible sever­it­ies of injury. This list is not exhaust­ive, so feel free to add more.

Table 2 – Potential Injury Severities

Possible Injury Severity (Se)
Fall on same level – Fatality 4
Fall on same level – Broken wrist 3
Fall on same level – Broken col­lar­bone 3
Fall on same level – Torn rotat­or cuff 2
Fall on same level – Bruises 1
Fall on same level – Head Injury 3
Fall on same level – Head Injury 4

How do we score this using a typ­ic­al scor­ing tool? We could add each of these as line items in the risk register, and then assess the prob­ab­il­ity of each, but that will tend to cre­ate huge risk registers with many line items at very low risks. In prac­tice, we decide on what we think is the most likely degree of injury BEFORE we score the risk. This res­ults in a single line item for the haz­ard, rather than sev­en as would be the case if we scored each of these poten­tial injur­ies indi­vidu­ally.

We need a prob­ab­il­ity scale to use in assess­ing the like­li­hood of injur­ies. At the moment, no pub­lished scor­ing tool that I know of has a scale for this, so let’s do the simple thing: Probability (Ps) will be scored from 0 – 100%, with 100% being a cer­tainty.

Going back to the second equa­tion, what we are really doing is assign­ing a prob­ab­il­ity to each of the sever­it­ies that we think exist, some­thing like this:

Table 3 – Potential Injuries and their Probabilities

Possible Injury (I) Severity (Se) Probability (Ps)
Fall on same level – Fatality 4  0.0075%
Fall on same level – Broken wrist 3  5%
Fall on same level – Broken col­lar­bone 3  5%
Fall on same level – Torn rotat­or cuff 2  5%
Fall on same level – Bruises 1  90%
Fall on same level – Head Injury 3 1%
Fall on same level – Head Injury 4   0.0075%
Fall on same level – Lacerations to hands 2 90%

The per­cent­ages for fatal­it­ies and frac­tures we taken roughly from [1]. Ok, so we can look at a table like this and say that cuts and bruises are the most likely types of injury in this case. We can either decide to group them for the over­all risk score, or we can score each indi­vidu­ally, res­ult­ing in adding two sep­ar­ate line items to the risk register. I’m going to use the oth­er para­met­ers from [2] for this example, and devel­op an example risk register, Table 4. In Table 4,

Se = Severity

Pr = Probability of the Hazardous Event

Fr = Frequency and Duration of Exposure

Av = Possibility to Avoid or Limit Harm

The algorithm I am using to eval­u­ate the risk is R = Se x [Pr x (Fr + Av)] [1]. Note that where I have com­bined the two poten­tial injur­ies into one line item (Item 1 in the register), I have selec­ted the highest sever­ity of the com­bined injur­ies. The less likely sever­it­ies, and in par­tic­u­lar the fatal­it­ies, have been ignored. You can click on  Table 4 to see a lar­ger, more read­able ver­sion.

Table 4 - Example Risk Register
Table 4 – Example Risk Register

Note that I did not reduce the Se scores in the Final Risk Score, because I have not made changes to the slip/​trip and fall haz­ards, only to the like­li­hood of the injury occur­ring. In all cases, we can show a sig­ni­fic­ant risk reduc­tion after mit­ig­a­tion. I’m not going to get into risk eval­u­ation (i.e., Is the risk effect­ively con­trolled?) in this par­tic­u­lar art­icle, but the fact that you can show a sig­ni­fic­ant risk reduc­tion is import­ant. There are lots of con­sid­er­a­tions in determ­in­ing if the risk has been effect­ively con­trolled.

Conclusions

Consideration of the prob­ab­il­ity of cer­tain kinds of injur­ies occur­ring must be con­sidered when estim­at­ing risk. This pro­cess is largely undoc­u­mented but nev­er­the­less occurs. When risk ana­lysts are con­sid­er­ing the sever­ity of injury from any giv­en haz­ard, this art­icle gives the read­er one pos­sible approach than could be used to select the types of injur­ies most likely to occur before scor­ing the rest of the risk para­met­ers.

References

[1] D. Nix, ‘Evaluation of Problems and Challenges in CSA Z434-​14 Annex DVA Task-​Based Risk Assessment Methodology’, 2015.

[2] National Floor Safety Institute (NFSI), ‘Quick Facts – Slips, Trips, and Falls’, 2015. [Online]. Available: http://​nfsi​.org/​n​f​s​i​-​r​e​s​e​a​r​c​h​/​q​u​i​c​k​-​f​a​c​ts/. [Accessed: 21- Jul- 2015].

[3] ‘Safety of machinery – Functional safety of safety-​related elec­tric­al, elec­tron­ic and pro­gram­mable elec­tron­ic con­trol sys­tems. IEC 62061.’, International Electrotechnical Commission (IEC), Geneva, 2005.

 

Digiprove sealCopyright secured by Digiprove © 2015
Acknowledgements: International Electrotechnical Commis more…
Some Rights Reserved

The Probability Problem

This entry is part 9 of 8 in the series Risk Assessment

In Occupational Health and Safety (OHS), risk is a func­tion of sever­ity of injury and the prob­ab­il­ity of the injury occur­ring. Understanding the sever­ity por­tion is usu­ally fairly easy, although advanced tech­no­lo­gies, like lasers for instance, take advanced haz­ard ana­lys­is meth­ods to assess cor­rectly. Risk vs. Cost and EffortProbability, on the oth­er hand, is a  chal­lenge. Mathematically, prob­ab­il­ity cal­cu­la­tions go from sub­limely simple to insanely com­plex. The simplest forms, like the prob­ab­il­ity of get­ting a num­ber from 1 to 6 when you throw a single die, can be under­stood pretty eas­ily. On any one throw of the die, you have a 1 in 6 chance of get­ting any one par­tic­u­lar num­ber, assum­ing the die is not weighted in any way. When we’re talk­ing about OHS risk, it’s nev­er that easy.

The First Problem: No Data

Risk assess­ment can be done quant­it­at­ively, that is, using numer­ic data. This approach is often taken when numer­ic data is avail­able, or where reas­on­able numer­ic estim­ates can be made. The prob­lem with using numer­ic estim­ates in quant­it­at­ive assess­ments, is this: Math lends cred­ib­il­ity to the answer for most people. Consider these two state­ments:

  1. After ana­lyz­ing the avail­able inform­a­tion, we believe that the risk is pretty low, because it is unlikely that the react­or will melt down.
  2. After ana­lyz­ing the avail­able inform­a­tion, we believe that the risk of a fatal­ity in the sur­round­ing pop­u­la­tion is very low, because the prob­ab­il­ity that the react­or will melt down is less than 1 in 1 mil­lion.

Which of these state­ments sounds more ‘cor­rect’ or more ‘author­it­at­ive’ to you?

Attaching num­bers to the state­ment makes it sound more author­it­at­ive, even if there is no reli­able data to back it up! If you are going to attempt to use quant­it­at­ive estim­ates in a risk assess­ment, be sure you can back the estim­ates up with veri­fi­able data. Frequently there is no numer­ic data, and that forces you to move from a quant­it­at­ive approach to semi-​quantitative approach, mean­ing that num­bers are assigned to estim­ates, usu­ally on a scale, like 1 – 5 or 1 – 10 rep­res­ent­ing least likely to most likely, or a fully qual­it­at­ive approach, mean­ing that the scales are only descript­ive, like ‘unlikely, likely, very likely’. These kinds of assess­ments are much easi­er to make as long as the scales used are well designed, with clear descrip­tions for each incre­ment in the scale, because the data used in the assess­ment is the opin­ion of the assessors. Here’s an example, taken from Chris Steel’s 1990 art­icle [1]:

Table 1: LO Likelihood of Occurrence
Scale Value
Description
0
Impossible, can­not hap­pen
0.1
Almost impossible, pos­sible in extreme cir­cum­stances
0.5
Highly unlikely, though con­ceiv­able
1
Unlikely, but could occur
2
Possible, but unusu­al
5
Even chance, could hap­pen
8
Probable, not sur­prised
10
Likely, to be expec­ted
15
Certain, no doubt

Some people might say that this scale is too com­plex, or that the descrip­tions are not clear enough. I know that the sub­tleties some­times get lost in trans­la­tion, as I dis­covered when try­ing to train a group of non-​native-​english-​speaking engin­eers in the use of the scale. Linguistic chal­lenges can be a major hurdle to over­come! Simpler scales, like that used in CSA Z432 [2], can be easi­er to use, but may res­ult in gaps that are not eas­ily dealt with. For example:

Table 2: Avoidance [2,Table A.2]
Category
Description
Criteria
A2
Not likely
Cannot move out of way; or inad­equate reac­tion time; or machine speed great­er than 250 mm/​s.
A1
Likely
Can move out of way; or suf­fi­cient warning/​reaction time; or machine speed less than 250 mm/​s.

A scale like the pre­vi­ous one may not be spe­cif­ic enough, or fine enough (some­times referred to as  ‘gran­u­lar­ity’, or in this case ‘gran­u­lar enough’) too be really use­ful. There are soft­ware pack­ages for risk assess­ment avail­able as well. One pop­u­lar product called CIRSMA™, uses a scale that looks like this:

Table 3: Probability of the Hazardous Event
Parameter Selection
Description
Possible — Easily able to avoid
Normally used to describe haz­ard­ous motions or events that:
  • Occur in plain view of the exposed per­son, and occur at a speed of less than 125 mm /​ second;
  • Can be read­ily fore­seen or detec­ted by the exposed per­son before the haz­ard­ous event occurs;
  • Are a res­ult of the actions of the anti­cip­ated exposed per­son and are under the dir­ect con­trol of the exposed per­son; and
  • Circumstances can be eas­ily and read­ily mod­i­fied or cor­rec­ted to avoid harm once a haz­ard­ous situ­ation has mater­i­al­ized.
Possible — Potentially able to avoid
Normally used to describe haz­ard­ous motions or events that:
  • Occur in plain view of the exposed per­son, and occur at a speed of more than 125 mm /​ second but less than 250 mm /​ second;
  • Could pos­sibly be fore­seen or detec­ted by the exposed per­son before the haz­ard­ous event occurs;
  • Are a res­ult of the actions of the anti­cip­ated exposed per­son and are par­tially under the con­trol of the exposed per­son; and
  • Circumstances can pos­sibly be mod­i­fied or cor­rec­ted in order to avoid harm once a haz­ard­ous situ­ation has mater­i­al­ized.
Unlikely — Unable to Avoid
Normally used to describe haz­ard­ous motions/​events that:
  • Occur either in plain view of the exposed per­son and occur at a speed of more than 250 mm /​ second or not in plain view of the exposed per­son and occur at a speed of less than 250 mm /​ second;
  • Are not likely to be fore­seen or detec­ted by the exposed per­son before the haz­ard­ous event occurs; and
  • Are not a res­ult of the actions of the anti­cip­ated exposed per­son but could be par­tially under the con­trol of the exposed per­son.
Impossible — Injury is Unavoidable
Normally used to describe haz­ard­ous motion/​events that:
  • Regardless of the loc­a­tion of the haz­ard, occur at such a speed that the exposed per­son would have little or no oppor­tun­ity to escape harm;
  • Could not be fore­seen or detec­ted by the exposed per­son before the haz­ard­ous event occurs;
  • Are not a res­ult of the actions of the anti­cip­ated exposed per­son and are not under the con­trol of the exposed per­son; and
  • Circumstances can­not to be mod­i­fied or cor­rec­ted in order to avoid harm once a haz­ard­ous situ­ation has mater­i­al­ized.

A scale like this is more descript­ive than the CSA scale, but less gran­u­lar and a bit easi­er to use than the Steel table.

Probability is also influ­enced by Frequency of Exposure to the haz­ard, and each of the tools men­tioned above have scales for this para­met­er as well. I’m not going to spend any time on those scales here, but know that they are sim­il­ar to the ones dis­played in terms of gran­u­lar­ity and clar­ity.

The Second Problem: Perception

This is the really big prob­lem, and it’s one that even the greatest minds in risk assess­ment and com­mu­nic­a­tion have yet to solve effect­ively. People judge risk in all sorts of ways, and the human mind has an out­stand­ing abil­ity to mis­lead us in this area. In a recent art­icle pub­lished in the June-​2012 issue of Manufacturing Automation Magazine, Dick Morley talks about the ‘Monty Hall prob­lem’ [3]. In this art­icle, Morley quotes colum­nist Marilyn vos Savant from her ‘Ask Marilyn’ column in Parade Magazine:

Suppose you’re on a game show and you are giv­en the choice of three doors. Behind one door is a car, behind the oth­ers, goats. You pick a door, say, num­ber one (but the door is not opened). And the host, who knows what’s behind the doors, opens anoth­er door, say, num­ber three, which has a goat. He says to you, ‘Do you want to pick door num­ber two?’ Is it to your advant­age to switch your choice?”

Here is where things start to go astray. If you keep your ori­gin­al choice, your chance of win­ning the car is 1:3, since the car could be behind any of the three doors. If you change your mind, your chances of win­ning the car become 2:3, since you know what is behind one door, and could the­or­et­ic­ally choose that one, or choose one of the oth­er two. Since you know for cer­tain that a goat is behind door three, that choice is guar­an­teed. Choose Door Three and get a goat. But if you choose to change your decision, your chances go from 33% to 66% in one move, yet most people get this wrong. Mathematically it’s easy to see, but humans tend to get emo­tion­ally dis­trac­ted at times like this, and make the wrong choice. According to Morley, stud­ies show that pigeons are actu­ally bet­ter at this than humans! When we start to talk about risk in abstract num­bers, like ‘one fatal­ity per year per 1 mil­lion pop­u­la­tion’ or stated anoth­er way, ‘1 x 10-6 fatal­it­ies per year’ [4], people lose track of what this could mean. We like to ourselves with time frame attached to these things, so we might tell ourselves that, since it’s June now and no one has died, that some­how the risk is actu­ally half of what was stated, since half the year is gone. In fact, the risk is exactly the same today as it was on January 1, assum­ing noth­ing else has changed.

In a recent court case involving a work­place fatal­ity, one expert wit­ness developed a the­ory of the risk of the fatal­ity using the Human Factors approach com­monly used in the pro­cess and nuc­le­ar indus­tries. Using estim­ates that had no sup­port­ing data, he came to the con­clu­sion that the like­li­hood of a fatal­ity on this par­tic­u­lar machine was 1 x 10-8, or roughly two orders of mag­nitude less than being hit by light­ning. In an intern­al HSE report in the UK the fol­low­ing chart In OHS, we believe that if a haz­ard exists, it will even­tu­ally do harm to someone, as it did in this case. We know without a doubt that a fatal­ity has occurred. The manufacturer’s sales depart­ment estim­ated that there were 80 – 90 units of the same type in the mar­ket­place at the time of the fatal­ity. If we use that estim­ate of the num­ber of that mod­el of machine in the mar­ket­place, we could cal­cu­late that the risk of a fatal­ity on that mod­el as 1:80 or 1:90 (8 x 10-1 or 9 x 10-1), sig­ni­fic­antly great­er than the risk of being struck by light­ning, and more than sev­en orders of mag­nitude more than estim­ated by the expert wit­ness. Estimating risk based on unproven data will res­ult in under­es­tim­a­tion of the risk and over­con­fid­ence in the safety of the work­ers involved.

Communication

Once a risk assess­ment is com­pleted and the appro­pri­ate risk con­trols imple­ments fol­low­ing the Hierarchy of Controls, the resid­ual risk must be com­mu­nic­ated to the people who are exposed to the risk. This allows them to make an informed decision about the risk, choos­ing to do the task, modi­fy the task or not do the task at all. This is called ‘informed con­sent’, and is exactly the same as that used by doc­tors when dis­cuss­ing treat­ment options with patients. If the risk changes for some reas­on, the change also needs to be com­mu­nic­ated. Communication about risk helps us to res­ist com­pla­cency about the risks that we deal with every day, and helps to avoid con­fu­sion about what the risk ‘really is’.

Risk Perception

Risk per­cep­tion is an area of study that is try­ing to help us to bet­ter under­stand how vari­ous kinds of risks are per­ceived, and per­haps how best to com­mu­nic­ate these risks to the people who are exposed. In a report pre­pared at the UK’s Health and Safety Laboratory in 2005 [5], authors Williams and Weyman dis­cuss sev­er­al ways of assess­ing risk per­cep­tion.

One approach, described by Renn [6], attempts to chart four dif­fer­ent aspects of risk per­cep­tion in people’s think­ing.

Perspectives on Risk
Fig. 1 – Graphing Risk Perception Factors

An example of these factors plot­ted on a graph is shown in Fig. 2 below. The data points plot­ted on the chart are developed by sur­vey­ing the exposed pop­u­la­tion  and then chart­ing the fre­quency of their responses to the ques­tions.

Fig. 2 – Graphing ‘Dread’

There are two factors charted on this graph. On the ver­tic­al axis, ‘Factor 2’ is the per­cept­ab­il­ity of the risk, or how eas­ily detec­ted the risk is. On the hori­zont­al axis is ‘Factor 1 – The Dread Risk’, or how much ‘dread’ we have of cer­tain out­comes. In Fig. 3 below you can see the assign­ment of factors to the pos­it­ive and neg­at­ive dir­ec­tions on these axes.

Fig. 3 – Dread Factors

 

 

At this point, I can say that we are a long way from being able to use this approach effect­ively when con­sid­er­ing machinery safety, but as prac­ti­tion­ers, we need to being to con­sider these approaches when we com­mu­nic­ate risk to our cus­tom­ers, users and work­ers.

Conclusions

When you are think­ing about risk, it’s import­ant to be clear about the basis for the risk you are con­sid­er­ing. make sure that you are using val­id, veri­fi­able data, espe­cially if you are cal­cu­lat­ing a numer­ic value to rep­res­ent the prob­ab­il­ity of risk. Where numer­ic data isn’t avail­able, use the semi-​quantitative and qual­it­at­ive scor­ing tools that are avail­able to sim­pli­fy the pro­cess and enable you to devel­op sound eval­u­ations of the risk involved.

Need more help? Contact me! Doug Nix

References

[1]     C. Steel. “Risk estim­a­tion.” The Safety and Health Practitioner, pp. 20 – 22, June, 1990.

[2]     Safeguarding of Machinery, CSA Standard Z432-​1994 (R1999).

[3]     R. Morley. “Analyzing Risk: The Monty Hall prob­lem.” Manufacturing Automation, June, 2012. p.26.

[4]     J. D. Rimington and S. A. Harbison, “The Tolerability of Risk from Nuclear Power Stations,” Health and Safety Executive, Her Majesty’s Stationary Office, London, UK, 1992.

[5]     J. Williamson and A. Weyman, “Review of the Public Perception of Risk, and Stakeholder Engagement”,  The Crown, London, UK. Rep. HSL/​2005/​16, 2005.

[6]     O. Renn, “The Role of Risk Perception for Risk Management.” in P.E.T. Douben (ed.): Pollution Risk Assessment and Management. Chichester et. al (John Wiley & Sons 1998), pp. 429 – 450

Digiprove sealCopyright secured by Digiprove © 2012 – 2014
Acknowledgements: See ref­er­ence list at the end of the more…
Some Rights Reserved