On today’s episode, we discuss whether risk matrices help us make better decisions. This is our first listener-submitted question!
In order to guide our discussion, we will use the paper Further Thoughts on the Utilities of Risk Matrices.
Topics:
Quotes:
“The assumption is that we use risk matrices, because they help us, in some way, to make decisions.”
“...What you’re representing on the matrix is less information than you started with: It’s either less precision than you had or its not representing the full range of uncertainty…”
“We’ve got a lot of tools in safety and risk management...and it’s worth knowing how those tools are being used and how effective people find them…”
Resources:
Ball, D. J., & Watt, J. (2013). Further thoughts on the utility of risk matrices. Risk analysis, 33(11), 2068-2078.
Anthony (Tony) Cox Jr, L. (2008). What's wrong with risk matrices?. Risk Analysis: An International Journal, 28(2), 497-512.
David: You’re listening to the Safety of Work Podcast, Episode 8. Today, we’re asking the question, do risk matrices help us make better decisions? Let’s get started.
Hey everybody. My name’s David Provan. I’m here with Drew Rae and we’re from the Safety Science Innovation Lab at Griffith University. Welcome to the Safety of Work podcast. If this is your first time listening, then thanks for coming. The podcast is produced every week and the show notes can be found at safetyofwork.com. In each episode, we ask an important question in relation to the safety of work or the work of safety, and we examine the evidence surrounding it. So, Drew, what’s today’s question?
Drew: David, our question for this episode is do risk matrices help us make better decisions? This is our first listener question which comes from David from Canada, who ask where did modern methods of risk assessment came from and how scientific they are. How scientific risk assessment is, is a really big question. Too vague to answer in this one episode. But David also asked specifically about one risk assessment practice, the risk matrix, and that’s what we’re going to talk about today.
A bit of background, in the early insurance and decision science, there was this thing called a risk matrix, which was a genuine mathematical matrix that you could use for calculations about expected values. Their basic use was you put in frequency and likelihood and you could use them to work out how much an insurance product was work when it packaged up various risks and various opinions.
What we call a risk matrix today is a sort of passed to descendants of those mathematical tools. It’s a table, usually a 3x3, 4x4, or 5x5 table, it’s got severity of consequences along one side and likelihood of outcome along the other. The cells in the table are usually colored. Typical colors are some green, some yellow, and some red to represent low risk, medium risk, and high risk. The general idea is that you start with a list of risks or hazards. Then you place each one on the table by estimating its severity and likelihood, and that then provides an indication of the priority of the risks.
Sometimes people will say directly that the reason we do this is as a decision tool. Sometimes people say that it’s not a decision tool, it’s a communication tool. But my thinking is that people communicate for a reason and that reason is usually to get a decision made even if it’s just an endorsement of what’s already been decided. In another case, the assumption is that when use risk matrices because they help us in some way can make decisions. We used to make decisions without matrices, today lots of decisions are made using risk matrices. It’s getting to the point where risk matrices are explicitly required by standards and in some places even by regulations or legislation.
The question we want to ask today is, do risk matrices help us make better decisions. Before we get into the [...] today, David, your thoughts on whether this is the right question to ask.
David: Before I answer that question, maybe I had one goal for today’s podcast which was try not to look like a fool because you have done so much research and writing in this space of risk assessment. My answer to that question now about risk matrices is that I think we think of them in a positivist kind of way. We are trying to understand a true value of risk and use that to make very rational or logical or objective decisions about where we should allocate resources and what we should do. My experience as we talk through the podcast is probably going to be risk matrices serve potentially more of a social purpose in organizations, to actually help people talk about risk as opposed to actually landing on what is the true risk.
Ultimately, in any case, whether matrices are estimating the right level of risk or whether they’re helping talk about it, at the end of the day, it’s all about making decisions. I think you’ve chosen probably the most essential question for risk matrices.
Drew: Yeah. When we talk about risk assessment, people often adopt fallback positions. They start off with this very positive approach and they say, “We use risk matrices to help us make decisions.” And when you point out problems with that they say, “Oh, we don’t really use them to make decisions. We use them to help us make decisions,” among many other factors. And then when you point out the problems with that, they say, “It’s not really about making decisions, it’s more just about communicating and socializing risk.”
It’s really important when we are calculating safety activities of any sort to be clear about what we think the benefit is and then do honestly evaluate against that original claim that we are making. Once we’ve decided on that original claim, we can then accept the fallback claim and start evaluating that claim instead. I think let’s start with the most positive thing you could say about risk matrices. It’d be really good if we did have a tool that would help eliminate human irrationality and help us make better decisions. Let’s genuinely ask that question, do risk matrices help us when we can’t make decisions?
David: Yup. For our listeners to put themselves in that mindset for this podcast, you’re facing an unfamiliar situation, you genuinely don’t know what to do and you whip out your risk matrix, you look at what’s in front of you, you apply it onto the criteria that you’ve established in your matrix and you then are able to make a decision that should be better than if you didn’t have that risk matrix to help you through that process.
The paper that we’ve chosen to review today is a paper titled, Further thoughts on the utility of risk matrices. The authors of the paper are David Ball and John Watt. At the time of the publication, Professor Ball was the director of Centre of Decision Analysis and Risk Management at Middlesex University. And Professor Watt is no co-director of that center. Also, I’ll link in the show notes one of the key reference papers titled, What’s Wrong with Risk Matrices by Professor Tony Cox at the University of Colorado. Because we’re able to use that paper to actually add some more science and more empirical research behind the main paper that we’re talking about today.
Drew, do you want to tell us about what sort of paper this is?
Drew: I have to admit that as academics, we don’t make things easy for practitioners when we write papers about risk assessment. We write lots and lots of papers and if you go searching for them, there are four main things that you’re going to run into. The first type of paper presents some new techniques for analyzing risk. These are very common papers. There are literally thousands of proposed risk assessment techniques.
The reason there are so many is that it’s a common mistake for new safety researches to just think that they see some sort of gap in the literature that they could fill by proposing a new technique or by modifying an existing technique or automating an existing technique. They do that without really understanding the risk assessment landscape. For a typical example of this C1 PhD thesis by Dr. Andrew Ray many years ago, others said it’s a very typical early researcher mistake.
The second type of paper is a variation on the first. It doesn’t just present a modified technique but it applies that technique to make claims about real world risks. These papers are even more common and they really pollute and dilute the literature. If you’re looking for information about a particular risk or about a particular type of risk assessment, you’re likely to run into one of these papers. The cardinal rule here is that you can’t validate a technique and use the technique at the same time. A paper that presents a new technique and claims to a major risk using that technique just doesn’t make sense.
The third type of paper seeks to provide information about some specific risk. And if done well, these should be really useful to practitioners. These give the scientific answers to questions like, how much should we really worry about a particular chemical (one of the most dangerous tools on construction sites)? What’s the most likely way to get injured by a train? The challenge in reading those papers is just knowing whether the techniques that they’ve used to measure the risk are reasonable techniques, that they’re providing some general real world information about risk.
The final type of paper is the sort that we are looking at today. These are the ones that a paper’s about how much you can trust risk assessment as a scientific or practical tool. Some of these are discussion papers, some of them are arguments papers, almost like [...] papers, and others contain some collection and analysis of real world data. Both the papers we are looking at today are mainly discussion papers and mathematical argument papers as you can probably tell from the titles. But the further thoughts about risk matrices paper actually includes a couple of interesting small experiments as well.
David, before we’re going to jump in here, have you looked at any sort of risk assessment stuff in your own research or in your own teaching?
David: My research into the role of safety professionals in organizations came across a lot of examples of safety professional involvement in risk assessment processes. It’s a very essential process to life as a safety practitioner. It was really interesting because in my research I was observing an interview and I was able to ask the question the practitioners not just what task you’re doing, but why you’re it, how you’re doing it, who’s involved, what’s the outcome, why is it a priority. It was really interesting in that I can’t recall any of the discussions that I had with people about their involvement in risk assessment activities coming down to the driver being we need to make a better decision here or we need to use the risk matrix is going to help us make a better decision.
Most of the reasons that the risk matrix was being used in a risk assessment process or just the risk assessment process more broadly was because of an administry [...] and/or political reasons. I’ll give two examples just to make that clear. The first was a team of safety practitioners that were involved in a risk assessment to justify a decision that a management team had already made, the management team had made a decision to make you change the operation and then they ask the safety practitioner to do a risk assessment to confirm that our decision is okay. That was one example.
The other example was a group of practitioners involved in a health risk assessment. The central activity in that risk assessment process was how can we make sure that this assessment or evaluation ends up being a particular box on the matrix so that we don’t have to go through all of their process of reporting it to senior management and doing all of the extra work that comes with that landing in a particular box.
Those are just two examples of how risk matrices when they hit organizational life they turn into things that administry or social or political as well as instrumental in trying to improve decision making.
Drew: That certainly matches some of my own observations when I’ve been studying major accidents. Is that people talk about risk assessments in hindsight as failed opportunities to understand the risk or to take action about the risk. If you look at risk assessments in context, very often they were being performed not as genuine decision-making activities, but in order to fill some social or political or regulatory function.
Taside, let’s talk about these specific criticisms of these risk matrices that come out of these papers. In each case, these are logical arguments, but then they are backed up by various types of evidence or analysis. The first big criticism we’ll label as the loss of information idea. This is mostly dealing with the question about whether risk matrices put hazards into the right order. The problem comes that risk matrices fundamentally contain less information in the matrix than we started with when we’re putting information in. Let’s consider two examples of that.
Let’s say we’ve got a hazard where we very precisely narrow the likelihood and the consequence of hazard, when we put that into a risk matrix, we put it into a broad category bin and that loses the precision that we had. Then let’s look at the other extreme where the likelihood and consequence are not precisely known. Then there are these big bins of uncertainty and the risk matrix doesn’t capture that uncertainty either. In fact, what we know about our hazard could position it across multiple cells and the risk matrix has now way to show that. It forces this to put that hazard into one particular cell.
Even worse than these two problems, hazards don’t just have one possible outcome with one possible likelihood. They could’ve range of possible outcomes each with their own likelihood. To put the hazard inquiries matrix, we pick one outcome and the likelihood of that one outcome and that can be really misleading. For example, if you’re in a moderate speed single vehicle car crash, you will probably have minor injuries, you might also die, but death has a much lower likelihood. Focusing on one of those two outcomes, the high probability of a minor injury or the low probability of death. Neither of those gives you a good picture of the full risk if the car crash.
David: Yeah. I think that’s where risk matrices immediately fall down where you are trying to access the likelihood of a consequence as opposed to potentially the likelihood of a particular event. You’re not getting much on the right hand side of the bowtie for those who are familiar with that kind of a technique through a risk matrix. I suppose that our listeners probably sitting there thinking, “Oh, but we thought numerous ways that our organization attempt to deal with the arbitrary or subjective nature of just picking a cell or picking a likelihood number or picking a consequence number.” I know that I have been in organizations who have tried everything from “We expect you to assess both the most likely and the worst case,” or, “We only want you to assess the worst credible case,” or, “We start to try to skew our evaluation calculations to assess two times consequence plus likelihood to try to get an evaluation that’s weighted towards the consequences.”
To any of these types of methods that organizations employ make up for any of the uncertainty that you’ve described?
Drew: Not really because most of those tell you a way of using the uncertainty, but they still have this fundamental problem that way you’re representing on the matrix is less information than you started with. It’s either less precision that you had or it’s not representing the full range of uncertainty because it’s forcing you to make a decision that you might not have the information to have. Even worse, as we go through these next criticisms, everyone of those fixes (and if you’ve got a particular fix in your own organization, maybe keep that in mind) is going to make these problems that come next even worse.
David: Yes. With the analogy be something like through the idea of putting a long technical report into a set of bullet points on a PowerPoint slide.
Drew: I can see from grin on your face (which doesn’t come across the podcast) you’re just trying to trigger me off here. The number of major accidents, where if you have to list under the true causes, we use PowerPoint in our risk assessment meeting, that’s exactly what we’re doing. Is we’re trying to reduce complexity in order to make decisions.
David: I can think of Colombia and I can also think of Nimrod just to start as without having any knowledge of disasters or anywhere near the knowledge of disasters that you have.
Drew: Yeah. Colombia was an accident caused by PowerPoint that predates the existence of PowerPoint.
David: What’s the next big criticism?
Drew: The second big criticism is technical consistency. This is not a visual medium but you’re going to have to bear with me for a moment. Risk matrices are supposed to be a rough semi-quantitative, semi-qualitative view of a real world that’s actually quantitative. For the matrix to make sense, you really should be able to draw two lines through the matrix. One line separates the green from the yellow and the other line separates the yellow from the red. After it lets the decision we’re trying to make. At every point on the line should have the same amount of risk, or otherwise it doesn’t make sense.
On the risk matrix, these lines are bumpy because we’ve got boxes. And in the real world, those lines are curves, they are smooth curves. When we take things from the quantitative real world into the semi-quantitative matrix, these curved lines are actually going to be crossing through the middles of some boxes and the edges of other boxes.
In Cox’s paper, he asked, “How much this matters?” He suggested that there’s a few basic consistency rules that everyone should be able to agree with. For example, the same curved line, the line of equal risk shouldn’t go through both green and red cells, otherwise you’re saying that these things one is unacceptable, one is acceptable, whereas actually they have the same level of risk. For the same reason, there shouldn’t be entire cells that appear on the wrong side of the line.
Now they are very simple rules and you’d think that even within those rules there’ll be a lot of flexibility, but Cox did the maths for small matrices and he was able to show that there’s exactly one sensible coloring scheme for 3x3 matrices. There’s exactly one sensible colorings theme for 4x4 matrices. And for 5x5 table, there’s a massive 2 sensible color schemes, but both of those use only three colors. Any other possible coloring of red, green, or yellow is going to break the rules leading to gross inconsistencies.
When I read that, I blinked because I didn’t actually know that. I have to go back and check his maths and work it out. The logic is really solid. I consider myself a bit of an expert on risk assessment and I didn’t even know that these rules existed. How many real world matrices out there do you reckon actually follow Cox’s coloring rules?
David: I must admit I was in the same camp as you, I hadn’t heard of that until today. We can basically say if you’ve got a 5x5 or less matrix and you’ve got 4 colors, then you’re breaking the mathematical rules. I see a lot of different configurations of colors on 5x5 matrices in a lot of different organizations. I’ve been involved in moving colors on risk matrices in organizations throughout my career without applying much science there. Mainly, what you’re trying to do, we just go, “Oh, we’ve got too many risks in this red box for our high consequence low likelihood thing, so we might make that box orange instead of red because we don’t want to do all the reporting and the assurance that’s associated with all these things that might never happen.”
I think there’s a real red flag—excuse the pun—for organizations around their risk matrices. Maybe just check whether the colors actually makes sense based on the science and don’t go moving stuff around unless you’re prepared to have grossly inconsistent results in your risk assessments.
Drew: This speaks directly to whether these matrices are good for helping us make decisions. The first criticism is that they reduce information, and that reduces the [...] to make good decisions. The second criticism is that the technique can actually introduce inconsistencies. It can directly lead to us making inconsistent decisions. The third big criticism is about how humans use the matrix. This isn’t the mathematical property, this is about human judgment. That’s what the experiments in the paper we’re looking at this week.
In these experiments, the participants were risk management or occupational health and safety students. Normally, I’m pretty skeptical about student experiments. It’s what a lot of psychology departments do, is they use students as their experimental subjects and the result is we have all of these psychological phenomena that are really only prefer your white Anglo-Saxon, Protestant American psychology students.
But in this case, the whole point of risk matrices is: (a) they’re supposed to be used by amateurs, and (b) they’re expected to be used by people who are doing risk management. You expect risk management students to have at least an above average understanding of risk concept. I don’t think this is not an unfair participation group.
In the first task, 50 students were asked to place 3 for from hot scenarios onto a 5x5 matrix. The three scenarios were a railway platform, a public footpath at the top of a cliff, and a medieval bridge at a historical site. The students were given photos and information about each site that will be in a few minutes just to put them on to the matrix.
David: For our listeners, if they can imagine the task that these people are given photograph, people walking across the bridge, given a matrix, given a bit of information and said, “Assess the risk of falling from height.”
Drew: This is something you can do in your own organization, just for fun if you is, grab your own organization’s risk matrix, grab a couple of hazards, take photos on your phone of what hazards look like, write a short paragraph description and just go around seeing where everyone puts those same hazard on through the same matrix. Just to get a feel for how consistent these decisions are made in your own organization.
That’s the first task. In the second task, 21 students from the same group were given everything from the first task. Including all of the outputs, they knew what every single student had rated for every single hazard. They were asked to write an essay where they commented on those ratings and then assigned and justified their own new ratings. This was set as an assessment task. They had several weeks to complete it, they could go on the internet, they could look for expert information, anything they like to do the task properly.
The results from this experiment showed that firstly, for all three scenarios, the results were widely scattered. In fact, they were so scattered that when you look at the plots of the results, it’s impossible to put the three hazards into end order. If this was a decision making through which one of these hazards should we deal with first, the task obviously could not help you with that decision.
David: You’re always quite surprised just how scattered those results were and particularly on both the likelihood. It was almost an even distribution across all of five cells on the matrix. Just as many people have almost put an answer in every one of those boxes. That’s a big range from this is not going to happen through to this is almost certain. People who had a similar level of capability in risk assessment looking at the same photograph with the same information.
Drew: Yeah. It’s supposed to be because when you’re doing undergraduate occupational health and safety training, presume the same is true for post graduate. You thought basic things about falls from heights. What the height about reach you can expect some injuries to be unlikely, some injuries to be almost certain. The people knew how high each of these things were. The very least you’d expect them to just pull out their notes or mentally remember their notes, where they get a rough estimate to how high the bridge is, how high the fall is and give that answer. But these people with the same training all get different answers.
David: I think that’s a relatively simple issue and if we struck light that out to where our organizations are also using some of these risk matrices to make more difficult decisions, we’re on technical systems like what’s the risk of some corrosion on a particular piece of process piping or something like that. And when you start thinking about much more technical situations, then I start to worry even more about just what the range of answers you’re going to look like in those scenarios.
Drew: Yeah, absolutely. The others expected that things were going to get more consistent in the second task because now the students have the opportunity to actually look up the answer. They can go online, they can look up these particular sites, these particular locations, they can look for similar locations, they can look for typical evidence about typical fall risks. And many of the students did in fact do this research. We’ve got their essays. We know what they said, we know that they did go and get that evidence. On average, the hazards were just as scattered. The only change between the two tasks was that on average, all of the hazards were shifted downwards in risk. One of the hazards shifted down a bit more than the other hazards. But all of the hazards were just as scattered.
The authors drew three main findings from these experiments. The first one was that assessors assign very different ratings to the same hazard. That was true in this experiment. That’s consistent across all risk assessment research. Even when we have very expert assessors. The second one, which I think is a little bit new, was that this scatter isn’t due to lack of information. The scatter didn’t change between when they had a few minutes to when they have the opportunity to get expert information. The third one comes from the essays that they wrote, that the scatter isn’t just a difference in estimation. It reflects deep underlying differences not just about how they understood the hazards, but what they included as in scope and out of scope for risk assessment, what mattered and didn’t mattered, and how they conceived and understood the very notion of risk. All of that varied between the people during this assessment.
David: Those are really interesting findings. Let’s talk about practical takeaways. We’ve introduced this matter in your practical takeaways throughout the podcast so far. But let’s start by reflecting the fact that it’s always worth asking whether our decision tools and writing are actually helping us make better decisions.
I think our listeners could all reflect on just what decisions are being made off the information in your risk matrices or your risk assessments? Are they actually informing decisions in real time? How is the information being used? And then, you can start to talk to people in the organization about how they’re using the risk matrix and are they using it to make decisions. We’ve got a lot of tools in safety and risk management within our organization and it’s worth knowing how those tools are being used and how effective peoplefind them to be.
Drew: This really brought to mind that there’s a common practice if someone’s having difficulty in making a decision, where you ask them, “Okay, think about what matters to you in this decision. Why don’t you pick some criteria and rate each of your options against those criteria?” Typical one is you buy a new car. What’s important to you? Is it value for money? Is it speed? Is it good looks? Is it how much it can carry? Is it how much it cost to maintain? Let’s write down each one of those and against each one of those options, let’s rate each of the cars.
There’ve been experiments that have shown that doing that formal defined process actually makes people less happy with their final decision, not more. When you come back a few weeks later and ask them, “Do you regret the buying decision?” They are more likely to say they regret it if they followed that careful process than if they just informally used their own judgment. That’s what we need to be really careful with any form of risk assessment, is things can look more methodical. But that doesn’t necessarily mean even if it’s going to make us make decisions that we’re happy with, let alone decisions which are objectively better decisions.
David: I think that’s really important once you’re there. Also, the hidden uncertainty, the thing that you don’t know like you said, you see the information in the matrix and you hear what people are saying about the risk, you just don’t know what their underlying assumptions are. Getting underneath that hidden uncertainty is really, really important. Drew, do you want to talk about that?
Drew: Yeah. Let’s imagine that we did this assessment not with 50 students working independently. But we broke the students up into groups of three. And we only pick one of those answers. In that one answer, there would probably be a clear ordering of the risk. We go away thinking this team has made a decision and the riskiest one is option B, the second riskiest is option A, the third riskiest is option C. That’s typical in what we do in organizations, is we just do risk assessment for each problem once with one team and it looks like we’ve got an answer. It’s only when you repeat the task over and over (in this case getting 50 people doing it independently) that you see that that one answer isn’t actually a clear ordering, it’s just rolling the dice and picking from many possible orderings. It’s just an arbitrary decision, not a careful one.
David: What can people practically do then? What would you advice them? Would you advise people to have the same groups repeat risk assessments in the organization? Would you advise people to go and get multiple groups to assess the same risk and compare what they come up with? How would you approach this?
Drew: Honestly, all of those different things, all they do is reveal to you how uncertain your decision is. Doing that is certainly a good idea if you don’t believe me. If you think that your risk assessment is definitive, you think it’s doing a good job, then yes, go out and get multiple people to independently do the same risk assessment and see for yourself how random it is.
But for a practical takeaway, the important thing is that once you accept that risk assessment is a bit arbitrary, once you accept that even the methodical things are just random preferences, then the thing to do is focus on risk reduction, don’t focus on risk. You should really only care about decision making tools if the end result is using those tools, is that you are better are risk reduction. If they’re taking up taking up time and attention, and they’re taking away focus from what can we do to reduce this risk, then skip the risk assessment part of it, skip using the risk matrix and jumpstart to, we got five hazards here, forget about what order they’re in, which one of these five has hazards books the best thing we can do for each one.
David: Yeah. Just thinking on the spot, I think I can see in your risk matrix which has just got two cells a risk that we need to do something about and risk that we’re happy that we don’t have to do anything about. If it’s in the box that says we need to do something about it, then get on and do something about it. I think that was one of the things that came out a lot of my research, an advice that I give to a lot of safety professionals which is make activities in your organization that you’re performing, go directed risk reduction activities.
Drew: I love the idea of the two box matrix because I think it’s going to make people really uncomfortable. People are going to really dread putting things in that box of things that we don’t need to do anything more about. You should be aware that that is essential what you’re doing when you use that risk matrix. Expanding it out to 5x5 cells and then deciding not to do anything about it is exactly the same as having two boxes and putting in the box that says, nope, we’re not going to do anything about it.
David: Yeah, it’s a good place to put all of those injuries where someone rolls their ankle getting out of a car on a flat concrete road. Lots of those types of things that we consume organization resources around, which are probably best in that box, which is just don’t do anything about it.
Drew: And if it is something serious, if it’s something that we can and do something about, then no risk matrix should be making us change our minds about that.
David: I think the follow on from that practical takeaway about using the risk matrix simply to know what you need to do something about and not getting too hung up over exactly what your cell at sitting in the matrix. The risk matrix in your organization will have a lot of social or at least a lot of political value in your organization. I’m assuming that most of our listeners will working in organizations, the organization will have a risk matrix and it will be central to a lot of reporting and a lot of conversations about safety and risk in the organization.
It will be something that you’ll use and be able to use to bring certain risks to the attention of certain people and processes in the organization. I know that I’ve been involved in doing that with all of the right intention of using the matrix and using particular cells in the matrix to highlight risks that were, in my opinion, not that well understood by the organization. I’m not encouraging our listeners to game the matrix, but I’m encouraging our listeners to understand the social and political value of these tools and to use them to increase understanding about if safety needs to be directed.
Drew: One thing that I’m a fan of is drawing big circles that cover multiple boxes. It takes a little bit of bravery to do that, but I think, honestly, that is what we’re trying to represent and show, is our current state of knowledge of this hazard is that it could be anywhere from acceptable to unacceptable. If people want to move the boxes in, they’re going to have to do something to reduce the size of that circle, which involves doing more investigation, or doing more understanding about the risk, in order to understand and control it, in order to put it into a box.
David: I really like that. I think from some of my reading last year when I was teaching the Understanding Risk subject in the Masters Program at Griffith around the uncertainty or the nature of uncertainty. We need to understand the likelihood and consequence, but more so we need to understand uncertainty. By putting that big circle or that big oval, depending on whether it’s the likelihood or the consequence that you’re most uncertain about, I really like that idea.
I hadn’t thought about that or heard of that idea before, but that would be something that I’d really advice our listeners to do. For risk that you’re completely convinced are, in my particular cell, put them in there. Once you don’t quite know, just put a big circle over about four or five cells and use that to prompt the discussion about how much information you have about that risk to make your decision.
Drew: So this makes me ask the question, do risk matrices help us make better decisions? The answer to that question is, we can’t say for sure, but we’ve got three very good reasons to think that risk matrices actually make it harder to make good decisions. Friend of the pod, Tom Concun, suggested that we ought to add a segment at the end of these podcasts. Add asking some questions. This might be things that you, our listeners, already know the answer to, or where you got some paper you’d like to us to read, or where you can point us to some things that we can cover on future shows.
It’s something that I’d certainly like to know, is whether people genuinely like risk matrices. I heard that lots of people use them, but do we use them because we see value or because we all feel that we have to? Maybe there are some people out there, maybe people even listening to the show, who find some benefit to risk matrices. There might be some benefit that we haven’t thought about, or there might be some way of using risk matrices or fixing risk matrices that doesn’t run into the problems that all of Watt and Cox have pointed out in their two papers. Let us know if you use risk matrices and why is it that you use it? Is it by choice? And if so, what’s the benefit that you see in them?
David: Yeah. I’d really like to know how organizations are using them to make decisions outside of just reporting and collating information. I don’t see too much but these days where it’s actually used in real time decision making, in a somewhat objective kind of way where a group of people come together and use it to facilitate the decision making process. If anyone’s got examples of where their organization’s using it to facilitate with decision making process, then I’d be really interested in that.
Drew: That’s it for this week. We hope you found this episode thought-provoking and ultimately useful in shaping the safety of work in your own organization. Join us on LinkedIn for discussion about the episode, or send any comments, questions, or ideas for future episodes to feedback@safetyofwork.com.