What I Know About Risk Management I Learned from Surfing
April 10, 2013
“Hey little boy, she’s gonna make you a man…”
- Sean Collins, Surfing Hall of Fame & Surfline creator
Surfing is risky business. There are uncertainties and sometimes danger. The costs can be serious injury, maybe death. However, the rewards can be high: tropical beaches, the perfect wave, surfer girls, and the endless summer.
Lesson 1: Risk management is key to survival.
I learned this from experience, mostly bad experience and bad advice, from my surfing buddies Demus and Pliers. Corollary: It is better to learn from the bad experiences of others.
Warm white sand, clear water, cool breezes through the palms and waves like glass; this was our favorite beach. It was perfect except for a few dangers that changed frequently. There were riptides, sharks and, worst of all, locals with bad attitudes. Demus and Pliers lived on the beach and were familiar with the conditions. As soon as I arrived at the beached I would ask Demus about the conditions. His response had two parts, one for the waves and one for the surfing dangers. He would say things like Good/Medium or OK/Pretty High.
This sounded good. Demus and Pliers were the experienced experts so I didn’t question them. Over time I began to learn more from my own bad experiences. When the dangers were High it was always better to stay out of the water, when the dangers Pretty High it was generally better to stay out of the water (unless the waves were Good),and when the dangers were medium it was occasionally better to stay out of the water.
This worked well for a while. Then it happened when least expected. Demus told me the waves were OK and the dangers were Pretty Low. I was subsequently caught in a riptide, attacked by a shark, and verbally abused by locals with bad attitudes. It seemed my basic risk management model had some flaws.
Lesson2: Understand and reevaluate your risk management model often, especially if your survival depends on it.
Corollary: Always question the experienced experts, especially the ones that live on the beach.
I took the initiative to gather more information about this method to manage surfing dangers. Demus told me that this was a common method. It is considered best practice. Everyone used it because it was qualitative, intuitive and easy to implement and modify. The only exception was a small group of surfers from the nearby university where they had access to high maintenance quantitative supermodels.
Lesson 3: Qualitative methods are usually developed and modified in isolation outside the areas of research.
Corollary: Anyone can develop their own version of a qualitative method and often do, even Demus and Pliers. Law 3: Demus and Pliers (like many risk managers) do not mix well with quantitative supermodels.
I implemented an enhancement. I requested second opinions from Pliers. More expert opinions should bring clarity. The next time I went to the beach I asked both Demus and Pliers about the current surfing dangers. Demus said Pretty High, while Pliers said Pretty Low.
Lesson 4: Qualitative estimates are understood and used in very different ways by different people.
Corollary: Two experienced experts will likely provide three or more opinions.
The research showed that qualitative estimates are subject to a variety of biases like overconfidence, representativeness bias, and insensitivity to prior probabilities among others. This didn't sound good. So I contacted other experienced experts at nearby beaches and learned that they used scoring systems with numbers. Risk analysis based on real numbers, this sounded good.
I invested a little to make Demus my Surfing Risk Officer (SRO). He worked closely with Pliers to develop my own Surfing Risk model. Demus identified three Key Risk Factors: Riptides, Sharks, and Locals with Bad Attitudes. Then Pliers converted the estimates for each Key Risk Factor from Low, Pretty Low, Medium, Pretty High and High to a corresponding ordinal number from 1 to 5. Demus and Pliers also assigned a probability to each Key Risk Factor. Now this was looking like a supermodel:
Surfing Risk = (p1 * Riptide) + (p2 *Sharks) + (p3 *Locals with Bad Attitudes)
The value for each Key Risk Factor was an average of the qualitative estimates provided by Demus and Pliers. p represented a probability from 0 to 1.0.
The next time I went to the beach Demus checked his spreadsheet said that the Surfing Risk was 5.6. Just to make sure I asked Demus how he got the number. Simple, Riptides were 2 with probability 0.5, Sharks were 3 with probability 0.3 and Locals with Bad Attitudes were 5 with probability 0.8. I check this against the recommended actions in my decision table where 15-13 is high, 12-10 is pretty high, 9-7 is medium, 6-4 is pretty low and 3-0 is low. The Surfing Risk was Pretty Low so I decided to paddle out.
I spent the day surrounded by Locals with Bad Attitudes and a couple of Shark. This was not good. Demus and Pliers explored other ways to improve the Surfing Risk supermodel. They integrated a common approach that adds weights to the Key Risk Factors. The weights could be adjusted to account for the differences that Riptides, Sharks, and Locals with Bad Attitudes had on my well-being. This helped some. While my decision making improved, the way the scoring method manipulated ordinal numbers was still a source of errors that led to some unexpected predicaments.
Pliers had an interesting solution. Locals with Bad Attitudes were four times more risky Sharks. The problem could be mitigated effectively and efficiently by spending four times more on mace than shark repellent. This sounded good, but I often had too much Shark repellant and not enough mace.
Lesson 5: Ordinal and cardinal numbers are different and should be treated differently.
Corollary: Beware of misbehaving supermodels.
Scoring methods based on ordinal numbers can introduce errors. Ordinal scales add ambiguities that +-affect the quality of risk analysis. Ordinal scales indicate relative order, not actual units of measure. They do not indicate magnitude. However, most scoring methods add and multiply values on ordinal scales as if they were real measures like price, length, distance, weight, depth, speed, etc. They are not.
The High, Pretty High, Medium, Pretty Low, Low ratings used to analyze risks related to surfing dangers are like the 5-star rating system for movies. A 2-star movie is simply better than a 1-star movie, not twice as good. It also doesn’t mean watching two 1-star movies are as good as watching one 2-star movie. It is not uncommon for Surfing Risk Officers to make critical decisions and allocate resources based on this type of reasoning.
The gnarly professor from the nearby university referred to common scoring methods with ordinal scales as "worse than useless" and "worse than random" (L. A. Cox, "What's Wrong with Risk Matrices?", Risk Analysis 28(2), 2008). I was not completely convinced. I believed that the Surfing Risk model has some value. It also has some weaknesses that need to be analyzed and addressed with better quantitative methods and probabilities. While I search for a well behaved supermodel I intend to spend more time scuba diving. In future blogs I will discuss what I learned about risk management while wrestling sharks and being trapped in an underwater cave.