Sunday, October 7, 2012

What's wrong with risk management?

Here's one of those provocative titles you see from time to time. This one, however, is from Matthew Squair (formerly DarkMatter and now Critical Uncertainties), and so it carries a bit of cache:
All You Ever Thought You Knew About Risk is Wrong

And, so getting to the points, there are two:

Point 1
In a word or two, it's a matter of utility (that is, perceived value vs risk) and the extremity of risk vs affordability

The St Petersburg Paradox, first posed by 18th century mathematician Daniel Bernoulli, is that in the face of constant expected value, we can not expect gamblers (or decision makers) to be immune to the potential for catastrophe between one risk scenario and another. The fact that scenario expected values are perceived differently, even though they are not, is the bias behind the idea of utility.

Example: if your project has a 10% chance of costing the business a million dollar loss on a project failure, is that any different than a project with a 1% chance of costing the business a ten million dollar loss? Or, another project with 0.1% chance of putting the business out of business with $100M in losses? At some point, there is capitulation: enough is enough. Sponsors won't take the risk, even though the expected value is constant--($100K)--and modest and equal in all three situations.

Thus, someone makes a utility judgment, applying their perceived value (fear in this case) to an otherwise objective value and coming up STOP! Expected value, as a calculated statistic, obscures the extreme possibility that may render the project moot.

Point 2:
We've all been taught that rolling a die 100 times in sequence is statistically equal to rolling 100 die one time. This property is called ergodicity--meaning statistics are stationary with doesn't matter when you do the rolling, the stats come up the same.

This idea that parallel and sequential events are statistically equivalent underlies the validity of the Monte Carlo simulation (MCS). We can do a simulation of a hundred project instances in parallel and expect the same results as if they were done in sequence; and, the average outcome will be the same in both cases.

But, what about the circumstances that afflict projects that are not time stationary: those circumstances where is does matter when in time you do the work? There's always the matter of resource availability, timing of external threats (budget authorization, regulatory changes), and perhaps even maturity model impacts if the project is long enough.

Consequently, when doing the MCS, it's a must to think about whether the circumstances are ergodic or not. If not, and if material to the outcome, then the MCS must be leavened with other reserves and perhaps major risk strategies must be invoked.

Maybe everthing you know about risk management is not quite right!