Saturday, December 10, 2016

More on the risk matrix

In 1711 Abraham De Moivre came up with the mathematical definition of risk as:

The Risk of losing any sum is the reverse of Expectation; and the true measure of it is, the product of the Sum adventured multiplied by the Probability of the Loss.

Abraham de Moivre, De Mensura Sortis, 1711 in the Ph. Trans. of the Royal Society

I copied this quote from a well argued posting by Matthew Squair on Critical Uncertainties entitled Working the Risk Matrix.  His subtitle is a little more high brow: "Applying decision theory to the qualitative and subjective estimation of risk"

His thesis is sensible to those that really understand that really understanding risk events is dubious at best:
For new systems we generally do not have statistical data .... and high consequence events are (usually) quite rare leaving us with a paucity of information.

So we end up arguing our .... case using low base rate data, and in the final analysis we usually fall back on some form of subjective (and qualitative) risk assessment.

The risk matrix was developed to guide this type of risk assessments, it’s actually based on decision theory, De’Moivres definition of risk and the principles of the iso-risk contour

Iso-risk contour

Well, I've given you De’Moivres definition of risk in the opening to this posting. What then is an iso-risk contour?

"iso" from the Greek, meaning "equal"
"contour", typically referring to a plotted line (or curve) meaning all points on the line are equal. A common usage is 'contour map' which is a mapping of equal elevation lines.

So, iso-risk contours are lines on a risk mapping where all the risk values are the same.

Fair enough. What's next?

Risk matrix

Enter: decision theorists. These guys provide the methodology for constructing the familiar risk matrix (or grid) that is dimensioned impact by probability. The decision guys recognized that unless you "zone" or compartmentalize or stratify the impacts and probabilities it's very hard to draw any conclusions or obtain guidance for management. Thus, rather than lists or other means, we have the familiar grid.

Each grid value, like High-Low, can be a point on a curve (curve is a generalization of line that has the connotation of straight line), but Low-High is also a point on the same curve. Notice we're sticking with qualitative values for now.

However, we can assign arbitrary numeric scales so long as we define the scale. The absence of definition is the achilles heel of most risk matrix presentations that purport to be quantitative. And, these are scales simply for presentation, so they are relative not absolute.

So for example, we can define High as being 100 times more of an impact than Low without the hazard of an uncalibrated guess as to what the absolute impact is.

If you then plot the risk grid using Log Log scaling, the iso-contours will be straight lines. How convenient! Of course, it's been a while since I've had log log paper in my desk. Thus, the common depiction is linear scales and curved iso-lines.

Using the lines, you can make management decisions to ignore risks on one side of the line and address risks on the other.

Common problems

There are two common problems with risk matrix practices:
  1. What do you do with the so-called "bury the needle" low probability events (I didn't use 'black swan' here) that don' fit on a reasonably sized matix (who needs 10K to 1 odds on their matix?)
  2. How do you calibrate the thing if you wanted to?
 For "1", where either the standard that governs the risk grid or common sense places an upper bound on the grid, the extreme outliers are best handled on a separate lists dedicated to cautious 360 situational awareness

For "2", pick a grid point, perhaps a Medium-Medium point, that is amenable to benchmarking. A credible benchmark will then "anchor" the grid. Being cautious of "anchor bias" (See: Kahneman and Tversky), one then places other risk events in context with the anchor.

If you've read this far, it's time to go.

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!
Read my contribution to the Flashblog

Wednesday, December 7, 2016

Storyboards ... if you can't draw it...

A great technique for writing proposals, papers, or books -- as I do a lot -- is to storyboard the ideas.

My favorite expression: "If you can't draw it, you can't write it!"

Here's Einstein on the same idea:
I rarely think in words at all. A thought comes and I may try to express it in words afterwards
If you're not a storyboard person, check out this website for insight...

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!
Read my contribution to the Flashblog

Sunday, December 4, 2016

Data quality, data parentage

My early career was in technical intelligence, so I was struck by this phrase applicable not only to that domain, but to my present domain -- project management:
"The value of [information] depends on it's breeding. .. Until you understand the pedigree of the information you can not evaluate a report. We are not democratic. We close the door on intelligence without parentage."
John LeCarre

Some years ago, Chapter 11 of the PMBOK was rewritten to include "data quality" as an element of risk understanding and analysis. Certainly, some of the motivation for that rewrite was the idea of information parentage -- information qualities.

The idea here is not that data has meet a certain quality standard -- though perhaps in your project it should -- but that you as project manager have an obligation to ascertain the data qualities. In other words, accepting data in a fog is bound to be troublesome.

If some attributes are unknown, or unknowable, at least you should do the investigation to understand whether or not the door should be closed. After all: there's no obligation to be democratic about data. Autocrats accepted!

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!

Read my contribution to the Flashblog

Thursday, December 1, 2016

Kahn Academy on game theory

Chapter 12 of my book, "Maximizing Project Value", posits game theory as tool useful to project managers who are faced with trying to outwit or predict other parties vying for the same opportunity.

When John von Neumann first conceived game theory, he was out to solve zero-sum games in warfare: I win; you lose. But one of his students challenged him to "solve" a game which is not zero-sum. To wit: there can be a sub-optimum outcome that is more likely than an outright win or loss.

For this most part, this search for compromise or search for some outcome that is not a complete loss is throughout the business world, the public sector (except, perhaps, elective politics), and certainly is the situation in most project offices.

The classic explanation for game theory is the "prisoner's dilemma" in which two prisoners, both arrested for suspected participation in alleged crimes, are pitted against each other for confessions.

The decision space is set up with each "player" unable to communicate with the other. Thus, each player has his/her own self interest in mind, but also has some estimate of how the other player will react. The decision space then becomes something like this:
  1. If only you confess, you'll get a very light sentence for cooperating
  2. If you don't confess but the other guy does, and you're found guilty, you'll get a harsher sentence
  3. If both of you confess, then the sentence will be more harsh than if only you cooperated, but less harsh than if you didn't cooperate
  4. If neither of you confess, risking in effect the trust that the other guy will not sell you out, you and the other prisoner might both go with a fourth option: confess to a different but lesser crime with a certain light sentence.

From there, we learn about the Nash Equilibrium which posits that in such adversarial situations, the two parties often reach a stable but sub-optimum outcome.

In our situation with the prisoners, option 4 is optimum -- a guaranteed light sentence for both -- but it's not stable. As soon as you get wind of the other guy going for option 4, you can jump to option 1 and get the advantage of even a lighter sentence.

Option 3 is actually stable -- meaning there's no advantage to go to any other option -- but it's less optimum than the unstable option 4.

Now, you can port this to project management:
  • The prisoners are actually two project teams
  • The police are the customer
  • The crimes are different strategies that can be offered to the customer
  • The sentences are rewards (or penalties) from the customer

And so the lesson is that the customer will often wind up with a sub-optimum strategy because either a penalty or reward will attract one or the other project teams away from the optimum place to be. Bummer!

There are numerous YouTube videos on this, and books, and papers, etc. But an entertaining and version is at the Khan Academy, with Sal Khan doing his usual thing with a blackboard and and voice over.

And, you can read Chapter 12 of my book: "Maximizing Project Value" (the green/white cover below)

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!
Read my contribution to the Flashblog

Monday, November 28, 2016

Agile in the critical systems space

I read a recent posting about agile from a very odd corner of the PM space for an agile conversation to be: CriticalUncertainties, a (conservative) blog about critical safety and failure (or fail safe) requirements in complex systems.

But, nonetheless, we get this input from critical systems safety expert Mathew Squair:
When you don’t know what to do, don’t sit down and plan what you don’t know, get people moving, talking, collaborating and making stuff. Then out of that activity you’ll find the information will emerge that will allow you to make decisions......
As Tom Peters points out we need to understand whether our methodologies have an inherent bias for action or a bias for planning, and then whether the situation is complex (but understood and stable) where planning will pay off or uncertain (with high novelty and volatility) where talking, thinking and looking at the small grain issues to build a picture of where we are is what we ought to be doing.

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!
Read my contribution to the Flashblog

Monday, November 21, 2016

Understanding project statistics

Chapter 14 from the GAO Cost Estimating Manual on "Cost Risk and Uncertainty" is a good read, easily understood, and very practical in its examples.

Here's one illustration that I particularly like.  When you look at it, it's understood in a moment that the repeated random throw of two dice generates a probability density function [PDF] that has a bell-shape curve.

Statisticians call this phenomenon the Central Limit Theorem: random occurrences over a large population tend to wash out the asymmetry and uniformness of individual events, such that a "central tendency" occurs around the more probable outcomes.  A more 'natural' distribution ensues.  The name for this somewhat natural distribution is the Normal distribution, more commonly: the bell curve.

Here's what it looks like to a project manager.  
Regardless of the distribution risks in either cost or schedule as adopted by work package managers for each individual work package, in the bigger picture at the summation will tend to be a bell-shaped distribution of risk.

Consequently,  the project manager's doesn't really need to understand the parameters of variation for each work package. The Central Limit Theorem does all the work. Triangles, Rayleighs, and even Binominal distributions are become bell shaped in the big picture.

  This diagram is (again) from Chapter 14 of GAO's manual:

If the risk analyst generates these data from a simulation, like a Monte Carlo simulation, then the numeric statistics like variance and standard deviation are usually reported, along with the cumulative probability more commonly called the "S" curve.  In the diagram, on the right side, we see the cumulative curve plotted and labeled on the vertical axis as the confidence level.  With a little inspection, you will realize that the cumulative curve is just the summation of the probabilities of the bell curve that is adjacent on the left.

The GAO manual, and especially Chapter 14, has a lot more information that is well explained.  Give it a read.

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!
Read my contribution to the Flashblog

Thursday, November 17, 2016

Adding staff -- slowly

First, we have "Brooks Law", as given in the classic case study "The Mystical Manmonth" by Dr. Fred Brooks:

"Adding staff to a late project makes it later"

I was no more thinking about that idea, than I read this missive in a history of the Civil War that I am engaged with:

“The veterans looked across the open ground at the newcomers with complete and unconcealed skepticism and hostility. In every line of their bearing—in the set of their jaws, the tilt of their heads, the look about their eyes peering out from under those valued hatbrims—they expressed for all to see the age-old, impersonal, unformulated feeling of the veteran for the recruit:

We have had it and you have not, and until you have been where we have been and have done what we have done we do not admit you to any kind of fellowship.

Excerpt From: Catton, Bruce. “Glory Road.” This material may be protected by copyright.

OK, that might be tougher than the normal project team might be, but in my experience, until there is bonding over a common stress, there's not cohesion, and maybe not even functional integration.

So, as in war and most other things, to speed assimilation along, sometimes a bonding experience is needed. Thus, all the bonding games, etc, but it often works. Else, just put everybody in the deep end. Survival will do all that is necessary

And, did I mention virtual teams: there's really not a difference, not really.

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!
Read my contribution to the Flashblog