Monday, February 25, 2013

Plausibility and probability


Ponder this for a moment:
Adding detail to the description of an uncertainty makes it less probable (than a more abstract version), though it makes it more plausible

Now move forward to that first tool of choice among risk managers, the risk register. The logic is this:
As detail abounds and abstraction abates, plausibility augments but probabilities must be adjusted; and, in general, probabilities go down.
Try this on for an example:
Scenario -- you are looking at number of interns or co-ops with an idea of placing them in various roles in your project, perhaps business analysis, engineering, or training and business readiness.

Adel is 22 years old, very smart, technically accomplished, studied (among many subjects) philosophy and music, has a strong sense of who she is, and is very detail oriented. She has leadership presence and personal charisma
 
Adel is a professional; she could be successful in some aspects of engineering, finance, business analysis, etc. or even marketing.

The question -- What do we think of Adel? Among those professional roles mentioned, which is the most likely? If engineering, then is she more likely a software designer/engineer who applies the female touch to typically qualitative requirements (needs and wants), or else is she a professional in some other aspect of engineering?

Most of us would look at it this way:
  • Music is a good segue to software; music has patterns, logic, rhythm, and detailed structure -- all attributes of software as well
  • Philosophy is about understanding beliefs, values, ways to think, and ideas -- all elements of human factors
  • And, she's technically accomplished
Most of us would conclude: Adel is very plausibly a software engineer; perhaps she would be very good at human factors engineering. Finance and business analysis seem just too dry for someone who's into philosophy and music. Of course, you can't dismiss marketing out of hand....

Yes! that's it. Adel is most likely a software designer/engineer. That's the highest probability in the distribution of plausible possibilities.

Not so fast
What about other engineering disciplines? Sure, it's possible, but is it probable? Look at how plausible the fit is to software design engineering.

Venn et al
Let's look at the Venn diagram:
  • Among all roles for women, Adel is a professional
  • Among all professionals, Adel is possibly an engineer
  • Among all engineers, Adel is possibly a software design engineer
  • Among all software design engineers, Adel is possibly a specialist in human factors engineering
Every time we add detail, the plausibility improves (given Adel's profile), but the share of the population goes down each time. If this were red balls in an urn with white balls, each level of detail would mean fewer red balls.

What if we are just comparing two abstractions of about equal detail, as in marketing v engineering? The rule does not apply: we've not changed the level of detail from one to the other. We're just comparing two possibilities.

Moving to the risk register
Every time we are challenged to add detail to the risk register, we may well improve plausibility (why else add detail?), but the rule is: that finer grain is less probable than the more abstract idea from which it was derived.

So, what's the problem here? The problem is we confuse plausible with probability. Just because an uncertain event seems very plausible doesn't correspondingly mean it is very probable.

Read more
This discussion is given in much richer detail in Daniel Kahneman's book: "Thinking fast and slow".