Monday, July 23, 2012

EVM and disjunction

The situation is disjunctive when everything must go right; nothing must fail. To say it another way: either failure or success is possible, but not at the same time. And disjunctive congnition occurs when one thing is given two distinctly different appearances or interpretations (we call it a 'success' but it's really a failure).

So much for the dictionary. Why should project managers care?

Well, for one thing, there's a cognitive bias to be concerned with that can affect many project situations:
Disjunctive bias: There is a bias towards the understating the likelihood of at least one failure and thereby overstating the likelihood of complete success

What's the usual countermeasure, the usual risk response? Make every single constituent very reliable so that nothing is allowed to fail.

Feel comfortable? Well, the problem here is that the math is against you.

The probability of at least one failure among a number of independent components (call them work packages, for convenience) is a binominal problem in probabilities, given by this messiness (click to see).

So, for example, consider this table of trouble, where
N = number of components
K = number of successes
N-K = number of failures

 N  K  N-K Prob of success  Prob failure Prob at least N-K failures
6 5 1 90% 10% 35%
4 3 1 90% 10% 29%
2 1 1 90% 10% 18%
6 5 1 99% 1% 6%
4 3 1 99% 1% 4%
2 1 1 99% 1% 2%
60 59 1 90% 10% 1%
60 59 1 99% 1% 33%
60 57 3 99% 1% 2%

The last couple of rows appear counter-intuitive. But, remember this is about a least 1 failure (or 3).
  • Third from the bottom tells us the likelihood of only one failure is not good
  • Second from the bottom tells us the likelihood of only one failure is good
  • The bottom row tells us that 3 failures is pretty improbable.

But still, with the high reliability, it's remarkably probable to get at least one failure in 60 objects, even though not three!

So, what if these objects are work packages, and the success/failure is 0/100% evaluation of the earned value?

What the table tells us is that even with hugely reliable work package managers, the likelihood of at least one failing is much higher than the individual failure rate, but the likelihood of more than one such failure is quite low. To wit: at least one failure: 33%; but three failures: only 2%

Thus, the cognitive disjunctive bias: who would have predicted 1 chance in 3 of getting a WP failure (on a WBS of 60 objects) when there is only 1 chance in 100 that a WP will fail?

If you want more about how this is calculated, visit the Khan