Wednesday, March 19, 2014

When the evidence doesn't compute...

Oops! Something went wrong... the project analysts followed this standard paradigm in three steps, given as:
1. There is a scientific or policy question of some theoretical or practical importance.
2. Researchers gather data on relevant outcomes and perform a statistical analysis, ideally leading to a clear conclusion ..
3. This conclusion informs policy.
Think of this as PLAN A.

But Andrew Gelman asks: "What happens if Step 2 fails?"

Well, we could take Jurgen Appelo's advice: "Measure everything; Conclude nothing". Off hand, I don't see how that moves the ball very well, or very far, but hey!.. Appelo has his reasoning.

Nonetheless, Step 2 does fail -- in fact, in the project domain Step 2 fails fairly often -- and then what's a PMO to do? Yogi Berra's advice comes to mind:"When you come to a fork in the road, take it!"

Enter PLAN B:
Here's Plan B: Invoke your policies and protocols for decision making in the face of uncertainty. If you are a bit short on these, take a look at the work of Tversky and Kahneman: they've written some very informing papers on this very topic.

In short, even if the conclusion is not clear, not unambiguous, not not self-conflicting (wicked), make a judgment as informed as possible and press on! Franklin Roosevelt: "Try something!"

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!
Read my contribution to the Flashblog