Monday, October 13, 2014

Ask a SME... or Ask a SME?

It seems like the PM blog sphere talks constantly of estimates. Just check out #noestimates in Twitter-land. You won't find much of substance among thousands of tweets (I refrain from saying twits)

Some say estimates are not for developers: Nonsense! If you ask a SME for an estimate, you've done the wrong thing. But, if you ask a SME for a range of possibilities, immediately you've got focus on an issue... any single point estimate may be way off -- or not -- but focusing on the possibilities may bring all sorts of things to the surface: constraints, politics, biases, and perhaps an idea to deep-six the object and start over with a better idea.

How will you know if you don't ask?

Some say estimates are only for the managers with money: Nonsense re the word "only". I've already put SMEs in the frame. The money guys need some kind of estimate for their narrative. They aren't going to just throw money in the wind and hope (which isn't a plan, we all have been told) for something to come out. Estimates help frame objectives, cost, and value.

To estimate a range, three points are needed. Here's my three point estimate paradox:
We all know we must estimate with three points (numbers)... so we do it, reluctantly
None of us actually want to work with (do arithmetic with) or work to (be accountable to) the three points we estimate

In a word, three point estimates suck -- not convenient, thus often put aside even if estimated -- and most of all: who among us can do arithmetic with three point estimates?

One nice thing about 3-points and ranges, et al, is that when applied to a simulation, like the venerable Monte Carlo, a lot washes out. A few big tails here and there are no real consequence to the point of the simulation, which is find the central value of the project. Even if you're looking for a worst case, a few big tails don't drive a lot.

But, here's another paradox:
We all want accurate estimates backed up by data
But data -- good or bad -- may not be the driver for accurate estimates

Does this paradox let SMEs off the hook? After all, if not data, then what? And, from whom/where/when?

Bent Flyvbjerg tells us -- with appropriate reference to Tversky and Kahneman -- we need a reference class because without it we are subject to cognitive and political maladies:
Psychological and political explanations better account for inaccurate forecasts.
Psychological explanations account for inaccuracy in terms of optimism bias; that is, a cognitive predisposition found with most people to judge future events in a more positive light than is warranted by actual experience.
Political explanations, on the other hand, explain inaccuracy in terms of strategic misrepresentation.

So that's it! A conspiracy of bad cognition and politics is what is wrong with estimates. Well, just that alone is probably nonsense as well.

Folks: common sense tells us estimates are just that: not facts, but information that may become facts at some future date. Estimates are some parts data, some parts politics, some parts subjective instinct, and some parts unknown. But in the end, estimates have their usefulness and influence with the SMEs and the money guys.

You can't do a project without them!

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!
Read my contribution to the Flashblog