Saturday, August 5, 2017

When to stop?

You're doing something. Fair enough
What if there's no obvious end-point?
  • Like finding the "just-right" SME for the team ... you never know when that is going to happen
  • When should you stop looking at candidates?
Actually, there's a computer science answer to this.
Look no farther than the "optimal stopping" algorithm!

As explained by authors Christian and Griffiths*, there are two ways to make a mistake trying to find the "best" SME ... assuming you are interviewing candidates in random order
  1. Stop too soon, hiring "suboptimally", missing out on the best candidate you never get to interview
  2. Stop too late, passing up the pick of the litter hoping there is an even better candidate
Obviously, the first person you interview is the "best" so far ... there's only one. The next one you interview has a 50/50 chance of being the best; and the third person has a one in three chance, etc. But how long to go on with this?

After a lot of research, and numerous academic papers, the computer science community has arrived at an algorithm (of course they have): "Look, then leap". 

Look, then leap
Step 1: set a fixed time to look over the field. Don't choose anyone (or anything) in the "look" phase
Step 2: after the look phase, leap! Leap on the best candidate that then comes along, or go back to a "best" in the look phase if they are still available.

What's the chance that you'll get the "best" candidate? About one chance in three ... actually, long term, 37%. Here's an abbreviated chart of the odds, as developed by validation of the optimal stopping algorithm:
  • 3 candidates; take the best after 1; 50% chance of getting the best
  • 6 candidates; take the best after 2; 43% chance of the best
  • 10 candidates; take the best after 3; 40% chance of the best
  • Huge number; take the best after 37% of huge; 37% chance of getting the best
Actually, look before leaping is not bad advice for a whole host of activity!
* "Algorithms to Live by", Brian Christian and Tom Griffiths

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!
Read my contribution to the Flashblog