Pert v Monte Carlo ... some still care

---

A note from a reader of some of my stuff on slideshare.net/jgoodpas:

"... I just read your slide presentation on using stats in project management and am a bit confused or perhaps disagree ... that using Monte Carlo will improve the accuracy of a schedule or budget based on subjective selections of activity or project
a) optimistic, realistic and pessimistic ($...Time) values,
b) a probability distribution (3 point triangular) and
c) arbitrary probability of occurrence of each value.

If I understand your presentation Central Limit Theory (CLT) and the Law of Large Numbers (LLN) when applied using Monte Carlo simulation states that accuracy is improved and risk more precisely quantified.

It seems to me that this violates a law that I learned a long time ago...garbage in-garbage out. Is this true? ...."

I replied this way about accuracy:

• Re 'accuracy' and MC Sim (Monte Carlo simulation): My point is (or should have been) the opposite. The purpose of MC Sim is to dissuade you that a single point estimate of a schedule is "accurate" or even likely. Indeed, a MC Sim should demonstrate the utility of approximate answers
• Project management operates, in the main, on approximations... ours is the world of 1-sigma (Six Sigma is for the manufacturing guys; it doesn't belong in PM).
• Re the precision/accuracy argument: My favorite expression is: "Measure with a micrometer, mark with chalk, and cut with an axe", implying that there is little utility in our business (PM) for precision, since most of our decisions are made with the accuracy of chalk or axes.
• A MC Sim gives you approximate, practical,and actionable information about the possibilities and the probabilities (all plural) of where the cost or schedule is likely to come out.
• And, to your point, the approximate answers (or data) should be adequate to base decisions under conditions of uncertainty, which is for the most part what PMs do.

So, what about MC Sim itself?

• Re your contention: " ... selecting the distribution, optimistic, pessimistic and realistic values must be accurate for Monte Carlo to provide more accurate results." Actually, this contention is quite incorrect and the underlying reason it is incorrect is at the core of why a MC Sim works.
• In a few words, all reasonable distributions for describing a project task or schedule, such as BETA (used in PERT), Normal (aka Bell), Rayleigh (used in many reliability studies, among others), Bi-nominal (used in arrival rate estimates), and many others are all members of the so-called "exponential family" of distributions. You can look up exponential distributions in Wikipedia.
• The most important matter to know is that, in the limit when the number of trials gets large, all exponentials devolve to the Normal (Bell distribution).
• Thus, if the number of trials is large, the choice of distribution makes no difference whatsoever because everything in the limit is Normal.
• If you understand how to do limits in integral calculus, you can prove this for yourself, or you can look it up on Wikipedia

How large is large?
It depends.

• As few as five sometimes gives good results (see image) but usually 10 or more is all you need for the accuracy needed for PM.
• Consequently, most schedule analysts pick a triangular distribution, which does not occur in nature, but is mathematically efficient for simulation. It is similar enough to an exponential that the errors in the results are immaterial for PM decision making purposes.
• Some other picks the uniform distribution like shown in the image; again for mathematical convenience

Should I worry about 'when' and 'where'?

• Now the next most important matter to know is that a sum of exponentials (BETA, Rayleigh, whatever) -- like would be the case of a sum of finish-to-start tasks -- has the same effect as a number of trials.
• That is, if the project is stationary (does not matter when or where you look at it), then a string of repeated distributions (as would be in a schedule network) has the same characteristics as a single distribution tried many times.
• And, each member of the string not be the same kind of distribution, though for simplicity they are usually assumed to be the same.
• Thus, whether it's one distribution tried many times, or one distribution repeated many times in a string of costs or tasks, the limiting effect on exponentials is that the average outcome will itself be a random variable (thus, uncertain) with a distribution, and the uncertainty will be Normal in distribution.

And, about that garbage

• The usual statement about GIGO assumes that all garbage has equal effect: Big garbage has big effects; and little garbage has little effects; and too much garbage screws it all up.
• This is certainly the case when doing an arithmetic average. Thus, in projects, my advice: never do an arithmetic average!
• However, in a MC SIM, all garbage is not equal in effect, and a lot of garbage may not screw it up materially. The most egregious garbage (and there will be some) occurs in the long tails, not in the MLV.
• Consequently, this garbage does not carry much weight and contributes only modestly to the results.
• When I say this I assume that the single point estimates, that are so carefully estimated, are one of the three estimates in the MC Sim, and it is the estimate that is given the largest probability; thus the MLV tends to dominate.
• Consequently, the egregious garbage is there, has some effect, but a small effect as given by its probability.

What about independence and correlations?

• If the distributions in the MC Sim are not independent then results will be skewed a bit -- typically, longer tails and a less pronounced central value. Independence means we assume "memoryless" activities, etc so that whether a string of tasks or one task repeated many times, there is no memory from one to the other, and there is no effect on one to the other other than in finish-to-start there will be effects on the successor start time.
• Correlations are a bit more tricky. In all practical project schedules there will be some correlation effects due to dependencies that create some causality.
• Again, like loss of independence, correlation smears the outcome distribution so that tails are longer etc.
• Technically, we move from the Normal to the "T" distribution, but effects are usually quite inconsequential. 

Where to get the three points if I wanted them?

• My advice is in the risk management plan for the project, establish a policy with two points:
  1. All tasks on the calculated single point critical path will be estimated by experts and their judgment on the task parameters go in the sim
  2. All other tasks are handled according to history re complexity factors applied to the estimated MLV (most likely value, commonly the Mode): that is, once the MLV for a non-critical task is estimated, factors set by policy based on history, are applied to get the estimates of the tails.
• Thus, a "complex task" might be: optimistic 80% of MLV and pessimistic 200% of MLV, the 80% and 200% being "policy" factors for "complex" tasks.
• This is the way many estimating models work, applying standard factors taken from history. In the scheduling biz, the issue is not so much accuracy -- hitting a date -- as it is forecasting the possibilities so that some action can be taken by the PM in time to make a difference.
• Providing that decision data is what the MC Sim is all about. No other method can do that with economic efficiency

PERT v MC Sim
Now, if the schedule is trivial, there is no real difference in the PERT and MC Sim. Take, for example, a dozen tasks in a tandem string of finish-to-start.

• The PERT answer to the expected value of the overall duration and the MC Sim will be materially identical.
• The sum of the expected values of each task is the expected value of the sum. So, the SIM and the PERT give identical answers insofar as an "accurate" expected value.
• But that's where it ends. PERT can give you no information about the standard deviation of the outcome (the standard deviation contains the reasonable possibilities about which you should be worried), nor the confidence curve that lets you assess the quality of the answer. 
• If your sponsor asks you run the project with an 80/20 confidence of finishing on time, where are you going to go for guidance? Not to PERT for sure.
• And, if your sales manager says bid the job at 50/50, you have the same issue... where's the confidence curve going to come from?
• And, most critically: PERT falls apart with any parallelism in the schedule in which there is no slack.

Thus, some 30 years ago PERT was put on the shelf, retired by most scheduling professionals

I'm almost out of gas on this one, so I'll end it here.

---

The image shows the outcome of 5 independent uniform distribution summed, as in a finish-to-start schedule sequence.

---

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!

Read my contribution to the Flashblog

Introverted ... and at a conference

---

OMG! If you're an introvert, attending a 3-day conference where you're expected to network and bring back new relationships, contacts, and business intelligence. Looking forward to that has got to be nothing but dread. Three days of small talk and useless chit chat about the weather? God awful!

Agree?

So, a few useful ideas are found here in a hbr.org essay entitled: "How Introverts Can Make the Most of Conferences" by reporter Dana Rousmaniere interviewing Susan Cain, author of "Quiet: The Power of Introverts in a World That Can’t Stop Talking" 

• Take a recharge break. Take more than one. Leave the building for a change. Even for extroverts, conferencing is exhausting
• When approaching a stranger, try to land on a topic of mutual interest. That way, you don't have to talk about the weather
• Open the conversation with something that is common in the environment, like the speech just heard.
• "Relax into extroversion" when you find a topic that stimulates your curiosity. That is: extend the conversation
• Be a speaker or panel member, or submit a paper to the proceedings. If there's such an opportunity, grab it! Nothing like a little public speaker exposure... it gets easier with experience.

And, to add something to Ms Cain's advice, you can also be in your company's product booth, or a walk-around person handing out souvenirs.

And, remember this, many of the people you will meet are likewise introverts looking for way to break in. So, most of whom you approach will be in your situation and looking for a connection.

In other words, it's a target-rich environment.

---

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!

Read my contribution to the Flashblog

Plausible or probable?

---

Ponder this for a moment:

Adding detail to the description of an uncertainty may make it seem less probable (the more you know, the less you think it will happen than a more abstract version), though it may make it more plausible

Now move forward to that first tool of choice among risk managers, the risk register. The logic is this:

As detail abounds and abstraction abates, plausibility augments but probabilities must be adjusted; and, in general, probabilities go down.

Try this on for an example:
Scenario -- you are looking at number of interns or co-ops with an idea of placing them in various roles in your project, perhaps business analysis, engineering, or training and business readiness.

Adel is 22 years old, very smart, technically accomplished, studied (among many subjects) philosophy and music, has a strong sense of who she is, and is very detail oriented. She has leadership presence and personal charisma

Adel is a professional; she could be successful in some aspects of engineering, finance, business analysis, etc. or even marketing.

The question -- What do we think of Adel? Among those professional roles mentioned, which is the most likely? If engineering, then is she more likely a software designer/engineer who applies the female touch to typically qualitative requirements (needs and wants), or else is she a professional in some other aspect of engineering?

Most of us would look at it this way:

• Music is a good segue to software; music has patterns, logic, rhythm, and detailed structure -- all attributes of software as well
• Philosophy is about understanding beliefs, values, ways to think, and ideas -- all elements of human factors
• And, she's technically accomplished

Most of us would conclude: Adel is very plausibly a software engineer; perhaps she would be very good at human factors engineering. Finance and business analysis seem just too dry for someone who's into philosophy and music. Of course, you can't dismiss marketing out of hand....

Yes! that's it. Adel is most likely a software designer/engineer. That's the highest probability in the distribution of plausible possibilities.

Not so fast
What about other engineering disciplines? Sure, it's possible, but is it probable? Look at how plausible the fit is to software design engineering.

Venn et al
Let's look at the Venn diagram:

• Among all roles for women, Adel is a professional
• Among all professionals, Adel is possibly an engineer
• Among all engineers, Adel is possibly a software design engineer
• Among all software design engineers, Adel is possibly a specialist in human factors engineering

Every time we add detail, the plausibility improves (given Adel's profile), but the share of the population goes down each time. If this were red balls in an urn with white balls, each level of detail would mean fewer red balls.

What if we are just comparing two abstractions of about equal detail, as in marketing v engineering? The rule does not apply: we've not changed the level of detail from one to the other. We're just comparing two possibilities.

Moving to the risk register
Every time we are challenged to add detail to the risk register, we may well improve plausibility (why else add detail?), but the rule is: that finer grain is less probable than the more abstract idea from which it was derived.

So, what's the problem here? The problem is we confuse plausible with probability. Just because an uncertain event seems very plausible doesn't correspondingly mean it is very probable.

Read more
This discussion is given in much richer detail in Daniel Kahneman's book: "Thinking fast and slow".

---

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!

Read my contribution to the Flashblog

What flavor is your scope creep?

---

Can there be scope creep in Agile? Doesn't agile define it away in a stroke: "Scope is whatever is prioritized in the backlog that fits within the budget (OPM, other people's money) and the time. The backlog changes all the time, but that's not creep, it's just backlog management."

What I just wrote is a "best value" definition of scope. But, sometimes it doesn't sell.

I credit my agile students with these observations about "creep", to wit:

Project scope creep: it may be well and good that a true pure play on agile has no conception of scope creep, because the backlog is simply rebuilt as a zero-base-backlog (ZBB) after every iteration -- if not after each iteration, then certainly after every release.

Consequently, after every release, you should be able to stop and be satisfied you've delivered the best possible stack of high value objects. Amen!

Resource scope creep
On the other hand, if you can't get commitment for dedicated teams because the resources are needed elsewhere, then you are saddled with the overhead and inefficiency of rolling out and rolling in resources. This is resource scope creep -- spending more on resources' training, recruiting, and assimilation than the budget allows.

Clearly, this is a different flavor than project scope creep.

And, it's been around since forever!
Who's not heard of "resource plug-and-play"? The plug-and-play process: just define a role, line up the role with appropriate resumes, pick one, plug him/her in, and you're off to the races!

• For more on plug-and-play, see the work of F.W. Taylor and "management science" wikipedia.org/wiki/F._W._Taylor

Agile to the rescue!
The fix is in: a pure play in agile means having fully dedicated teams that use the inevitable white space in the team schedule to improve the product -- more testing, more refined refactoring, working completely through the technical debt -- but not to get off the team and go elsewhere, only to return later.

Ooops!
The idea of giving up on matrix and other plug-'n-play resource schemes is pretty much opposed by managers not willing to give real agile a try.

Shifting from input to output
Again, it's an input/output focus tension... managing the white space by moving resources around is an input focus; leaving resources in place to use the white space for quality is an output focus.

Traditional schemes often put resource managers in charge of finding resources for the project manager. Obviously, those resource guys are going to focus on getting their job done according to their metrics. Agile schemes are a bit different: once the resource manager has given up the resource to an agile team, it's the team leader who's responsible for getting good productivity and managing the white space on behalf of a best-value product

---

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!

Read my contribution to the Flashblog

3-point estimates ... good for you!

---

Here's my three point estimate paradox:

We all know we must estimate with three points (numbers)... so we do it, reluctantly
None of us actually want to work with (do arithmetic with) the three points we estimate

In a word, three point estimates suck -- not convenient, thus often put aside even if estimated -- and most of all: who among us can do arithmetic with three point estimates?

But hey! Three point estimates are good for you ... they are the broccoli of projects

First question (are you whining? ): where does this really matter in the domain of project management?

• When adding up a budget or calculating the duration of a schedule
• When doing a risk register and someone says to multiply impact x probability

Second question: how do you go about it? Actually, you can build a couple of simple templates in a spreadsheet to do the arithmetic. Then fill in the 3 points, and presto! -- a summation or product (add or multiply) pops out.

The budget
To understand this, start with the basics:

• Suppose I have two work package budgets, one is fixed and one is uncertain. I use a 3-point estimate for the latter. I need the total budget
• As an example, with numbers:
• Fixed budget = 10
• Uncertain budget 3-pts: 5, 8, 12
• Possible totals: 10+5; 10+8; 10+12
• With no other information, pretend you are a Bayesian. You would say your "a priori" is that all 3 totals are equally possible. Thus: 1/3 * (15+18+22) = 18.3

Now, you can see it's easy to extend this to both numbers being uncertain: in that case, you would then do 9 possible totals and divide by 9.

The risk register
Multiplication is the issue with the risk register, typically I x P. Say what you will, but both I and P are uncertain, and usually P is the most uncertain. We usually have no real basis for assuming a probability since we usually have insufficient history to establish a reasonable frequency (1/P)

As an example with numbers, suppose I've been told that with some risk my budget is going to be cut by $50. For our purposes, we can take this impact as a single point "certainty". What's uncertain is whether or not there's much chance it will happen.

• Fixed impact = 50
• Uncertain probability (better thought of as a confidence of a some % or less): 10%, 30%, 50%
• Possible IxP products (this figure, or less): $5, 15, 25 (P is dimensionless, so IxP has the dimensions of I)
• Again we can do our Bayes thing (no information for the 'a priori') and come with the average IxP: 1/3 (45) = $15
• This extensible to uncertainty of both numbers wherein there will be 9 products

Monte Carlo
Is anybody going to do this on a real project with a spreadsheet? Likely not. But, this is the way a Monte Carlo simulation works and the MC Sim is common among projects run quantitatively.

---

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!

Read my contribution to the Flashblog

The case for "little data"

---

"Big Data" is the meme of the day, but most projects run on "little data", the sort of data that fits into the constraints of spreadsheets like Excel. It's everyday stuff that drives estimates, scorecards, dashboards, task assignments, and all manner of project analytics.

So, assuming you using Excel as a spreadsheet for doing actual calculations and data entry, and not a row-column table version of Word, you will find that you to do some analytics and data analysis from time to time.

Pivot tables are one spreadsheet data tool, but that's not the discussion today. Today, it's filters, which is the Excel name for the process and tool, but which the database people -- familiar with SQL for row by column -- would call a query.

And so, how to do a filter in Excel, something practical for the project manager? There are Youtube's galore on the subject, but here's a neat, step by step, illustrated process that goes from the simple to the advanced.

Just what the PMO need to get into the data business

Some other rules
Beyond what you will read in the linked article, there are a few data rules that will make life simpler

• Every column should have a header or title that is unique, even if just column1, column2, etc
• Only one data value in a cell. Thus, first and last names should be in separate columns; so also city and state. But maybe also captain and ship's names. This is called "normalizing" the data
• Keep the static data in separate row/column areas from the data that changes. So, if a ship sails to San Francisco on a certain date, the ship's description goes in one area; the city description in another; but the city/date/ship is dynamic and belongs in a third area.
• Don't put 'word processing' paragraphs or labels in the middle of the data. In other words, maintain the integrity of row by column
• It's good to have at least one column that is guaranteed to be uniquely valued in each cell, like a row ID
• If you can avoid using "spaces" in the data, that's good. It makes the query more sure. So, "column1" instead of "column 1"

 

---

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!

Read my contribution to the Flashblog

Get ahead by leaving behind ...

---

Nick Tasler has a posting on the HBR Blog Network with an intriguing title: "To Move Ahead You Have to Know What to Leave Behind" 

His theme is summed up this way in an early paragraph:

The Latin root of the word "decide" is caidere which means "to kill or to cut." (Think homicide, suicide, genocide.) Technically, deciding to do something new without killing something old is not a decision at all. It is merely an addition.

So, what do the hoarders among us do? We save everything; kill -- or throw away -- nothing! Actually, we find deliberately making these trade-offs -- keep this; discard that -- so energy absorbing (Tasler says: exhausting) that we just layer on new change.

This behavior leads to what Tasler calls "trickle down trade-offs" in which there is sort of a push-down stack with the most current issue on top and the not-dealt-with issues more or less far down the stack.

Many of have seen this in action -- and we joke about planning an exit strategy before the stuff at the bottom finds its way back to the top!

No! .. that's not management -- that's escape. To be effective, we must all acquire instinct and be thought of as 'trained killers' of excess (think: Lean!)

---

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!

Read my contribution to the Flashblog