Jim Benson has a thought provoking post on the Five Pathologies of estimating. Each is likely present in an organization that has not moved up the maturity scale. Maturity levels in the CMMI paradigm. The post, like many notions in the agile world, starting with the Manifesto assumes the software development process is broken – ML 1. With that assumption there is little to direct the effort towards identifying the rot causes and taking corrective actions, as found in highly maturity levels.

Let’s See What Corrective Actions Will Remove the Symptoms produced by a Root Cause

Each item in the post could be a root cause or just a symptom of the root cause. A full Root Cause Analysis would be needed for a specific domain, but I’ll make suggestions below that can be broadly applied to each. These responses are extracted from Steve Loving’s “Mitigation’s for the Planning Fallacy” given at the PMI Silicon Valley Symposium.

• Guarantism – The belief an estimate is actually correct. The commitment to an estimate as a fact, that is we guarantee the price of the work will be $X.XX

• First let’s establish a fundamental principle of all estimating processes. No point estimate is correct in the absence of a confidence level.

• All numbers on projects are random numbers. Only the check out price at the grocery store is a fixed number.

• Is that $600 plumbing quote a 80% confidence number or a 30% confidence numbers.

• When the receiver of the quote doesn’t ask for the confidence level, and further ask for the uncertainties in that quote – the reducible and irreducible uncertainties – then the provider of the quote of off the hook. And the receiver of the quote has not locked that number in her mind.

• This is the anchoring and adjustment problem

• CORRECTIVE ACTION – no quote accepted without a statistical confidence based on past performance or some parametric assessment

•  Promisoriality – The belief that estimates are possible. This is labeled the Planning fallacy

• Use of the Planning Fallacy without also reading the solutions to the Planning Fallacy is a Root Cause.

• Bent Flyvbjerg has much to say on the Planning Fallacy and the corrective actions.

• Reference Class Forecasting, Parametric Modeling, simple wide band Delphi and many other estimating techniques can address the Planning Fallacy

• CORRECTIVE ACTION – Use of the Planning Fallacy and NOT reading the solutions, is a low maturity behaviour.

• Swami-itis – The belief that an estimates is a basis for sound decision making. All projects are probabilistic process driven by the underlying statistical network of interconnected activities. Fail to recognize that modeling this system is the root cause of naive and ill-informed estimates.

• Our domain – software intensive systems – is driven by Systems Engineering and Monte Carlo Simulation of the System of Systems.

• Tools are available – some simple some complex. But the paradigm is systems engineering drives our thought processes.

• Craftosis – The assumption that estimates can be done better. We believe we will get better at our estimates.  That estimation is a skill.  This may be true,  our estimating skills can be honed.  However,  the planning fallacy and the realities of how we work put a cap on the accuracy we are able to attain.

• We’e back to the misuse of the Planning Fallacy.

• So let’s revisit the planning fallacy foundations so we don’t misuse it again

• The context of planning provides many examples in which the distribution of outcomes in past experience is ignored. Scientists and writers, for example, are notoriously prone to underestimate the time required to complete a project, even when they have considerable experience of past failures to live up to planned schedules. – – Kahneman and Tversky (1979)

• Kahneman and Tversky tell us as does Flyvbjerg – don’t do stupid things on purpose. Don’t underestimate when you have direct experience from the past that those underestimates were wrong.

An Interlude at Address the Planning Fallacy

The Planning Fallacy is a set of cognitive biases present across all levels of expertise and all subject matters

• Planners/PMs/Project teams are exhibiting the Planning Fallacy when they

• Use internal, idealized scenarios about the future.

• Ignore past information

• Fall prey to false optimism.

• Engage in the estimating processes with unacknowledged high motivational forces.

• The Planning Fallacy has two components

• Anchors that influence the Planning process – data  introduced early in planning, even spurious data, that then wrongly  influence estimates

• Related to the Planning Fallacy, found in novices and experts alike, are limits in cognition when probabilistic thinking is involved – this becomes an issue when project teams must deal with likelihood of risk events. This is seen in the Healthcare.gov project. Identifying and Managing Project Risk: Essential Tools for Failure-Proofing Your Project, Second Edition. By Tom Kendrick, 2009.

• The Planning Fallacy has several dimensions

• Plans

• Past

• Optimism

• Motivation

• Anchor

• Models

• Solutions to the Planning Fallacy are straight forward

• Premortum -What were all the possible pitfalls in this imaginary project failure?

• Reference Classes – The Reference Class is a database of projects. A project team can compare their project to projects in the database. There are several external database for most IT projects.

• Most Likely Development – Look at parts of the project that have the highest probability of cost overrun, schedule overrun, or negative environmental impacts.

• Bayesian analysis – adjust judgment of the probability of success of the planning efforts.

• Explicit model – simulation models, including Monte Carlo Carlo and Method of Moments.

• Structured techniques – processes they rely on data and detail repeated multiple times throughout estimation phases

• Decomposition – using the Work Breakdown Structure to force more details, and illuminate faulty estimating models and resulting biases

• Scrum – Stories and tasks are ranked and scored by the Scrum team using story points.

• Theory of Constraints – use real time mitigation for projects via buffer management.

OK, Back to the Post

• Reality Blindness – The insistence that estimates are implementable In business,  we create estimates and plans and begin work immediately,  rarely with the expectation that those plans will change.

• This is a very naive process, a low maturity level in the CMMI sense.

• Plans change, that’s why it’s called a plan.

• When plans changes for all the right, and possibly wrong, reasons, estimates must be adjusted.

• In our formal Federal Acquisition Regulation, Baseline Change Requests mandate a new Estimate to Complete and Estimate At Completion.

Summary

The post showed the symptoms of poor planning processes. The root causes are well know and the Corrective Actions readily available. The challenge is to have the will and fortitude at actually do the right thing.

This is a challenge in our high maturity space and defense business. So that’s the real problem. It is not what many suggest – that estimating can’t be done. It can. But you have to want to do it.

Like nearly every root cause it comes down to the people, page 18 below. Without a Root Cause Analysis, the original post just points out the symptoms and provides not means to take corrective actions. Leaving 2 out of the 3 steps missing for making improvements to our probability of project success.

Some Resource Materials

Related articles