This is a follow up post in our series about bidding with confidence. This blog goes deeper into three-point estimating and confidence scoring.
You are producing a cost estimate for a new proposal. The lead engineer Jessica who did a marvelous job of delivering the prior work has a history of overly optimistic estimates when it comes to cost to complete: “better to promise four days and deliver in six than to be conservative and promise it in four weeks and deliver in three”, she would say.
The answer is not to simply go to Jessica and take her input as gospel, and then leave it up to the delivery manager, who often has no idea how you came up with the original estimate, to ‘inherit’ the budget and start thinking of reasons why things have changed and the original estimate is no longer valid….
If you want to submit your bid with confidence, you need to understand what it takes to deliver the project, the specific risks that can derail execution, and to what degree of confidence can the project be delivered within the estimate. We believe there are five different, but complementary, ways to estimate with confidence. They are as follows:
Three-point estimating - best case, most likely and worst-case estimates
Confidence scoring - assigning a confidence score to each estimate
Risk registry - Creating a register of project delivery risks and cost reduction opportunities
Independent Cost Estimates (ICE)
Overall performance risk
Of course, good estimating relies on a clear definition of objectives and scope, as well as clearly documenting assumptions, risks, and constraints. But in addition, we recommend that you use one or more of these techniques next time you submit a large project-based proposal. How you do that is explained below. And next week’s part 3 blog will explain how to create a risk register with your estimate and how two or three of these methodologies can be smartly combined to give an overall picture of confidence,
Three-point Estimating
This is perhaps the simplest way to bid with confidence and involves your subject matter expert positioning their estimate (hours, full-time equivalent staff, dates/durations, number of lines of code, number of web-pages, number of drawings etc.) as the “most likely outcome”. Then they must back this up with a “best-case” (lowest number of hours or any other input) and a “worst-case” scenario. Let’s say that Jessica needs to estimate the number of developers to build a new app. Based on previous similar work, the most likely outcome is 4 people for 3 months. But potentially 3 stronger resources could get the job done in 10 weeks, or it could take 5-6 people from 3-4 months to complete the work. Assuming a straight-forward 8-hour day, working 20 days per calendar month, and ignoring public holidays, the three-point estimate is:
Best-case = 3 x 10 x 5 x 8 = 1,200 hours
Most likely = 4 x 3 x 20 x 8 = 1,920 hours
Worst-case = 5.5 x 3.5 x 20 x 8 = 3,080 hours.
One important caveat - Jessica often does not know the cost drivers or risks which will result in the worst-case scenario, and how they impact cost.
Aggregating Three-point Estimates
When you collect estimates from lots of different people for different work packages, tasks, or work breakdown structure (WBS) elements within your proposal you could in theory add up the most-likely estimates to derive the total. It is not wise, however, to simply add up all the best and worst-case estimates to derive a project-wide range because it is extremely unlikely that all the worst-case outcomes would occur at the same time - unless they are all related to the same underlying cause.
Instead, the most common approach is to run a Monte Carlo simulation over all those estimates. A more in-depth discussion on Monte Carlo simulations and other strategies for generating the risk-adjusted cost profile will be discussed in Part 3 of this blog series.
For three-point estimates specifically, we use a PERT distribution for doing the sampling of probable outcomes in the Monte Carlo simulation. The PERT distribution was specifically designed for the use case of best-case, worst-case, and most likely.
As you can see in the illustration using our example, the most probable outcome is around the most likely and it then gives lesser and lesser chance the closer we get to the two extremes.
Assigning a Confidence Score
As an alternative to three-point estimates, you can instead use confidence scoring. On a scale of 99% (“perfect”) to 1% (“who knows?”) How confident are you? The confidence again indicates a spread of the possible outcomes. In this case, we use a normal distribution with a standard deviation. The more confident, the smaller standard deviation, as illustrated below.
However, asking Jessica whether she is 80% or 60% confident or even worse what the standard deviation would not work in a practical sense. Each individual person would have a different way of expressing confidence, however, in our platform you can define rules that derive that confidence automatically. As an example, if the system analyses the material cost for a specific part, the system reviews the historical data and based on algorithms calculate the confidence score. It might find a valid quote for $1,200, it might find 22 historical PO’s with similar unit costs, or it might find no historical cost at all, each resulting in a different confidence score.
Furthermore, the system can also help Jessica derive confidence in labor estimates by looking at the estimating source that she uses. For example:
90% confidence score could be assigned if Jessica indicate that almost identical prior work has been performed and the actuals for those have been used to estimate this effort
75% confidence could be assigned if we used sizing metrics, namely the number of software lines of code (SLOCs) times a known ratio of how many hours it takes per SLOC based on the previous five years of history
60% confidence - A detailed project plan was created and resourced in a project scheduling tool
The exact confidence scores and estimating sources are of course configurable, and the above just serve as examples.