Quote from sachinm on 13 October 2023, 12:47 pm

To get the ball rolling here, have you considered the role of "Optimism Bias" in developing the "Should Cost", where there are Known-Unknowns in scenarios with low project definition and high uncertainty.

Project appraisers have the tendency to be over optimistic. And so, those producing cost models can be influenced by their heuristics and cognitive biases, potentially undermining their capacity to produce accurate and robust forecasts. This is a demonstrated, systematic, tendency for project appraisers to be overly optimistic, especially in the absence of data (i.e., scope) by providing their own insight based on their experience to “fill in the blanks”.

This Optimism bias is the demonstrated systematic tendency for appraisers to be over-optimistic about key project parameters, including capital costs, operating costs, project duration and benefits delivery. Over-optimistic estimates can lock in undeliverable targets.

Explicit adjustments should therefore be made to the estimates of a project’s costs, benefits and duration, which should be based on data from past or similar projects, and adjusted for the unique characteristics of the project in hand. To redress this tendency HM Treasury's Green Book suggests that appraisers should make explicit, empirically based adjustments to the estimates of a project’s costs, benefits, and duration. The Green Book (2022) guidance provides cost and time uplift percentages for generic project categories which should be used in the absence of more robust primary data.

The Supplementary Green Book Guidance on Optimism Bias (HM Treasury 2003) with reference to the Review of Large Public Procurement in the UK (Mott MacDonald 2002) notes that there is a demonstrated, systematic, tendency for project appraisers to be overly optimistic and that to redress this tendency appraisers should make explicit, empirically based adjustments to the estimates of a project’s costs, benefits, and duration.

However, often these corrections for Optimism Bias are used as another source of financial contingency...

Rather, HM Treasury recommends that these adjustments be based on data from past projects or similar projects elsewhere, and adjusted for the unique characteristics of the project in hand. In the absence of a more specific evidence base, HM Treasury encouraged departments to collect data to inform future estimates of optimism, and in the meantime use the best available data. In response to this, the Department for Transport (DfT) contracted Bent Flyvbjerg in association with COWI to undertake the consultancy assignment "Procedures for dealing with Optimism Bias in Transport Planning".

This Guidance Document is the result of the assignment's stated objectives below:

1. provide empirically based optimism bias up-lifts for selected reference classes of transport infrastructure projects; and
2. provide guidance on using the established optimism bias uplifts to produce more realistic forecasts for the individual project's capital expenditures.

How have you dealt with "Optimism Bias" in your projects?

For example, Reference Class Forecasting (RCF) offers a robust statistical and project-oriented methodology for calculating uplifts, employing a systematic top-down approach rooted in historical data and outcomes from analogous, previously executed projects.

How have you used "Reference Class Forecasting (RCF)" in your projects to manage complexity, and uncertainty?