This was the heading of an article in the magazine “New Civil Engineer”, published on 14 November. Metric Views looks at the effect of removing optimism bias on the estimate of the cost of road traffic sign conversion prepared by the UK Department for Transport in 2005.
The article begins:
“Infrastructure clients are planning to ditch Treasury-backed rules for calculating the contingency budgets of major infrastructure projects.
They have produced a report which concludes that using the Treasury optimism bias mechanism to anticipate cost overruns is costing billions of pounds.
The proposal has been made by the Infrastructure Risk Group, an industry-led group with representatives from major infrastructure clients.”
The article concludes:
“It (scrapping optimism bias) would also lead to the earlier identification of specific risks, avoiding fear of the unknown being priced into optimism bias.”
Readers may wish to be reminded how optimism bias was included in the estimate of the cost of converting speed, directional and distance signs prepared by the UK Department of Transport (DfT) in November 2005.
The sums (reference 1) looked like this (amounts in £ million):
Cost of conversion (England only) 241
Extra 10% for number of signs increased 24
Disposal costs @ £10 per sq m 4
(Total) 269
Plus extra 16% for rest of UK 312
Plus supervision, preparation & design @ 25% 390
Plus optimism bias @ 45 – 65% 565 – 644
Preparatory work 100
Publicity 10 – 15
Project management 10
Total 680 – 760
This related to the DfT’s estimate of a total number of signs in England of 417 000, increased by 16% for the rest of the UK, giving a cost of about £1500 per sign. This compared unfavourably with the average actual cost of converting speed limit signs in the Republic of Ireland in January 2005 of £100 per sign (reference 2).
Removing optimism bias reduces the total to £510 – 515 million or about £1060 per sign – still astonishing, and showing just how much the DfT calculations lack credibility.
Reference 1. The DfT’s estimate is available in the National Archives at:
http://webarchive.nationalarchives.gov.uk/20090902164442/http://www.dft.gov.uk/pgr/roads/tss/general/estimatingcostconversion
with the detailed numbers at:
www.dft.gov.uk/adobepdf/165240/183423/estimatingcostconversion
Reference 2. “Metric signs ahead”, published by UKMA in 2006, paragraph 6.10. Available as a free download on UKMA’s web site: www.ukma.org.uk
Metric Views 
It should also be pointed out that the starting figure in the above calculation (£241 million) was already a gross exaggeration, as it assumes the most expensive methods of doing the job (e.g. total replacement in a single year rather than using vinyl overlays and spreading the cost over several years to combine it with routine maintenance).
Even if “optimism bias” were appropriate for major projects like HS2 or Heathrow third runway, it is not relevant to minor projects like sign conversion, which would be done on a “schedule of rates” so that you know the cost fairly accurately before you start.
For a fuller analysis see pages 65-67 of “Metric signs ahead” at http://www.ukma.org.uk/msa-summary (scroll down and click on “Publication”).
LikeLike
While I am completely in favour of scrapping optimism bias, I can also understand the rationale for incorporating it. For very many years, as a quantity surveyor, I used to do preliminary and detailed cost estimates for a wide variety of projects. In the early phases, involving a preliminary estimate, some degree of bias had to be incorporated (which we called a design contingency), as there were so many unknowns at these early stages.
The key however was that as more information became known, the contingency, or bias, was progressively reduced, until by the time the project went to tender, it was at zero. (There were other contingencies to cover for unknown site conditions and the like, but these were relatively small, rarely over 5%.)
Allowing for the unknown in the early phases can put you into a quandary. Too much contingency or bias, and the client accuses you of being forced to delete things that could have been incorporated within the established budget. Too little, and you end up looking very silly when the tenders come in over your estimates. The key, as Erithacus stated above, is to start with some sensible parameters in the first place, and then add in bias, or contingency, based on the level of information available at any one point in time. As more information becomes available, any bias should be reduced accordingly.
In the case of converting Britain’s road signs, the original estimates were indeed ridiculous (politically motivated as we all know), and the optimism bias just compounded that. When it comes to politics, there are lies, damned lies and statistics. And politicians just love statistics.
LikeLike
I can understand most of the above; there is also a 2.4% pa for inflation presumption.
What I do not understand, and never will, is why when such a ridiculous estimate is pronounced, the media do not react adversely.
Sure the media are there to make money, not to inform nor educate the likes of us (me).
It seems then, that upsetting the government propaganda programme is not in the media interest. Nor it seems that of any other watchdog body, union (jobs) or organisation (in this case motoring). Surely keeping a government honest would be in the interests of all of us.
I sometimes wonder if I really am living in an educated, developed country of enlightenment. But then, I have seen that other side of the fence too, it is not pretty.
So I guess it must just be me.
But I wish to pose a question. You and I use spread sheets, Gantt charts and other project software to produce a meaningful set of budget figures. Is it not reasonable that government departments release (publically, on line that is) at least a basic set of factual, checkable, criteria to show some of the workings behind big projects? Or any project come to that.
LikeLike
In addition to the points made above.
(1) The true cost of replacing a road sign for the purposes of changing it to metric is not simply what it costs to buy the sign and do the job. All signs are replaced eventually so the replacement will postpone the day those same costs would have been incurred anyway (i.e give it a new lease of life).
(2) No allowance was made in the DfT estimate for scale economy.
LikeLike
The result of the failure to convert the UK’s road signs is the total disconnect between the metric system of weights and measures taught in schools and used throughout the economy, the health service, the building industry and life in general and the outdated imperial system still used on the roads. On arriving in the UK and seeing its imperial road signs you could be forgiven for thinking you had gone back in time to a pre-metric age. Some metric signs do exist, though usually next to signs in imperial units giving a patchy and piecemeal approach that seems to say the UK does not know what its system of weights and measures is or should be. When the changeover to metric in the UK began over forty years ago, there was never any plan to exclude the nation’s road signs. Today we have two generations who have had to muddle through as far as weights and measurements are concerned and who in many respects are now functionally innumerate as a result, a complaint voiced by many prospective employers of those applying for work where numbers are involved. Every country needs a proper system (one proper system) of weights and measures. Why does the UK not have one? When will we have proper metric road signs?
LikeLike