Getting a grip on GRIP.

[whole diary]

[grahame]

You can look ahead to all sorts of number

Your forecast
Your target
What is necessary level for it to "work"

and you can do so over all sorts of time periods and measure the numbers in all sorts of different ways

Ticket sales
Passenger numbers (not the same thing?)
Income from the service
Income from ticket sales that use the service as part of the journey
Benefits to the local community

[TonyK]

You can also employ cheerleaders in the style of Muhammad Saeed al-Sahhaf to big up the service once it is running. Bristol's MetroBust's propaganda department has been known to get all sniffy whenever the success of the Severn Beach line is mentioned, and continued to extol the success of its £60 million-plus busway, even when it wasn't open.

[Red Squirrel]

...
How many of these, though meet Trowres' criterion of not having 'achieved the forecast demand'?

Bearing in mind that most of the projects will have had more than one demand forecast and some were quite a long time ago, that isn't going to be an easy question to answer. The report I quoted earlier listed 23 stations studied (i.e. for which they could find the forecasting reports!) of which 10 had outcome passenger numbers below forecast. Of the 23, nine were within +/- 20% ... demand forecasting is difficult! I would suggest, however, that the spread of figures given falls far short of supporting the idea that schemes always turn out better than forecast.

Thanks; the report makes interesting reading!

Of the 10 stations that didn't achieve their demand forecasts, the worst appears to be Newcraighall at 60% below par. Of the 13 that exceeded them, the best was Ebbw Vale Parkway at 450% over par. So whilst it's not true to say that new stations always do better than anticipated, the bell curve looks like it is centred to the right of 0%.

To check this (please bear with me and my 'O' Level standard maths) I took a simple average of the percentages in Fig.3.2: taken as a whole the reopening schemes in the report did 33% better than forecast.

Maybe that's a crude measure; percentages can be misleading. So I added up the total number of passengers forecast for all the schemes, and the actual numbers:

((5443769/5126661)*100)-100 = 6.2%

So it seems to me that whichever way you cut it, re-openings tend to do better than forecast. But I will try to avoid sloppy use of words like 'always' in future!

_{Edit: Corrected sum - RS}

[IndustryInsider]

Aylesbury Vale Parkway is an interesting one as it opened early in June 2009 rather than planned in 2010.  The report does gives some detail, but as it was being built to provide a station for several large house build projects nearby and it opened before most of those had been built.  The recession led to long delays on some phases of those estates, some of which have only recently been completed and usage has been strongly increasing since 2010.  That perhaps explains that one?

[grahame]

Of the 10 stations that didn't achieve their demand forecasts, the worst appears to be Newcraighall at 60% below par. Of the 13 that exceeded them, the best was Ebbw Vale Parkway at 450% over para

... taken as a whole the reopening schemes in the report did 33% better than forecast.  ...

Maybe that's a crude measure....

((5126661/5443769)*100)-100 = 6.2%

One thing it highlight is how difficult even those with appropriate background find it to forecast how a new station will do.  Figures can be wildly out!

Aylesbury Vale Parkway is an interesting one as it opened early in June 2009 rather than planned in 2010.  The report does gives some detail, but as it was being built to provide a station for several large house build projects nearby and it opened before most of those had been built.  The recession led to long delays on some phases of those estates, some of which have only recently been completed and usage has been strongly increasing since 2010.  That perhaps explains that one?

Now that is a very interesting case.  When significant estate of new houses is built, there's often a pull towards getting a bus service in early so that people can use it from first occupancy, and a pull towards getting in late to reduce the early carriage of fresh air.  Best approach (IMHO^▸ (in my humble opinion)) is to get the bus / train early so that people use it as they move in and don't buy a second car due to the (early) lack of public transport, which then has to compete for the market it's come to late. On that basis, thank goodness for Aylesbury Vale Parkway opening early. 

[Red Squirrel]

...On that basis, thank goodness for Aylesbury Vale Parkway opening early. 

This is an argument FoSBR^» (Friends of Suburban Bristol Railways - site) put to WECA^» (West of England Combined Authority - about) with regard to Henbury. Fell on deaf ears, sadly; WECA say the BCR^▸ (Benefit Cost Ratio) doesn't stack up until the houses are built.

[TonyK]

This is an argument FoSBR^» (Friends of Suburban Bristol Railways - site) put to WECA^» (West of England Combined Authority - about) with regard to Henbury. Fell on deaf ears, sadly; WECA say the BCR^▸ (Benefit Cost Ratio) doesn't stack up until the houses are built.

Was there ever a better demonstration of the classic chicken and egg? Wherever the final agreed site of Henbury station is to be, it will sit sit somewhere close to one of the two worst traffic bottlenecks on routes into Bristol, and will see use from Day One. The opportunity exists at the moment to spur something off towards the obsolescent Mall without having loads of houses either side, which makes too much sense for WECA. As well as that, an initial service from Henbury could service Ashley Down, possibly Horfield / Lockleaze too, with some form of alternate stations pattern at Stapleton Road and Lawrence Hill to complement the Severn Beach line. Also, having a transport link in place could spur the builders into action in a way that the possibility of a replacement bus route next year, should First Bus survive that long, would not.. I thought WECA was supposed to be enabling things like that.

[Trowres]

Responding to RS again:

Of the 10 stations that didn't achieve their demand forecasts, the worst appears to be Newcraighall at 60% below par. Of the 13 that exceeded them, the best was Ebbw Vale Parkway at 450% over par. So whilst it's not true to say that new stations always do better than anticipated, the bell curve looks like it is centred to the right of 0%.

To check this (please bear with me and my 'O' Level standard maths) I took a simple average of the percentages in Fig.3.2: taken as a whole the reopening schemes in the report did 33% better than forecast.

Maybe that's a crude measure; percentages can be misleading. So I added up the total number of passengers forecast for all the schemes, and the actual numbers:

((5443769/5126661)*100)-100 = 6.2%

So it seems to me that whichever way you cut it, re-openings tend to do better than forecast. But I will try to avoid sloppy use of words like 'always' in future!

Well, demand can't go below -100%, but it surely can go above +100%, so there is an inbuilt tendency for the arithmetic mean to be positive. However, I think your second approach is fair. Now, put that 6.2% in the context of the following statement from the 2010 report:

Given the paucity of industry guidelines for forecasting demand for new stations,
and the limited evidence base, we took the view that demand forecasts were
reasonably accurate if they were within +/- 20% of observed demand.

Now, I have two questions (neither of which would be easy to answer, but as a line of argument they might work):

1. Of the station proposals that were assessed and didn'tachieve the required BCR^▸ (Benefit Cost Ratio), how many were close enough that (say) a 20% better demand outcome would have flipped the business case to "GO"?

2. If the Wolmar approach was followed, how many of those otherwise rejected schemes would go on to be economic flops?

Having apparently become an advocate of forecasting, I think a further question is needed:
3. How much expenditure is justified on forecasting, as a percentage of the expected scheme cost?

I am not even attempting to answer that one tonight!

[Trowres]

WECA^» (West of England Combined Authority - about) say the BCR^▸ (Benefit Cost Ratio) doesn't stack up until the houses are built.

    

As a capital project, the BCR is presumably evaluated over a 60-year period. Even allowing for the discounting applied to future costs and revenue, this makes it sound as if the BCR is balanced on a knife edge. (2.00001 ?).

One might enquire if the demand modelling encapsulates the higher uptake of public transport if it is provided before travel habits of new housing dwellers become established.

[ellendune]

As a capital project, the BCR^▸ (Benefit Cost Ratio) is presumably evaluated over a 60-year period. Even allowing for the discounting applied to future costs and revenue, this makes it sound as if the BCR is balanced on a knife edge. (2.00001 ?).

Since the BCR is based on discounted* costs and benefits and all the capital costs are are presumably at the start of the project, immediate benefits will have more impact on BCR than benefits in the future.  So a delay in housing will cause a delay in benefits.  However, given the parlous state of economy I wonder if a realistic discount rate (I know the treasury sets one) is being used.  If the sort of discount rate being used for calculating annuities for pensions at the moment were used the BCR might look a whole lot better.

* Discounting reduces the value of future costs and benefits in the future by a factor.  So the initial costs and benefits are not reduced the first year costs and benefits are reduced by dividing by 1 + the discount rate.  The second year costs and are reduced by dividing by the square of 1+ the discount rate etc.  So the smaller the discount rate the less the impact. 

[ellendune]

It appears that the treasury discount rate is still at 3.5% (so 1 + discount rate is 1.035) which was set in 2003.  Much has changed in the world since 2003 DB^» (Deutsche Bahn - German State Railway - about) pension valuations are typically done on a rate around 0.5% to 1% above 20 year gilt rates which are between 0% and 0.5% at the moment!  So arguably the discount rate for should be between 0.5% and 1.5%.  That would dramatically change the BCR^▸ (Benefit Cost Ratio) for many schemes.

[Red Squirrel]

...demand can't go below -100%, but it surely can go above +100%

A very good point. I worked out that percentage on my way to deriving the more meaningful figure based on actual numbers, but it would probably have strengthened my argument had I not mentioned it!

...this makes it sound as if the BCR^▸ (Benefit Cost Ratio) is balanced on a knife edge. (2.00001 ?).

If I've read this report correctly, the BCR for MetroWest 2 (which includes Henbury) is 1.88: 'Moderate Value for Money'. But the report goes on to say:

...it is considered that there are a number of factors which could improve the BCR such as additional growth and alignment with demand from existing and emerging developments.

I note that of the capital cost of £54.163 million, the S106 contribution is £2.3 million plus land.

[grahame]

...demand can't go below -100%, but it surely can go above +100%

A very good point. I worked out that percentage on my way to deriving the more meaningful figure based on actual numbers, but it would probably have strengthened my argument had I not mentioned it!

If you go for the median rather that the mean, you rise from 6.2% over to 7% over.  23 stations listed, and sorted by growth percentage, that 7% over target is in the middle.

[IndustryInsider]

It would be interesting to see how the growth forecasts vs reality for stations opened in the last yen years stack up.  Forecast modelling should become more accurate over time.

[stuving]

It would be interesting to see how the growth forecasts vs reality for stations opened in the last yen years stack up.  Forecast modelling should become more accurate over time.

For errors of timing that's what I'd expect - the world is often slow to do what's predicted, especially (though not only) where railways are involved. But for other errors, I'd expect rather that they would grow with time, as the world changes in ways never foreseen and often not foreseeable.