Melbourne Metro Tunnel Project—Phase 1: Early Works

Tabled: 6 June 2019

2 Strategic planning for the Melbourne Metro Tunnel

Before the Victorian Government funds a public sector project, agencies must show that they have examined the need for the investment, understood the underlying context of the need, and can propose strategic interventions and project options to solve the problem and deliver benefits to the community and the economy.

This strategic conceptualising and planning process helps agencies to develop a detailed business case, which is the typical vehicle used to seek government approval and funding of a project.

This part examines the strategic planning, option identification and business case development used for the Melbourne Metro Tunnel Project.

For this component of the audit, we examined whether the transport problem that the Melbourne Metro Tunnel Project is expected to solve was adequately understood in the 2016 business case.

To achieve this, we focused on the following agencies:

  • DPC
  • DoT
  • DTF
  • PTV.

2.1 Conclusion

The agencies involved in the strategic planning for the Melbourne Metro Tunnel Project demonstrated a good understanding of Melbourne's future rail transport needs, as expressed in the business case.

Although these agencies understood the problem and therefore the need for the project, their response to the need only considered a tunnel as an option for detailed analysis. Other potentially viable non-tunnel options to solve the identified transport problem were not analysed in a meaningful and comparative manner.

Options analysis

DoT did not undertake a comprehensive, comparative analysis of any option other than a new asset option—that is, a city bypass rail tunnel. This meant government was not fully informed as to whether a non-tunnel option could have provided similar benefits to the overall rail network at an equivalent or potentially lower cost than the current Melbourne Metro Tunnel Project.

A potential 'existing network improvement' option could have involved heavy investment in the existing overland rail network, using HCMTs and high-capacity signalling.

We consider that DoT could have provided a clearer picture for decision-makers by presenting separate analyses of the costs and benefits of a tunnel project compared to investing in the existing overland rail network, and a combination of these options.

Modelling of passenger demand

The business case used two strategic passenger demand models—VITM and a Strategic Check Model—a commercially available strategic transport forecasting model.

The sensitivity of these models to changes in the supply of public transport services or fares, which could affect passenger behaviour, is low when compared to the typical ranges that international guidelines expect from strategic transport models.

This means that there is a risk that both models could have inaccurately forecast the timing, location, quantity and rate of growth of future patronage for the new tunnel and stations. This could also affect the accuracy of the proposed economic benefits of the project, which heavily relied on forecast patronage data from VITM.

Benefits framework

The 2016 business case captured the expected network-wide benefits from the Melbourne Metro Tunnel Project well and showed that a rigorous analysis was conducted to identify and attribute project benefits.

However, the benefits plan in the business case has some weaknesses. It does not describe a risk management strategy, nor does it allocate responsibility for ongoing benefits management. This could make it challenging in the future to measure whether the project has delivered its expected benefits, and thus define the level of value for money achieved.

2.2 Melbourne Metro Tunnel Project business case

Aim of the project

The 2016 business case's key objectives were to:

  • ease pressure on a strained public transport system
  • meet public transport patronage growth due to the city's growing population
  • provide the 'backbone' to establish a future metro-style rail network.

The project aims to achieve these by connecting the Cranbourne/Pakenham and Sunbury lines—the metropolitan rail lines that service two of Melbourne's largest growth corridors to the south-east and north-west—to create a Sunshine–Dandenong line. This is meant to increase the number of services on the new dedicated corridor and enable future expansion of the metropolitan network by removing these services from the City Loop and other central area tracks.

The project also aims to create capacity through the inner core of the network for five other metropolitan lines—Werribee, Frankston, Craigieburn, Upfield and Sandringham—and enable an increase in services to better meet demand across the metropolitan rail network.

Analysing the transport problem

The business case details the anticipated population growth of Melbourne and the trend towards increased demand for public transport on trains.

The high-level problems identified were:

  • chronic overcrowding and unreliable rail services, which reduce Melbourne's liveability and access to jobs and key activity precincts
  • physical transport network constraints, which reduce Melbourne's economic prosperity and productivity
  • insufficient public transport services, which impact access into and around central Melbourne and limit the potential for urban renewal.

The business case sufficiently describes and analyses these problems.

Strategic options to address the problem

When addressing potential solutions to the three identified high-level problems, the business case makes a distinction between high-level strategic options and lower-level options to implement a solution based on the preferred strategic option. This two-step approach is consistent with DTF guidance.

The business case considered three strategic options to address the identified problems:

  • Strategic option 1—current state: current operations with productivity improvements, without significant investment beyond currently planned expenditure.
  • Strategic option 2—demand and productivity management: conduct demand management and productivity improvements on existing assets or systems without significant investment.
  • Strategic option 3—increase supply: increase capacity for access to the CBD through significant capital investment in public transport assets or road alternatives.

The business case assessed the strategic option to increase supply as the most favourable strategic response.

Options assessments of various capital projects were used to develop detailed solutions that could respond to the chosen strategic option, with the Melbourne Metro Tunnel Project listed as one of the detailed solutions.

The business case identified that this strategic option provided the most comprehensive medium to long‐term solution to meet increasing patronage demand and capacity constraints on the rail network and increasing supply would provide better public transport for priority CBD development precincts.

Incomplete analysis of capital options

Only strategic option 3 considers significant investment as a possibility—and this option does not consider significant new investment to better use or augment the existing overland rail network.

Therefore, transport agencies did not adequately analyse whether a similar (i.e. $11 billion) or lesser amount of money could have delivered similar or greater benefits than the Melbourne Metro Tunnel Project if directly invested across the existing rail infrastructure network.

An appendix to the business case, which subjected 13 capital investment options to a high-level qualitative ranking analysis, included separate elements of an overland rail network option (HCMTs, high-capacity signalling, and an overland route improvement described as 'viaduct widening').

Although the qualitative shortlisting process used did not result in a detailed analysis in the business case of this combination of potential capital projects, HCMTs and improvements to track and power supply along the Pakenham/Cranbourne corridor are now being delivered by other projects.

Longer, higher-capacity trains will increase the productivity of each train trip. The new seven-car HCMTs will carry 1 100 passengers per train, which is 20 per cent more than older trains. This means that some capacity benefits for the Dandenong–Sunshine corridor will be available early, by using HCMTs on the existing overland rail route before the tunnels open in 2025.

These capital projects are funded under the HCMT PPP and the Cranbourne–Pakenham Line Upgrade Project and are key enablers for many of the wider network benefits that the Melbourne Metro Tunnel Project expects to deliver.

This will deliver important enabling infrastructure and rolling stock, which will support the Melbourne Metro Tunnel to achieve expected operational benefits across the wider rail network.

These current investments are similar to what would have been required for heavy investment in the overland rail network—and therefore imply that this option was not only feasible, but required.

We consider that DoT should have given more objective comparative analysis of heavy investment in the overland rail network as part of the detailed capital options development process.

This would have given decision-makers a better understanding of the relative costs and benefits of an overland rail network investment approach versus the current tunnel project.

2.3 Concept design and modelling of passenger demand

After the government approved the Melbourne Metro Tunnel Project business case in February 2016, RPV commissioned the development of a concept reference design by a private sector joint venture, which RPV appointed as their technical adviser.

Purpose of the concept design

The main purpose of the concept design was to demonstrate a technically feasible way for the Melbourne Metro Tunnel Project to be constructed.

A secondary purpose was for the concept design to be used for any required environmental assessments and planning approvals. The design included maps of the proposed tunnel alignment and architectural drawings of the stations.

Part 3 of this report discusses the reference design and EES process in detail.

Patronage demand modelling

The purpose of transport modelling is to evaluate the performance of complex transport systems and to identify an expected capacity range for patronage demand.

Models used in Victorian transport planning

Transport agencies in the Victorian public sector use transport simulation models to test and identify the benefits or drawbacks of changes to the transport network. These models are not limited to rail infrastructure and simulations can be performed for any transport mode.

Strategic simulations used for transport modelling are commonly known as four-step models and are an accepted international approach for this type of predictive modelling.

The four steps refer to the following:

  • Determine the total number of trips to be made in the simulation.
  • For each of these trips, determine a trip start point and a trip end point.
  • Distribute the trips between the start points and the end points to either a public transport mode of travel or a 'highway' mode of travel (i.e. to a private vehicle).
  • Assign the trips to a particular path through either the public transport system or the 'highway' system, as applicable.

These types of models attempt to simulate the future but, as Figure 2A shows, many sources of uncertainty can have a material impact on their accuracy.

Figure 2A
Notional sources of uncertainty in predictive forecasting models

Figure 2A shows notional sources of uncertainty in predictive forecasting models

Source: Willumsen, Luis G. (2015). 'Dealing with Uncertainty in Demand Modelling and Forecasting', presented at the New Zealand Modelling User Group Conference, Auckland, 10 September 2015.

The uncertainty associated with a model's output is driven by four factors:

  • The base year data used in the model may become out of date over time.
  • The future data that could replace the base year data is itself uncertain.
  • The imperfect quality of the model may introduce a level of uncertainty.
  • The scenarios explored by the models introduce a level of uncertainty as time passes and the model's scenarios become increasingly out of date.

These four factors of uncertainty increase exponentially with time, meaning that forecasts more than a few years into the future become increasingly uncertain and therefore cannot, in general, be relied on.

Models used to support Melbourne Metro Tunnel Project strategic planning

To support analysis in the 2016 business case, PTV and its contractors simulated the predicted future operations of the Melbourne Metro Tunnel on two strategic computer-based modelling tools, VITM and a Strategic Check Model.

This modelling effort sought to calculate passenger demand for proposed and existing stations after the Melbourne Metro Tunnel Project is operating.

Victorian Integrated Transport Model

VITM was the primary strategic demand modelling tool used for the Melbourne Metro Tunnel Project analysis.

VITM is a four-step strategic transport simulation tool that transport agencies use to model the overall transport system in Victoria. VITM calculates the total number of trips between different points in Victoria. This can be used to determine increased demand for transport due to population growth or an increase of jobs in a particular location.

A private company developed VITM in 2011 for DoT, which owns the model and supports it with a mix of departmental staff and contractors.

Strategic Check Model

The Strategic Check Model is another four-step strategic simulation tool.

Although the modelling for the Melbourne Metro Tunnel Project primarily used VITM outputs, PTV used the output from the Strategic Check Model to assess the accuracy of outputs from VITM.

Finer-grain models

RPV used VITM-generated passenger demand forecasts to inform two station-level passenger modelling tools called ClicSim (City Loop and Inner Core Simulation) and STEPS. RPV used these simulations to support the detailed design process and refine the concept designs for the stations.

We discuss these finer-grain simulations in Part 3 of this report.

Analysis of the strategic forecasting models

Our analysis of the models found that key decisions about the construction of demand forecasting models are not well documented.

Similarly, calibration and validation processes are not transparent or timely and the rationale for many data assumptions included in VITM and used for the 2016 business case is not clear, because DoT did not document these in detail.

The accuracy of a predictive model's sensitivity to change can be checked by testing a feature in the model and then seeing how the model predicts any subsequent demand changes. This is known as 'output elasticity' and can show how well the model performs when predicting change.

Our commissioned expert research found that elements of the modelling—such as the sensitivity of the model to the provision of extra services or an increase in fares—may cause an under-forecast of the rate of expected demand growth. We judged this in comparison to the expected values from other comparable international guidelines, which are based on observed international experience.

We also found that the Strategic Check Model, which PTV used to check VITM outputs, has more acute sensitivity issues than VITM. The sensitivity of VITM is just within, and the Strategic Check Model is well outside, the range of international guidelines, when judged against both models' observed output elasticities.

There is a risk, therefore, that neither model is adequately sensitive to impacts of public transport system changes, and therefore could under-forecast the future rate of patronage demand growth. This could mean that more passengers use the public transport system more often or earlier than the model predicts.

Lack of disclosure of uncertainty in forecasting models

DoT does not adequately disclose the inherent bands of uncertainty in the various strategic transport models used in Victoria in briefings to key decision-makers, public documents, or government announcements or websites when referencing modelling outputs. Although an appendix in the 2016 business case considered different modelling scenarios, the modellers did not vary the inputs of each scenario to arrive at a band of uncertainty for each scenario.

Models can generate a precise level of detail, but because of uncertainty, there is a risk that they are not accurate in representing current or future situations.

Relevant recent examples of how data from models is used without a discussion of the inherent uncertainty include:

  • modelling for the Melbourne Metro Tunnel Project business case, which was presented as a sole modelling output rather than offering a range of potential modelling scenarios such as low/medium/high or conservative/median/aggressive
  • passenger-per-day public announcements for new rail projects, which often use a single expected patronage figure, with no discussion of any possible plus or minus percentage variance.
Recent reviews of the models

The models used for demand forecasting for the Melbourne Metro Tunnel Project were peer reviewed, as required by DTF's High Value High Risk process. We found, however, that the peer review reports were high level and did not critique the basis or rationale for some of the fundamental assumptions that drive each model's outputs.

DoT advises that assumptions were queried by the peer reviewers, but we saw limited documentary evidence of this.

A review by an international engineering and consultancy firm, commissioned by Infrastructure Victoria in 2016, found that the models used by DoT, including VITM, were satisfactory in the way they performed their calculations.

However, the technical review also found that the current set of forecasting tools needed sustained investment, as well as enhancements in construction and usage, to improve decision-making and investment choices.

DoT has not yet implemented this report's suggested further enhancements, but advised that it is undertaking work to improve specific elements of the VITM model, and will do a major model recalibration over the next two years.

The 2016 review provides a useful road map for the wider transport portfolio to improve rigour and cohesion in the way it uses demand forecasting models. We are also aware of two other external reviews of transport modelling commissioned by DoT in 2014 and 2016 that make a range of useful recommendations about VITM's construction and operation, as well as broader governance and resourcing improvements.

DoT should implement the enhancements recommended in these three strategic and technical reviews and apply them to the various transport demand and simulation models used across the wider transport portfolio, as applicable.

Other implications from our review

Due to many machinery-of-government changes in the transport portfolio in recent years, we observed that corporate memory is sometimes patchy and human capital is stretched in the transport modelling area. This leads to fragmented technical knowledge about the models and limited documentation on past key decisions.

The governance processes for the calibration and validation of the VITM model are not transparent and some assumptions used in the models are not clearly documented or, when documented, are complex and difficult to understand.

Our previous audits have found that population growth forecasts in general—including the Victoria in Future population forecasts from DELWP, which have been used as inputs to the modelling for the Melbourne Metro Tunnel Project—have been historically inaccurate and tend to underestimate growth.

However, these forecasts are still used as key data input into long-run patronage demand models, so any previous error in population forecasting can carry forward as a trend error for future patronage demand assumptions and forecasts.

2.4 Expected benefits from the proposed solution

The business case outlined a number of benefits that the Melbourne Metro Tunnel Project is expected to deliver.

Economic benefits

The Melbourne Metro Tunnel Project is expected to enable rail users to commute more easily to and from the CBD, thereby increasing the accessibility of economic opportunities and high-quality jobs and services. It should also enable businesses in the CBD to access a broader range and wider pool of workers.

During construction, the Melbourne Metro Tunnel Project is expected to create an extra 3 900 jobs across Victoria, and approximately 4 700 jobs across Australia. The 2016 business case analysis argued that the overall economic benefits outweigh the costs of the project.

The business case identified these financial benefits in the form of net present values and benefit–cost ratios, which rely on outputs from VITM. If this modelling is incorrect, the benefit–cost ratio for the project may be incorrect.

Substantial benefits could also have been achieved through better use of existing assets. However, only cursory discussion or analysis of this option is made in the business case.

It is therefore unclear whether the benefit–cost ratio of the Melbourne Metro Tunnel Project outweighs the benefit–cost ratio that better use of existing assets—through targeted improvement investments such as investing in the existing overland route assets and using high-capacity signalling and HCMTs—could have delivered.

Passenger benefits

Modelling for the business case estimated that the Melbourne Metro Tunnel Project could increase the peak capacity of the network by 39 000 passengers in each morning and afternoon peak period.

Modelling for the project has also estimated substantial travel time savings for both metropolitan and regional commuters travelling to St Kilda Road and Parkville, with a maximum saving of up to 25 minutes for commuters on the Cranbourne/Pakenham line and up to 15 minutes on the Gippsland line.

Figure 2B shows the projected increase in peak capacity and time savings for all metropolitan train lines as a result of the Melbourne Metro Tunnel Project.

Figure 2B
Expected increase in peak capacity and time savings

Metropolitan train line

Increase in peak capacity (%)

Increase in passengers
(per week)

Time saving to St Kilda Road (minutes)

Time saving to Parkville (minutes)

Alamein

Not reported

Not reported

10

10

Belgrave/Lilydale

Not reported

Not reported

10

10

Craigieburn

27

54 000

8

8

Cranbourne/ Pakenham

45

121 000

25

15

Frankston

15

36 000

15

15

Glen Waverley

Not reported

Not reported

5

10

Hurstbridge

Not reported

Not reported

5

10

Sandringham

48

72 000

5

10

Mernda

Not reported

Not reported

5

10

Sunbury

60

113 000

20

20

Upfield

17

45 000

8

8

Werribee/ Williamstown

24

63 000

5

10

Source: RPV (Metro Tunnel website).

Service benefits

The business case details a range of benefits that improve service reliability and capacity of the metropolitan rail network, including:

  • easing congestion and facilitating future growth
  • freeing up the City Loop and capacity in the inner core of the metropolitan rail system
  • minimising the overall impact of incidents and improving the resilience, punctuality and reliability of services.

The 2016 business case also identified some benefits for trams on the busy St Kilda Road/Swanston Street corridor, with the new rail line expected to take up some of the heavy north–south passenger tram loads.

This would allow for future redistribution of some tram routes off the corridor to better serve growth in the western area of the CBD, via a new 200-metre connecting tram track known as the Park Street Link.

Benefits framework for the project

A benefits management process is expected to identify, track and measure benefits to ensure that a project's potential and anticipated benefits are actually delivered. An effective benefits management process is critical to achieving the outcomes sought from investments and is required for all major infrastructure projects in Victoria.

The 2016 business case clearly specifies two major benefits, namely:

  • greater productivity and economic growth for Melbourne
  • a more liveable Melbourne.

Both benefits are linked to a specific project outcome—the completion of the Melbourne Metro Tunnel Project. Public transport users and non-users will benefit from increased productivity and economic growth, and improved liveability.

However, the business case does not explain the weighting of these benefits (or the process used to choose them):

  • greater productivity and economic growth for Melbourne—60 per cent
  • a more liveable Melbourne—40 per cent.

Figure 2C shows the Benefit Management Map.

Figure 2C
Melbourne Metro Tunnel Project Benefit Management Map

Figure 2C shows the Benefit Management Map.

Note: PT = public transport. GFA = gross floor area.
Source: DEDJTR, Melbourne Metro Business Case, 2016.

Analysis of the benefits framework

Overall, the benefit management framework is well designed and:

  • identifies benefits that logically flow from the proposed outcomes of the Melbourne Metro Tunnel Project
  • defines a target group for the benefits
  • shows the outlined benefits as clearly attributable to the project
  • has benefits that are discernible and have measurable key performance indicators.

The 2016 business case followed the applicable DTF guidance and captured the expected benefits well. The business case also shows that DoT performed a rigorous analysis to correctly identify and attribute the expected project benefits.

We found, however, some deficiencies in the benefit management process. The business case does not describe a risk-management strategy, nor does it allocate responsibility for benefit management.

We have found in recent transport-related audits that it is important to assign ongoing evaluation responsibilities for projects to entities that will exist well beyond the project's completion.

Allocating clear benefit management responsibility, with adequate resources, mitigates the risk that agencies focused on delivery will neglect or avoid the final stages of benefit management and post-completion evaluations for major investments.

Many benefits assumptions relate to unfunded projects

Many benefits from the Melbourne Metro Tunnel Project are expected to accrue to the wider metropolitan rail network rather than just the Sunbury to Pakenham/Cranbourne corridor, which the business case shows will only get a modest frequency increase of four trains per peak hour when the tunnels open.

Many assumptions in the wider benefits framework are also based on projects that the government has not yet approved or funded.

During this audit, DoT told us that the business case's economic analysis was based on a distinct modelling approach used in Victoria, where assumptions about the future development of the network are built into any underpinning analysis. DoT says that this is done to give a more realistic comparison of the transport network and economy, both with and without the project, and to moderate an overly optimistic view of a project's potential benefits.

We understand DoT's rationale for this modelling and analysis approach, but consider that relying on unfunded projects to define a future network state is a key risk to defining and thus realising the overall benefits.

Including unfunded projects in modelling assumptions potentially reduces the extent and attribution of benefits arising from the project under analysis. This is because the model assumes that other notional unfunded projects will provide some other benefits and absorb some demand growth. However, this assumption is only valid if the assumed projects proceed within the model's assumed time frame.

Infrastructure Australia's December 2016 review of the business case concluded that it was 'confident that the benefits of the project will exceed its costs and that the proposed solution will provide a net benefit to the Australian economy'.

However, Infrastructure Australia also raised some concerns with DoT's modelling approach and noted that although the 'approach is useful for the purpose of integrated long-term transport planning, it is unconventional for economic evaluations, and could understate the [benefit–cost ratio] if the unfunded substitute projects assumed in the base case do not proceed'.

DoT and the central agencies—DTF and DPC—will need to carefully manage this issue around unfunded assumed projects to ensure the significant investment in the Melbourne Metro Tunnel Project delivers the expected benefits to society and the economy, as described in the business case.

Recent rail project announcements

Many recent rail project announcements, such as the Suburban Rail Loop, Melbourne Airport Rail Link, and the Wyndham Vale and Melton electrification projects, have materially altered the 2016 business case assumptions and analysis for the tunnel project.

Due to these announcements, many assumptions around patronage demand projections, rail operating concepts, future rail network configurations and wider project benefits relate to a future network strategy that is no longer correct.

DoT is the client for the project, but despite the many material changes to the Melbourne Metro Tunnel Project's wider context, it has not yet adjusted its assumptions about how the tunnels will operate and the benefits that will be delivered after completion.

DoT will need to review and refresh the original assumptions. This could occur as part of the modelling and business case development processes for any new projects so that agencies can better understand network-wide effects and their interaction with the tunnels.

Once assumptions are refreshed, this should inform an updated benefits management plan which DoT, as project client, should actively manage and oversee.

Back to Top