Follow up of Managing the Level Crossing Removal Program

Have the Department of Transport and the Major Transport Infrastructure Authority effectively addressed recommendations made in our 2017 audit Managing the Level Crossing Removal Program?

Video presentation

Video transcript

Key facts

What we found

We consulted with the audited agencies and considered their views when reaching our conclusions. The agencies’ full responses are in Appendix A. 

Summary of progress and outcomes

The table on the next page summarises DoT’s and MTIA’s progress on each of our 2017 recommendations. Appendix C contains the full list of recommendations from our 2017 audit.

Of the 10 recommendations:

• seven have been fully implemented
• one has been partially addressed, with no further work planned
• two are still in progress and further work is needed to ensure that the underlying issues are addressed.

There is a detailed assessment of each recommendation throughout the report.

Agencies' progress on addressing our 2017 recommendations

Note: Refer to Appendix C for the full recommendations.

Back to top

Established in 2015, the LXRP is one of the government’s major transport infrastructure projects. It aims to reduce pressures on the transport network by easing road congestion and travel delays caused by level crossings. It also intends to improve safety by decreasing the chance of accidents involving trains and road users or pedestrians.

Our 2017 audit Managing the Level Crossing Removal Program examined the effectiveness of the LXRP and made 10 recommendations to improve it. 

1.1 Risks posed by level crossings

A level crossing is an intersection where a rail line crosses a road or path at the same level. 

Level crossings use boom gates to manage the flow of road and foot traffic across the rail line. When boom gates are down for extended periods of time, traffic can become congested. This can increase risk-taking behaviours by drivers and pedestrians.

Between 2005 and 2015, there were more than 149 level crossing collisions involving a train and a vehicle or pedestrian in metropolitan Melbourne. Of these, 38 resulted in fatalities and 22 in serious injuries. 

The alternative to a level crossing is constructing an overhead bridge or underground tunnel to separate the rail line from road and foot traffic.

1.2 The LXRP’s evolution

LXRP1: the first 50 sites

Following its election in 2014, the Victorian Government pursued its pre-election commitment to remove 50 of Melbourne’s most dangerous and congested level crossings by 2022. These 50 sites are listed in Appendix D. At the time, the project's estimated cost was $6.9 billion. 

In 2015, the government established the Level Crossing Removal Authority (LXRA), which was an administrative office within the then Department of Economic Development, Jobs, Transport and Resources (DEDJTR).

LXRA was responsible for delivering LXRP1 and achieving:

• more reliable and efficient transport networks by addressing congestion and delays caused by level crossings
• better connected, liveable and thriving communities by reducing delays and increasing the attractiveness of living and investing in areas surrounding removed crossings
• safer communities by removing conflict points between trains and road users and pedestrians to reduce the number of crashes. 

LXRP2: a further 25 sites

In October 2018, the government announced that it would remove a further 25 level crossings through LXRP2. 

The official LXRP covers 75 sites. However, MTIA is removing a further three sites that need to be removed at the same time as one of the official project sites. Appendix D contains the full list of all LXRP sites and their progress to date.

Project budget

As at 30 June 2020, MTIA has spent $6.2 billion of its $14.8 billion total funding. MTIA is forecasting that it is on track to complete all 75 level crossing removals within budget.

To date, the Department of Treasury and Finance (DTF) has approved the release of $1.28 billion of the project's risk provision to address cost escalation pressures. The remaining risk provision is $879 million.  

Chapter 3 discusses how MTIA is using its procurement and packaging approach to minimise project costs.

1.3 Changes to government agencies since our 2017 audit

Since our 2017 audit, machinery of government changes have altered the roles and responsibilities of the agencies delivering the LXRP.  

As Figure 1A shows, on 1 January 2019, the government abolished DEDJTR. It replaced it with DoT and the Department of Jobs, Precincts and Regions. DoT also absorbed the functions of:

• VicRoads, which managed the state’s road networks
• Transport for Victoria (TfV), which integrated the planning and coordination of the state’s transport system 
• Public Transport Victoria (PTV), which coordinated the public transport network and managed its integrity.

At the same time, the government abolished LXRA and transferred responsibility for the LXRP to MTIA—a newly formed administrative office within DoT. 

MTIA is also responsible for delivering the government’s other major transport infrastructure projects, including the Metro Tunnel, West Gate Tunnel, North East Link, Regional Rail Revival and Melbourne Airport Rail.

FIGURE 1A: Machinery of government changes relevant to this audit

Note: In 2013, VicRoads began planning works to remove 10 level crossing sites. The government transferred responsibility for this to LXRA in 2015.
Source: VAGO, based on information from DoT and MTIA.

Since the machinery of government changes:

• DoT is responsible for delivering the six recommendations we made to DEDJTR
• MTIA is responsible for delivering the four recommendations we made to LXRA.

Throughout this report we refer to DoT and MTIA, except for when we discuss former agencies’ specific actions. 

1.4 Project delivery

MTIA is delivering the LXRP using an alliance contracting model. Rather than allocating individual contracts to remove each level crossing, MTIA groups multiple sites into packages and then contracts program alliances to deliver them. 

Program alliances

Following a competitive tender process, MTIA established four program alliances in 2017 to deliver different work packages:

• North Eastern Program Alliance (NEPA)
• North Western Program Alliance (NWPA)
• Southern Program Alliance (SPA)
• Western Program Alliance (WPA).

In 2018, NEPA split into two streams for road and rail:

• NEPA Rail
• NEPA Road.

This change was designed to maximise the expertise of the construction companies that are part of NEPA.

In 2019, NEPA Rail and NEPA Road changed their names to the South Eastern Program Alliance (SEPA) and the Metropolitan Roads Project Alliance (MRPA) respectively.

Legally, there are still four program alliances. However, in practical terms MTIA is working with five. Each program alliance has different program directors and leadership teams. 

Appendix D shows the full list of LXRP2 sites and Appendix F lists their program alliance allocation.

Procurement approach

At the time of our 2017 audit, LXRA planned to use a deferred price contracting structure to procure contractors to remove the remaining 32 level crossings.

Under this structure, program alliances would competitively bid for and complete an initial package of works of between one and four level crossing sites. If LXRA was satisfied with an alliance’s performance, it would then ask it to develop a formal proposal for delivering additional sites. This meant that only the initial works package would be subject to full price competition. 

In February 2019, MTIA introduced a new delivery strategy and allocated the remaining LXRP1 sites and 25 additional LXRP2 sites to the program alliances. 

While MTIA has now allocated all 75 sites, program alliances still need to provide a detailed works proposal and costings before they are awarded a construction contract for a package of sites. MTIA can reallocate sites to a different program alliance if it is not satisfied with an alliance’s works proposal or past performance.

We discuss MTIA’s approach to procurement and packaging in Chapter 3.

1.5 Options for removing level crossings 

As shown in Figure 1B, there are four options for separating the rail line from the road to remove a level crossing. These are known as grade separation options.

MTIA analyses high-level issues, benefits and constraints for each level crossing site before selecting an indicative preferred option.

FIGURE 1B: Grade separation options

Option	Description	Potential negative impacts	Potential benefits
Rail over road	A rail bridge is built over the road. The road remains at the existing level.	Train stations may need to be modified to suit the new rail level.	It can improve pedestrian access and create opportunities to use the area beneath the rail line.
Rail under road	A rail tunnel is built beneath the existing road.  The road remains at the existing level.	Nearby train stations may need to be modified or rebuilt to suit the new rail level.	Additional pedestrian or cycling bridges can be built to further improve access across the lowered rail line.
Road over rail	The rail line remains at the existing level.  A road bridge is constructed over the rail line.	Service roads and alternate access options need to be built. Pedestrian access to train stations needs to be maintained.	Modifications to train stations are usually not needed because the rail level does not change.
Road under rail	The rail line remains at its existing level.  An underpass is built beneath the rail line for the road.	Service roads and alternate access options need to be built. Pedestrian access to train stations needs to be maintained.	Modifications to train stations are usually not needed because the rail level does not change.

Source: VAGO, based on images and information from MTIA.

Selecting a replacement option

To determine which grade separation option to use for each level crossing site, MTIA considers:

• road functionality and rail operational requirements
• road and rail horizontal and vertical alignments
• construction challenges
• station locations and opportunities to improve the surrounding precinct
• requirements for land acquisition.

MTIA can also choose a hybrid approach that combines elements of different options. 

1.6 Applying the HVHR framework 

The HVHR framework is a series of project assurance checks that DTF manages. These checks scrutinise major infrastructure projects to increase the likelihood that they will achieve their stated benefits on time and within budget. 

DTF uses an assessment tool to determine a project’s risk profile. It considers a project HVHR if it is:

• high risk according to its risk assessment tool
• medium risk according to its risk assessment tool and has a total estimated investment of between $100 million and $250 million
• low risk according to its risk assessment tool, but has a total estimated investment of more than $250 million 
• identified by the government as needing the level of scrutiny applied to HVHR investments.

According to DTF’s criteria, the LXRP is a HVHR project. While it is low risk, it requires a total estimated investment of more than $250 million.

Investment life cycle and HVHR guidelines and gateway reviews

Like other HVHR projects, the LXRP is subject to DTF’s investment life cycle (ILC) and HVHR guidelines. The guidelines outline the government’s process for investment decision-making and project delivery across three stages:

• the business case
• procurement
• delivery.

HVHR projects need to complete ongoing quarterly progress reports for OPV. They also undergo external gateway reviews to identify risks to their budget or delivery. These reviews occur at the following six ’gates’:

• gate 1: concept and feasibility
• gate 2: business case
• gate 3: readiness for market
• gate 4: tender decision
• gate 5: readiness for service
• gate 6: benefits analysis.

The ILC and HVHR guidelines recommend that if a project involves multiple physical sites, then the responsible agency should package gateway reviews for efficiency. For the LXRP, works packages that meet the $250 million HVHR threshold undergo gateway reviews, rather than the whole project. 

Back to top

Conclusion

Our 2017 audit found that DEDJTR did not analyse if the 50 level crossing sites that the government had committed to remove were the most dangerous and congested. We also found that DEDJTR and LXRA did not develop a business case to outline the project’s intended benefits before starting construction. 

Since then, LXRA developed and applied a transparent process to select sites for LXRP2. This new process has improved the project’s cost-effectiveness because it uses delivery efficiency as one of the criteria for site selection.

However, DEDJTR and LXRA did not develop a new business case for LXRP2. This means that the government did not have vital information about the project's expected economic benefit before it made its decision to invest. 

2.1 LXRA’s site selection process for LXRP2

In our 2017 audit, we found that DEDJTR did not assess the value of removing the first 50 level crossings because they were based on the government’s 2014 election commitment.

DEDJTR and LXRA did not provide detailed advice to the government on the project's expected benefits before construction began. This meant that the government could not be assured that the selected sites met the project’s stated objective to remove the most dangerous and congested level crossings. 

We recommended that LXRA improve its site selection process for any future level crossing removals. 

Progress and outcomes

From mid-2017 to late 2018, LXRA developed, refined and applied a transparent site selection process for LXRP2. Figure 2A shows a timeline of LXRA’s work on LXRP2 during this period. 

LXRA used three principles in its initial planning work as criteria to select LXRP2 sites:

• safety
• movement
• place.

Following advice from DoT, MTIA refined this to include a fourth principle—delivery efficiency.

LXRA used these principles to select the most dangerous and congested sites for LXRP2. LXRP2 also minimises delivery time and cost by removing sites that are next to LXRP1 sites or other major projects. 

Unlike its process for LXRP1, LXRA provided DEDJTR and the government with a range of different project proposals before finalising the LXRP2 sites. These options varied in size from 21 to 45 sites and ranged from $4 billion to $16 billion in cost. 

MTIA’s advice to the government for LXRP2 was a significant improvement to LXRP1. This is because it provided the government with a comprehensive understanding of what it could achieve for different investment levels. 

Selecting LXRP2 sites 

LXRA developed a site prioritisation framework to respond to our recommendation. This framework outlines LXRA’s four principles for prioritising level crossing removals. Figure 2B describes these principles and the underpinning factors that LXRA used to assess each site. 

As Figure 2C shows, LXRA’s framework has a five-stage process to determine which sites to recommended for removal. 

In stage one, LXRA analysed 276 level crossings using data from several sources, including the Australian Level Crossing Assessment Model (ALCAM) and VicRoads’ traffic data. 

In stage two, LXRA used the safety, movement and place principles to categorise each site as having a ‛very high need’ or a ‛high need’ for removal. For example, Figure 2D shows how LXRA defines high or very high need for the movement principle.

In stage three, LXRA identified and prioritised the sites that met the threshold for very high need and high need for more than one principle. 

As Figure 2E shows, LXRA then considered these sites using the delivery efficiency principle in stage four. 

LXRA used the following factors to assess if removing a level crossing would have a delivery efficiency benefit: 

In stage five, LXRA produced a consolidated list of 25 sites to submit to the government for approval. Figure 2F shows that each site meets at least one of the selection principles or criteria for delivery efficiency, adjacency or future need.

FIGURE 2F: LXRP2 sites’ prioritisation ratings for principles, delivery efficiency, adjacency and future need

Source: VAGO, based on information from MTIA and ALCAM.

Balancing the site selection principles 

While the site selection principles were not formally weighted, LXRA prioritised sites that scored highly on the safety and movement principles. This means that LXRP2 is more closely aligned with the project’s overall objective of removing dangerous and congested level crossing sites than LXRP1. 

Safety

LXRA used data from ALCAM to consider the number of past traffic incidents and the risk of future incidents occurring at each site. 

By using the safety principle to prioritise sites, LXRA selected:

• the three most dangerous level crossing sites in the state (Fitzgerald Road, Ardeer; Mount Derrimut Road and Robinsons Road, Deer Park). While ALCAM classifies these sites as ‛non-metropolitan’, LXRA selected them over metropolitan sites with higher movement or congestion ratings because the risk scores for these sites are significantly higher than those ranked fourth and below
• 18 sites in ALCAM's top 100 riskiest sites in the metropolitan area 
• two sites that were rated as very high against the safety principle (Greens Road and Webster Street in Dandenong). ALCAM ranked these sixth and eleventh in its list of the state’s most dangerous sites.

Movement 

The movement principle focuses on sites that have high levels of traffic congestion. Congestion can increase the risk of accidents and risk-taking behaviour at level crossings. 

LXRA used data from TfV, VicRoads and ALCAM to assess sites against the movement principle. 

LXRP2 sites have an above average level of daily trains and vehicle traffic. They also have a higher number of predicted annual collisions. In particular, LXRP2 sites have:

• 160.4 trains per day on average, compared to the state average of 19.55
• 10 400 vehicles per day on average, compared to the state average of 1 822
• 64 per cent more predicted annual collisions on average.

A further 32 per cent of LXRP2 sites have a medium to high risk of collisions occurring. The level crossing with the highest level of traffic in the state (46 000 daily vehicles) is Bell Street, Preston. This site is part of LXRP1 and is adjacent to three LXRP2 sites.

Place 

The place principle considers a site’s impact on pedestrian and cycle paths. It also assesses how the level crossing affects access to important local community facilities, such as schools, medical facilities and local shops. While LXRA consistently applied the place principle in its site assessments, it did not select any of the LXRP2 sites based on this principle alone. 

LXRA identified five sites as having a high need according to the place principle. Of these five sites, four had a delivery efficiency need. The other site—Mount Derrimut Road, Deer Park—also had very high movement and safety needs. 

Delivery efficiency

In our 2017 audit, we highlighted the absence of delivery efficiency in the criteria for LXRP1. We suggested that LXRA should consider the potential cost and time savings of removing sites located close together. 

By adding delivery efficiency as one of its four site selection principles, LXRA took a more strategic and future focused approach for LXRP2. It selected LXRP2 sites that minimised construction costs and rail service disruptions.

In our 2017 audit we identified five sites that LXRA should have considered for removal because they were adjacent to other level crossing removal sites: 

• three sites in Chelsea (Swanpool Avenue, Chelsea Road and Argyle Avenue)
• two sites in Glen Huntly (Neerim Road and Glen Huntly Road).

LXRA included all of these sites in LXRP2. 

As Figure 2G shows, there is close association between LXRP1 and LXRP2 sites.

LXRA also considered delivery efficiency by:

• selecting eight sites in LXRP2 based on delivery efficiency alone 
• ensuring that 80 per cent of LXRP2 sites met the delivery efficiency principle. Of these sites, 56 per cent are adjacent to other LXRP2 sites.
• selecting five sites that are located near current and future major transport projects, such as the Cranbourne Line Upgrade and Metro Tunnel Project. 

Figure 2H shows examples of how LXRA used the delivery efficiency principle to select sites.

FIGURE 2H: Examples of delivery efficiency outcomes for LXRP2 sites 

Source: VAGO, based on information from MTIA.

As Figure 2A shows, LXRA developed several options before submitting its final 25 site option to the government. The key difference between the site options it developed in March 2018 and its final list was determined by the delivery efficiency principle. Some sites that LXRA had initially considered for removal narrowly missed inclusion in LXRP2 because they did not have a high delivery efficiency need. However, none of these sites had a very high safety need. 

2.2 Business case and advice to government

The primary purpose of a business case is to provide the government with enough information to make an informed investment decision. Preparing a business case is one of DTF’s key requirements for all HVHR infrastructure projects. 

In our 2017 audit, we found that DEDJTR did not complete a business case until the LXRP had been underway for two years. Consequently, DEDJTR did not follow DTF’s HVHR framework to provide the government with a range of project options.

We also found that DEDJTR did not follow the HVHR framework to update the business case to reflect significant project changes. While DEDJTR added two level crossings to the project after completing the business case, it did not update it. 

Progress and outcomes

We found that DoT and MTIA only partially addressed our recommendation. DoT did not complete a new business case for LXRP2. Instead, DoT and MTIA approached LXRP2 as an extension of the original project. They considered a new business case unnecessary as the government had already made its decision to invest in an expanded project. 

While MTIA did not follow DTF’s ILC and HVHR guidelines for completing a business case, its advice to government for LXRP2 did include most of the information that HVHR projects require. This included a clear definition of the problem that the investment will address and estimated costs and delivery timelines. 

However, MTIA did not complete a cost-benefit analysis for the full 75-site project. Cost-benefit analysis is a valuable tool for government decision-making that quantifies the economic benefits of an investment decision. While the LXRP is on track to achieve its intended safety and congestion benefits at each individual site, a cost-benefit analysis would have enabled government to better understand the expected economic benefits of its almost $15 billion investment.

Advice to government for LXRP2 

A business case is the first step in DTF’s ILC and HVHR guidelines, which state that the agency delivering a project should:

• establish the need that the project aims to address
• define the project’s intended benefits
• explore project options
• estimate costs
• outline the delivery process.

As part of the 2019–20 state budget process, LXRA made a funding submission for LXRP2. While LXRA did not complete a business case for LXRP2, most of the key information it provided to the government aligned with the requirements of the business case stage of DTF’s ILC and HVHR guidelines, including the project’s:

• cost estimates and budget, including budget allocations, scope, milestones and estimates
• site prioritisation framework, including LXRP2’s overall benefits and alignment, prioritisation process and selection options 
• key considerations, including stakeholders, future needs, corridor completion, land use and network importance
• delivery process, including formal responsibilities, agreements and risk mitigation strategies. 

Gateway review process

By not completing a business case, MTIA missed the first three gates in DTF’s gateway review process, which provide early and independent scrutiny of a project’s deliverability. Instead, MTIA established an internal review process that broadly covered gates 1 to 3. This process included a: 

• detailed assessment of the project’s progress by internal MTIA staff and external consultants contracted by MTIA
• review by MTIA’s executive team and DTF representatives, which focused on the budget estimate for each works package
• final stage where the contracted program alliance presented detailed works package information and costings for MTIA’s executive review team and DTF representatives, to assess.

DTF’s gate 4 reviews are designed to confirm that a project’s procurement and tender evaluation stages are fulfilling its objectives, benefits plan and statutory requirements. 

MTIA has completed 11 gate 4 reviews that cover four works packages. The independent gateway reviewers engaged by DTF gave each package a ‘green’ project compliance rating. This means that the reviewer found no significant issues that threatened the cost, quality or timeliness of a work package’s delivery. 

2.3 Assessing options for removing level crossings

Our 2017 audit raised concerns that LXRA was not using a consistent approach to assess grade separation options for all 50 sites. 

As part of LXRP’s business case, which was finalised in 2017, LXRA developed a grade separation options assessment framework. However, prior to this, LXRA and VicRoads selected the first 20 sites without clear criteria. 

In its grade separation options assessment framework, LXRA included a multi criteria analysis tool to help it select the remaining 30 sites. However, we found that this framework did not weigh the selection criteria.

We also found that LXRA used a different approach for Frankston line sites, which considered the impact of removal options on adjacent properties. LXRA did not use this criterion to assess any grade separation options for LXRP2. 

Progress and outcomes

Since our 2017 audit, MTIA has transparently and consistently applied its grade separation options assessment framework for LXRP2 and the remaining LXRP1 sites. The framework uses a three-stage approach to select a grade separation option for each site: 

• Stage 1: MTIA conducts a preliminary assessment to identify an ‘indicative’ option to estimate a site’s removal costs.
• Stage 2: MTIA conducts a detailed site investigation, design assessment and consults with the local community.
• Stage 3: MTIA submits its preferred option to the Minister for Transport Infrastructure for final approval. 

In response to our 2017 audit, MTIA completed detailed site investigations to reassess grade separation options for the remaining LXRP1 sites. MTIA has not used the approach it took for LXRP1 sites on the Frankston line to select any other grade separation options. 

In late 2018, the government announced the 25 LXRP2 sites and specified an indicative grade separation option for each. 

MTIA advised us that over the course of the LXRP, it has developed its knowledge about what options are feasible for different sites. Consequently, it has changed fewer grade separation options after its Stage 1 assessments. This means that MTIA’s community consultations only focus on options that are achievable within engineering and budget constraints. 

Back to top

Conclusion

Our 2017 audit found that LXRA’s procurement approach may not achieve value for money because not all sites would be subject to full price competition. While LXRA had set up mechanisms to minimise costs, these were untested. 

MTIA has since changed its procurement approach. While this has further reduced price competition, MTIA is using its benchmarking tool effectively to manage costs. It is also incentivising the program alliances to share lessons learnt to achieve cost savings across the whole project. 

3.1 MTIA’s procurement approach for LXRP2

At the time of our 2017 audit, LXRA had planned to apply full price competition to 25 per cent of the 32 remaining LXRP1 sites and procure the rest through a deferred pricing contract structure.

We found that this could pose a risk to achieving value for money because not all sites would be subject to full price competition.

We also found that it could encourage program alliances to engage in loss-leading behaviours, such as bidding low in initial tendering to secure an advantage when bidding for the remaining sites. 

Progress and outcomes

Following the government’s announcement in 2018 that it would remove a further 25 level crossings, MTIA developed a new procurement strategy to integrate these sites into the existing project. 

MTIA’s new procurement strategy allocated all of the remaining LXRP1 sites and the LXRP2 sites to one of the five existing program alliances. This approach differed from LXRA’s original plan to leave some sites unallocated to drive competition among program alliances for the remaining sites. Appendix F shows the program alliance packages for LXRP1 and LXRP2.

Since 2017, the infrastructure construction market in Victoria has continued to experience unprecedented demand. This has shifted market dynamics and has encouraged contractors to seek higher profits, more favourable risk allocation and be more selective in the projects they bid for. 

However, the program alliances had already agreed to the key costs of their LXRP works packages, including corporate overheads, profit percentages and labour on costs for staff. By providing the program alliances with a stable pipeline of works, MTIA has secured better resource allocation and planning because contractors have their resources locked in over the short to medium term.

MTIA’s procurement approach also mitigates the risk of delays due to competition with other projects over access to the same section of rail track because MTIA and DoT can plan works well in advance.

Project assurance review

In August 2019, OPV completed a project assurance review on the LXRP. It found that MTIA’s procurement model was a ‛good basis’ for delivering the 75 level crossing removals by 2025. 

OPV acknowledged that MTIA’s packaging approach is less competitive than a traditional tender process. However, OPV found that MTIA is successfully using several mechanisms to achieve efficiencies. These mechanisms include a cost benchmarking tool, independent price estimator and a performance reward system that incentivises program alliances to collaborate. We discuss these mechanisms further in Section 3.3. 

3.2 Packaging sites for removal

The LXRP’s 2017 business case considered two options for packaging the then 30 remaining level crossings sites:

• a corridor approach, which grouped sites along the same rail corridors
• a discipline-based approach, which grouped sites that required similar types of works (such as modifying stations, power, signalling, and rail track).

LXRA determined that the corridor approach effectively balanced time pressures and the need to manage service disruptions. As a result, it packaged most of the remaining LXRP1 sites using this approach.

In our 2017 audit, we found that LXRA did not consider which sites had the highest priority for removal from a safety or rail efficiency perspective in its packaging approach. 

We recommended that LXRA evaluate its packaging approach as the project progressed to ensure that it could incorporate lessons learnt into its future packaging decisions.

Progress and outcomes

MTIA has continued to take a broadly corridor-based approach to packaging sites. However, this approach is more flexible and considers the benefits of grouping sites according to construction technique and proximity to other work sites. This approach is allowing MTIA to balance cost, timeliness and the expertise of different program alliances.

In February 2019, the government approved MTIA’s new site allocation framework, which outlines how it allocates works packages to the five program alliances. The framework builds on LXRA’s initial corridor-based approach and includes the following new criteria for packaging sites:

• construction technique, which involves allocating works based on the expected grade separation option
• capability, which involves allocating works to program alliances based on their past performance
• proximity, which involves allocating works based on whether a site is near another major transport infrastructure project site
• capacity, which involves allocating works based on workforce skills or availability.

MTIA did not formally review its packaging approach when it developed its site allocation framework. However, MTIA did incorporate lessons learnt from LXRP1, such as packaging sites to minimise disruption for commuters and reducing costs by grouping sites that use similar construction techniques.

As discussed in Chapter 2, delivery efficiency was one of MTIA’s core principles for selecting LXRP2 sites. This principle required MTIA to consider the cost and time savings of selecting sites that were close to other LXRP sites or major projects. 

This is an important shift since our 2017 audit and shows that MTIA is proactively incorporating lessons learnt from earlier removals into LXRP2. By packaging sites that are close enough to be removed during the same construction period, MTIA can minimise disruptions to public transport users and local residents. 

3.3 Benchmarking and minimising costs

Our 2017 audit found that while using program alliances with partial price competition saves time, it removes competitive tension. To address this risk, LXRA developed a number of mechanisms to minimise costs, including a benchmarking tool.

At the time of our 2017 audit, LXRA had only applied the benchmarking tool to its procurement for NEPA. 

Progress and outcomes

Since our 2017 audit, MTIA has fully embedded its benchmarking tool into its process for awarding additional works packages. The tool is working to ensure that packaged works are cost-efficient by requiring program alliances to submit a target outturn cost (TOC) that is broadly in line with the LXRP benchmarking tool. MTIA has also successfully implemented other mechanisms to manage costs, including the JCC. 

Awarding additional works packages

MTIA has allocated all of the remaining sites to program alliances. However, program alliances still need to develop a detailed project proposal and TOC before they are formally awarded an additional works package.

MTIA has developed a framework to assess value for money when awarding additional works packages. This framework includes:

• developing a benchmark cost estimate using its benchmarking tool
• engaging an independent estimator to examine the costs proposed by a program alliance
• assessing a program alliance’s performance using performance monitoring reports
• assessing non‐price elements, including a program alliance’s proposed approach to construction, delivery, continuous improvement, risk management and stakeholder management. 

Benchmarking tool

MTIA’s benchmarking tool is a database that includes cost information from previous level crossing removals, the Regional Rail Link Project and VicRoads. MTIA adds new data over time, which ensures that the tool can provide realistic construction cost estimates.

MTIA uses the benchmarking tool to develop a benchmark cost estimate for each additional works package. MTIA provides program alliances with a high-level price to assist them when developing their cost estimate.

MTIA incentivises program alliances to submit a TOC that is less than the benchmark by increasing their performance award if they are successful. 
In this way, MTIA’s benchmarking tool encourages program alliances to compete against MTIA when developing cost estimates for additional works packages, instead of competing against each other.

Independent estimator

MTIA also appoints an independent estimator to examine TOCs submitted by program alliances. 

The independent estimator uses their own benchmarking data to assess if a program alliance’s TOC is realistic and based on accurate assumptions. 

Using an independent estimator provides assurance that both MTIA and program alliances are using robust and comprehensive price information. 

Site reallocations

To date, MTIA has reallocated two additional works packages to a different program alliance due to cost and performance issues: 

These reallocations demonstrate that MTIA’s benchmarking tool is working as intended. They also indicate that MTIA is taking a proactive approach to monitoring value for money.

Sharing lessons learnt

While there is now less direct competition between program alliances, MTIA has successfully incentivised them to share lessons learnt and re-use ideas to achieve cost savings. 

Typically, contractors are in competition with each other and are keen to protect their intellectual property because it gives them a competitive advantage when bidding for work.

By tentatively allocating all of the remaining sites, MTIA has encouraged program alliances to work together and share information to help achieve cost efficiencies across the whole project. 

The JCC is one mechanism that program alliances can use to discuss their work and share ideas. The JCC includes 13 subcommittees, which are organised by discipline or topic, such as sustainability. The JCC has an online document sharing site that all program alliances can access to share information. 

MTIA modelled this approach on a similar framework used for the Regional Rail Link project. It has further strengthened this approach by offering program alliances incentives to collaborate.

In 2018, OPV found that the JCC was a key mechanism for encouraging a culture of sharing and continuous improvement across the LXRP.

At the time of our 2017 audit, LXRA’s performance reward regime included two indicators that focused on improvement—one for continuous improvement and another for innovation. In 2018, LXRP replaced these two indicators with:

• ‛adding value’, which combines the continuous improvement and innovation indicators and measures if program alliances share learnings about better practice 
• ‛adoption’, which rewards a program alliance when it uses a solution from another alliance, such as the same station design.

These indicators encourage program alliances to share ideas, rather than rewarding them for developing unique design solutions. By doing this, the LXRP has the potential to achieve cost savings through economies of scale because program alliances can adopt similar designs that require the same standard components and materials. 

Back to top

Conclusion

In 2017 we found that LXRA was using unclear KPIs to measure the benefits of individual level crossing removals. We also found that LXRA was not progressively monitoring whether the project was achieving its intended benefits.

MTIA has improved the amount and quality of data it uses to measure KPIs for individual sites. It has also introduced a process to annually monitor the project’s overall progress. As a result, MTIA now has a comprehensive understanding of how the project is progressing and whether it is on track to achieve its intended benefits.

4.1 KPIs for level crossing removal sites

Benefits management is a core part of effective project delivery. It involves setting defined objectives and KPIs at the start of a project and monitoring if these are achieved. 

Under the LXRP benefits management plan, MTIA uses 14 KPIs to measure if each individual site is achieving its intended benefits. However, our 2017 audit found that these KPIs did not set clearly defined benchmarks for improvement. 

Many of the KPIs were binary measures. This means that simply removing a level crossing would be considered an improvement. For example:

• 100 per cent of sites with road-based public transport will have improved punctuality (KPI 1.3b)
• 100 per cent of sites will have improved access to local activity centres and major services (KPI 2.3b).

In 2017, we recommended that MTIA’s benefit reports include a discussion of the results and the extent of any improvements, rather than simply stating if a KPI has been met. 

Progress and outcomes

MTIA advised us that following our 2017 audit, it considered if it could develop benchmarks for each KPI. However, it determined that it was not feasible to fully model data for every KPI at every site to create a specific benchmark. Instead, it decided to focus on providing more detailed information and commentary.

At the time of our 2017 audit, LXRA had four draft site benefits reports, but these included limited data. 

Since our 2017 audit, MTIA has increased the level of detail and analysis in its site benefits reports. 

As a result, the reports now provide more comprehensive information about the extent to which an individual level crossing removal has achieved its intended benefits.

At June 2020, MTIA has:

• completed 10 individual site benefits reports
• 19 reports in draft 
• a further five sites where drafting has not commenced. MTIA advised us that this is because traffic patterns have not returned to normal yet. Drivers avoid level crossings sites during construction, and it takes some time for them to return once the area reopens.

The 10 completed reports include the four reports that were in draft at the time of our 2017 audit. Each individual site report is approximately 35 pages and provides contextual information and an assessment of the extent to which each KPI has been met. For example, instead of just stating that removing a crossing as improved travel time for vehicles (KPI 1.1a), the reports specify the minutes saved.

This qualitative and contextual material is more meaningful than a binary report that simply states if the KPIs have been met. 

Appendix E lists the 15 KPIs that MTIA set for the LXRP benefits management plan.

KPI results

Of the 10 sites with completed benefits reports, MTIA has:

• achieved 70 per cent (105) of the KPIs
• not met 21 per cent (31) of the KPIs
• not finalised a further 9 per cent (14) of the KPIs.

Figure 4A shows these results.

The completed site benefit reports show improved travel times and an increased level of traffic flow. For example: 

• the number of vehicles travelling through the 10 sites has increased by 24 per cent on average (KPI 1.1b—reducing congestion and improving traffic flow)
• the average improvement in travel time for vehicles at the 10 sites is one minute during the morning and evening peak periods (KPI 1.1a)
• Blackburn Road, Blackburn, had the greatest improvement for vehicle throughput with an increase of 60 per cent for northbound peak hour traffic in the morning.

Of the unmet KPIs, 16 are due to a lack of pre-removal data for sites that VicRoads was initially responsible for. As the LXRP progresses, there will be fewer sites without suitable baseline data, and the quality of MTIA’s KPI reporting will therefore improve.

Other unmet KPIs are also due to data gaps. For example, MTIA cannot yet conclude against KPI 3.1a because it requires three years’ worth of data to assess if a level crossing removal has had any negative safety outcomes.

4.2 How MTIA monitors the project’s outcomes

In our 2017 audit, we found that there was insufficient data to make an informed judgement on how well the project was progressing towards achieving its intended outcomes.

At the time, LXRA advised us that when it completes all of the removals, it intends to combine the individual site benefit reports into a holistic project report. However, we found that waiting until the project is complete would mean that LXRA would have limited insight into how the LXRP is progressing towards realising its benefits.

Progress and outcomes

Following our 2017 audit, MTIA developed the LXRP Benefits Framework. This framework is designed to increase the transparency of the LXRP’s overall progress, outcomes, benefits and performance.

Project benefits reports

The LXRP Benefits Framework requires MTIA to annually report on the LXRP’s overall progress. Complementing the site benefits reports, these project benefits reports provide a progressive and timely assessment of whether the project is achieving its overall objectives.

The project benefits reports also aim to assess the wider economic, social and environmental benefits that the project has created or enabled, which are broader than the site specific KPIs. 

MTIA measures and reports on the project benefits across four domains, as outlined in the 2017 business case:

• Communities are more mobile and connected.
• Communities are more productive.
• Communities are more vibrant and safer.
• Communities are more sustainable.

MTIA measures the benefits of each domain at two levels:

• the project level, which aggregates information about all sites
• the corridor level, which describes the combined benefits of removing level crossings along a specific rail corridor. For example, the Cranbourne to Pakenham rail corridor.

MTIA has completed two project benefit reports—one at 30 June 2018 and one at 30 June 2019. 

The 2019 report includes data analysis and economic modelling undertaken by an external consultant. Based on data from 26 removed sites, the 2019 report found that the project is achieving:

• 7.6 per cent higher travel time savings for road and train users at removed level crossing sites than forecast
• 17.5 per cent higher reliability savings than forecast.

This means that travel time benefits across the whole transport network are 3.5 per cent higher than the original business plan forecast.

While the annual project benefits reports provide a useful point-in-time assessment, the results are uneven. This is because the individual sites do not contribute equally to the project’s overall benefits. The removal of some sites will have a greater impact on the network than others. 

For example, while the 2019 report found that overall travel time benefits are 3.5 per cent higher than the original business plan, this is a decrease compared to the benefits reported at 30 June 2018. At that time, LXRA had data for nine sites and observed that travel time benefits were 17 per cent higher than forecast in the business case.

The project benefits reports are not publicly available. However, MTIA does include some information from case studies on its website. Given that the business case for the first 50 sites was not completed until LXRA had started construction, providing public updates on the LXRP’s benefits would improve transparency around the project. 

Gate 5 and 6 reviews 

In addition to MTIA’s annual project and individual site benefits reports, DTF’s gateway reviews are providing independent assurance that the LXRP is achieving its intended benefits.

As discussed in Chapter 2, all HVHR projects must go through DTF’s gateway review process. In December 2018, MTIA received a joint gate 5 and 6 report for 10 LXRP1 sites. 

MTIA advised us that DTF combined the gate 5 and 6 reviews for some packages. This was because some packages included sites that had already been operational for two years by the time the review started, which limited the benefits of a standalone gate 5 review.

The 10 sites that DTF reviewed in 2018 were part of packages 1, 2 and 4, which VicRoads started and LXRA completed. LXRP packages are listed in Appendix F. 

The independent gateway reviewer engaged by DTF assessed the 10 sites’ overall delivery confidence as ‛good’. It also noted that LXRA had successfully managed a high volume of work, a challenging timeline and the current ‛heated’ construction market. 

While the gate 5 and 6 report was positive, it did not cover any sites that were fully delivered by LXRA/MTIA through the program alliance model. This means that MTIA is yet to fully compare the benefits of its contracting approaches. 

MTIA advised us that it has not set the next gate 5 and 6 review date with DTF. It estimates that this will take place in early 2021. This review will assess the benefits of MTIA’s new procurement approach compared to the earlier approach. 

It is important that DTF and MTIA continue to complete timely gate reviews so they can identify any issues that could impact the LXRP achieving its benefits as early as possible.

OPV's major projects performance reports

In addition to internal reports and gate reviews, MTIA and DoT provide information to OPV for inclusion in its quarterly major projects' performance reports. These reports to government highlight emerging risks and issues in major infrastructure and information technology projects, which includes the LXRP.  

In its June 2020 report, OPV noted that LXRP is experiencing cost pressures related to the escalating cost of labour and materials. However, OPV gave the project an overall 'amber' risk rating. This means that it considers that MTIA is managing the identified risks and no government action is needed at this time.  

As discussed in Chapter 3, MTIA is using several mechanisms to minimise costs.

Back to top

Conclusion

Our 2017 audit found significant weaknesses in PTV’s network integrity controls and the process it used to manage changes to engineering standards. 

While DoT has introduced new systems to govern standards and assure network integrity, 2019 machinery of government changes have made further work necessary. Until this work is complete, there is a risk that project delivery agencies and contractors may be unsure about the standards their projects need to meet.

DoT is also yet to develop system-level network requirements, which means it lacks a holistic view of the public transport network’s technical needs. DoT does not have a documented plan to complete this work in a timely way.

5.1 Network integrity controls

Strong network integrity controls support a transport system to work as a coherent whole. They also ensure that individual infrastructure projects fit seamlessly into the network. 

In our 2017 audit, we found that PTV did not have adequate resources to ensure the transport network’s integrity. We also noted that while PTV had established a new network integrity governance framework in November 2017, it was too early to tell if it would be effective.

Progress and outcomes

During 2018 and 2019, PTV and DoT progressively monitored their network integrity controls. They also made changes to improve how the controls ensure that major infrastructure projects integrate with the public transport network. 

While DoT introduced a new network and project governance framework in March 2020, work on this recommendation is still in progress:

Assurance process for infrastructure projects

In 2019, DoT introduced a network integrity assurance management procedure. This procedure outlines the assurance activities that DoT and infrastructure delivery agencies need to complete throughout a project’s life cycle to ensure it will integrate with the overall public transport network.

Under the assurance procedure, DoT conducts reviews throughout a project’s life cycle to confirm that the delivery agency is managing network risks. DoT also assesses how effectively the project will integrate with the existing public transport network. 

MTIA is following the assurance procedure for LXRP works packages and providing DoT with the evidence it needs to complete its assurance reports.

Network and project governance framework

In January 2019, machinery of government changes transferred the responsibility for the road network from VicRoads to DoT. In March 2020, DoT introduced a network and project governance framework to integrate road and rail network integrity controls.

While infrastructure delivery agencies’ project assurance responsibilities have stayed the same, the framework has replaced two key governance bodies with new committees.

DoT has established a change management process to ensure that infrastructure delivery agencies and contractors understand the new framework. DoT is updating its key policies, including its network integrity assurance management procedure, to align with the new framework. 

5.2 Network standards

As part of its network integrity and assurance role, DoT is responsible for setting network technical standards (NTS). These define the specific engineering and technical requirements that transport infrastructure and assets need to meet to ensure compatibility and consistency across the entire transport network. In addition to NTS, rail operators also develop their own engineering standards that relate to the parts of the network they are responsible for.

In 2017, we found that PTV did not have NTS. Instead, Victoria’s rail network relied on the Victorian Rail Industry Operators' Group standards, which PTV did not oversee. 

This meant that PTV did not have control over new standards or changes to existing standards. Consequently, it could not manage the risk that ad hoc changes could affect the network’s integrity. 

We also found a risk that changes to engineering standards could have time and cost impacts on in-progress projects that had been approved using earlier standards. The agencies delivering these projects would need to alter works or seek waivers from the new standards.

Progress and outcomes

Since 2017, PTV and DoT have endorsed 13 NTS, which cover all of the rail network’s major technical components. These include track, traction power, public transport precincts and operational control management systems. DoT and PTV are developing a further five NTS.

In addition, DoT has introduced a standards governance framework to oversee changes to NTS. However, DoT still has work to do to address the risks raised in our 2017 audit:

New standards governance framework

Before construction begins on an infrastructure project, the delivery agency outlines the project’s agreed outcomes. It also defines the minimum system requirements that contractors need to meet to ensure that the project seamlessly integrates with the transport network.

NTS and engineering standards change over time. If changes have significant safety impacts, then contractors may need to change their plans to meet the new requirements. 

As part of its franchise agreement with rail operator Metro Trains, PTV established a standards governance group in early 2018. This group ensures that Metro Train’s engineering standards align with the network’s overall technical standards. 

In 2019, DoT built on this governance group and introduced an expanded standards governance framework. The new standards governance framework:

• manages the development and maintenance of NTS
• manages the relationship between NTS, operator-developed engineering standards and related user policies and guides
• includes several DoT-run governance committees that:
  • endorse new or revised standards
  • ensure that any changes align with network requirements
  • determine if in-progress projects need to adopt a new or changed standard
• clearly defines the roles and responsibilities for DoT, project delivery agencies, rail operators and contractors.

MTIA advised us that while DoT's new standards governance framework is a significant improvement since our 2017 audit, some changes are still causing confusion for delivery agencies and contractors. 

One example of ongoing confusion is Metro Train’s recent engineering directive on axle counters. 

FIGURE 5A: Case study: Metro Trains’ directive on axle counters

Source: VAGO, based on information from DoT and MTIA.

With the introduction of DoT's network and project governance framework in March 2020, DoT has changed the names and responsibilities of the committees involved in standards governance. To avoid confusion, DoT needs to update its standards governance framework to reflect these changes. 

Network requirements

While DoT has developed NTS, it advised us that it is still developing higher level network requirements, including a Victorian Rail Plan that will provide the agreed architecture for the network's configuration and service levels.  Until that work is completed, DoT lacks a consolidated understanding of the public transport network’s technical needs to guide project development and future system-wide improvements. 

Back to top

We have consulted with DoT and MTIA and we considered their views when reaching our audit conclusions. As required by the Audit Act 1994, we gave a draft copy of this report, or relevant extracts, to those agencies and asked for their submissions and comments. 

Responsibility for the accuracy, fairness and balance of those comments rests solely with the agency head.

Response provided by the Secretary, DoT

Back to top

Back to top

Our methods

For this follow-up audit we assessed whether agencies: 

• made accurate attestations about implementing our 2017 recommendations 
• have taken timely action to address our recommendations
• have plans to address incomplete recommendations 
• are monitoring and reviewing the impact of their actions to implement our recommendations
• made transparent and appropriate decisions in cases where they decided not to implement recommendations
• have addressed the performance issues we identified, and made recommendations about, in our 2017 audit. 

We conducted our audit in accordance with the Audit Act 1994 and ASAE 3500 Performance Engagements. We complied with the independence and other relevant ethical requirements related to assurance engagements. We also provided a copy of the report to the Department of Premier and Cabinet.

Unless otherwise indicated, any persons named in this report are not the subject of adverse comment or opinion.

FIGURE C1: Our 2017 audit recommendations

Note: See Section 1.3 for changes to responsible agencies.
Source: VAGO.

Back to top

FIGURE D1: The first 50 level crossing removal sites (LXRP1)

Note: *Level crossing not included in original list. MTIA added these later for delivery efficiency with LXRP1 sites.
Source: VAGO, based on information from MTIA.

FIGURE D2: LXRP2 level crossing removal sites

Source: VAGO, based on information from MTIA.

Back to top

FIGURE E1: Site assessment framework—LXRA benefits and KPIs

Source: VAGO, based on information from MTIA.

Back to top

FIGURE F1: LXRP1 works packages

Note: Packages 1 to 4 preceded the program alliance framework.
Note: *sites allocated to NEPA rail.
Note: **sites allocated to NEPA road.
Source: VAGO, based on information from MTIA.

FIGURE F2: LXRP2 program alliances

Note: *Greens Road is now allocated to the Cranbourne Line Duplication's program alliance.
Note: **Station Street is now allocated to the Rail Infrastructure Alliance program alliance.
Source: VAGO, based on information from MTIA.

Back to top