Tendering of Metropolitan Bus Contracts

Conclusions

The state has not yet secured full value for money from the MMBF despite achieving almost $33 million in cost savings.

While the project to establish the MMBF contract was based on sound planning and was run in accordance with Victorian Government Purchasing Board (VGPB) requirements, the contract's potential is not currently being realised due to PTV's failure to resolve longstanding issues impacting the implementation of the related performance regime.

PTV needs to urgently address the lack of reliable performance data, as it compromises its capacity to effectively manage the contract, and achieve the state's broader reform objectives.

The Department of Economic Development, Jobs, Transport and Resources (DEDJTR) and PTV's slow progress in preparing to reform the 27 private metropolitan bus contracts which are expiring in 2018 is of particular concern. Despite the scale and complexity of this exercise, and the significant opportunity cost to the state from failure, substantive planning to address these issues has only recently commenced.

Urgent and sustained action is required, to avoid limiting the state's procurement options in 2018, and compromising any opportunity to leverage better value from these services through increased competition.

Findings

Performance of the MMBF

Although the MMBF has been operating for over 18 months, it cannot yet be determined whether it is achieving all of its objectives and full value for money for the state.

MMBF expenditure for 2013–14 was almost $33 million, or 18 per cent, less than the amount PTV estimated it would have spent if the previous contracts had continued.

However, MMBF cost savings alone are not sufficient to demonstrate that value for money has been achieved as this relies on the contract also delivering improved services to more customers.

The MMBF contract establishes an incentive and penalty regime to drive achievement of the operator's service improvement obligations. Effective performance monitoring by PTV is therefore critical to ensure these obligations are met, and to realise the contract's full potential.

While the MMBF performance regime is conceptually sound, PTV has yet to effectively implement it. In particular, PTV has failed to:

• reach a timely agreement with the operator on the standard for determining incentive payments for improvements in bus patronage
• resolve longstanding data reliability issues impacting the operation of the performance regime by April 2015 when it was due to be fully implemented
• withhold payments for instances of non-performance to encourage timely corrective action.

These circumstances are compromising achievement of the state's reform agenda, including PTV's capacity to reliably assess and manage the contractor's performance.

PTV recently advised that full implementation of the performance regime has been delayed.

PTV recognises that overcoming this issue is critical because it will otherwise not be able to implement the financial incentives and penalties that are due to begin after a total rebuild of the current bus timetable—known as the Greenfields timetable.

It has therefore proposed an alternative method for calculating service reliability and punctuality to the contractor, but this had not yet been implemented at the time of audit. However, we found that PTV's proposal is deficient as it relies heavily on self-reporting by the operator to determine the reliability of bus services. PTV did not have a plan or procedures in place to audit and verify the accuracy of this information.

Urgent action is required by PTV to address this issue as it risks compromising the integrity of the performance regime and of related incentive payments.

Reforming remaining bus contracts

Progressing broader reform of the remaining 27 bus service contracts will be a complex and challenging exercise for the state. These contracts do not contain any clear end-of-term rights for the state to run a competitive tender process for these services. The state's right to do so is a longstanding issue that is contested by some members of the industry. Any change in the delivery arrangements for these services will require extensive planning and lead time to assure the availability and transition of any critical assets and infrastructure.

Our 2009 audit, Melbourne's New Bus Contracts, recommended that the former Department of Transport provide early advice to government on the strategic options and constraints for future metropolitan bus contracts. While the department and PTV implemented this recommendation for the MMBF, it has not been addressed for the next tranche of bus contracts.

In July 2014, the former Department of Transport, Planning and Local Infrastructure and PTV established a joint working group that commenced initial planning for the next round of bus contract procurements. The working group commissioned initial research on bus structures in other jurisdictions and potential reform options.

An interdepartmental steering committee comprising PTV, DEDJTR, the Department of Treasury and Finance and the Department of Premier and Cabinet was also established in early 2015 to inform development of a procurement strategy for the government's consideration by August 2015.

However, progress on this initiative has been slow. Consequently, the risk is now high that a project of this significance, scale and complexity may not be effectively planned and delivered in the time remaining.

In turn, this also risks limiting the state's options in 2018. If delays mean insufficient time is left to pursue an open tender, the state may find that renegotiating with incumbent operators is the only option. This will likely compromise any immediate opportunity for the state to achieve improved value from these contracts.

Tendering the MMBF

The MMBF competitive tender process was guided by appropriate procurement principles and probity requirements, and resulted in a contract that is capable of delivering improved services and value to the state.

PTV's strategic planning for the MMBF procurement was clearly focused on producing a value for money outcome, but did not fully comply with required VGPB policies and guidance. In particular, PTV did not develop a business case or final strategic procurement plan for the MMBF. This created a risk that state decisions about the procurement were not soundly based, although our assessment of supporting documentation indicates that key costs, benefits, risks and options were adequately identified.

During tender assessment, PTV introduced the targeted improvement process—an additional assessment step that was not contemplated in the evaluation plan. Despite this, the process was managed in accordance with VGPB requirements, and assisted with optimising value from the bids and producing a value for money outcome for the state.

Managing the MMBF agreement

PTV's contract management policies and practices for the MMBF are inadequate and are further compromised by the above-noted deficiencies in performance monitoring. PTV does not have a complete contract management plan in place for the MMBF agreement. While routine tasks and processes are documented in a Contract Administration Manual and Process Administration Guides, they do not clarify where responsibility lies for specific tasks and key decisions—including for resolving complex or contentious issues, the approach for conducting regular contract reviews, and the related information management requirements.

This situation, combined with PTV's willingness to negotiate extensions to some contract deadlines rather than withhold payments or invoke penalties, risks reducing the value for money delivered by the contract and undermining achievement of the state's related reform objectives.

PTV advised that it has only waived the withholding of payments for some delayed initiatives which were offered by the contractor and formalised in the contract, but which have no impact on core services to customers. However, a key goal of introducing the performance-based MMBF contract was to create incentives to improve performance. While deadlines may need to be extended in some circumstances, the contract enables PTV to reasonably reduce payments in response, but it has never done so. This is inconsistent with the goals of the MMBF contract and the related reform agenda.

Additionally, PTV has not assessed the value for money implications of the contract variations that have so far been approved. This risks undermining MMBF's value, as it means there is little assurance that implemented variations support achievement of MMBF's objectives.

Recommendations

That Public Transport Victoria:

1. promptly rectifies all barriers to implementing the Melbourne Metropolitan Bus Franchise performance regime, including data reliability issues
2. documents and assesses the rationale for all decisions to waive the withholding of payments for non-performance
3. closely monitors the delivery of key offer commitments and proactively addresses any slippage from contractual time lines
4. systematically audits and verifies the reliability of performance data provided by the operator underpinning incentive payments.

That Public Transport Victoria and the Department of Economic Development, Jobs, Transport and Resources:

5. advise government on all key risks, options and required actions for reforming the metropolitan bus contracts expiring in 2018
6. develop a business case assessing the cost and benefits of alternative reform options and related strategies for all future bus service procurements in accordance with Victorian Government Purchasing Board guidance.

That Public Transport Victoria:

7. separates the role of probity auditor and advisor on future procurements.

That Public Transport Victoria and the Department of Treasury and Finance:

8. use the lessons from the Melbourne Metropolitan Bus Franchise's targeted improvement process to develop guidance on how it may be applied more broadly in future procurements.

That Public Transport Victoria:

9. ensures that contract management processes are in place prior to commencement for any new service contracts
10. establishes a contract management plan for the Melbourne Metropolitan Bus Franchise in accordance with Victorian Government Purchasing Board guidance
11. establishes knowledge management and document control processes to ensure critical information and reasons for decisions are readily available to the necessary staff
12. establishes a process for systematically monitoring and verifying the completion of required Melbourne Metropolitan Bus Franchise contract management tasks
13. consistently enforces the terms of the Melbourne Metropolitan Bus Franchise agreement in accordance with the contract.

Submissions and comments received

We have professionally engaged with Public Transport Victoria, the Department of Economic Development, Jobs, Transport and Resources and the Department of Treasury and Finance throughout the course of the audit. In accordance with section 16(3) of the Audit Act 1994 we provided a copy of this report to those agencies and requested their submissions or comments.

We have considered those views in reaching our audit conclusions and have represented them to the extent relevant and warranted. Their full section 16(3) submissions and comments are included in Appendix A.

1.1 Introduction

1.1.1 The role of buses in Melbourne

Buses are an important part of Melbourne's public transport system providing local and cross-town travel and connections to rail and tram networks. They also provide an alternative to travelling by car. Buses are often the only readily accessible form of public transport available for people living in Melbourne's middle and outer suburbs. As Figure 1A shows, around 82 per cent of Melbourne dwellings are within 400 metres of a bus route compared to around 20 and 30 per cent for trams and trains respectively.

Figure 1A
Melbourne dwellings within 400 metres of public transport, 2010

Source: Victorian Auditor-General's Office from Public Transport Victoria's, Network Development Plan—Metropolitan Rail, December 2012.

In the 10 years since 2004–05, metropolitan bus patronage has been steadily increasing overall, as Figure 1B illustrates.

Figure 1B
Total metropolitan bus boardings by year

Source: Victorian Auditor-General's Office from information provided by Public Transport Victoria.

1.1.2 Overview of Victoria's bus industry

Melbourne's bus network is extensive—it consists of more than 300 routes covering over 5 500 kilometres, and is serviced by approximately 1 700 buses. The arrangements underpinning the delivery of these services are set out in:

• 27 private contracts with 11 operators established in 2008, covering around 70 per cent of bus services
• a contract established in 2013 with a single operator, covering roughly 30 per cent of services—this contract is the Melbourne Metropolitan Bus Franchise (MMBF) agreement.

In 2013–14, bus services were the second largest transport expenditure for the state. As Figure 1C shows, 26 per cent, or around $0.6 billion, of the total paid to transport service providers was for operational payments to metropolitan bus service providers.

Figure 1C
Payments to transport service providers

Source: Victorian Auditor-General's Office from information provided by Public Transport Victoria.

1.1.3 Evolution of Victoria's bus industry

Private operators

From 1869 to 1974, bus services were provided by private, self-funded companies which developed bus routes, determined their own schedules and fares, and bought bus depots and vehicles. In 1974, the government introduced regulated fares across the whole metropolitan public transport system and began subsidising bus operators. This marked the start of contractual relationships between government and bus operators.

Despite bus operators becoming increasingly financially dependent on state subsidies throughout the 1980s and 1990s, the bus industry retained the view that its history of private investment entitled it to exclusive rights to operate the routes they had developed. Accordingly, the industry disputes the government's right to competitively tender services. However, our 2009 report Melbourne's New Bus Contracts noted that the state does not accept this position. The issue has never been legally tested and remains a potential risk and barrier to reform.

The bus industry has not been exposed to the same open market competition as Melbourne's other public transport modes. Instead, contracts have historically been renewed by negotiating with incumbent operators only. Throughout the 1990s, successive reports by the then Industry Commission, the Victorian Commission of Audit and VAGO noted that these negotiated contracts lacked incentives for operators to minimise costs and improve services.

State established routes

In the early 1990s the state sought to increase bus services while setting conditions for future bus industry reform.

The state established and privatised two companies—the National Bus Company (NBC) and Melbourne Bus Link (MBL)—to run new bus services independent of the private operators. As the private operators could not claim the same historic ownership rights over the routes they operated, the state had more control over the related procurement and contracting decisions.

Further additional routes—the SmartBus orbital routes—were established by the state in 2006 under an interim contract with the former Department of Transport, with the intention of offering the routes to tender in December 2012.

2008 bus contracts

On 1 July 2008, 29 metropolitan bus contracts came into effect. These were established through negotiation with the incumbent operators and fell into the following two categories:

• Two contracts covering around 30 per cent of services were let to NBC and MBL for a five-year term, from 1 July 2008 to 30 June 2013. These were interim arrangements, allowing the state time to run an open tender to establish successor operators. The services covered by these contracts are now part of the MMBF.
• 27 contracts for the remaining 70 per cent of services were let to existing private operators. These were established for a term of seven years, from 1 July 2008 to 30 June 2015, with an embedded right to extend a further three years to 2018.

The 27 contracts for the remaining 70 per cent of services are traditional fee‑for‑service style arrangements, with fixed routes and limited incentives or penalties available to encourage performance improvements or innovation. This has not changed significantly since the late 1980s.

1.1.4 Previous audits of Melbourne's bus services

Previous VAGO audits in 2009, 2012, 2013 and 2014 have highlighted ongoing challenges with the delivery and performance of Melbourne's bus services.

Melbourne's New Bus Contracts, 2009

This audit focused on the 27 private contracts, and found that while there were likely to be significant gains from the new contracts, the state had not fully achieved its procurement objectives.

The key areas identified where the new contract regime failed to deliver on the procurement objectives include:

• an absence of competitive tendering
• no right to tender existing local services and no agreement to tender at the end of contract
• limited access to strategic assets for government
• no rights over existing local bus services assets
• no access to operators' financial data and open book access for most operators.

The audit also found the operational performance regime for bus services needed to be strengthened to provide greater assurance about the on-time running of buses.

Public Transport Performance, 2012

This audit examined the performance of public bus, tram and train services across Victoria and found that the former Department of Transport was not prepared for the rapid growth in public transport patronage between 2004 and 2009.

The audit also found that information for measuring bus performance fell short of expectations for sufficiency and reliability. While the department introduced a rolling program of audits of contractual compliance for metropolitan buses, it had not implemented VAGO's 2009 recommendation to use its contractual rights to examine and verify operators' records of on-time running.

Developing Transport Infrastructure and Services for Population Growth Areas, 2013

This audit assessed the effectiveness of state agencies in planning and delivering transport infrastructure and services for population growth areas. It examined whether planning effectively identified current and future transport needs, and whether implementation and funding strategies supported the timely delivery of required transport infrastructure and services.

The audit found that residents in growth areas generally have less frequent and direct bus services compared to those in metropolitan Melbourne. It also noted that significant investment of more than $10 billion is required to address the current infrastructure and service backlog, and additional recurrent funding of $197 million per year is needed to improve bus services across metropolitan Melbourne.

Coordinating Public Transport, 2014

This audit examined how well Public Transport Victoria (PTV) is managing the coordination of trams, trains and buses, and specifically how its activities and existing contractual arrangements support the achievement of seamless travel within and between different modes.

The audit found that many of Melbourne's bus routes currently have long wait times, indirect routes and do not operate on schedules designed to harmonise well with the rail network or other bus routes. This reduces their usefulness for commuters and impedes patronage growth.

The audit also highlighted that the limitations with the tracking technology used on most buses mean that PTV is highly reliant on self-reporting by operators for determining penalties and other incentive payments. The absence of robust performance information and heavy reliance on self-reporting by bus operators means it is not possible to reliably use financial incentives to improve bus services.

1.2 Melbourne Metropolitan Bus Franchise

The MMBF consists of:

• 41 routes servicing the eastern suburbs and Melbourne CBD, plus approximately 107 school special services
• nine routes servicing the western suburbs, south eastern suburbs and the Melbourne CBD, plus approximately 15 school special services
• three SmartBus orbital services, which are premium cross-town bus services linking radial train lines and key activity areas in Melbourne's middle and outer suburbs.

1.2.1 Key features and benefits

A key feature of the MMBF is the performance-based incentives included in the franchise agreement, similar to those used for tram and train services. The MMBF contract incorporates financial incentives and penalties linked to key performance targets, designed to improve services and patronage performance.

The expected benefits of the MMBF agreement include:

• improved public transport coordination through timetable changes
• timetable and route changes to better meet customer needs
• enabling the MMBF network to evolve with community demands
• improved communication with customers.

These benefits were expected to be realised in two stages:

• Between the commencement of the contract in August 2013 and April 2015, the new operator was expected to develop an understanding of the MMBF business and customers, and to develop changes intended to improve the business model and customer experience.
• From April 2015 more significant changes are expected when the operator's rebuilt timetable takes effect. This is expected to result in improvements in punctuality, patronage, and coordination with other public transport modes.

1.2.2 Reform agenda

In approving the open tender for the MMBF in September 2011, the then government also approved a reform agenda and determined that it would consider MMBF as having achieved value if it:

• reduced costs for the state
• delivered improved services for more customers
• progressed reform of the bus industry.

Figure 1D summarises the main priorities of the reform agenda.

Figure 1D
Key reform agenda priorities

Source: Victorian Auditor-General's Office.

These reform priorities were reflected in the MMBF agreement, making it fundamentally different to any other contracting arrangement within Melbourne's bus network. It contains a suite of incentives and penalties designed around performance targets—including reliability, punctuality, patronage growth and customer satisfaction. It also gives the operator flexibility to design and propose route changes in collaboration with PTV, and requires the operator to review all routes and do a total rebuild of the current bus timetable—known as a Greenfields timetable—in 2015.

These reforms are intended to establish an environment for wider reforms when the contracts for the remaining 70 per cent of services expire in 2018.

1.2.3 Objectives and outcome of the tender

The MMBF agreement was awarded through a competitive open tender process, referred to as the Metropolitan Bus Services Project (MBSP). This process was expected to reduce the cost to the state and increase competition in Melbourne's bus market.

The MBSP had the following objectives:

• to encourage sustainable competition in the provision of services
• to establish new contracts that:
  • provide safe, integrated, accessible and reliable services
  • support the efficient operation of passenger transport services
  • promote innovation in the delivery of services
  • provide value for money.

The open tender process for the MMBF was approved in September 2011 and on 26 April 2013 the government announced the chosen operator.

The MMBF agreement became operational in August 2013. It is worth approximately $1.718 billion in nominal terms over 10 years and comprises 50 bus routes, 100 school special routes, 504 buses and around 1 000 drivers operating from seven depots across Melbourne.

1.3 Roles and responsibilities

Public Transport Victoria

PTV became operational in April 2012. Its primary objective is to plan, coordinate, provide, operate and maintain a safe, punctual, reliable and clean public transport system. PTV's core functions include managing ongoing network improvements, planning for future public transport needs and ensuring public funds are spent prudently and efficiently.

PTV oversees public transport operators who are responsible for the day-to-day operation of services and, subject to PTV's specifications, for improving scheduling to support better connectivity between transport modes.

PTV is responsible for planning, tendering, negotiating and managing all bus service contracts. While PTV was responsible for managing the MMBF procurement process, initial planning occurred prior to its creation, and was the former Department of Transport's responsibility.

The Department of Economic Development, Jobs, Transport and Resources

The MBSP was initiated by the former Department of Transport in April 2009, now incorporated within the Transport Division of the Department of Economic Development, Job, Transport and Resources (DEDJTR). However, when PTV became operational in April 2012, most of the staff in the public transport division transferred to PTV along with responsibility for MBSP.

DEDJTR is currently part of a bus services reform joint working group with PTV. This group is responsible for formulating plans on the future of bus services, contractual and legislative issues, options for dealing with expiring contracts, and any other elements of reform.

The Department of Treasury and Finance

The MMBF was designated a complex procurement and was therefore subject to additional oversight from the Department of Treasury and Finance (DTF) and the Treasurer under the High Value High Risk framework. DTF worked with PTV in trying to achieve value for money procurement outcomes at both the working group and steering committee level.

1.4 Audit objectives and scope

This audit examined whether the state has effectively secured value for money from the new MMBF arrangements by assessing whether:

• the project to establish the MMBF was based on sound planning and was conducted in accordance with purchasing, probity and ethical conduct requirements and guidelines
• management of the franchise agreement is delivering value for money and improved performance.

The audit also examined whether appropriate planning to optimise value for money in future metropolitan bus contracts is underway, including progress on implementing the lessons learned from the MMBF procurement.

The audit includes PTV, DEDJTR and DTF due to their involvement in the MMBF procurement process and future bus contract planning.

1.5 Audit method and cost

The audit involved:

• desktop research
• consultation with agencies and stakeholders
• qualitative evaluation against best practice principles.

The audit was performed in accordance with the Australian Auditing and Assurance Standards. Pursuant to section 20(3) of the Audit Act 1994, unless otherwise indicated, any persons named in this report are not the subject of adverse comment or opinion.

The total cost of the audit was $475 000.

1.6 Structure of the report

The report is structured in the following parts:

• Part 2 discusses the performance of the MMBF
• Part 3 discusses the tendering process
• Part 4 discusses the adequacy of contract management of the agreement.

2.1 Introduction

The state used an open tender procurement method to establish the new Melbourne Metropolitan Bus Franchise (MMBF) because it expected that this would result in a better service to customers at a reduced cost to the state.

It also sought to use the MMBF to test the savings achievable through increased competition and the potential benefits of an enhanced performance-based contract as a precursor to extending these reforms to the rest of Melbourne's metropolitan bus industry.

The state expected the MMBF would achieve value if it:

• reduced costs for the state
• delivered improved services for more customers
• progressed reform of the bus industry.

This Part of the report examines whether the MMBF is on track to achieve improved value for money for the state.

2.2 Conclusion

Although the MMBF has been operating for over 18 months, it cannot yet be determined with any certainty whether it is achieving full value for money for the state.

The MMBF has proven less expensive than previous arrangements by reducing the cost to the state in 2013–14 by almost $33 million. However, as it also sought to improve services for more customers and progress reform of the bus industry, this metric alone is not sufficient for demonstrating value for money.

Public Transport Victoria (PTV) failed to establish reliable performance data by April 2015 when the performance regime was due to be fully implemented. This means that the performance regime could not be effectively applied as originally intended to manage the contractor's performance. While full implementation of the regime has since been delayed pending approval of the Greenfields timetable refresh, PTV needs to urgently address these issues as they are compromising its capacity to reliably manage the contractor's performance, including achievement of the state's reform agenda.

PTV's rationale for waiving the withholding of some operator payments for delays in delivering required service improvements is also at odds with the MMBF's reform goals, and is a missed opportunity to enforce the principles of the contract and leverage better value for the state.

Substantive action is still required to establish the preferred procurement method for the remaining metropolitan bus contracts that expire in 2018. Consequently, there is a significant risk that the state will miss the opportunity to significantly progress reform of the bus industry and achieve better value for money.

2.3 Reduced costs for the state

The actual cost of MMBF services is less than under the previous procurement arrangements, and is in accordance with the tender price and advice provided to the state by PTV following tender evaluation.

As shown in Figure 2A, MMBF expenditure for 2013–14 was almost $33 million, or 18 per cent, less than the amount PTV estimated it would have spent if the previous contracts had continued.

Figure 2A

2013–14 actual MMBF expenditure compared to tender estimate and previous contracts

Source: Victorian Auditor-General's Office based on PTV financial reports.

The MMBF cost savings have been consistent with PTV's estimates, and are expected to total approximately $380 million over the 10-year life of the agreement. These savings are attributable to efficiencies achieved by merging five contracts into one, as well as the new operator's proposed innovations and willingness to accept a greater share of the risk for poor service delivery.

However, MMBF cost savings alone are not sufficient for assessing its achievement of value for money. As noted in Part 1 of this report, the state determined in 2011 that MMBF will have achieved value for money if it:

• reduced costs for the state
• delivered improved services for more customers
• progressed reform of the bus industry.

2.4 Improved services for more passengers

A key goal of the state's reform agenda in establishing the MMBF incentive-based contract was to provide the state with the means to continuously improve and grow bus services through clearly defined performance standards and monitoring arrangements.

In this context, the MMBF contract established key obligations for the operator to:

• continuously improve bus punctuality, reliability, safety and customer services—including the provision of information to passengers
• increase bus passenger numbers
• identify opportunities to improve existing bus services—including improving connections between buses and other public transport modes.

The MMBF contract contains an incentive and penalty regime to support the achievement of these obligations. Effective monitoring by PTV of the operator's performance, including enforcing penalties when necessary, is therefore critical for assuring these obligations are met, and for realising the contract's potential to deliver better services.

Although the performance mechanisms are soundly based, they are impacted by longstanding data reliability issues, and delays in achieving full implementation. These issues risk compromising achievement of value for money from the contract, and PTV's capacity to effectively manage the contractor.

PTV failed to resolve longstanding data reliability issues by April 2015, when they were originally due to be fully implemented under the contract. It also failed to reach a timely agreement with the operator on the standard for determining incentive payments for improvements in bus patronage—in effect delaying the operation of this regime by 12 months.

PTV advised towards the end of this audit that implementation of the performance regime and related Greenfields timetable has since been delayed, and that it has begun work to reach agreement with the operator on a proposal to address these data issues.

It will be critical for PTV to effectively resolve these issues as they are compromising achievement of the state's reform agenda—including PTV's capacity to reliably assess and manage the contractor's performance.

2.4.1 Performance regime

The MMBF performance regime comprises the Patronage Incentive Regime (PIR), Operational Performance Regime (OPR) and Reliability Regime (RR), and uses financial incentives and penalties attached to service targets.

The regime is intended to encourage the operator to improve performance and increase passenger numbers. The performance regime supports achievement of the state's reform agenda by:

• providing financial incentives for improved performance
• encouraging innovative ways to improve services
• shifting some of the financial risk for poor service to the operator.

However, PTV failed to resolve longstanding data reliability issues by April 2015, which meant that the OPR was not able to be implemented as planned. It also failed to reach a timely agreement with the operator on the standard for determining incentive payments for improvements in bus patronage—in effect delaying the operation of this regime by 12 months.

PTV recently advised that full implementation of the OPR has since been delayed as the state has yet to approve the Greenfields timetable refresh proposed by the contractor.

As a result, a key aspect of the MMBF agreement remains inoperative, which compromises the achievement of the reform objectives—including value for money.

Patronage Incentive Regime

The PIR is intended to provide the operator with a financial incentive to introduce service improvements and other innovations that lead to increased passenger numbers above a benchmark agreed with PTV. Conversely, poor services leading to a decrease in passengers results in a financial penalty.

The MMBF contract and PIR measure patronage by myki touch‑ons. However, due to concerns with the reliability of myki data, and protracted negotiations with the operator over setting the PIR benchmark, it was not operational until February 2015—more than 12 months after it was due to be implemented under the contract. This issue is examined further in Part 4 of this report.

PTV resolved the myki data issue in July 2014. However, agreement with the operator was not reached until January 2015. The failure to reach a timely agreement on the passenger benchmark is unsatisfactory. While it is clear that passenger numbers have been increasing, the absence of an agreed benchmark meant PTV could not be certain that the operator was meeting its expectations during this period. Similarly, the operator did not necessarily know if more work was needed to further increase passenger numbers.

Operational Performance Regime

An MMBF service is considered punctual if it is observed at the specified service monitoring location, and had arrived there no more than 4:59 minutes late and departed no more than 59 seconds early.

PTV set an interim OPR punctuality benchmark of 70 per cent for all services at the start of MMBF operations, based on its estimate of the previous operators' performance. It proposes to set the permanent benchmark at 90 per cent when the new Greenfields timetable commences—at which point the OPR's financial penalties will start to be applied. The OPR penalises the operator by up to $2 million annually if it fails to meet its punctuality benchmark. It also provides for PTV to terminate the contract if punctuality drops below 75 per cent for a year without due cause. However, reliable bus punctuality data is not yet available—impeding the operation of the OPR.

The contract initially required the Greenfields timetable to be implemented by April 2015, but this has been delayed. As noted earlier, PTV is currently working with the contractor to resolve reliability issues with the data.

Reliability Regime

Reliability is the number of service kilometres scheduled compared to the number of service kilometres actually travelled, expressed as a percentage. The RR came into force at the commencement of the contract.

If reliability falls below 95 per cent for a quarter, or below 99 per cent for three quarters within a 24-month period, PTV can immediately terminate the contract. Reliability below 98 per cent in a quarter, or below 99 per cent for three quarters in 24 months, triggers a noncompliance event, compelling the operator to commit to remedial action.

However, data concerning bus reliability has been problematic.

Absence of reliable performance data

The effective operation of the performance regime depends on the availability of accurate and reliable data.

The MMBF agreement's OPR and RR were designed to use data gathered by PTV's bus tracking system (BTS). While PTV knew there were issues with the reliability of the BTS in June 2011 when it was developing the OPR and RR, it assumed that these issues would be overcome by March 2012, prior to the OPR and RR becoming operational. However, this did not occur and PTV's progress in addressing this issue has been unsatisfactory.

Our 2014 audit Coordinating Public Transport noted that PTV advised that it expected the new BTS to be fully operational by July 2014, however, this has yet to occur.

The BTS data issues stem from the number of services being monitored. The OPR and RR require 100 per cent of MMBF services to be monitored by the BTS. However, PTV now recognises that no technology solution is capable of 100 per cent functionality as there will always be periods, however brief, where hardware breaks down or software fails.

PTV has improved the monitoring of services throughout the MMBF contract from 89 per cent in 2014, to the current level of 93 per cent. While this is encouraging, the unmonitored services affect the bus reliability calculation required under the contract as these services are recorded as missed service kilometres, even if the service ran as scheduled. Punctuality is similarly affected as a bus is considered not punctual if it is not observed at a contract monitoring point, even if this is due to a BTS malfunction.

PTV documents supplied in March 2015 acknowledge that while this improvement in coverage better supports operational and service planning activities, it is not sufficient for assessing the punctuality of services as required by the contract.

PTV has proposed an alternative method to calculate service reliability and punctuality to the contractor, but this had not yet been implemented at the time of audit. PTV recognises that overcoming this issue is critical because it will otherwise not be able to implement the financial incentives and penalties that are due to begin after the Greenfields timetable refresh. PTV could not demonstrate that it was on track to address this issue by April 2015 when the Greenfields timetable refresh was initially due to occur.

Furthermore, we found that PTV's proposed alternative method for calculating reliability and punctuality is currently deficient as it relies heavily on self-reporting by the operator for determining reliability. PTV did not have a plan or procedures in place to audit and verify the accuracy of this information.

Urgent action is required by PTV to address this issue as it risks compromising the integrity of the performance regime and of related incentive payments.

PTV is also aware of a reliability issue with its myki touch-on data. At times myki machines can temporarily stop functioning—preventing bus passengers from touching on. While the system flags when a myki machine stops operating and for how long, PTV has no practical way of assessing the number of passenger boardings that are not recorded. However, PTV believes the completeness and accuracy of touch-on data is not materially affected.

2.4.2 Offer commitments

The MMBF requires the operator to deliver 21 service improvement projects—described as offer commitments—by time lines the operator committed to in its tender response. If the operator fails to deliver a project on time PTV can withhold payments until a satisfactory remedy has been implemented or remedial plans have been agreed to.

PTV has not exercised its contractual powers in relation to the operator's failure to deliver some projects as committed. This risks diminishing the effectiveness of the offer commitments performance incentive mechanism.

Projects not completed by due date

Four offer commitment projects have not been completed by the time frames originally agreed by the operator. Three projects had their due dates extended, as the operator was not able to complete these projects on time. Had PTV not chosen to extend the deadlines, it would have been able to withhold more than $200 000 of contract payments and require the operator to remedy the underperformance.

Additionally, PTV did not take any action for the operator's failure to deliver on its bus refurbishment offer commitment. At the end of the first contract year, PTV could have withheld up to $180 000 for buses that had not been refurbished.

PTV advised that it has only agreed to extend some contract time lines for initiatives offered by the contractor and now formalised in the contract, but which have no impact on core services to customers. PTV also wanted to build a business relationship with the operator.

While PTV has the discretion to withhold payments in response to these circumstances, it has never done so. This approach does not provide appropriate incentives for the operator to comply with its contractual obligations, and risks compromising achievement of the state's reform goals.

Delivery of key improvement projects

Three key offer commitment projects require the operator to reduce fare evasion, develop and implement a new bus timetable known as Greenfields, and increase customer satisfaction. As these projects directly relate to the reform agenda, they are important for assessing whether the state is getting better value for money from the MMBF.

Fare evasion

The fare evasion project requires the operator to reduce the fare evasion rate on its services to 5 per cent by 1 August 2015 and then to 2.3 per cent by 30 June 2021. By May 2014, the fare evasion rate had risen to 13.1 per cent from 10.1 per cent in October 2013. In October 2014 the rate was 8 per cent indicating that reaching the target rate will be challenging for the operator.

While it is evident that the operator has taken a range of remedial actions, failure to achieve this result would entitle PTV to formally notify the operator that its noncompliance with the MMBF risks termination of its contract, or withholding of contract payments until a satisfactory remedy has been implemented.

In August 2014, the operator raised concerns that the methodology of the survey that PTV uses to calculate the level of fare evasion was not representative of the areas it operated. Although PTV subsequently reviewed and made some minor adjustments to the survey, it is unclear if these have been accepted by the operator.

Greenfields timetable

A total rebuild of the current bus timetable, known as the Greenfields timetable, is another key offer commitment. By having the operator develop and propose a rebuilt timetable, PTV aims to improve the efficiency of these bus services and increase patronage at no additional cost to the state. Implementation of the Greenfields timetable was scheduled for April 2015. However, this has now been delayed, as the state has not yet given its approval to proceed.

The initial step toward Greenfields implementation was the July 2014 timetable refresh. The operator implemented a suite of service changes, including route changes, route cancellations and timetable changes. The timetable refresh was not as extensive as the full Greenfields implementation will be, but was still a much more significant change than bus patrons are accustomed to. These changes met with passenger resistance and criticism of the consultation processes—following passenger complaints and highly critical media coverage over the cancellation of one bus route, the then Minister for Public Transport intervened to reinstate this service.

The operator subsequently acknowledged that improved communication to passengers about the changes needed to occur, such as providing more information at bus stops, more media releases and earlier availability of printed timetables. The operator submitted its draft business case for the Greenfields timetable on time. However, as this was before the full impact of the July 2014 refresh was known, it did not take into account the concerns raised by customers. PTV advised that the draft was subsequently finalised after considering passenger feedback on the July 2014 service changes and proposed Greenfields timetable.

PTV further advised during the audit that, while it is not possible to achieve 100 per cent acceptance by the public of proposed service changes, it was confident the operator would implement the Greenfields timetable in April 2015 as originally intended by the contract. However, the state has since delayed its implementation. Failure to implement it successfully, and learn from past experiences, will have negative consequences in terms of service improvement.

Customer satisfaction

The operator is required to improve its overall customer satisfaction rating to 80 per cent or more by 31 December 2016. If this result is not achieved, the operator will be required to implement remedial plans to address the factors negatively impacting on the rating.

Although PTV reviews the results of quarterly customer satisfaction surveys and discusses the results with the operator, it does not complement this work with specified interim targets to more rigorously track the operator's progress towards meeting the target.

2.4.3 Service specifications

Service specifications are performance standards set by the MMBF agreement that include:

• general obligations covering the quality of service provision
• ensuring staff are properly trained and experienced, and that drivers are appropriately authorised
• maintaining sufficient numbers of buses to meet operational requirements
• ensuring bus depots and other bus-related infrastructure meets all legislative requirements.

PTV has not defined the qualitative elements of the general obligations. For example, the operator is required to perform its obligations in a 'timely and expeditious way' and in a 'proper, competent, courteous, safe and reliable manner' but these standards are not defined. Nor does PTV undertake any form of monitoring of the operator's compliance with the service specifications, preferring to address issues as they arise.

As such, PTV is not as well placed to identify and respond to any noncompliance or to challenge the operator where PTV believes its standards have not been met.

2.5 Progressive reform of the bus industry

The MMBF was intended to form the basis for further reform of the Victorian bus industry. In this regard, the next round of procurements will be for the remaining 70 per cent of metropolitan services that are currently provided through 27 private contracts that expire in July 2018.

PTV and the Department of Economic Development, Jobs, Transport and Resources' (DEDJTR) slow progress in determining the preferred procurement option and tight project time frame for establishing follow-on contracts risks the state not achieving further bus industry reform and improving value for money.

2.5.1 Procurement risks

Establishing new contracts to replace the 27 that expire in July 2018 will be a complex and time-consuming project. The number and severity of risks facing this project will depend on how much further the state elects to pursue its reform agenda.

Absence of clear end-of-term rights

Throughout the 1990s, successive reports by the then Industry Commission, the Victorian Commission of Audit and VAGO all noted that the practice of establishing contracts through negotiation with incumbent operators, without exposing them to competition, minimises the state's ability to gain best value for money. Moreover, the Victorian Government Purchasing Board's guidelines are based on the presumption that all major procurements will be open for tender to ensure the market is fully tested, and that all qualified providers can compete for government work.

The 27 private contracts do not contain any clear end-of-term rights for the state or an explicit right to offer the services through tender. However, the bus operators claim their financial investment in establishing the services gives them proprietary rights over their routes, and deny the state has the right to offer their routes to tender. The state does not share this view, but has not yet tested its position.

Consequently, pursuing an open tender could lead to the current operators taking legal action against the state, thereby potentially delaying any tender process until a settlement is reached or a legal ruling is obtained.

Negotiating contractual reforms

If the state elects to establish new contracts through negotiation with incumbent operators, it is unlikely to be able to pursue contractual reforms as extensive as those sought during the MMBF's procurement.

When contracts were negotiated in 2008 the incumbent operators were resistant to major reforms, including open book access to their financial data, establishing end‑of‑term rights and asset rights. Our 2009 audit of this process, Melbourne's New Bus Contracts, found that only modest cost reductions were achieved, and that on a like-for-like basis, the costs of the contracts negotiated in 2008 were very close to the costs of those they replaced.

2.5.2 Timeliness of planning

As the state faces risks no matter which procurement option is chosen, early advice to government is critical to ensure that all risks are appropriately assessed during the decision-making process.

Our 2009 audit recommended the former Department of Transport provide early advice to government on the strategic options and constraints for future metropolitan bus contracts. While the department and PTV implemented this recommendation for MMBF, it has not been addressed in relation to the next tranche of bus contracts.

The project to establish the MMBF required multiple approaches to government over 14 months before the decision to proceed with the open tender was reached. A further 22 months elapsed before the MMBF commenced operating.

Achieving further bus industry reform through new contracts will be much more complex, high risk and time intensive, and consequently will be likely to require more than a three-year lead time. PTV and DEDJTR established a joint working group in July 2014 to commence preliminary planning for the next round of bus contact procurements. The working group commissioned initial research on bus structures in other jurisdictions and potential options for reform. An interdepartmental steering committee comprising PTV, DEDJTR, the Department of Treasury and Finance and the Department of Premier and Cabinet was also formed in early 2015 to inform the development of a related procurement strategy for the government's consideration by August 2015.

However, progress on this initiative has been slow, meaning the remaining time for planning and implementing this project is ambitious and not commensurate with the significant scale and complexity of the task.

This situation risks limiting the state's options in 2018—if delays result in insufficient time to pursue an open tender, negotiating with incumbent operators may be left as the only way to proceed. Such an outcome will likely compromise any opportunity for the state to achieve improved value for money from these bus contracts.

Recommendations

That Public Transport Victoria:

1. promptly rectifies all barriers to implementing the Melbourne Metropolitan Bus Franchise performance regime, including data reliability issues
2. documents and assesses the rationale for all decisions to waive the withholding of payments for non-performance
3. closely monitors the delivery of key offer commitments and proactively addresses any slippage from contractual time lines
4. systematically audits and verifies the reliability of performance data provided by the operator underpinning incentive payments.

That Public Transport Victoria and the Department of Economic Development, Jobs, Transport and Resources:

5. advise government on all key risks, options and required actions for reforming the metropolitan bus contracts expiring in 2018.

3.1 Introduction

The state chose an open tender process to establish the Melbourne Metropolitan Bus Franchise (MMBF) to advance reform of Melbourne's bus industry, and to maximise value by increasing competition in the Victorian bus market.

The former Department of Transport (DOT) managed the strategic planning and most of the tender planning, until Public Transport Victoria (PTV) became operational in April 2012. DOT personnel involved in tender planning were transferred to PTV, which assumed responsibility for the remaining tender planning and the tender conduct. The Department of Treasury and Finance (DTF) was also directly involved throughout the planning and procurement stages, with representatives sitting on the project and tender steering committees, as well as the tender evaluation panel.

PTV employed a two stage approach to market, involving an expression of interest (EOI) followed by a request for tender (RFT) issued to shortlisted applicants. The MMBF's value and its significance to the state as a critical element of Melbourne's public transport network led the state to apply additional scrutiny to the tender under the High Value High Risk (HVHR) process. PTV also committed to complying with Victorian Government Purchasing Board (VGPB) requirements to ensure the tender was run in accordance with better practice.

Figure 3A provides an overview of the MMBF procurement process.

Figure 3A

The MMBF procurement process

Source: Victorian Auditor-General's Office.

3.2 Conclusion

Strategic planning for the MMBF tender was clearly focused on producing a value‑for‑money outcome, but did not fully comply with the required VGPB policies and guidance. Despite this, our assessment of supporting documentation indicates that the key costs, benefits, risks and options were adequately identified and considered.

The competitive tender process was guided by the appropriate procurement principles and probity requirements, and resulted in a contract that is capable of delivering value for money.

While the targeted improvement process (TIP) introduced at the end of the RFT assessment was not planned for, it contributed to achieving a value-for-money outcome for the state.

3.3 Strategic planning for the procurement

VGPB sets policies and minimum standards for departments' procurement of non‑construction goods and services. These cover procurement planning, conduct, awarding of contracts, and contract management.

VGPB has recently moved to more principles-based policies under the procurement reform process. However, the MMBF procurement was planned and implemented between 2010 and 2013 and was subject to the more prescriptive VGPB policy requirements in place at that time.

3.3.1 Approach to government and procurement options

DTF has provided advice to successive governments since the 1990s on the need to seek competitive reform of the bus industry to deliver better services for commuters and better value for money for the state. Its advice to the Treasurer and government during 2011 clearly demonstrated the case for competitively tendering the MMBF procurement. This advice also outlined the strategic advantages of tendering these bus services to support longer-term reform—including plans for the future tendering of the remaining 70 per cent of metropolitan bus services.

DTF and the former DOT considered a range of procurement options, including direct negotiation with incumbent operators, and tendering the services provided by NBC, MBL and SmartBus as individual packages. In March 2011 these options were presented to government, along with a recommendation to proceed with an open tender. The government did not initially accept this recommendation and instead sought additional advice on the merits of negotiating with the incumbent operators.

In September 2011, after considering additional advice from DTF and the former DOT, the government approved the use of an open tender and the application of the HVHR process. It also approved the reform agenda—a series of reform goals that the MMBF project was to pursue. Responsibility for future MMBF project approvals was also delegated to the Treasurer and Minister for Public Transport. This decision marked the end of strategic planning and the start of project development.

3.3.2 Developing a business case

The former DOT did not develop a business case or a final strategic procurement plan (SPP) for the MMBF. VGPB policy in place at the time required an SPP for procurements over $10 million, and for high risk or complex acquisitions regardless of value.

The SPP is intended to detail how the project will meet the identified business needs, and is supported by a business case. The business case should contain all the information needed to decide whether to support a proposed procurement activity, including the costs and benefits of the project, the proposed procurement method and conduct, and how the project intends to achieve the best value for money.

Our 2009 audit, Melbourne's New Bus Contracts, stated that for new major bus procurements the former DOT should use VGPB guidance to develop a business case as part of the SPP, because the business case format is a clear and disciplined way to compare the costs, benefits and risks of the procurement options.

This departure from VGPB requirements created a risk that government decisions on the procurement project were not based on comprehensive advice. However, our assessment of MMBF-related Cabinet and ministerial submissions indicates that key costs, benefits, risks and options were properly identified and considered. In particular, the final approved submission to Cabinet in September 2011 contained the critical elements of an SPP, including a description of how the procurement meets the business need, procurement options for achieving the desired outcomes, details of stakeholders and departmental resources, and risk management issues.

This informed the development of the EOI and RFT documentation.

3.4 Procurement planning

Between project approval and the release of the EOI in June 2012, the former DOT, DTF and PTV continued planning the practical conduct of the tender. This was informed by the approved reform agenda, VGPB and HVHR policy requirements, and a market sounding exercise undertaken in November 2011. This culminated in the development of a procurement conduct plan (PCP) as an overarching framework for managing the tender process.

3.4.1 Procurement conduct plan

The PCP was approved by the steering committee in May 2012. It documented the governance structure, and the required procurement processes to be followed to achieve an impartial, transparent and accountable tender process. It provided an adequate framework for addressing probity risks, as it described the need to engage a probity auditor and clear processes for dealing with conflicts of interest and for ensuring the confidentiality of submitted tender information. It also documented communication protocols with tenderers and related information management requirements.

A project management plan was also developed providing an additional reference for the project team on required procedures that was in accordance with the PCP.

PTV also established a risk management framework, consisting of a risk and issues management plan, risk register and risk action plans in accordance with HVHR requirements. There is evidence to indicate that the risks were monitored by the project team, recorded on the risk register and were regularly reported to the steering committee.

3.4.2 Market sounding

The former DOT canvassed interest from potential bidders in a market sounding exercise undertaken in November 2011. Through this process DOT sought comments from potential tenderers on risks and issues the project team had identified and to gauge the level of market interest. The market sounding exercise confirmed that the open tender could deliver a contract that met the reform agenda objectives.

3.4.3 Governance

Project governance should set a firm framework to guide project success, create transparency and confidence in decision-making, and provide clarity around roles and responsibilities. The PCP and project management plan clarified governance arrangements for the Metropolitan Bus Services Project (MBSP). These were implemented as intended, however, the independence of the probity auditor was at risk of being compromised at times, as the auditor also provided probity advice. This issue is discussed in Section 3.5.5. The governance arrangements are outlined in Figure 3B.

Figure 3B

MBSP governance structure

Source: Victorian Auditor-General's Office from information provided by Public Transport Victoria.

3.4.4 External review and approvals

Under the HVHR framework, a project is required to undergo two Gateway reviews during the procurement stage. It also requires projects to obtain the Treasurer's approval at key points of the process, including the issuing of EOI and RFT documents to the market.

Evidence reviewed indicates that PTV complied with the HVHR requirements and underwent reviews at Gate 3 and Gate 4. The gate reviews did not raise any substantive issues. The Treasurer approved the release of the EOI and RFT documentation and the preferred tenderer prior to contract award.

3.5 Procurement conduct

In open tender processes VGPB policy emphasises the importance of ensuring all bidders are treated fairly, have access to the same information, and that standards of probity, confidentiality and information security are applied.

PTV had sound processes in place for the EOI and RFT phases, and the unanticipated TIP was adequately managed and did not compromise the tender process. The TIP is examined in Section 3.5.3.

The key VGPB requirements for an open tender process and PTV's compliance with these is summarised in Figure 3C.

Figure 3C

Key VGPB procurement policy requirements

Source: Victorian Auditor-General's Office.

3.5.1 Expression of interest process

The EOI evaluation plan approved by the steering committee in May 2012 accords with VGPB policy. It set out the requirements for the EOI evaluation process—including evaluation roles and responsibilities, document control procedures, criteria and methodology, and reporting and debriefing requirements. The plan also documented arrangements for managing probity such as conflicts of interest, receipt and registration of responses and respondent communication. EOI documents were issued to the market on 6 June 2012 following the Treasurer's approval as required under the HVHR process. Responses closed on 19 July 2012.

The initial EOI evaluation panel report was prepared in August 2012 consistent with the requirements set out in the EOI evaluation plan. It ranked the respondents and documented a recommended shortlist comprising the four highest ranked submissions. The report included evaluation panel scores and a high-level summary of evaluation team findings against each of the criteria. The steering committee considered the report on 20 August 2012, and directed the evaluation panel to refine its assessment, particularly in relation to the fourth, fifth and sixth ranked submissions, as the basis of their ranking was unclear.

Further analysis led to a re-ranking of the bids, which did not materially alter the outcome for the previously four highest ranked bids. However, this further analysis improved the rank of the tenderer who would later be selected as the operator from fifth to equal fourth. A supplementary assessment report was prepared which recommended shortlisting the five highest ranked responses. The steering committee approved the panel's recommendation on 23 August 2012.

The RFT was released to the five shortlisted parties on 29 October 2012. Two of the parties withdrew from the process—due to the parties being awarded contracts in other jurisdictions, meaning they no longer had capacity to commit to the MMBF—resulting in three proposals being submitted.

3.5.2 Request for tender process

VGPB guidance provides that an RFT evaluation plan should be prepared and approved prior to the close of the tender period. While the MMBF RFT evaluation plan was prepared and approved by both the project director and steering committee chair, final approval of the plan was on 1 February 2013, one day after the RFT period closed. However, PTV took the appropriate steps to mitigate any probity breach in this instance. Project team members were not given access to submissions until after the evaluation plan was approved by the steering committee chair. The probity auditor also noted that the version of the RFT evaluation plan approved by the chair was unchanged from the version approved by the project director.

The RFT evaluation plan was prepared in accordance with VGPB guidelines. However, the plan did not include the TIP, as it was not part of the planned evaluation process and was added after the evaluation commenced.

3.5.3 Targeted improvement process

The TIP involved inviting shortlisted bidders to reconsider certain cost areas that PTV assessed were capable of improvement. In presenting the TIP to the steering committee, the evaluation panel outlined the following stages:

• Stage 1: Targeted improvement—all shortlisted respondents were given tailored areas specific to their proposal to focus on and each given a two-week response period.
• Stage 2: Clarification—focused exclusively on the highest ranked proposal and on ensuring the draft contract was close to being executable.

By engaging all shortlisted bidders during this evaluation stage, the TIP maintained competitive tension and assisted with optimising value for money from the bids. The steering committee and probity auditor endorsed this approach in March 2013.

3.5.4 Approval of preferred tenderer

The RFT Final Evaluation Report was prepared and endorsed by the steering committee in April 2013. The report showed the panel's overall rank for each respondent at the completion of the initial evaluation process, and after the TIP. The probity, legal and financial aspects of this outcome was also independently scrutinised and assured.

The tender process concluded on 24 April 2013, when the Treasurer approved selection of the preferred bidder following endorsement of this recommendation by the PTV board and Minister for Public Transport.

3.5.5 Probity

PTV effectively managed most probity requirements for the project. Established procedures largely conformed to guidelines set out in the VGPB policy for the Conduct of Commercial Engagements. Specifically, PTV:

• developed a PCP which adequately outlined the standards of conduct for the process
• provided tenderers with the same information
• established clear processes for identifying and managing conflicts of interest
• engaged a probity auditor to provide an independent assessment of the procurement conduct.

The tailored approach adopted with tenderers during the TIP process did not breach VGPB guidelines, as the information sought from individual tenderers was commercially sensitive.

PTV's PCP outlined how VGPB's probity principles were to be applied to the procurement. Figure 3D outlines how PTV complied with these.

Figure 3D

Application of probity standards

Source: Victorian Auditor-General's Office.

Probity auditor

The probity auditor's terms of reference were to provide a limited or negative assurance review of the process. The probity auditor provided two limited assurance reviews covering 15 November 2010 to the end of the EOI stage, and from 19 September 2012 to the selection of the preferred bidder.

PTV did not separate the probity advisor and probity auditor roles. The probity auditor also provided probity advice including:

• providing probity briefings to project team members
• advice on managing conflict of interests
• advice on probity requirements in steering committee meetings.

Although VGPB policy and associated guidelines allow the probity auditor and probity advisor roles to reside with a single entity, VAGO believes combining these roles weakens the perception of independence. This is consistent with our finding in VAGO's 2009 performance audit of Melbourne's New Bus Contracts.

Future bus service procurements should engage a separate probity auditor and probity advisor.

Recommendations

1. That Public Transport Victoria and the Department of Economic Development, Jobs, Transport and Resources develop a business case assessing the cost and benefits of alternative reform options and related strategies for all future bus service procurements in accordance with Victorian Government Purchasing Board guidance.
2. That Public Transport Victoria separates the role of probity auditor and advisor on future procurements.
3. That Public Transport Victoria and the Department of Treasury and Finance use the lessons from Melbourne Metropolitan Bus Franchise's targeted improvement process to develop guidance on how it may be applied more broadly in future procurements.

4.1 Introduction

Effective contract management ensures that all parties meet their obligations under the contract, and is necessary to secure value for money. Key elements of effective contract management include:

• policies that clarify roles and responsibilities for contract management and decision-making
• staff with appropriate skills who understand the objectives of the contract
• monitoring and management of contractor performance throughout the life of the contract
• consistent and professional application of contract provisions
• ensuring that payments are made in accordance with the contract and related performance standards.

Effective contract management also involves responding to changing circumstances and implementing contract variations as required. Contract variations should be used to ensure changed circumstances do not reduce the value of the contract, and that innovations and opportunities to increase value for money can be exploited.

This Part of the report assesses Public Transport Victoria's (PTV) contract management policies, processes and systems, and whether the Melbourne Metropolitan Bus Franchise (MMBF) agreement is being managed effectively.

4.2 Conclusion

PTV did not have any contract management processes in place to track performance and ensure compliance with the contract terms for the first four months of MMBF operations. As a result, it was not in a position to ensure that the contract delivered value for money during this time.

PTV's contract management policies and practices for the MMBF are inadequate, and risk undermining the value of the MMBF agreement. The lack of a contract management plan has led to inconsistency in the application of some contract terms.

These inconsistencies and PTV's willingness to negotiate extensions on contract deadlines risk reducing the value of the contract, and created a precedent for the operator to negotiate away penalties.

PTV has not assessed the value-for-money implications of the approved contract variations. This risks undermining MMBF's value, as it means there is little assurance that variations support the achievement of MMBF's objectives.

4.3 Systems, policies and processes

Effective contract management systems, policies and processes enable a contract to be administered efficiently, correctly and consistently. A contract management plan is the primary source for this—Victorian Government Purchasing Board guidance highlights the need for a contract management plan for all critical and high risk procurements.

PTV advised it has an organisation-wide contract management framework, and a quality management system certified to AS/NZS ISO 9001:2008 Quality management systems – requirements, which governs its approach to managing all contracts with operators.

This framework is supplemented by specific policies and procedures applicable to the MMBF.

4.3.1 Policies and processes

PTV does not have a complete contract management plan in place for the MMBF. Contract management processes specific to the MMBF are documented in a Contract Administration Manual (CAM) and 17 supporting Processes Administration Guides (PAG). The CAM is intended to assist PTV staff to understand the obligations of the franchise agreement and guide them through the related workflow processes. The PAGs contain workflow processes for routine contract management tasks.

The CAM and PAGs are sufficient for routine contract management and administrative processes, such as calculating payments. However, they were never intended to operate without an MMBF contract management plan. Key elements missing from the CAM and PAGs include:

• Clear roles and responsibilities—the CAM lays out a general chain of responsibility within PTV for the contract, but does not sufficiently detail where responsibility lies for specific tasks and key decisions. There is no decision‑making process for complex or contentious issues, which increases the risk of delays due issues being escalated unnecessarily.
• Guidance on contract reviews and a review strategy—there is no documented process for conducting regular contract reviews and no related review strategy.
• Requirements for data and information management, including files and recordkeeping—there is no documentation detailing the MMBF's information or knowledge management processes, policies or strategies.

Without an MMBF-specific document covering these elements, the knowledge needed to effectively manage the contract is likely to be lost due to staff changes. This risk has materialised and is covered in Section 4.4.5.

While the CAM and PAGs, in concert with PTV's contract management framework, offer sufficient guidance for routine contract management, they are not sufficient to ensure that the contract is managed effectively and consistently throughout the life of the agreement.

4.3.2 Contract management system

PTV's contract management system, Affinitext, is designed to facilitate effective contract management by breaking down the MMBF's complex processes into simple tasks that are assigned to individuals. PTV advised that MMBF franchise management staff regularly review and update Affinitext on the status and completion of due tasks, which on occasions involves physical inspection. However, it is not clear that this has systematically occurred.

Contract management staff advised that all tasks are usually completed, but acknowledged they do not systematically verify completion, or if sufficient evidence has been provided.

Affinitext is routinely updated following approved contract variations. However, there are no established procedures for ensuring the CAM and PAGs are similarly updated. Consequently, there is insufficient assurance that the existing guidance material is up to date. This issue is examined further in Section 4.5.

4.3.3 Establishing processes and transition

There is no evidence that PTV had any contract management processes in place for the first four months of MMBF operations. This means that PTV was not in a position to ensure that the contract delivered value for money when services commenced, as it did not have the mechanisms needed to track performance and ensure compliance with the contract terms.

Our 2009 audit Melbourne's New Bus Contracts recommended that for new contracts PTV should engage more effectively with operational staff to complete the contract management documentation, and to plan for the transition to the new arrangements before existing contracts expire. This recommendation was partially implemented for MMBF transition. Operational staff were involved during all stages of the tender assessment, contract development and transition planning—however, PTV still failed to finalise its contract management processes prior to the start of services.

4.4 Contract management

PTV is not managing the MMBF contract effectively, undermining the potential for the state to realise the full value possible from the agreement. Contract terms are being inconsistently enforced, with PTV declining to apply penalties. PTV has been entitled to withhold over $380 000 in payments and seek underperformance remediation plans and commitments, which it has declined to do.

4.4.1 Transition

The transition from the five previous contracts to the MMBF contract occurred without any major interruption to services. However, as there were no MMBF-specific contract management processes in place, it is not possible to assess whether all aspects of the contract were being effectively managed during this time.

A Transition Steering Group was established to manage the transition, which was primarily concerned with risks and issues that could impede the commencement of operations. The group did not consider the implications of not having a contract management framework in place.

The Transition Steering Group managed the operational commencement risks well, ensuring mitigation or management plans were developed for significant risks, and requiring regular updates from those responsible for each risk. Successful management or mitigation strategies included:

• shifting the service commencement date from a weekday to a Sunday, to ensure start-up issues affected the fewest customers
• running several combined first day of service rehearsals with the operator and PTV staff.

4.4.2 Application of contract terms

Some of the MMBF contract terms and penalties are being applied inconsistently. PTV chose to extend the completion deadlines for three offer commitments—bus rebranding, the establishment of an operations control centre, and the installation of automatic passenger counters. Instead of withholding contract payments of $200 000 and making the operator develop and implement a plan to remedy the underperformance—as the contract entitles PTV to do—it chose to extend the completion deadlines.

A fourth offer commitment—the refurbishment of buses over 13 years old—was not met for the first contract year. Again, PTV had the option of withholding payment of $180 000 and making the operator develop and implement a plan to remedy the underperformance, it did not do so.

PTV advised that it has only waived the withholding of payments for some delayed initiatives offered by the contractor and now formalised in the contract, but which have no impact on core services to customers. However, a key goal of introducing the performance-based MMBF contract was to create incentives for improving performance. While deadlines may sometimes need to be extended, the contract enables PTV to reasonably reduce payments in response, but it has never done so.

PTV's practice of extending contractual deadlines rather than withholding payments is inconsistent with the state's reform objective of establishing an incentive-based contract. Contractual penalties are an important mechanism for encouraging improvements in performance. PTV's failure to consistently enforce them risks compromising the contract's value and achieving the state's reform objectives.

4.4.3 Data issues and relationship management

As noted in Part 2 of this report, ongoing issues with data reliability are impeding PTV's capacity to measure and monitor performance and effectively manage the contract. Of particular concern is the delay in setting the patronage benchmark, required to operate the Patronage Incentive Regime, which was supposed to come into effect from 1 January 2014, but was ultimately not set until February 2015 because of a disagreement between PTV and the operator.

PTV initially set the patronage benchmark based on six months instead of 12 months of data. PTV believed that the data collected by the myki systems for the first six months of the year was not reliable. The MMBF contract allows the benchmark to be set using less than 12 months of data if the data is unreliable and both parties agree to the alternative calculation. The operator disagreed with this approach. Once the benchmark was set, the operator was paid a performance incentive of over $300 000 for exceeding its patronage target in 2013–14. It is concerning that the dispute was allowed to run for so long.

4.4.4 Greenfields

Full implementation of the MMBF's performance incentive and penalty regime will not occur until after the introduction of a rebuilt timetable, referred to in the contract as the 'Greenfields timetable'. The Greenfields timetable is a redesign of MMBF services, including adjusted timings, some redesigned routes and cancellation of duplicated or low-patronage routes. It was scheduled to be introduced in April 2015, however, the state has recently delayed its implementation.

The introduction of the Greenfields timetable is the point at which the MMBF agreement is capable of operating at its full value. The MMBF agreement was designed to improve services to customers, meet the demands of more passengers, and deliver improved punctuality. Therefore, the Greenfields timetable redesign is intended to deliver better targeted services than those currently operating.

Under the MMBF agreement, the Operational Performance Regime (OPR) comes into force when the Greenfields timetable commences. This activates penalties of up to $2 million each year for underperformance. PTV believes its current data reliability issues will be resolved by the time Greenfields is implemented, however, any delays in resolution means PTV will be unable to assess the success of Greenfields or fully exploit the OPR.

4.4.5 Knowledge management

PTV does not have effective knowledge management processes in place for MMBF, despite having sufficient knowledge and document management systems.

PTV uses TRIM as its record-keeping system, but the system has not been used to manage MMBF information and knowledge effectively. Specifically, document control within TRIM has not been enforced meaning there are multiple versions of key documents and no clear indication of which is the final authoritative version. During the conduct of this audit, PTV was unable to readily produce final, approved versions of several key documents relating to the tender process and the management of the MMBF agreement.

This poses a significant risk for PTV, as new or inexperienced personnel may inadvertently rely on incorrect information contained in draft documents when managing the contract and formulating related decisions.

This situation means that there is an overreliance on the knowledge of individuals, which has not been routinely captured in a way that successors or the wider PTV organisation can make use of. For example, PTV referred many questions relating to the conduct of the tender to an individual who no longer works for PTV. Personnel will continue to change and new staff will have difficulty sourcing information or reasons for previous decisions.

The lack of effective knowledge management also has implications for future bus service procurement and reform. The lessons learnt during the MMBF procurement are likely to be lost, as the key documents and reasons for decisions cannot be readily located.

4.4.6 Contract management training and support

PTV does not routinely supply contract management staff with specific contract management training. Line staff mainly learn their duties on the job, with the higher level managers relying on their previous experience and training. Affinitext and TRIM training is provided for staff, with refresher training and in-house support available, but this is generic system training. There is no specific training or formal induction for staff involved in managing the MMBF agreement. PTV provided contract management training for relevant staff in February 2015.

4.5 Contract variations

Contract variations are sometimes necessary to respond to changes in the operating environment, or to take advantage of innovations or unanticipated circumstances. However, they should not reduce the value of the contract to the state, nor should they be used to escape consistent underperformance. All contract variations should be assessed to ensure value for money is maintained, and that key elements of the contract are not undermined.

There have been two variations to the MMBF agreement. While both variations were approved at the appropriate level and in accordance with MMBF provisions, there is no evidence that PTV analysed how the value of the contract would be affected. The impact of these variations on the original value proposition of the contract remains unclear.

The first contract variation was executed in August 2013, three months after execution of the franchise agreement. The primary purpose of this variation was to put into effect the financing and depot arrangements finalised following execution of the MMBF agreement. This is not unusual in large projects—financing arrangements are seldom concluded before a signed contract is in place. However, this variation also made changes to other aspects of the contract:

• Patronage growth targets—the benchmark figure for the increase in the patronage that will occur each year decreased by approximately 113500 passengers.
• Real monthly operating payment (RMOP)—the total amount paid for services each month was reduced by approximately $29600 per month.

The second variation, executed in June 2014, primarily concerned additional depot details finalised after the start of operations. However, it also changed other aspects of the contract:>

• RMOP—this payment was increased by approximately $39000 per month above the price set by the first variation, cancelling out the savings gained in the first contract variation. The overall effect of this was to increase the price of services by approximately $9500 per month, or 0.1percent, above the original agreement. PTV has noted that this increase was to take into account a cost that it previously understated in information it provided to tenderers.
• Offer commitments—the requirements and deadlines for the refurbishment and rebranding the operator's fleet were extended, as detailed in Section 4.4.2.

In the absence of a contract management plan, no guidance is available to MMBF staff on how to ensure value for money is maintained in respect of proposed variations. The Contract Variation PAG is insufficient, containing only high‑level process guidance to ensure variations are properly approved.

PTV needs to ensure that all decisions on contract variations are robust and take into consideration the overall effect of the changes on the contract's value proposition. In particular, variations regarding the performance monitoring regime should be carefully analysed to ensure expected performance standards are not reduced.

PTV has processes in place to ensure that Affinitext is updated following contract variations. However, these are generic Affinitext processes, and are not specific to MMBF. There are no processes in place to ensure the MMBF CAM and PAGs are updated following contract variations, and there is no evidence that these documents were reviewed or updated following the June 2014 contract variation. MMBF's contract management system and processes are at risk of diverging from each other as Affinitext is updated while the CAM and PAGs are not. This will render the CAM and PAGs ineffective, as their processes will no longer reflect variations to the contract.

Recommendations

That Public Transport Victoria:

1. ensures that contract management processes are in place prior to commencement for any new service contracts
2. establishes a contract management plan for the Melbourne Metropolitan Bus Franchise in accordance with Victorian Government Purchasing Board guidance
3. establishes knowledge management and document control processes to ensure critical information and reasons for decisions are readily available to the necessary staff
4. establishes a process for systematically monitoring and verifying the completion of required Melbourne Metropolitan Bus Franchise contract management tasks
5. consistently enforces the terms of the Melbourne Metropolitan Bus Franchise agreement in accordance with the contract.

RESPONSE provided by the Chief Executive Officer, Public Transport Victoria

RESPONSE provided by the Secretary, Department of Economic Development, Jobs, Transport & Resources

RESPONSE provided by the Secretary, Department of Treasury and Finance