Management of the Provincial Victoria Growth Fund

Conclusions

RDV undertook a wide range of planning activities to develop and plan for implementing PVGF. However, some deficiencies meant that RDV could not demonstrate that the fund was soundly based. While there were criteria for prioritising initiatives to be funded, assessment decisions were not adequately documented and criteria were not applied consistently to all funding decisions. Also, there was no business case underpinning it.

The monitoring and reporting systems and processes for initiatives and programs under PVGF were generally sound. However, these focused on outputs rather than outcomes. RDV had effective arrangements in place to manage grant programs and to disburse funding to grant recipients. Deficiencies in planning and undertaking evaluations, particularly the lack of a final evaluation of PVGF, mean RDV is unable to demonstrate that the fund achieved its objectives and outcomes. There were weaknesses in RDV’s risk management, and its documentation and recording of all PVGF programs and activities was not complete.

However, RDV has improved its planning and management of RGF and addressed some of the deficiencies of PVGF, particularly with regard to evaluation planning and meaningful reporting on the achievement of outcomes.

Findings

Funding

PVGF provided $100 million funding to a range of programs and projects designed to deliver its objectives and intended outcomes. When PVGF was announced in 2005 it included $78.36 million for specific programs and initiatives set out in the Moving Forward plan (allocated funding). A further $21.64 million was set aside to respond to emerging issues and for prospective activities not initially identified in the plan (unallocated funding).

Provincial Victoria Growth Fund objectives and outcomes

PVGF’s key objectives were to increase support to regional councils, businesses and communities to market and promote provincial Victoria, identify and capture new business and investment opportunities, tackle skills shortages, manage change and plan for growth. The objectives are aligned with the broader policy framework established for the Moving Forward plan.

The expected outcomes were that economic activity would be stimulated and more people, jobs and investment would be attracted to provincial Victoria. However, relevant and appropriate performance measures were not established to enable the achievement of outcomes or the effectiveness of the fund as a whole to be assessed.

Selection and prioritisation of Provincial Victoria Growth Fund initiatives and programs

A long list of initiatives and programs were initially considered for inclusion in the Moving Forward plan (allocated funding). Prioritisation criteria were developed to support this process and, while the objectives of PVGF and the Moving Forward plan are reflected in the prioritisation criteria, assessment decisions are not documented. As a result, the rationale for funding proposed initiatives is not clear.

There is little supporting evidence to substantiate how the $21.64 million unallocated portion of the funding was distributed. DBI advised that initiatives were assessed, prioritised and approved by the fund’s interdepartmental committee (IDC) as required before seeking ministerial approval. However, DBI does not have documentation to verify how the assessment and prioritisation of initiatives occurred. As a result, RDV cannot demonstrate that the selection of projects was soundly based, particularly as a significant number of initiatives included in the fund did not have adequately developed business cases.

Business cases

There was no business case developed for PVGF. In addition, the departments of Premier and Cabinet, and Treasury and Finance advised government that they did not support a third of the 26 proposals in the Moving Forward submission. This is because they did not have fully developed business cases and could not be adequately assessed against other Budget priorities. Nevertheless, all of the proposals in question were approved by government and included in the starting initiatives when PVGF was announced. There is no evidence that business cases were developed for these proposals to confirm their merit following the decision to commit to them.

Consequently, it was not demonstrated that there was adequate consideration of the need, options, costs, time frames, risks and expected benefits normally canvassed in a business case to allow decision-makers to assess whether a preferred option best met the need. The limited evidence means there was not a reliable basis to review whether funding decisions were adequately informed, consistent and soundly based. It is also not clear how achievement of the outcomes could be adequately assessed for these initiatives in the absence of expected benefits.

Implementation and evaluation plans

Implementation and evaluation plans (IEP) were developed for each initiative under PVGF. These were intended to identify high-level risks and risk management strategies, an evaluation framework, and performance measures and targets to measure the outcomes of the initiatives.

The IEP for PVGF itself was inadequate. It did not identify a risk management strategy, contain adequate measures and targets to allow for meaningful reporting, nor include an evaluation framework at the time it was endorsed. However, a risk management plan and evaluation framework for PVGF were subsequently developed.

Similarly, half of the 26 IEPs for the initial initiatives funded under PVGF did not adequately identify high-level risks, and 18 (or 69 per cent) did not include an evaluation framework. None of the 26 IEPs outlined the basis for the targets adopted. Consequently, performance measures set in the plans were not sufficient to understand or demonstrate the impacts on PVGF’s intended outcomes.

Performance measures and targets

The capacity of PVGF’s performance management framework to provide meaningful information on the fund’s intended outcomes or impacts was limited. Performance measures focused on activities over outcomes, and established targets were inconsistent and unclear. It was also unclear how individual initiative and program objectives were expected to contribute to the achievement of PVGF objectives and outcomes.

Governance

There is no documented evidence that the IDC established to oversee PVGF adequately carried out its assessment, prioritisation and evaluation responsibilities. This limited the effectiveness of its monitoring and review roles. DBI advised that the IDC did carry out its functions as required, but is unable to provide any documentation verifying this.

Monitoring and reporting

RDV established a sound monitoring and reporting framework for PVGF and effective arrangements were in place to manage grant programs and the disbursement of funds.

There is no single, comprehensive source of public information on overall PVGF expenditure disaggregated by funding recipient, initiative, program and project. RDV’s annual report describes the initiatives funded within PVGF and provides details of the grant payments made from the fund. However, it does not identify initiatives and programs for which grants were provided. This lack of transparency limits the ability to scrutinise PVGF’s performance.

Six-monthly reporting under IEPs was the main mechanism for reporting and monitoring implementation of initiatives under PVGF. This reporting focused mainly on the achievement of milestones and expenditure, and did not adequately report on impacts and outcomes. Progress reports did not include information on the performance measures identified in the IEPs.

RDV used the information from individual progress reports to compile collated information on PVGF initiatives for reporting to the government covering progress against milestones, budgets and outputs. However, these reports provided limited information on progress towards the achievement of outcomes.

Evaluation

RDV developed a ‘global’ evaluation framework for the Moving Forward plan which was endorsed by PVGF’s IDC as its applicable evaluation standard. Nevertheless, in the early stage of implementing PVGF, RDV did not adequately prepare to evaluate PVGF and did not apply a structured and integrated approach to evaluation. Together with the output-focused performance reporting, RDV never assessed PVGF’s overall effectiveness by completing a final evaluation. As a consequence, it cannot demonstrate that the fund fully achieved its intended objectives and outcomes.

Part way through the life of PVGF, RDV engaged consultants to undertake an interim evaluation of the Moving Forward plan. Published in 2009, this review provided only limited information on PVGF’s performance because of issues with the number of complete and operational projects, and limitations in data and performance measures.

RDV and two other agencies evaluated four PVGF initiatives during the life of the fund. Collectively, these initiatives accounted for $10.5 million or 11 per cent of PVGF’s expenditure of $92.4 million as at 30 June 2011. These evaluations provided some understanding of outcomes and the achievement of objectives for the programs themselves, but they did not demonstrate how they had contributed towards achieving PVGF’s overall outcomes or how the available results were incorporated into the Moving Forward interim evaluation.

Risk management

RDV’s risk management framework for PVGF included elements required to effectively manage PVGF’s risks, but it was not adequately developed or applied, and there was little evidence that all risks were purposefully managed.

The risk register was reviewed biannually to coincide with progress reporting, however, the reports did not adequately document action taken to address identified and emerging risks. Progress reports did not contain sufficient information to provide assurance that risks were being managed. The basis for reducing some risk ratings is unclear.

There were instances where the risk management framework failed either to detect materialising risks or mitigate their impact. Indeed, the risk identified relating to not being able to determine the benefit/value-add of initiatives was confirmed by the Moving Forward plan interim evaluation.

Records management

The Public Records Act 1973 requires public sector entities to keep full and accurate records. Neither DBI nor DPCD has been able to locate the final PVGF progress reports within their document management systems. As a result, the final status of PVGF and its component initiatives and programs cannot be readily verified. Other record and document management issues identified during the audit include missing project information and data which appear to have been destroyed as part of Cabinet processes or lost in the transfer of RDV from DBI to DPCD.

In the later stages of this audit, DBI provided evidence relating to activities that occurred when it was responsible for managing PVGF. This included some new information which had not been provided by DPCD.

The Regional Growth Fund

RDV has improved planning with respect to RGF. In particular, it has considered the deficiencies of PVGF with regard to evaluation planning and the capability to meaningfully report on the achievement of outcomes. An RGF Policy Context and Operational Model has been developed which provides the policy and operational framework to guide implementation of the fund. This includes a strategic framework for RGF, objectives (outcomes), principles, and operational structures and governance arrangements. A high-level risk analysis indicates that a risk management strategy will be developed. A framework for reporting and evaluation has also been developed based on RGF’s strategic framework, the intent of the Regional Growth Fund Act 2011, and an evaluation program logic model.

RGF’s evaluation framework clarifies the relationship between its objectives, strategic action areas, program activities and expected outputs. RDV has engaged a consultant to undertake formative and mid-term evaluations between 2012 and 2014 to understand and document RGF’s outcomes and opportunities for improvement. A final evaluation of RGF is also planned, and program-level implementation action plans and individual evaluation plans are expected to be developed.

Recommendations

1. When developing future programs and initiatives, the Department of Planning and Community Development should:
   • develop robust business cases and processes to assess proposals to inform funding decisions, in line with Department of Treasury and Finance guidance
   • develop clear and measurable outcomes that are supported by relevant and appropriate performance measures and targets.
2. The departments of Planning and Community Development, and Business and Innovation, should act to assure compliance with the requirements of the Public Records Act 1973 and associated standards and policies, and implement changes as appropriate.
3. The departments of Planning and Community Development, and Business and Innovation, should make sure key records relating to the Provincial Victoria Growth Fund, including key project documentation, are complete.
4. The Department of Planning and Community Development should apply the lessons learnt from the Provincial Victoria Growth Fund in implementing the Regional Growth Fund to:
   • provide assurance that the fund and its major components are soundly based
   • develop a comprehensive evaluation framework supported by relevant and appropriate performance measures and targets
   • implement robust monitoring and reporting systems and processes
   • undertake mid- and end-term evaluations with a focus on demonstrating the achievement of objectives and outcomes
   • develop and apply a sound risk management framework
   • implement appropriate record keeping and document management systems.

Submissions and comments received

In addition to progressive engagement during the course of the audit, in accordance with section 16(3) of the Audit Act 1994, a copy of this report was provided to the Department of Planning and Community Development and the Department of Business and Innovation with a request for submissions or comments.

Agency views have been considered in reaching our audit conclusions and are represented to the extent relevant and warranted in preparing this report. Their full section 16(3) submissions and comments are included in Appendix C.

1.1 Introduction

Regional and rural Victoria is important to the state’s prosperity and liveability, contributing approximately $71 billion to Victoria’s economy annually. The sustainable growth of regional and rural areas of Victoria is dependent on strong economies, strong communities, and well-developed infrastructure.

Regional communities are experiencing increasing challenges such as managing population change, providing the infrastructure to support economic growth, community development, the survival of small towns, and addressing the impacts of climate change.

Commonwealth, state and local governments all have roles in addressing these challenges. The Victorian Government has had a range of policies and programs aimed at driving regional and rural development and addressing challenges facing rural communities.

1.2 Policy framework

A number of policies guided regional development programs.

Growing Victoria Together, 2001

Growing Victoria Together was the previous government’s ten-year strategy for making Victoria a stronger, more caring and innovative state. It included a commitment to more quality jobs, supporting thriving, innovative industries across Victoria, and linking and growing all of Victoria. Priorities for provincial Victoria were to build stronger regional communities, economies and infrastructure for sustainable growth.

Moving Forward: Making Provincial Victoria the Best Place to Live, Work and Invest, 2005

In November 2005, the government released its $502 million, five-year economic statement for rural and regional Victoria, Moving Forward: Making Provincial Victoria the Best Place to Live, Work and Invest (Moving Forward). The strategic direction of the Moving Forward plan was based on three key drivers—people, industry and innovation, and infrastructure.

Moving Forward was the government’s action plan for positioning provincial Victoria to respond to key challenges facing regional communities by:

• boosting economic activity and stimulating investment and job growth across provincial Victoria
• attracting more people to provincial Victoria and helping regional communities to manage the impacts of population growth
• providing provincial Victoria with the infrastructure, skills and industries to drive future economic growth
• increasing innovation, the adoption of new technologies and regional specialisation
• building dynamic, confident regional communities with the capacity to take up new opportunities, plan for the future and maintain quality lifestyles.

Moving Forward initiatives were funded from three sources:

• a $200 million extension of the existing Regional Infrastructure Development Fund (RIDF), which was created in 1999 to support the delivery of regional infrastructure and capital works projects
• a new $100 million Provincial Victoria Growth Fund (PVGF) to support initiatives to drive economic and population growth unrelated to infrastructure or capital projects, which was launched in November 2005 with a five-year life span
• additional funding of $202 million, which included initiatives funded from internal departmental budgets as well as new specific initiatives outside the RIDF and PVGF.

In June 2008, following consultations with regional stakeholders, the government released Moving Forward: Update – The Next Two Years, 2008 to 2010. This was designed to respond to emerging challenges such as the demand for new skills and infrastructure, the effects of drought, and an increasingly competitive global environment. No new funding was provided as part of the Moving Forward update and PVGF provided a funding source for the development of new initiatives.

While under development, the Moving Forward plan was known as the Provincial Victoria Economic Statement. However, in this report it is referred to as Moving Forward, regardless of which stage of its development is being discussed.

1.3 The Provincial Victoria Growth Fund

The creation of PVGF was part of a package of initiatives intended to drive employment, population and investment growth in provincial Victoria. PVGF had specific objectives and outcomes that were aligned with the strategic direction of the Moving Forward plan.

The key objectives of PVGF were to provide increased support to help regional councils, businesses and communities to manage and plan for change and growth, capture new business and investment opportunities, tackle skills shortages, and market and promote provincial Victoria. The expected outcomes of PVGF were to stimulate economic activity and to attract more people, jobs and investment to provincial Victoria.

PVGF was designed to complement RIDF, which provided support for capital works, transport, education and information and communication technology (ICT) infrastructure, and tourism facilities under the Moving Forward plan. PVGF was intended to support activities and projects designed to drive population, investment and business growth. It consisted of $78.36 million in funding allocated to specific programs set out in the Moving Forward plan, and unallocated funding of $21.64 million for prospective activities not included in this plan. The original Moving Forward plan stated that this would provide an opportunity for departments to seek funding for program proposals that were not included in PVGF but which met its objectives.

PVGF contained programs and projects managed by a number of government departments and agencies including:

• the Department of Victorian Communities (now the Department of Planning and Community Development)
• the Department of Innovation, Industry and Regional Development (now the Department of Business and Innovation)
• the Department of Sustainability and Environment
• the Department of Primary Industries
• the Department of Treasury and Finance
• Tourism Victoria
• Workforce Victoria.

Some initiatives were stand-alone and delivered by individual departments, such as the Investing in Provincial Victoria campaign and the Gold Workforce Development Plan. Others, such as the Provincial Economic Partnerships program and the Extension of Community Regional Industry Skills program, provided funding to local councils, community organisations or business and industry through a grant application process.

PVGF funding ceased in June 2010. However, $13.8 million was carried forward to 2010–11 to finalise commitments under the fund.

Ready for Tomorrow: A Blueprint for Regional and Rural Victoria, 2010

In June 2010, the government released Ready for Tomorrow: A Blueprint for Regional and Rural Victoria. This $630.7 million plan was designed to bring together the work of regional councils, businesses and community groups to support the regions and governments to manage growth, respond to changing environments and enhance productivity, liveability and sustainability. Following the 2010 change of government, the blueprint was replaced and a new fund to support regional and rural Victoria was launched—the Regional Growth Fund (RGF).

Regional Growth Fund, 2011

To be delivered over eight years, the $1 billion RGF is the government’s current commitment to drive regional development. It was established by the Regional Growth Fund Act 2011 with two long-term goals of developing a prosperous and thriving regional Victoria with more opportunities for regional Victorians, and improving the quality of life for regional Victorians. It aims to achieve these goals by:

• providing better infrastructure, facilities and services
• strengthening the economic, social and environmental base of communities
• creating new jobs and improving career opportunities
• supporting the planning and development of projects
• leveraging increased investment.

This audit has not assessed RGF but has examined the extent to which lessons learned from PVGF’s performance have been adequately considered and applied to RGF.

1.4 Roles and responsibilities

1.4.1 Regional Development Victoria

Regional Development Victoria (RDV) is the government’s lead agency for developing rural and regional Victoria. Established under the Regional Development Victoria Act 2002, RDV is a statutory body within the Department of Planning and Community Development. It aims to support regional Victoria to build stronger economies and communities through increased employment, new investment and better infrastructure.

Prior to machinery-of-government changes in 2011, RDV was a statutory body within the former Department of Innovation, Industry and Regional Development, now the Department of Business and Innovation.

PVGF was administered by RDV and overseen by the Minister for State and Regional Development (the minister). It was delivered by RDV and other state government departments and agencies.

1.4.2 Interdepartmental committee

PVGF was overseen by an interdepartmental committee (IDC) chaired by RDV. The IDC was responsible for considering and recommending operational guidelines, assessing, prioritising, and providing appropriate recommendations to the minister on unallocated funding applications, and monitoring and evaluating PVGF’s performance.

RDV was also responsible for preparing reports on the progress of PVGF for consideration by the IDC and referral to the minister for consideration by government.

1.5 Audit objective and scope

The audit assessed the effectiveness of the PVGF management framework and whether intended outcomes were achieved by examining whether:

• funding decisions were soundly based and transparently linked to the objectives of PVGF
• governance arrangements supported effective cross-government coordination, including oversight of PVGF and funded initiatives
• performance reporting and evaluations demonstrated achievement of PVGF’s objectives.

The audit included the Department of Planning and Community Development and the Department of Business and Innovation.

1.6 Audit method and cost

The audit was conducted in accordance with section 15 of the Audit Act 1994 and Australian Auditing and Assurance Standards.

The cost of the audit was $365 000.

1.7 Structure of the report

The report is structured as follows:

• Part 2 discusses planning for PVGF
• Part 3 looks at managing and implementing PVGF.

2.1 Introduction

Comprehensive planning is a prerequisite for the effective delivery of programs and projects. Programs should be underpinned by a clear rationale and a sound business case that is supported by robust monitoring, reporting and evaluation to demonstrate effectiveness. Programs need clear and measurable objectives, outcomes, outputs and targets. Planning and implementation should be supported by appropriate governance and oversight arrangements. Establishing a robust framework for planning and implementation is essential to demonstrating that desired outcomes can be achieved.

2.2 Conclusion

While Regional Development Victoria (RDV) undertook a wide range of planning activities to develop and plan for implementing the Provincial Victoria Growth Fund (PVGF), it cannot demonstrate that the fund was soundly based. Deficiencies in performance measures and targets mean that RDV is not able to demonstrate that intended outcomes have been achieved. There is no evidence that the PVGF interdepartmental committee (IDC) adequately carried out its assessment, prioritisation, and evaluation responsibilities, which limited its monitoring and review activities.

2.3 Development of Provincial Victoria Growth Fund initiatives and programs

PVGF provided $100 million of funding to a range of programs and projects designed to deliver its objectives and intended outcomes. PVGF included funding for specific programs and initiatives set out in the Moving Forward: Making Provincial Victoria the Best Place to Live, Work and Invest (Moving Forward) plan (allocated funding), while some funds were set aside to respond to emerging issues and for prospective activities not initially identified in the plan (unallocated funding).

PVGF investment decisions should have been underpinned by a clear and transparent process for selecting initiatives and programs to be funded that was aligned with the objectives of the fund.

During 2004 and 2005, RDV undertook a range of activities to inform the development of a Cabinet submission for the Moving Forward plan, including determining initiatives to be included in the plan that would be funded through PVGF.

Allocated funding

To identify and prioritise proposed initiatives and programs for the Moving Forward plan, an IDC led by RDV held a number of brainstorming workshops. RDV also met with key departments to generate a long list of potential initiatives and programs for inclusion in the plan.

The long list of proposed initiatives was considered by a secretaries' working group, and subsequently by government, having regard to the following prioritisation criteria that they:

• drive employment, population and investment growth in provincial Victoria in the medium- to long-term
• improve or support the level of services and infrastructure in provincial Victoria to underpin population growth and sustainability
• do not impede the mobility of labour and capital between provincial Victoria and Melbourne
• support the development of the skills required for provincial Victoria to capture opportunities for economic growth
• support the comparative advantages of specific regions and contributes to regions’ ability to adapt to innovation and economic change
• do not create any additional regulatory burden on business or local government that might inhibit regional economic development
• provide value-for-money and robust costings
• have a role for government, and capacity to deliver.

While the objectives of PVGF and the Moving Forward plan are reflected in the prioritisation criteria, assessment decisions were not documented using this criteria. Therefore, the rationale for recommending proposed initiatives is not clear.

The Department of Premier and Cabinet (DPC) and the Department of Treasury and Finance (DTF) later analysed the initiatives against the same criteria and ranked them 1 to 5, where 1 means ‘strongly support’ and 5 means ‘do not support’.

The ranking process considered 133 separate initiatives for consideration under the Moving Forward plan. Of these, around 20 per cent were initially allocated PVGF funding. The rankings are summarised in Figure 2A.

Figure 2A

Summary of proposed initiatives' rankings

Source: Victorian Auditor-General’s Office based on information provided by the Department of Premier and Cabinet.

Unallocated funding

There is little supporting evidence to substantiate how the $21.64 million unallocated portion of the funding was distributed.

DBI advised that all initiatives considered for unallocated funding were:

• subject to an internal assessment and prioritisation process
• submitted to the IDC after RDV had confirmed that their objectives aligned with those of PVGF
• assessed, prioritised and approved by the IDC before submission to the minister for approval.

However, there is no evidence confirming these processes occurred. Meeting minutes indicate that the IDC considered and endorsed initiatives but it is not clear whether they were assessed.

DPCD has around 50 per cent of the concept proposals that were developed for new initiatives—the remainder cannot be found. DBI advised that following the development of concept proposals, more detailed submissions were developed and approved by the minister. However, DBI could not provide evidence of these submissions and could not verify its advice.

The lack of documented analysis outlining how potential initiatives and programs were identified, assessed and prioritised for possible inclusion in PVGF reflects poor administrative practice and means that RDV cannot demonstrate that the selection of projects was soundly based. This is more concerning, as a significant number of initiatives included in PVGF did not have adequately developed business cases. This is discussed in the following section.

2.3.1 Development of the initiative proposals

Investment logic

An investment logic map should be developed during the concept and feasibility phases of the investment lifecycle to define the logic for an investment. Instead, DBI developed an investment logic map for the Moving Forward plan in 2008 as part of evaluation planning—almost three years after its launch in November 2005. Given the timing of its development, it is not clear how the investment logic map developed by DBI could have informed the development of the plan or PVGF.

Business cases

A business case demonstrates the need for government investment in a particular initiative, program or project. It allows decision-makers to make a fully informed decision about the merits of a funding proposal. DTF guidance requires a strategic assessment of the business needs and the likely solution—an options analysis to determine which options will provide the best solution—and development of a business case to determine whether there is a compelling case for investing.

At the time PVGF was developed, DTF’s August 2003 Gateway Initiative Business Case Development Guidelines were applicable to proposals and initiatives seeking government funding. Although these guidelines applied to asset proposals and initiatives, it is clear that both DPC and DTF expected business cases to be developed for PVGF initiatives. In July 2008, DTF developed its Investment Lifecycle Guidelines, which were designed for all investments with a total estimated investment of over $5 million, but could be applied to investments of any type, complexity and cost.

In October 2005, government endorsed the Moving Forward plan. To inform government decision-making, DPC and DTF provided a briefing which included an overview of the plan, a summary of proposed initiatives, and recommendations relating to individual initiative proposals in the plan. Although DPC and DTF supported PVGF, they noted that they had not seen a business case, and were concerned that eligibility criteria and comprehensive governance arrangements had not yet been developed for the fund.

DPC and DTF did not endorse a third of the 26 proposals in the submission because they did not have fully developed business cases and could not be adequately assessed against other Budget priorities. Notwithstanding this advice, all of the proposals were approved by government and included in the starting initiatives when the fund was announced. There is no evidence that business cases were developed for these proposals to confirm their merit following the decision to commit to them.

Consequently, it is not demonstrable that there was adequate examination of the need, options, costs, time frames, risks and expected benefits normally canvassed in a business case to allow decision-makers to assess whether the preferred options best met the needs. This means there was no reliable basis from which to determine whether funding decisions were adequately informed, consistent and soundly based. ‘Concept proposals’ developed after the fund had commenced operation also did not include the scope and depth of information required to allow fully informed funding decisions to be made. Neither is it clear how the outcomes could be adequately assessed for these initiatives in the absence of business cases outlining the expected benefits.

2.3.2 Provincial Victoria Growth Fund funding and initiatives

Under PVGF, $78.36 million of funding was allocated to specific programs and initiatives set out in Moving Forward. While RDV managed a large number of initiatives included in PVGF, funding for the remainder was allocated to a range of departments and agencies for administration.

Initiatives were either stand-alone or contained numerous grant-based programs designed to disperse smaller amounts of funds among successful funding applicants. For example, the Extending the Make it Happen in Provincial Victoria campaign consisted of three programs: Investing in Provincial Victoria, Provincial Victoria Sponsorship and Provincial Economic Partnerships.

Appendix A provides details of the allocation of funding and initiatives in PVGF when the Moving Forward plan was released and a brief description of each initiative.

A further $21.64 million was set aside as unallocated funding to respond to emerging issues and for prospective activities not initially identified in the plan. This funding was available to provide grants to local provincial councils and other bodies for initiatives that met the objectives of PVGF. This unallocated funding was also used to support administration of the fund and to engage consultants who undertook evaluation activities. Appendix B outlines the initiatives supported through ‘unallocated’ funding after PVGF commenced, including a brief description and the expenditure under each initiative.

2.3.3 Implementation and evaluation plans

An implementation and evaluation plan (IEP) was developed for each initiative under PVGF to guide the implementation of initiatives and support evaluation activities. Government endorsed a global implementation plan for Moving Forward that articulated the requirements for developing IEPs for each initiative funded under the plan using a template developed by RDV with guidance from DPC. The IEPs were to identify high‑level risks and risk management strategies, an evaluation framework, and performance measures and targets to assess the achievement of outcomes.

The IEP for PVGF itself failed to develop a risk management strategy, did not contain adequate measures and targets to allow for meaningful reporting, and did not include an evaluation framework at the time it was endorsed. However, a risk management plan and an evaluation framework were subsequently developed for PVGF.

Similarly, half of the 26 IEPs for the initial initiatives allocated funding under PVGF did not adequately identify high-level risks, and 18 (or 69 per cent) did not include an evaluation framework. None of the 26 IEPs outlined the basis for the targets adopted. Consequently, performance measures set in the plans were not sufficient to understand or demonstrate the impacts on the fund’s intended outcomes.

A progress report template was created for use by agencies that were required to report every six months to RDV. RDV used these reports to prepare a consolidated report on the initiatives for the IDC and government.

2.4 Governance arrangements

A sound governance framework was required to support effective management of PVGF and provide adequate oversight of the fund and funded projects. Prior to the release of the Moving Forward plan, the structure and administrative arrangements for PVGF were endorsed by government. PVGF was overseen by the Minister for State and Regional Development, who was accountable for the overall statement of outcomes. RDV was responsible for PVGF’s administration and progress reporting.

In 2005 an IDC was established for PVGF. It was chaired by the chief executive of RDV, with representatives from RDV and the departments of Sustainability and Environment, Infrastructure (now Transport), Primary Industries, Victorian Communities (now Planning and Community Development), Premier and Cabinet, and Treasury and Finance.

The IDC’s terms of reference outline that its role was to:

• consider and recommend guidelines for the operation and administration of PVGF to ensure it reflected the policy objectives and context of Moving Forward and broader government objectives, and monitor and review it periodically in view of implementation and evaluation outcomes
• assess and prioritise applications for funding and provide an appropriate recommendation to the Minister for State and Regional Development, or in the instance of applications in excess of $2 million, countersigned by the Minister for Finance, with the exception of applications for funding of less than $10 000, which will be determined by the minister on advice from RDV
• review the PVGF implementation plan, and individual implementation plans for sub-programs and initiatives, for referral to the minister for approval
• monitor and evaluate the implementation of PVGF and consider biannual reports for referral to the minister for consideration by government.

There is no documented evidence that the IDC adequately carried out its assessment, prioritisation and evaluation responsibilities, limiting the effectiveness of its monitoring and review role. DBI advise that the IDC did carry out its functions as required, however it was unable to provide any documentary verification.

The IDC considered the general principles and guidelines for the grant components of PVGF prior to ministerial endorsement. For unallocated funds, the IDC considered IEPs for new initiatives, however there is no evidence showing how those initiatives were assessed or prioritised.

The IDC actively monitored PVGF and considered biannual reports for referral to the minister for consideration by government. However, apart from considering a global IEP for the fund, it did not effectively evaluate the implementation of PVGF. Evaluation of PVGF is discussed further in Part 3.

2.5 Performance measures and targets

Performance reporting is fundamental to accountability. It provides information on how effectively and efficiently agencies deliver programs, and how economically they use public funds.

Effective measurement of performance requires well-expressed objectives that clearly articulate the outcomes to be achieved. Underpinning performance reporting are performance measures that demonstrate whether agencies are meeting their objectives. Performance measures should be linked to program objectives and outcomes, and should provide adequate information on whether these objectives and outcomes are being met.

While the PVGF outcomes themselves were generally clear, they were not supported by relevant and appropriate performance measures to enable the achievement of outcomes and the effectiveness of the fund as a whole to be assessed.

Targets are an important way of measuring the progress of activities and the achievement of outcomes. In order to do this effectively, performance measures should be relevant, appropriate, measurable and clearly aligned with objectives.

PVGF’s performance management framework provided limited meaningful information on its intended outcomes or impacts. Performance measures focused on activities rather than outcomes, and established targets were inconsistent and unclear. It was also unclear how individual initiative and program objectives were expected to contribute to the achievement of PVGF objectives and outcomes.

RDV established performance measures for PVGF related to Moving Forward objectives in terms of increased productivity, more exports, population growth and increased jobs. However, these measures were focused on outputs alone and they were not sufficient to demonstrate the achievement of outcomes.

The PVGF IEP developed in 2006 contains specific performance measures for the fund. These are outlined and assessed in Figure 2B.

Figure 2B

Performance measures for the Provincial Victoria Growth Fund

Source: Victorian Auditor-General’s Office, based on information from Regional Development Victoria.

Targets for PVGF and component initiatives and programs were established. However, there were inconsistencies and the basis for establishing the targets is not clear. For example:

• RDV set a job creation target for PVGF of 100 jobs per year, despite setting a target of 750 jobs per year for the initiative Extending the Community Regional Industry Skills Program,which was one component initiative.
• Targets were not substantiated on the basis of the available quantitative evidence. While a target of 100 new jobs attributable to PVGF initiatives per year was established, the basis for selecting this target is not clear and it is not evident whether achievement of the target would constitute achievement of the PVGF’s desired outcomes.

In addition, RDV applied the same template for performance indicators and targets to all Moving Forward plan initiatives and programs, and this template did not contain any PVGF or initiative objectives. Therefore, the template used for reporting performance indicators and targets under PVGF did not provide a logical link to its own or component initiative objectives.

Recommendation

1. When developing future programs and initiatives, the Department of Planning and Community Development should:
   • develop robust business cases and processes to assess proposals to inform funding decisions, in line with Department of Treasury and Finance guidance
   • develop clear and measurable outcomes that are supported by relevant and appropriate performance measures and targets.

3.1 Introduction

Monitoring, reporting and evaluation are essential to the effective implementation of programs and demonstrating the achievement of intended objectives and outcomes. Robust frameworks for these activities are fundamental to transparency and accountability, and demonstrating the effective use of public funds. Appropriate risk management, document management and record keeping are required to effectively manage and administer initiatives and programs.

The Provincial Victoria Growth Fund (PVGF) ceased operation in June 2010. The $1 billion Regional Growth Fund (RGF), to be delivered over eight years, is the government’s current commitment to driving regional development. Although this audit has not assessed RGF, it has considered the extent to which lessons learned from PVGF’s performance have been adequately considered and applied to RGF.

3.2 Conclusion

While there was generally a sound monitoring and reporting framework for initiatives and programs established under PVGF, it focused on outputs rather than outcomes at the transaction level. Regional Development Victoria (RDV) had effective arrangements in place to manage grant programs and the disbursement of funding to grant recipients. Deficiencies in planning and undertaking evaluations, particularly the lack of a final evaluation of the fund, mean that RDV cannot demonstrate that the PVGF objectives and outcomes were achieved. Weaknesses in risk management and poor record management mean that not all risks were adequately managed and documentation and records for PVGF programs and activities are not complete. Early indications suggest improvements to RDV’s planning of RGF with regard to evaluation planning and the capability to meaningfully report on the achievement of outcomes.

3.3 Monitoring and reporting

Sound monitoring and reporting is central to effective program implementation and fundamental to effective oversight. It underpins key activities, such as evaluation, through facilitating the collection of data required to measure and demonstrate the achievement of objectives and outcomes. A range of monitoring and reporting activities took place under PVGF.

3.3.1 Monitoring and reporting against implementation and evaluation plans

Six-monthly reporting under implementation and evaluation plans (IEP) was the main mechanism for reporting and monitoring implementation of initiatives under PVGF. All initiatives were required to undertake this reporting to RDV, however, the reporting focused primarily on the achievement of milestones and expenditure without providing adequate information on impacts and outcomes. Progress reports did not include information on the performance measures identified in the IEPs.

RDV used the information from individual progress reports to compile collated information on PVGF initiatives for reporting to government covering progress against milestones, budgets and outputs. However, these reports provided limited information on progress towards the achievement of outcomes. This view is supported by the Department of Treasury and Finance, which in briefings to government consistently referred to the need for greater reporting on outcomes. This is evident from the fourth reporting period (1 July to 31 December 2007), when the plan was halfway through its implementation, to the eighth reporting period (1 July to 31 December 2009), when some progress towards the achievement of outcomes could reasonably have been expected.

3.3.2 Monitoring and reporting of grant recipients

Approximately 40 per cent of initiatives involved funds being disbursed across multiple organisations, which represented a contract management challenge. It is important that monitoring and reporting of individual projects provides assurance that outputs are delivered as reported.

Funding agreements with approved grant recipients should:

• outline the roles and responsibilities of each party, and require the parties to agree on this prior to the allocation of funds
• contain appropriate reporting requirements that enable monitoring of the progress of projects and the use of funds
• require clearly defined outputs to be delivered by the recipients
• require evidence that agreed milestones and/or outputs have been met before payment is made
• include procedures for withdrawing, suspending or recovering funds in the event of noncompliance, or if the project is not proceeding adequately
• require the recipient to submit a final report and/or financial acquittal on project completion.

To assess the effectiveness of RDV’s management of grant-based projects we examined a sample of project files from four PVGF grants programs:

• Provincial Victoria Sponsorship Program (five projects)
• Provincial Economic Partnership Program (seven projects)
• Support for Councils to Manage and Plan for Growth (six projects)
• Business Support to Enhance Investment Program, Community Regional Industry Skills Program II (CRISP II) (five projects).

Of the 23 grant programs examined for which RDV had direct responsibility, 19 were adequately managed in terms of:

• documenting obligations and funding in contracts or memoranda of understanding
• monitoring that funding obligations were met prior to disbursement of funds
• regularly reporting on payments and deliverables for initiatives, programs and projects.

However, for one grant program (CRISP II), there are gaps in the documentation necessary to confirm that all outputs had been verified.

It is not possible to verify outputs for four of the five projects for several reasons. In one instance, the grant was cancelled after the first instalment of $250 000 was paid because the grant recipient ceased operating due to deterioration in its financial circumstances. However, RDV had assured itself that the required outputs relating to the provision of training had been delivered prior to making this payment. The minister approved termination of the grant after the first instalment was paid. In another instance, the final payment was not claimed by the grant recipient as it estimated the cost of undertaking an audit associated with claiming the final payment of $12 500 was not worthwhile given its value. For the remaining two projects there is no evidence of the provision of final payments, as project files are incomplete. Therefore, all outputs are not able to be verified. In one case we were able to verify payment of the first instalment of $125 000 of a $375 000 grant. In the other case we were able to verify two payments totalling $250 000 of a $300 000 grant.

3.3.3 Whole of Provincial Victoria Growth Fund reporting

There is no single, comprehensive source of public information on overall PVGF expenditure disaggregated by funding recipient, initiative, program and project. RDV’s annual report describes the initiatives funded within PVGF, and provides details of the grant payments made from it. However, it does not identify initiatives and programs from which grants were provided. Information is provided on employment, capital investment and exports, but this is not aligned specifically with PVGF, where important information about the outcomes of the fund is missing. This lack of publicly available information limits the ability to scrutinise PVGF’s performance.

3.4 Evaluation

3.4.1 Evaluation planning

Effective evaluation planning is fundamental for measuring program performance and demonstrating the achievement of objectives and outcomes. Performance reporting should be sufficient to enable assessment of performance and to report achievements.

RDV developed a ‘global’ evaluation framework for the Moving Forward: Making Provincial Victoria the Best Place to Live, Work and Invest (Moving Forward) plan. The interdepartmental committee (IDC) endorsed this as the applicable evaluation framework for PVGF. However, RDV did not adequately prepare to evaluate PVGF by developing and then applying a structured and integrated approach to evaluation in the early stages of implementing the fund.

A more rigorous approach to demonstrating the link between individual initiative objectives and PVGF objectives and outcomes was required. However, from 2007, partway through the fund’s implementation, RDV undertook a range of evaluation planning and coordination activities, including developing an investment logic map (discussed in Part 2.3.3). RDV also established an evaluation working group that undertook a range of activities including overseeing the interim evaluation discussed below.

In the initial stages of implementing PVGF, reporting was appropriately focused on progress against time lines and budgets. It was intended that the focus would shift to reporting and measuring the fund’s impacts and achievement of outcomes, but this did not occur. Together with the output-focused performance reporting, PGVF’s overall effectiveness was never assessed and RDV cannot demonstrate that the fund fully achieved its intended objectives and outcomes.

3.4.2 Moving Forward plan interim evaluation

RDV engaged consultants to undertake an interim evaluation of the Moving Forward plan, which was published in 2009. The evaluation was intended to assess the outcomes of the initiatives at the mid-point of the plan, to identify achievements and highlight areas for improvement. However, failures to define and collect data, and limitations in data reliability and performance measures severely limited the usefulness of the evaluation.

The interim evaluation identified the following limitations:

• the multifaceted approach of the Moving Forward plan meant that the alignment between initiatives and objectives was not always obvious
• performance measures and outcomes for the Moving Forward plan were not clearly articulated from the outset
• while there were gaps in the data needed to understand the impact of projects, the data that was collected also varied in quality
• the extent to which the initiatives in the Moving Forward plan contributed to the strong population, employment and business growth experienced by regional Victoria between 2005 and 2008 could not be demonstrated.

While the interim evaluation provided some useful information about economic growth and community strengthening, it was impaired by the lack of quantitative information on impacts and outcomes. The interim evaluation:

• relied on survey information to fill the gaps in PVGF’s evaluation framework. The consultants received responses from 348 of the 974 projects under the Moving Forward plan. Of these, less than half (125) reported impacts that resulted in an economic benefit
• relied heavily on qualitative survey responses because only 30 of the 974 projects had generated data on the economic benefits
• showed that evidence associated with projects covered by PVGF was typically qualitative, with program and project managers describing the impacts without supporting data on outputs or outcomes
• made no observations about how the initiatives and programs within the Moving Forward plan could be made more effective. It focused on proving benefits rather than identifying opportunities for improvement.

3.4.3 Provincial Victoria Growth Fund initiative evaluations

The global evaluation plan developed by RDV identified that it was difficult to develop a framework to fit all initiatives and that departments should therefore use existing frameworks where they existed. The risk of this approach was that not all initiatives would be evaluated.

RDV and two other agencies evaluated four PVGF initiatives during the life of the fund. These were:

• Provincial Leadership Development Program—RDV
• Industry Workforce Development Strategy—Department of Business and Innovation (DBI)
• Boosting Local Public Private Partnerships—Department of Treasury and Finance
• Make it Happen in Provincial Victoria campaign—RDV.

Collectively, these initiatives accounted for $10.5 million or 11 per cent of PVGF’s expenditure of $92.4 million by 30 June 2011.

The evaluations provided some understanding of outcomes and the achievement of objectives for the programs themselves. However, the lack of evidence that initiatives were assessed in terms of their contribution towards the overall outcomes of PVGF or that the available results were incorporated into the Moving Forward interim evaluation was a weakness across all of the evaluations.

For example, the evaluation of the Make it Happen in Provincial Victoria campaign assessed levels of awareness of the campaign, but did not consider how these results had contributed to the achievement of related PVGF outcomes, such as population growth or job creation. The aim of the Make your Business Happen in Provincial Victoria 2006 campaign evaluation was to assess Melbourne businesses and business people’s awareness of the campaign message about opportunities to invest in provincial Victoria. Despite this, survey recipients comprised ‘Melbournians aged over 18’, and not specifically businesspeople. The evaluation made no recommendations for campaign improvements, despite the finding that the key message of the Investing in Provincial Victoria campaign—to encourage business and investment in the region—was not strongly associated with the campaign.

Only the Boosting Local Public Private Partnerships review made recommendations for program improvement.

3.5 Risk management

Risk management involves the identification, assessment and prioritisation of risks, followed by their active monitoring or management, with the aim of reducing the likelihood of them occurring, or their impact. A risk management framework includes systems for the ongoing identification, assessment, prioritisation and appropriate management of risks.

RDV’s risk management framework for PVGF included elements required to effectively manage PVGF’s risks. However, it was not adequately developed or applied and there is limited evidence that all risks were purposefully managed. RDV developed a risk register, endorsed by the PVGF IDC, that identified nine risks to the PVGF objectives and assessed the likelihood and potential consequences of the risks materialising. However, the basis for the identified risks was not evident, and there is little documented rationale or analysis to support the ratings ascribed to each risk. In addition, proposed actions and the reporting of these to mitigate each risk lacked sufficient detail to provide assurance they would be adequately managed.

The risk register was reviewed biannually to coincide with progress reporting, however the reports did not adequately document action taken to address identified and emerging risks. Progress reports contained insufficient information to provide assurance that risks were being adequately managed. There are instances where risk ratings were decreased, however, the basis for revising the risk ratings is not clear.

The risk management framework failed to detect all materialising risks or mitigate their impacts. In particular, we found one identified risk—benefit/value-add of initiatives cannot be determined—had materialised, which was confirmed by the Moving Forward plan interim evaluation discussed in Section 3.4. Implementation plans for initiatives and programs were not reviewed and updated to reflect changes in risk management strategies, and half of the 26 IEPs for PVGF’s original initiatives did not identify any risks, despite this being a mandatory requirement.

3.6 Records management

The Public Records Act 1973 requires public sector entities to keep full and accurate records. Responsibility for this rests with the head of the entity. Sound record management is integral to the effective administration of public sector agencies; it is essential to deliver effective and efficient services and facilitate open and accountable government.

According to RDV, the final progress reports for PVGF initiatives were provided to DBI in October 2010. However, neither DBI nor the Department of Planning and Community Development (DPCD) were able to locate these reports within their document management systems. As a result, the final status of PVGF and its component initiatives and programs cannot be readily verified.

Other record and document management issues identified during the audit included missing project information and data which appears to have been destroyed as part of Cabinet processes or lost in the transfer of RDV from DBI to DPCD.

In the later stages of this audit, DBI provided evidence relating to activities involving evaluation and reporting that occurred when it was responsible for managing PVGF. This included some new information which had not been provided by DPCD.

3.7 Lessons for the Regional Growth Fund

RDV has improved planning with respect to RGF. In particular, it appears to have considered deficiencies from PVGF with regard to evaluation planning and the capability to meaningfully report on the achievement of outcomes. This view is based on advice from RDV and the review of key RGF documentation which outlines how the fund would be managed.

RDV advised that the RGF was an election commitment and as a result there is no business case underpinning the fund. This means that there is no clear and documented basis for this major investment of $1 billion over eight years. In the absence of a business case, RDV has developed an RGF Policy Context and Operational Model to reflect the government’s commitments. This policy and operational framework to guide implementation of the fund includes:

• the strategic framework for the fund
• the strategic objectives (outcomes)
• the principles and operational structures, including operational streams, under which funds are to be distributed
• the governance arrangements, including clear roles and responsibilities for the Minister for Rural and Regional Development, the IDC, a regional policy advisory committee and an outline of regional level governance
• a high-level risk analysis, which indicates that an RGF risk management strategy will be developed
• a framework for reporting and evaluation based on RGF’s strategic framework, the intent of the fund’s Act and an evaluation program logic model
• administrative guidelines for each operational stream.

RDV suggests RGF’s evaluation framework makes specific the relationship between its objectives, strategic action areas, program activities and expected outputs and objectives. RDV has engaged a consultant to undertake formative and mid-term evaluations between 2012 and 2014 to understand and document RGF’s outcomes and opportunities for improvement. A final evaluation of the fund is also planned. There will also be program-level implementation action plans and individual evaluation plans.

RDV has established an executive officer-level Monitoring and Evaluation Committee, which meets monthly. It is supported by fortnightly meetings of an officer-level working group. This should provide close monitoring of RGF’s activities and, combined with the funds evaluation framework, improve the capability to report meaningfully on outputs and outcomes.

Recommendations

1. The departments of Planning and Community Development, and Business and Innovation, should act to assure compliance with the requirements of the Public Records Act 1973 and associated standards and policies, and implement changes as appropriate.
2. The departments of Planning and Community Development, and Business and Innovation, should make sure key records relating to the Provincial Victoria Growth Fund, including key project documentation, are complete.
3. The Department of Planning and Community Development should apply the lessons learnt from the Provincial Victoria Growth Fund in implementing the Regional Growth Fund to:
   • provide assurance that the fund and its major components are soundly based
   • develop a comprehensive evaluation framework supported by relevant and appropriate performance measures and targets
   • implement robust monitoring and reporting systems and processes
   • undertake mid- and end-term evaluations with a focus on demonstrating the achievement of objectives and outcomes
   • develop and apply a sound risk management framework
   • implement appropriate record keeping and document management systems.

Auditor-General’s response to the Department of Planning and Community Development

We considered all information provided by the Department of Planning and Community Development during the audit, including information provided late in the audit relating to risk management. Accordingly the audit summary (page xi) states: ‘There were instances where the risk management framework...’ and section 3.5 (page 25) of the report states: ...there is limited evidence that all risks were purposefully managed.’ These amendments took account of the information from the Department of Planning and Community Development and the report does not imply that no risks were managed effectively.