Outcomes of Investing in Regional Victoria

Regional Victoria is home to strong communities and plays a significant role in Victoria’s economy; however, it faces challenges compared with metropolitan Melbourne, including slow or negative economic growth in some areas. Successive governments have focused on improving economic and social outcomes in regional Victoria through significant programs and grants. As a result, Regional Development Victoria (RDV)—now part of the Department of Jobs, Precincts and Regions—has administered over $1 billion in grants since 2011 to improve regional outcomes.

This audit looked at the effectiveness of those grant programs in supporting regional Victoria, including whether RDV could demonstrate that it has improved outcomes in regional Victoria, and the quality of RDV’s delivery of these important government programs.

We have made seven recommendations to the Department of Jobs, Precincts and Regions focused on improving its management and evaluation of its regional grant funds and public transparency of allocated grants.

Interactive dashboard

We have created an interactive dashboard that provides information about the distribution of RDV’s grants to each regional local government area (LGA). The dashboard also provides a snapshot of economic and social indicators for those LGAs.

Dashboard

 

Back to top

ABS	Australian Bureau of Statistics
CBA	cost-benefit analysis
DJPR	Department of Jobs, Precincts and Regions
DPC	Department of Premier and Cabinet
DTF	Department of Treasury and Finance
EOI	expression of interest
GEMS	Global Engagement Management System
GRP	gross regional product
LGA	local government area
PVGF	Provincial Victoria Growth Fund
RDV	Regional Development Victoria
RDV Act	Regional Development Victoria Act 2002
RGF	Regional Growth Fund
RIDF	Regional Infrastructure Development Fund
RJIF	Regional Jobs and Infrastructure Fund
SEIFA	Socio-Economic Indexes for Areas
VAGO	Victorian Auditor-General's Office

Back to top

Gross Regional Product—the value of goods and services produced in an area.

Regional Victoria is home to strong communities and plays a significant role in Victoria's economy—it created 19 per cent of the state's Gross Regional Product (GRP) in 2017–18. However, regional Victoria also faces challenges compared with metropolitan Melbourne, including slow or negative economic growth, low job creation rates and slow population growth.

Successive state governments have focused on improving economic and social outcomes in regional Victoria through significant programs and grants. Regional Development Victoria (RDV)—a statutory agency in the Department of Jobs, Precincts and Regions (DJPR), formerly the Department of Economic Development, Jobs, Transport and Resources—is the lead agency responsible for developing regional Victoria and administering key regional development grant programs.

Since 1999, RDV has administered a series of regional development grant funds. The current fund, the Regional Jobs and Infrastructure Fund (RJIF), was set up in 2015–16 with the objective of 'growing jobs, building infrastructure and strengthening communities in regional Victoria'.

We audited two of the previous funds—the Regional Growth Fund (RGF) and Provincial Victoria Growth Fund (PVGF). These audits found that RDV could not demonstrate that the grants had supported regional growth outcomes due to deficiencies in the funds' administration. Our 2015 RGF audit made nine recommendations to RDV about the design and implementation of RJIF.

This report examines whether investment in regional Victoria through these three funds has improved economic and social outcomes for regional Victoria.

Conclusion

Program logic—an analysis approach that sets out the relationships between the objectives, resources, activities, outputs and outcomes a program is designed to achieve. Output—the services provided or goods produced by an entity. Outcome—the impacts on society of an organisation's outputs.

Having administered more than $1 billion in allocated funds since 2011, RDV still cannot reliably determine whether their grants have improved economic or social outcomes directly or indirectly, or whether any benefits have been sustained beyond the immediate injection of funds into a community. There is a high risk that its next evaluation, scheduled for 2021, also will not provide the answers to these questions.

RDV has improved its management of RJIF compared to previous funds, which was necessary given the serious issues identified in our two prior audits. To do so, it improved its organisational culture, increased its focus on program monitoring and better promoted the fund to intended applicants. However, several issues still hamper the successful management of RJIF:

• RJIF's evaluation framework has incomplete measures and targets and lacks a full program logic model and data plan.
• RDV advises it has spent less than the state's benchmark of 5 per cent of RJIF's funds on its administration, however it cannot substantiate this claim.
• RDV struggles to assess grants promptly, in part because it has not adopted a risk‑based approach to grant assessment.

RDV can also improve its transparency as it currently does not publish a comprehensive list of the grants awarded or complete information about how it assesses applications.

Findings

Achieving social and economic outcomes

RDV cannot show that its grant programs have improved social and economic outcomes because it has not successfully evaluated them. RDV has engaged multiple consultants to evaluate the various funds since 2009, each of which concluded that they could not systematically evaluate the funds due to issues including:

• inconsistencies between objectives and implementation
• a lack of targets and measures
• poor data quality
• challenges attributing change in social and economic indicators to the grant programs.

During our 2015 audit, RDV advised it would undertake a 'full-term evaluation' of RGF, however it was unable to do so due to various issues including a lack of appropriate data. Instead, RDV relied on a limited, case-study based evaluation of the $570 million RGF that could not demonstrate whether it had achieved its objectives.

RDV has developed an evaluation framework to assess RJIF outputs and outcomes and has dedicated resources to its implementation. This framework uses the dual approach of assessing changes in high-level indicators and undertaking a cost-benefit analysis (CBA) of individual RJIF-funded projects. It includes activities to mitigate the risk that RDV will be unable to evaluate RJIF, including data probes and an evaluation readiness report to determine whether individual projects can access necessary evaluation data. However, significant risks to a successful evaluation of RJIF remain:

• The evaluation framework uses an incomplete program logic approach, despite RDV committing to this in the RJIF business case. The program logic model does not clearly articulate how the fund's objectives link with its grant activity and evaluation framework. This has resulted in the fund's objectives having incomplete evaluation measures.
• RDV has not developed a consistent, comprehensive set of measures, with business rules for data capture. RDV therefore cannot be sure that its grant recipients will collect consistent data that will enable an overall evaluation of the fund.
• RDV has not set targets against which to measure the fund's success in delivering on its objectives.
• RDV plans to evaluate grants using a CBA approach to supplement its analysis of high-level indicators, but it has not outlined a methodology for how it will conduct this analysis to ensure consistency and enable an analysis of the whole fund.
• RDV has not developed a data plan—in line with its commitment in the RJIF business case—to ensure it consistently collects data about the grants and the baseline data needed to evaluate the fund.

Overall, RDV's evaluation framework carries significant risks that indicate RDV is unlikely to be able to undertake a localised, program specific or fund-level analysis of RJIF. Unless RDV addresses these issues, it will once again be unable to demonstrate the benefits of its regional grant fund and set up an evidence base for any future fund development.

Management of RJIF

RDV has undertaken a significant amount of work to improve its management of RJIF compared to earlier funds by focusing on cultural change and improving the content of its reporting framework. Timeliness issues have affected RDV's administration of RJIF and RDV cannot prove it has managed the fund's administration costs in line with state guidance.

Timeliness

RJIF grants support councils, businesses and other organisations to implement new projects to improve outcomes in regional Victoria. Many projects are time sensitive and rely on RJIF funding to proceed, but RDV is slow to process grants.

Overall, grants take nine to 12 months to assess and process, from expression of interest (EOI) submission to a funding agreement. RDV's reports show the assessment of a formal grant application and signing the agreement each take approximately three months. There are also additional steps prior to lodging a formal application, including:

• the applicant writing an EOI
• RDV assessing the EOI
• the applicant developing the formal application with RDV's support.

RDV advises that the time from formal application to approval is the best measure of performance because this is within their control. However, the full timeframe is relevant because a project cannot commence until RDV and the funded organisation sign the grant agreement, and the long lead-times caused significant dissatisfaction among grant recipients.

A key reason for this delay is that RDV applies the same grant assessment processes regardless of a grant's size or risk, despite significant variations in grant size—of the 694 grants approved to 31 December 2018, 266 were for less than $50 000 and five exceeded $20 million. RDV's processes, which assess some applications multiple times, and the need to seek the Minister for Regional Development's (the Minister) approval for all grants, increases the work required of applicants and RDV. In other areas of their regular operations, RDV regional directors have expenditure approval delegations of up to $150 000 while the Chief Executive Officer has $500 000, but they have no authority to approve RJIF grants regardless of value.

Adopting a risk‑based approach to assessing applications would enable RDV to improve its timeliness and streamline the application process for small grants, while dedicating more resources to higher-risk applications.

Grant administration costs

RDV was allocated 12.3 per cent of RJIF's funds to administer the grant program and some of its other service delivery activities. Department of Treasury and Finance (DTF) guidelines on grant administration recommend that agencies keep administration costs below 5 per cent. RDV advises that it meets this target but does not have documentation or data to support this claim.

In 2017, DTF recommended an interdepartmental capability review of RDV's grant management. DTF's view was that RDV needed to improve 'RJIF process and control processes to prevent funding shortfalls from occurring again' and 'to identify drivers of RDV's significant administration and project costs'. The government and RDV accepted this recommendation; however, RDV advised that the government did not approve the terms of reference and the review did not occur. This is a missed opportunity to evaluate RDV's funding and drive efficiency in its grant administration.

As part of its 2019 budget bid, RDV plans to separate its grant administration costs, capped at 5 per cent, from its other funding. Separating these costs would increase the transparency of its grant administration.

Data quality and documentation

Good quality data enables agencies to monitor and evaluate their performance against stated objectives. RDV uses a case management system—the Global Engagement Management System (GEMS)—to manage RJIF. RDV captures all grant-related activity in GEMS to provide data for internal monitoring and reporting. However, RDV does not have effective quality assurance processes to ensure it is accurately reporting on the outcomes of its grants.

We reviewed a selection of grants and identified several types of errors that have affected the quality of information RDV uses to monitor the program and report to government. One issue is that RDV incorrectly reports the number of jobs it expects to create through RJIF.

We sampled 10 cases and found errors in RDV's reporting of the number of jobs created in eight cases. After we raised this issue with RDV during the audit, RDV reviewed its data and increased the reported number of jobs RJIF created by 2 239. We retested the same sample after RDV's changes and found that RDV was still incorrectly reporting data on four of the eight cases originally identified.

We also found that RDV consistently documents its advice to the Minister and grant approvals, however, it does not consistently keep records for other aspects of its grant management, including:

• engagement with potential applicants, such as emails and meeting minutes
• grant application assessments and analyses of supporting documentation
• internal decisions to progress or not progress an application.

This lack of documentation reduces RDV's ability to effectively monitor RJIF's implementation and ensure the transparency and accountability of decision‑making.

An issue identified in the 2015 RGF audit was that RDV attributed the total number of jobs created and investment leveraged as RDV's achievement—even when its contribution was a small part of the overall project. We recommended that RDV develop guidelines for making appropriate adjustments to outcomes, to ensure accurate and fair reporting. RDV did not develop these guidelines and continues to attribute the full number of jobs created by a project to RJIF, regardless of the size of the grant relative to the project's total cost.

For example, RJIF contributed $1.5 million to a $55 million project—2.7 per cent of the total project cost—yet included all 357 expected jobs in RJIF output figures. This means RDV is overstating RJIF's impact. RDV advises it uses the phrase 'facilitated by RJIF' to communicate that multiple parties are involved in the project. However, this does not clearly communicate RDV's level of investment and its influence in creating the jobs for which it takes full credit in its internal and public reporting.

Distribution of RJIF grants

The most funded projects were those delivering election and budget commitments that were allocated to RJIF. Election commitment projects consumed $191 million and budget projects $101 million—58.5 per cent of the fund—to 31 December 2018.

Those grants were the key driver of 50 per cent of funds being delivered to regional city local government areas (LGA) to 31 December 2018—areas which may be more able to independently deliver growth through increasing populations, transport connections and business opportunities. In particular, 29.7 per cent of RJIF funding went to organisations in the Ballarat and Geelong LGAs. Most of those grants were to deliver election or budget commitments—81 per cent for Geelong and 91 per cent for Ballarat.

While a sizable part of RJIF was allocated to deliver election or budget commitments, we found no evidence that RDV uses available social and economic data—for example population and job growth figures—to target the remaining funds to where they might have the greatest impact.

RDV advises that targeting areas of greatest need in regional Victoria is not an objective of the fund. However, RJIF's business case identified the need for a nuanced approach as 'Regional Victoria has a number of pockets of entrenched disadvantage' and 'there is variation in how these challenges and opportunities impact upon different parts of country Victoria'.

RDV relies on its networks for information and government policy for direction. By omitting available data, it misses an opportunity to enhance its understanding of regional need. Incorporating data analysis would enable RDV to target grants to local needs and increase the likelihood of delivering strong outputs that align with the fund's objectives.

Transparency and accountability

Given that grants are public funds, it is important that the public has access to enough information to understand how the grants are allocated. We found two key issues that diminish the transparency and public accountability of RDV's management of RJIF.

Public information about awarded grants

At the commencement of our audit in September 2018—more than three years into RJIF—RDV did not publish a list of awarded grants on its website. RDV has since added a searchable list of grants. However, RDV has only published information about 335 grants to April 2019 despite the Minister having approved over 600 grants and it does not include the value of grants given to businesses. This omission contrasts with its earlier practice of publishing the value of RGF grants to businesses. RDV's annual report also includes an incomplete list of grants awarded in the relevant year and again does not include the value of grants to businesses.

RDV advises that it only publishes information about grants that the Minister has announced. It further advises that it does not include the value of grants to businesses to protect commercial information and avoid inadvertently establishing a benchmark for grants awarded to businesses.

The state's grant policy does not provide advice on disclosure. In the absence of state policy, the commonwealth guidelines are informative. They require disclosure unless specific reasons exist. These guidelines detail that if not fully disclosed, the agency must publish as much information as possible and the reasons why it did not make the remaining information public.

RDV's approach means the public and stakeholders cannot obtain a comprehensive understanding of the money distributed through RJIF. This approach diminishes the transparency of the fund and its accountability.

Making accountable decisions

RJIF has three programs. The Regional Infrastructure Fund invests in major infrastructure projects. The Regional Jobs Fund helps create and retain jobs. The Stronger Regional Communities Plan supports towns to attract families and young people. The three programs have 12 different streams.

RDV needs to accurately apply RJIF guidelines to ensure consistency and public accountability when it recommends grants for approval to the Minister. RDV assesses RJIF applications against the Regional Development Victoria Act 2002 (the RDV Act) and the relevant RJIF program and stream guidelines, which are available online. However, in recommending grants to the Minister, RDV deviated from the published guidelines in two key areas.

Firstly, while knowing that the government had allocated large projects to the Stronger Regional Communities Plan (the communities plan), RDV developed public guidelines capping the value of grants at $50 000—much less than the cost of the already committed projects. In its advice to government on the communities plan's guidelines, RDV did not include advice on this issue and did not make provisions to exceed the cap for projects allocated to the program. When RDV later assessed the applications for these grants, it did not advise the Minister that they would exceed the cap set out in the program's publicly available guidelines.

Based on RDV's guidelines, to the public it appears that $50 million worth of $50 000 grants were available, however this is not the case. Instead, $20.1 million was committed to 34 grants above $50 000 to deliver election or budget commitments—16 of which were over $500 000 and the largest of which was $3.4 million. RDV could have avoided this lack of clear and appropriate information by including the exceptions in the guidelines.

Secondly, RDV applies unwritten, subjective weightings in its assessment of grant applications that preference job creation and value for money outcomes. Applying weightings is relevant and can help guide decision-makers. However, by not documenting and publicising the weightings, RDV limits the transparency of its decision-making. Further, grant applicants cannot access all assessment criteria when preparing their submission, which potentially reduces their chance of success.

Recommendations

We recommend that the Department of Jobs, Precincts and Regions:

1. strengthen the Regional Jobs and Infrastructure Fund's evaluation framework to ensure it:

• has measures, supported by clear business rules and data definitions, against all the Regional Jobs and Infrastructure Fund's objectives
• supports consistent data collection from grant recipients to allow for the aggregation of results at state, local, stream and program levels
• details its cost-benefit analysis method (see Section 3.3)

2. improve data quality and the documentation of decision-making by:

• fully recording its engagement with stakeholders in its case management system, including calls, emails and meeting minutes
• using its case management system to record all grant documentation
• improving staff training on data quality, documenting decisions and using the Global Engagement Management System
• implementing a policy to guide data entry and documentation requirements
• implementing effective quality assurance processes, specifically for Regional Jobs and Infrastructure Fund output data (see Sections 2.3)

3. review and ensure that grant administration costs:

• have not exceeded 5 per cent of the Regional Jobs and Infrastructure Fund costs
• are effectively monitored and reported on in the future and remain below 5 percent of the fund's value (see Section 2.3)

4. streamline application and approval processes by adopting a risk-based approach (see Section 2.3)

5. improve transparency of grant programs by:

• publishing all assessment policies, criteria and weightings
• following its grant guidelines and processes, or reporting publicly when and why it diverts from its standard approach
• fairly and accurately reporting fund outputs and outcomes including the number of jobs the grant program creates prior to reporting (see Section 2.4)

6. maintain a list of all awarded grants including Regional Jobs and Infrastructure Fund and Regional Growth Fund that:

• is complete and up to date, by announcing all grants within 60 days of RDV and the recipient signing the grant agreement
• discloses information publicly, including the value of the grant awarded, unless the recipient could suffer commercial harm, determined in line with the considerations included in Victoria's Freedom of Information Act 1982
• clearly identifies rationale and evidence, on a case by case basis, for the decision of not releasing information about a grant (see Section 2.4)

7. use available social and economic data to understand the areas of greatest need and use this information when assessing grants, providing advice to government and undertaking strategic planning for any future fund (see Section 3.4).

Responses to recommendations

We have consulted with DJPR and we considered its views when reaching our audit conclusions. As required by section 16(3) of the Audit Act 1994, we gave a draft copy of this report to the agency and asked for its submissions or comments. We also provided a copy of the report to the Department of Premier and Cabinet (DPC).

The following is a summary of those responses. The full responses are included in Appendix A.

DJPR has accepted the seven recommendations from this audit and has undertaken to implement them by 31 December 2019.

Back to top

1.1 Regional Victoria's role in the economy and community

Regional Victoria is home to 24 per cent of the state's population. In 2017–18 it produced 19 per cent of the state's GRP despite challenges such as low population and job growth.

Since 2005–06, regional Victoria has experienced negative GRP growth in six years and growth of less than 1 per cent in another four. The gap in GRP per person between regional Victoria and metropolitan Melbourne is increasing, as shown in Figure 1A.

Figure 1A
Annual GRP growth per person in regional Victoria and metropolitan Melbourne

Source: VAGO, based on RDV's public information portal.

Victoria's Regional Statement 2015—the government's overarching policy to support regional Victoria—acknowledges that population growth can 'bring significant opportunities through new skills and economic activity'. Overall, regional Victoria has experienced significantly lower population growth than metropolitan Melbourne, as shown in Figure 1B.

Figure 1B
Population growth in regional Victoria and metropolitan Melbourne

Source: VAGO, based on RDV's public information portal.

1.2 Legislation and policy

Various pieces of legislation and policy have established Victoria's regional development grant programs over time, directed their management and provided guidance on better practice.

Regional Development Victoria Act 2002

RDV manages RJIF under the RDV Act. The government created each fund through amendments to the RDV Act—most recently, the Regional Development Victoria Amendment (Jobs and Infrastructure) Act 2015, which simultaneously established RJIF and abolished RGF.

Section 17 of the RDV Act sets out the five purposes for which grant funds can be used to support regional Victoria:

• to provide for better infrastructure, facilities and services
• to strengthen the economic, social and environmental base
• to create jobs and improve career opportunities
• to support the development and planning of local projects
• to support economic or community development.

Schedule 2 of the RDV Act lists the 48 LGAs and six alpine resorts in regional Victoria that can apply for RJIF grants.

The RDV Act requires the Minister's approval for all grants. The Treasurer must also approve grants of more than $5 million. The RDV Act has provisions for the Minister to delegate this power to RDV, however this has not occurred.

Policy guidance

DTF's Standing Directions of the Minister for Finance 2016 details agency responsibilities when distributing 'discretionary financial benefits'. Under these directions, agencies must maximise value for money and have effective and efficient administrative controls over their grant programs.

The standing directions require agencies to apply DTF's 2016 Investment principles for discretionary grants when developing and managing grant programs. These principles recommend agencies minimise administration costs, ensure that recipients' compliance costs are proportionate and that the program does not overreach into local governments' areas of responsibility or create an ongoing need for funding.

The Commonwealth Department of Finance produced the Commonwealth Grants Rules and Guidelines 2017 (the commonwealth guidelines), which—although only directly applicable to Commonwealth agencies—supply further guidance relating to the design, implementation, governance and transparency of grant programs.

1.3 Regional grant funds

Successive state governments have implemented initiatives to address the economic and social challenges facing regional Victoria. Figure 1C shows the regional grant programs administered by RDV since 1999.

Figure 1C
Key regional development grant programs

Note: RGF was an eight-year program that planned to distribute $1 billion in grants, but RJIF replaced it in 2015–16. At the time the government ceased RGF it had allocated up to $570 million for grants.

Source: VAGO, based on RDV information.

1.4 Regional Jobs and Infrastructure Fund

The government set up RJIF with the aim of 'growing jobs, building infrastructure and strengthening communities in regional Victoria'.

RJIF aims to deliver $500 million in grants over four years to 2018–19. Councils, businesses and non-government organisations in Victoria's 48 regional LGAs can apply for grants.

Figure 1D shows RJIF's three programs and their funding levels, objectives and 12 streams.

Figure 1D
RJIF's three programs and streams

Source: VAGO, based on RDV information.

RDV is responsible for assessing RJIF applications and making recommendations to the Minister for approval. Figure 1E shows RDV's assessment process for RJIF applications.

Figure 1E
Current RJIF application assessment process

 

Note: RDV recommends applications under the communities plan directly to the Minister without requiring Investment Committee approval.

Source: VAGO, based on RDV information.

Until 2017, RDV's Regional Infrastructure Development Committee evaluated Regional Infrastructure Fund applications. RDV chaired the committee, which included representatives from DPC, DTF and DJPR.

DJPR's Investment Committee evaluates applications for the Regional Jobs Fund (jobs fund). It includes senior public servants and independent experts responsible for assessing a range of investment opportunities. It meets regularly and is the final stage in deciding whether to recommend a grant to the Minister for consideration.

The Investment Committee has reviewed applications for both the Regional Infrastructure Fund and the jobs funds since 2017 to improve the consistency of decision-making between the two programs.

RDV refers applications for the communities plan directly to the Minister without the Investment Committee's consideration as these grants are smaller and focused on community building activities.

RDV uses the GEMS case management system to manage RJIF and to capture all information relating to a grant—enquiries and engagement with organisations, applications and supporting documents, assessments, ministerial briefs, grant agreements, implementation activities and payment. RDV's monitoring and reporting is based on the data entered in GEMS.

RDV plans to evaluate RJIF outcomes two years after the fund ends.

1.5 Roles and responsibilities

Many agencies play a part in social and economic development in regional Victoria.

Regional Development Victoria

RDV is the lead agency responsible for facilitating development across regional Victoria and has administered more than $1 billion through grant programs since 2011. RDV was part of the former Department of Economic Development, Jobs, Transport and Resources until 1 January 2019. Since then, RDV has been part of DJPR.

RDV manages RJIF by:

• providing public information about eligibility criteria and assessment processes
• working with potential applicants to develop grant proposals
• assessing and recommending grant applications to the Minister
• monitoring the delivery of funded projects
• reporting to government and the public on the fund's outputs
• monitoring and evaluating the fund's outcomes.

Councils

Councils have an obligation under the Local Government Act 1989 to promote the social, economic and environmental viability and sustainability of their municipal districts.

Regional councils can apply directly for RJIF grants. For example, RJIF partially funded the Geelong Library and Heritage Centre, alongside council and Commonwealth funding. Councils also support local businesses to apply for RJIF grants by facilitating engagement with RDV.

Other state entities

Multiple state entities, programs, and projects influence economic and social outcomes in regional Victoria, including:

• the Department of Environment, Land, Water and Planning, which manages eight Regional Growth Plans to provide direction for land use and development
• the Latrobe Valley Authority, which works with communities, businesses and relevant councils to increase jobs in the Latrobe Valley
• the Geelong Authority, which advises the Minister for Planning on strategies to attract investment to central Geelong to help create jobs and drive growth
• Regional Partnerships, which engage with their communities to identify local priorities and provide feedback to government for planning and budgeting.

1.6 Past audits

We have audited two of RDV's previous regional funds.

Management of the Provincial Victoria Growth Fund (2012)

The Provincial Victoria Growth Fund (PVGF) was the subject of our 2012 audit Management of the Provincial Victoria Growth Fund, which found an inadequate application of funding criteria and poor documentation of decisions. We also found RDV lacked a framework to evaluate PVGF and to incorporate the lessons learned into the design of RGF.

Regional Growth Fund: Outcomes and Learnings (2015)

Our 2015 audit Regional Growth Fund: Outcomes and Learnings examined RDV's management of RGF. It found that due to weaknesses in the fund's design and implementation, RDV could not show it achieved value for money or the program's objectives.

1.7 Why this audit is important

The ongoing development of Victoria's regional areas is important to the state's economic and social future. The regional economy plays a vital role in the overall Victorian economy.

Our audits of previous programs found deficiencies in their administration and made recommendations for the design of RJIF. It is important to assess if RDV is applying the lessons learned from its previous funds to improve its management of RJIF, and whether the state's current and historical investments in the regions is achieving the intended objectives.

1.8 What this audit examined and how

The audit evaluated whether state investment through various funds has improved outcomes in regional Victoria. We evaluated RDV's implementation of RJIF by focusing on its design, assessment processes, monitoring and evaluation. The audit also examined how RDV has evaluated earlier grant programs to ensure it is implementing better practice and helping drive regional development.

We engaged with five councils of varying sizes and regional stakeholders to gain added perspectives on RDV's effectiveness in administering RJIF.

We conducted our audit in accordance with Section 15 of the Audit Act 1994 and the Australian Auditing and Assurance Standards. The cost of this audit was $445 000.

1.9 Report structure

The remainder of this report is structured as follows:

• Part 2 reviews RDV's management of RJIF, including changes it has made and areas for further improvement.
• Part 3 evaluates whether the funds have improved outcomes in regional Victoria and highlights how RDV could use data to assess need and distribute grants.

Back to top

RDV's current regional grant fund is RJIF, which commenced in 2015 and will conclude in June 2019. We audited two previous funds and in 2015 made nine recommendations to RDV about the implementation of RJIF.

In this part we assess whether RDV's design of RJIF, decision-making, record‑keeping and public information have been effective, transparent and accountable.

2.1 Conclusion

RDV's delivery of RJIF is an improvement on its management of the earlier funds. RDV has driven cultural change, implemented better oversight mechanisms and increased the detail of its reporting.

However, RDV cannot show that it has managed the fund efficiently as it cannot demonstrate that it has kept RJIF administration costs—mainly staffing—to the state guideline of 5 per cent. RDV advises that this is because RDV staff undertake multiple functions like project management and maintaining its regional network as well as managing RJIF grants. Within this operating model, RDV did not establish processes to accurately track how much it spent managing RJIF.

RDV's assessment of RJIF applications has often been slow and the application process requires significant effort from applicants and RDV. The lack of a risk‑based approach to grant assessment is a key reason for the fund's high administrative burden.

Throughout RJIF's implementation, RDV has raised awareness of the fund by publicly promoting the fund and providing information about grant eligibility and processes. However, it could do more to improve RJIF's transparency by increasing the information it makes public about the grants awarded and RDV's assessment policies. This would enable stakeholders and the public to know where RDV has disbursed the money and help applicants understand how RDV assesses grant applications.

2.2 RJIF alignment with policy and better practice

Victoria and the Commonwealth have both issued guidance to agencies that manage grant programs. RJIF's design aligns with most of the guidance in state policy and some of the commonwealth guidelines.

Victorian guidelines—Investment principles for discretionary grants

DTF's Investment principles for discretionary grants include nine principles for grant management, including capping administrative costs at 5 per cent. Figure 2A shows how RDV has incorporated the principles into RJIF's design.

Figure 2A
RJIF compliance with DTF's Investment principles for discretionary grants

Principle	Assessment	VAGO analysis
Grants should only be used when they secure a government policy outcome.	✔	RJIF requires grants to align with government policy objectives.
Discretionary grants should not be used without first considering alternative policy mechanisms or existing grant programs.	✘	RDV implemented RJIF to deliver the government's 2014 election commitment of having a regional grant fund. The government approved both the RJIF business case and the RJIF guidelines. The business case did not consider or provide government with alternatives to achieve its policy objectives.
Grants should not be used with the principal objective of transferring revenue to local government.	✔	A range of entities, including councils, community organisations and businesses, received RJIF grants.
Care should be taken to ensure grants do not lead to state government overreach into local government's areas of responsibility, nor create an ongoing need for funding.	✔	RDV awards RJIF grants to specific projects and does not provide ongoing funding.
Grant programs should be designed to minimise administration costs.	✘	RDV cannot prove it has kept administrative costs below the recommended 5 per cent.
Where small grants are used, they should be administered by the organisation that is able to do so most efficiently.	✘	RDV assesses all grant applications and monitors project implementation in the same way regardless of grant size.
Accountability requirements imposed on grant recipients should be proportionate to risk.	✘	RDV does not have a risk-based policy to determine accountability requirements based on grant size or risk.
Grants can be disbursed by competitive, negotiated or allocated mechanisms.	✔	RDV uses an allocative grant model for RJIF, which is a supported model in the investment principles.
Better Grants by Design should be used to provide further guidance when designing and developing new grant programs.	✘	This document has the same principles as above but recommends capping administration costs at 10 per cent.

Note: Better Grants by Design is a DTF guide for agencies managing grant programs.

Source: VAGO.

Commonwealth guidelines—Commonwealth Grants Rules and Guidelines 2017

Competitive grant program—assessors compare applications against each other, and against the grant guidelines, to determine which applications offer the best outcomes and value for money. Allocative grant program—applications are assessed against the grant criteria only.

RDV could use the commonwealth guidelines to inform the design of any future regional fund, like the RJIF. The guidelines overlap with the Investment principles for discretionary grants in recommending agencies take a risk‑based approach and minimise the effort needed by all parties to process small grants.

RDV could use several added areas covered by the commonwealth guidelines to strengthen its management of RJIF:

• Competitive grants program—the guidelines recommend the use of a competitive grant process in most circumstances. RJIF is an allocative model that does not compare applications.
• Ministerial transparency—the guidelines require that Ministers tell Parliament when they award grants in their electorates or against departmental advice. This is not part of RJIF's legislation, policy, design or public reporting.

Implementing VAGO's 2015 recommendations

In 2015, we made nine recommendations to RDV to enhance the implementation of RJIF. RDV has taken steps to implement all the recommendations. However, based on our findings for this audit, there is a need for further action to fully implement the recommendations. Appendix B shows our assessment of RDV's performance in implementing the recommendations.

2.3 Implementing RJIF

Although not identical, RJIF's objectives and criteria are similar to those of the previous funds, including its focus on employment and social improvements, and the outputs and outcomes against which RDV measures performance.

In implementing RJIF, RDV has undertaken a significant amount of work to incorporate the recommendations of our 2015 audit. RDV has driven changes by implementing effective oversight mechanisms and focusing on cultural change.

RDV advises that it recognised the need to drive cultural change to improve performance, which it did by improving its staff engagement and willingness to examine problems. Examples include:

• engaging a consultant to evaluate cultural issues
• developing a cultural priority initiative plan to monitor the implementation of projects driving cultural change
• conducting culture change workshops
• staff engagement activities
• developing and monitoring a new people and culture framework.

Despite previous audits and multiple reviews, RJIF's delivery has not been without challenges and some issues with RDV's performance have continued into RJIF.

Governance

Our 2015 audit found significant issues with RDV's governance structure and its ability to report on the fund's progress and outcomes. In response, RDV has put in place a range of management frameworks and reporting processes.

Management structures

RDV uses a range of committees to manage RJIF and assess grants, prior to making recommendations to the Minister. Three key bodies oversee RJIF's implementation:

• The RJIF Implementation and Monitoring Committee is made up of executive level staff responsible for managing RJIF's delivery. This body has also monitored the implementation of our 2015 audit recommendations.
• The Regional Infrastructure Development Committee and Investment Committee comprise RDV staff, staff from other departments and non‑government experts who review applications.
• The Shark Tank is made up of RDV executives reviewing grant applications prior to referring them to the Investment Committee.

These committees have enabled RDV to manage the delivery of the fund by progressing grants systematically. RDV expects to have processed grants totalling the allocated $500 million by its close in June 2019.

Monitoring and reporting

RDV management rely on a range of reports to manage the fund's delivery. RDV's reporting framework outlines the range of reports produced, their time frames and recipients. RDV has adapted and improved some of these reports from the RGF, while implementing others for RJIF. The reporting framework includes:

• a range of monthly and quarterly reports designed for different audiences, such as RDV executives and the Minister
• a major projects report, providing updates on significant projects and new grants over $1 million
• tracking of projects linked to election or budget commitments
• a new projects report.

RDV has also integrated risk reporting into its reporting framework, including:

• a dashboard report of the number of projects with risks to delivery
• a case-based report about high-value projects and election commitments that face risks to delivery, which includes mitigation strategies
• a strategic risk register, which includes mitigation strategies.

Since the RGF, RDV has improved its reporting to provide more relevant and detailed information. RDV's reports now include information on:

• the value and number of grants distributed
• the number of jobs created
• the amount of outside investment leveraged by grants
• some measures of timeliness
• distribution of funds per person by LGA
• pipeline reporting to the regional level
• progress and risks to the delivery of key funded projects
• strategic risks to the fund.

Although the reports do allow RDV to oversee RJIF, issues with the accuracy of the data they include undermines their effectiveness. We discuss this issue in detail later in this part. Figure 2B describes one example of how RDV's data quality issues affect its reporting.

Figure 2B
Inaccurate data for jobs created

RDV's June 2018 quarterly report showed RJIF grants were expected to create 5 166 jobs. However, in the data extract used to produce the report, we found that 81 per cent of the projects did not list any expected jobs. We reviewed 10 of these grants and found that across eight cases there were 465 expected jobs—recorded in signed grant agreements—that were missing from the data extract and therefore not included in its reporting. RDV advises that in response it has undertaken work to encourage officers to correctly enter jobs data into GEMS. As a result, in preparing the September 2018 quarterly report RDV added 2 239 expected jobs, of which only 278 came from projects newly entered into the system. We reviewed the same 10 grants in February 2019 and found that four still had errors that affected RDV's reporting. This shows a lack of effective quality assurance of the data RDV relies on.

Source: VAGO, based on RDV reports.

RJIF grant administration costs

In August 2017, DTF recommended a joint capability review of RDV by DTF, DPC and DJPR arising from errors in the advice it provided to government during the transition from RGF to RJIF. DTF requested that the review evaluate RDV's fund management capabilities and 'significant administration and project costs'. The government and RDV agreed to the review. RDV advised that DPC and DTF developed the terms of reference in consultation with RDV, but the government did not approve these and consequently the review did not occur.

Government policy in 2015, at the time RDV prepared the business case, was for the maximum grant administration costs to be 10 per cent of the allocated funds. In 2016, DTF's guidelines revised this cap to 5 per cent.

RDV advises that its allocation from RJIF funds both RJIF's grant administration and other 'service delivery' activity. RDV defines grant administration as activity related directly to assessing and administering grants. It defines service delivery as including managing regional networks, supporting economic development, providing advice to the public on grant options and overseeing the implementation of election and budget commitment projects funded through RJIF.

RDV advises that while it will have expended $61.45 million by 30 June 2019, only 5 per cent of the total fund—$25 million—will have gone towards grant administration. However, RDV does not have adequate information to confirm the cost break up between the two categories. We reviewed the available information and could not determine with certainty what proportion of the $61.45 million RDV has spent on grant administration compared to service delivery.

As such, RDV cannot know if it has followed state guidelines on grant administration costs in administering RJIF. To provide greater clarity in future, RDV has used its 2019 budget bid to separate these costs, seeking 5 per cent of future funds for their administration and separate funding for its service delivery activities. RDV's 2015 RJIF business case did not separate the level of funding needed to administer the fund versus supporting RDV's other costs. If implemented, this would improve the transparency of the fund's administration costs and enable RDV to determine whether it is complying with state guidelines. RDV would need to develop effective systems to track its expenditure under this arrangement.

Transition to RJIF

RDV gave inadequate advice to government concerning the transition from RGF to RJIF, which created a temporary funding shortfall and consequent delays in grant approvals.

The previous government planned to distribute RGF as $1 billion over eight years, from 2011 to 2019. In 2015, RDV recommended that the new government reprioritise RGF's remaining $500 million to fund RJIF when it changed over between the two funds. However, RDV did not advise the government about $226 million in outstanding commitments for RGF projects.

As a result, RDV was unable to finalise RJIF funding agreements until the government finally decided to fund the full $500 million promised for RJIF grants alongside the committed RGF projects. While the government resolved this issue, RDV delayed progressing grants which affected applications in 2016 and 2017.

Timeliness and risk-based processing

RJIF has experienced significant timeliness issues, resulting in delayed project implementation and a high administration burden for RDV and applicants. RDV's lack of a risk-based grant processing approach and issues with finalising RJIF's funding arrangements are some causes of these delays. RDV officers, councils and other stakeholders have described a slow approval process for RJIF grants ranging from nine to 12 months.

Our analysis of available data found an average of:

• 314 days between an EOI submission and a signed grant agreement
• 202 days between an applicant lodging a formal application and a signed funding agreement.

Application assessment processes for each RJIF program involve multiple reviews prior to grant recommendations to the Minister. RDV applies each program's full process to all applications without applying a risk-based approach based on factors such as grant size.

In a risk-based approach, RDV would apply varying levels of assessment based on a project's risk profile, including the amount of funding requested or the project complexity. Factoring in risk would enable RDV to efficiently process low-risk grants while focusing resources on assessing applications for projects of higher risk. This approach would also align RJIF's management with the state policy guidance discussed earlier in this section.

RDV has identified that its processes contribute to the timeliness issue. An internal review team recommended steps to improve practices, including:

• setting time line targets for each grant process stage
• standardising the EOI process across funding streams
• developing a grants training program for newer staff.

RDV is implementing these changes in the fourth year of RJIF's administration. RDV needs to incorporate these and any further process improvements into any future fund to ensure it benefits from the efficiency gains.

RDV customer satisfaction results also show that timeliness has had a negative impact on the delivery of RJIF. Figure 2C shows that while overall dissatisfaction levels are low, timeliness is a key issue and driver of the overall dissatisfaction result.

Figure 2C
RJIF grant recipient client dissatisfaction

Source: VAGO, based on RDV data.

Timeliness was particularly an issue in years two and three of RJIF, which RDV attributes to its inability to finalise applications in 2016 and 2017 due to the temporary funding shortfall. This would have impacted the middle years of RJIF but grant recipients have expressed dissatisfaction with processing times in all years of the fund's management.

RDV's ability to adopt a risk-based approach is in part limited by the Minister's decision not to delegate the power to approve RJIF grants to RDV officers. As a result, RDV must prepare a brief to the Minster for each grant application, regardless of size or risk.

This contrasts with financial delegations in other areas of RDV. For example, regional directors have financial delegation up to $150 000 and the CEO up to $500 000. Delegating grant approval to these levels—or at a lower threshold such as $50 000—would enable RDV to manage low‑risk grants internally.

Figure 2D shows that 63.5 per cent of grants were for less than $150 000.

Figure 2D
Percentage of all RJIF grants up to 31 December 2018 by financial threshold

Source: VAGO, based on RDV data.

Adopting a risk-based approach would also enable RDV to increase its cost efficiency in processing grants. While it is not possible to determine the actual cost of administering RJIF, at the 5 per cent benchmark the average cost of processing a grant was $31 520 to 31 December 2018. This compares to a median grant value of $60 714 and the fact that 214 of the 694 grants were for less than the average cost. This indicates that a considerable proportion of grants had a high administrative cost relative to the value of the grant, which RDV should be able to reduce by using a risk-based approach to processing low‑value grants.

Value-for-money outcomes

RDV has developed measures to assess grants' value for money and includes this information where appropriate in:

• assessments of applications
• DJPR's Investment Committee consideration
• ministerial briefings
• monthly and quarterly reporting
• cabinet and budget briefs.

RDV produced a position statement outlining four criteria—equity, efficiency, effectiveness and economy—which RDV advises is its framework to deliver value for money. It includes ways RDV will assess RJIF's value for money outcomes at the fund and grant levels.

RDV's measure for equity is based on the geographical spread of grants and providing 'additional support in those areas under-represented'. RDV has not undertaken an analysis of available data to identify areas of highest need and target grants to drive equity. We discuss this further in Section 3.4. Instead, the delivery of election and budget commitments has driven grant distribution.

To measure effectiveness, RDV intends to evaluate whether approved grants are in line with RJIF guidelines, which include a criterion that they align with government objectives. In reviewing a selection of cases, we found projects delivering election or budget commitments under the communities plan that do not follow the program's guidelines. This is discussed in Section 2.4.

When assessing a grant, RDV uses two measures that link most closely to its goal of achieving economy and efficiency:

• "Dollars per job"—the ratio of a grant's value per job created.
• Total project cost to grant ratio—the total cost of the project compared to the grant.

The total project cost includes the RJIF grant and any other sources, including private sector or other government investment.

RDV advises that it may support higher total project cost to grant ratios to deliver projects in specific sectors or disadvantaged regions due to the perceived value of the project or potential jobs. However, RDV has not documented its assessment criteria under this approach, which has led to inconsistency in the value for money delivered by RJIF grants. This practice also reduces the fairness of RDV's assessment of applications and means the disbursement of funds is not transparent. Figure 2E shows that RDV has recommended grants with a large range of value for money outcomes.

Figure 2E
Range of RJIF value-for-money outcomes to 31 December 2018

Measure	Lowest value-for-money project	Highest value-for-money project	Overall
Dollars per job	$185 034	$833	$8 065 median per job
Total project cost to grant ratio	Fully funded	$70 total project to $1 of RJIF funding	$4 total project to $1 of RJIF funding

Note: The dollars-per-job calculation only includes jobs fund grants, as grants in other programs may not include jobs as an output.

Source: VAGO, based on RDV data.

Data quality and documentation

RDV has ongoing issues with its recording and reporting of data and the way it attributes jobs created by its grants. Multiple consultancies and audits dating back to 2009 have identified issues with RDV's data quality.

Inaccurate data and RDV's reporting

RDV's inconsistent record-keeping in GEMS undermines its ability to monitor and report accurately to the Minister and the public.

We reviewed a selection of cases and identified several types of errors in GEMS, which will affect the quality of the information RDV includes in its reports. The errors were:

• not recording the number of expected jobs created in the correct field
• incorrectly recording the total value of projects
• not recording that a grant is linked to an election commitment
• inconsistently using goods and services tax exclusive and inclusive figures for grant amounts
• attributing the grant to the incorrect LGA.

The most common and significant issue was RDV's reporting of the total 'jobs created' figure based on its extraction of data in GEMS as discussed in the section on monitoring and reporting.

The fact that we found these issues from reviewing only a limited selection of cases demonstrates poor data quality and a lack of quality assurance activities. Further organisation‑wide effort—including a review of all cases—is necessary to improve RDV's data quality and ensure that other errors are not present.

These data errors affect the quality of information RDV uses to recommend grants, monitor the program and report to government.

Estimating jobs created figures

RDV estimates the number of jobs its grants will create in its internal and external reporting, rather than relying on validated figures. RDV uses different methods to count the total number of jobs, as shown in Figure 2F.

Figure 2F
RDV counting jobs

Report	Jobs included	Source
Internal reporting	Direct jobs, indirect jobs and construction jobs	Grant application
Annual report	Direct jobs	Grant agreement
Evaluation	Direct jobs	Validated numbers

Source: VAGO, based on RDV information.

RDV advised that it engages with organisations while discussing grant applications to test the accuracy of the number of claimed jobs, but we found no record of this in the cases we reviewed.

Consistent with its practice as part of RGF reporting and in contrast to our recommendation from the 2015 audit, RDV continues to attribute the full number of jobs created by a project to RJIF, regardless of the size of the grant relative to the project's total cost.

For example, RJIF contributed $1.5 million to a $55 million project—2.7 per cent of the total project cost—yet included all 357 expected jobs in RJIF output figures. This overstates RDV's influence in creating jobs in regional Victoria. RDV advises it uses the phrase 'facilitated by RJIF' to communicate that multiple parties are involved in the project. However, this does not clearly communicate RDV's level of investment and its influence in creating the jobs for which it takes full credit in its internal and public reporting.

Shortly after project completion, RDV verifies if the grant recipient did create the promised jobs. A full evaluation of RJIF's outcomes would provide RDV with an opportunity to confirm whether grant recipients had sustained the jobs created through RJIF projects. RDV could then publicly report against these figures to demonstrate the outcomes of its funds.

Leveraging external investment

RDV has overstated the amount of other funds RJIF grants have leveraged in multiple budget briefs, most recently in its 2019 bid to establish a new fund. It did this even after we advised RDV that this issue had occurred in its briefings to government in 2017.

RDV calculates RJIF's 'funding ratio' by comparing the value of RJIF grants to the total value of projects, which includes the RJIF grant. In this metric, the total project cost compared to RJIF grant is approximately four to one. RDV advised government that it has facilitated 'a total investment of $4.16 for every dollar of RJIF grant funding'. In fact, based on that calculation it had leveraged $3.16 of private or other government funding for each RJIF dollar.

Documenting grant decision-making

RDV inconsistently documents its decision-making, which reduces accountability and makes evaluation challenging.

In late 2018, an internal RDV review identified that a lack of policy guidance for staff on how to document grants resulted in 'inconsistent practices which impacts the timely and efficient storage and retrieval of key documentation'.

The different assessment processes and documentation RDV has used over time and across programs to deliver RJIF compound the issue of inconsistency. This will make aggregating and comparing grants more difficult when it evaluates the fund. The consultants that attempted to evaluate RGF identified this barrier to evaluation.

We reviewed a selection of files—some in GEMS and other documentation provided by RDV—for 40 grants valued above $2 million and found inconsistencies in RDV's documentation of its evaluation process. Based on the available documents, it was not possible to assess how RDV had critically evaluated the application and supporting documentation. The specific issues identified were:

• not independently verifying information provided by the applicant such as the claimed economic impacts or building costs
• not documenting its analysis of information provided by the applicant
• copying content from the application into its internal documentation and Ministerial briefs.

We also reviewed a random selection of 40 grants in GEMS to test whether RDV documented the pre-application stage. Doing this was one of the recommendations of our 2015 audit. We found RDV had recorded:

• engagement with an organisation prior to application—for example emails, meetings, phone calls—in only 30 per cent of cases
• internal assessments of EOIs and supporting documentation in only 20 per cent of cases.

From 2017, RDV implemented meetings of senior officers to review applications prior to providing them to the Investment Committee. RDV refers to these meetings as the 'Shark Tank' and advises that they improve the consistency of applications across different offices and the completeness of the supporting information. RDV did not record any meeting minutes for these discussions until late 2018, so it is difficult for RDV to assess the impact this practice has on its management of RJIF.

RDV advised it did not record the meeting outcomes because it did not consider them as a decision point. However, after the commencement of this audit RDV did start documenting these minutes. Keeping records of meetings in which RDV considered grants would help retain knowledge within RDV.

2.4 Transparency and accountability

Ensuring transparency and accountability is essential if the public is to have confidence in RDV's administration of grant funds. While RDV publishes information about the fund, it does not provide a complete report of the grants funded or how it processes grant applications.

Public information about the fund

RDV delivers information about RJIF through its website, offices, regional networks and Regional Partnerships. The publicly available information allows potential applicants to understand their basic eligibility and directs them to RDV's regional offices to obtain detailed information and support to develop applications.

Since 2016, RDV has implemented the Regional Partnerships, which enables it to engage with a wide range of regional stakeholders, identify regional needs, promote RJIF and identify grant opportunities. GEMS data shows that two of the 694 grants approved to 31 December 2018 are linked to the Regional Partnerships.

RDV established a goal to equitably distribute the RJIF, which it plans to assess in part by the availability of public information and its promotion of the fund. RDV customer satisfaction results over the past four years show that more than 90 per cent of grant recipients were satisfied with the accessibility and clarity of the information about RJIF. This indicates success in this area, but only based on information from stakeholders who obtained grants. Our engagement with councils and stakeholders reflected this result as they advised us that RDV's regional offices maintain positive and constructive relationships with them.

Public reporting of awarded grants

The disclosure of awarded grants enables the public and stakeholders to understand how departments and Ministers are distributing state funds. RDV does not provide comprehensive public information about the grants it manages, including who received them, for what project and at what value. RDV also does not provide public information about the grants awarded under previous funds.

Victoria does not have a policy on grant disclosure, and RDV practices are consistent with the information other government grant programs make public. However, this does not provide transparency to the public. In the absence of state guidance, RDV could use the commonwealth guidelines, which state that agencies must publish as much information as legally possible about grants. If an agency withholds information, it must publicly declare this and the reason why. The guidelines also advise agencies to take all reasonable steps to ensure that future grant agreements contain provisions that do not prevent the disclosure of information.

RDV's website and annual report

RDV's website has provided varying amounts of information over time about grants awarded under its regional funds. Under RGF, RDV published a list of grants and the value awarded. However, this practice has changed and RDV no longer publishes information about RGF grants on its website.

Until October 2018, RDV did not publish a list of RJIF grants on its website. A list is now available, but it is incomplete because RDV only publishes information about grants the Minister has announced. As at April 2019, the website listed 335 of the more than 600 grants the Minister has approved. Appendix C shows the value of RJIF and RGF grants to each LGA.

RDV also does not publish the value of grants to businesses under the jobs fund. RDV advises that it has an unwritten policy to withhold the value of these grants to maintain commercial confidentiality and avoid establishing a benchmark for grants, which could interfere with providing funding based on a business's need. Although this is a risk, it is one managed by other departments in their tender and contracting operations to ensure transparency and should be manageable in the context of a grant fund.

RDV's annual report also does not detail all grants awarded within the year and does not disclose the value of grants to businesses. As it only includes information for one year, the reports are more limited than RDV's website in providing transparency.

Transparency of grant assessment

RDV usually applies consistent processes to assessing grants within each RJIF program. We did not identify significant variation in how RDV assessed applications related to election or budget commitments compared to other applications in the selection of cases we reviewed.

However, we did find two cases in which RDV deviated from its regular practices:

• In 2018, RDV processed a grant within one day, but due to subsequent developments RDV did not disburse the grant.
• In October 2018, RDV fast-tracked a grant by obtaining 'out of session' approval from the Investment Committee, which ensured that the Minister could approve and announce the grant prior to the 2018 election caretaker period. The grant was a 2014 election commitment, which RDV had been processing since then without finalising the grant prior to 2018.

In both cases, RDV diminished the transparency and accountability of the fund by diverting from its regular processes.

Undocumented assessment policies

RDV applies weightings—which it has not documented in policy or publicised—when it evaluates applications:

• RDV's assessment favours applications that create jobs, whereas the jobs fund's guidelines value both job creation and retention. RDV's preference potentially disadvantages small LGAs where job retention is more important and likely than job creation. Small councils confirmed this in our discussions with them.
• RDV has applied two ways to assess an application's value-for-money—focusing on grants that deliver more jobs for the grant money invested or that leverage more funds from other sources.

The use of these policies can aid decision-makers to better assess and rank applications. However, by not documenting and publicising the policies, RDV risks being inconsistent and is not transparent.

Stronger Regional Communities Plan funding

RDV developed the RJIF business case, including the communities plan after the 2014 election. In the business case, RDV identified 10 election commitments valued between $100 000 and $8 million that the government had allocated to the communities plan to deliver. After the government approved the business case, RDV developed guidelines for the communities plan including a cap of $50 000 per grant, which the government approved as RDV advised that the limit would 'allow for sufficient projects to be supported'.

RDV developed the guidelines and continues to publish them, despite knowing that projects connected to election commitments and subsequent budget commitments would exceed the grant cap. RDV did not develop exceptions within the guidelines to allow for these projects. Since the program started, RDV has assessed 34 grants delivering election or budget commitments that exceed the program's $50 000 cap. RDV's grant recommendations to the Minister did not advise her that the grants did not comply with the program's public guidelines. This diminishes the transparency of the program as there is one set of guidelines for public applications and another, unwritten set for applications tied to election commitments.

Figure 2G shows an example of a grant funded above the $50 000 cap.

Figure 2G
The communities plan grant approved above $50 000

In January 2016, the Minister approved the allocation of $1.4 million from the communities plan to Sport and Recreation Victoria—then part of the Department of Health and Human Services—to support the delivery of a sport tournament. RDV officers described the allocation as a 'grant'. However, unlike other grants, the organisation did not submit a formal application and RDV did not document the reason it did not receive an application. The briefing to the Minister recommended approval on the basis that the grant was an election commitment and that it 'aligns with the objectives and requirements' of the communities plan. However, the briefing identified employment and business opportunities as objectives of the communities plan, which is inconsistent with the public guidelines. This advice resulted in the Minister approving a grant that had limited alignment with the program's objectives and that was above the funding cap.

Source: VAGO, based on RDV information.

Back to top

In this part, we assess whether RDV can demonstrate the outcomes the grants it has administered have achieved in improving outcomes in regional Victoria.

We also highlight the way RDV could use available social and economic data to target fund expenditure to drive better outcomes for regional Victorians.

3.1 Conclusion

Despite the significant investment to date, RDV cannot demonstrate its impact on social and economic outcomes in regional Victoria. Consultants engaged to assess PVGF and RGF outcomes have been unable to comprehensively evaluate those funds due to deficiencies in their design and inconsistent record-keeping. Due to this lack of effective evaluation, RDV did not have an evidence base to inform the design of the RJIF.

In response to these historical issues, RDV has put effort into improving its monitoring and reporting of RJIF and developing RJIF's evaluation framework. However, significant risk remains that it will not be able to evaluate RJIF's outcomes after the fund finishes in 2019. These risks include:

• the evaluation framework not incorporating a complete program logic approach and failing to align the fund's objectives with its activities and evaluation measures
• a lack of effective measures for some of RJIF's objectives
• a lack of targets for all of RJIF's objectives
• not having set criteria for how it will undertake the CBA analyses, which RDV is relying on as a key element of its fund-level evaluation framework
• not having developed a data plan to specify and control the data it needs from grant recipients to ensure it will be able to aggregate the information to conduct an analysis at the fund-level.

Unless RDV addresses these challenges, it may be again unable to fully evaluate RJIF and assess whether it has achieved its objectives.

RDV has also not undertaken a local or regional level needs analysis using social and economic data to consider how to best target its efforts and grants to maximise RJIF's intended outcomes. RDV has not linked the distribution of RJIF funds to any identified areas—for example, LGAs or industries—and instead over half its funds have been given to projects delivering election or budget commitments. A thorough analysis of available data, along with RDV's regional networks and greater authority to distribute funds based on need, would enable RDV to target grants to areas and projects that help achieve the greatest impact in regional Victoria.

3.2 Outcomes of RDV's previous funds

Our two audits of RDV's previous funds and several consultant reports commissioned by RDV have identified persistent issues affecting evaluation. RDV could not evaluate PVGF and RGF due to failures in program design, data retention and documentation. These issues persisted from PVGF into RGF, and some remain in RJIF despite RDV, consultants and VAGO identifying the issues as early as 2009.

Previous attempts at evaluation

In 2009 RDV engaged a consultant to evaluate PVGF and the Regional Infrastructure Development Fund (RIDF). It found that PVGF had generated some economic and social improvements, but there were challenges in attributing outcomes to the funds. The consultant found that issues with the alignment of projects to the fund's objectives, a lack of measures, and data quality issues affected its ability to evaluate PVGF's outcomes. RDV did not undertake any further analysis of PVGF outcomes. The evaluation of RIDF found issues with data quality, a lack of baseline data and challenges in attributing outcomes to the fund.

RDV engaged a consultant in 2013 to undertake an interim evaluation of RGF, but RDV was not satisfied with the report. During our 2015 audit, RDV committed to undertaking a comprehensive review of RGF after the fund finished and engaged another consultant in 2016 to undertake this review. The consultant concluded it could not comprehensively evaluate RGF due to the poor design of the fund's evaluation framework and the lack of data—issues similar to those that impacted the PVGF evaluation.

The consultant recommended instead that RDV evaluate a selection of grants to identify the lessons it could learn from RGF. RDV accepted this advice and engaged another consultant in 2017 to undertake this sample-based evaluation. This evaluation concluded that the grants usually achieved their output goals and recommended that RDV measure a project's success using a cost-benefit analysis. This review could not provide a comprehensive, fund-level assessment of RGF's impact in regional Victoria.

As a result, RDV has been unable to fully evaluate the outcomes of PVGF and RGF and cannot demonstrate the outcomes of more than $600 million worth of investment in regional Victoria. This is a missed opportunity for RDV to assess how and in what form grants can improve regional Victorian outcomes, which it could use to design future funds.

3.3 RJIF's outcomes

Good program design includes clear objectives, measures, targets and methods to conduct the evaluation. For RJIF to successfully deliver outcomes—and prove its success—its design and implementation needs these key components.

Similar to our experience at the time of the RGF audit, many of RJIF's projects are still ongoing, so it is difficult to evaluate all its outcomes. However, the consistent lack of a detailed evaluation framework is a concern.

It is possible to apply a program logic model to assess whether RJIF's design and implementation mean it is likely to achieve its intended outputs and outcomes, and whether RDV will be able to evaluate them. This approach requires us to consider:

• RJIF's strategic objectives
• the resources available to RDV to achieve those objectives
• the outputs that RJIF has delivered and how they align with its objectives
• the outcomes that RJIF will deliver and how RDV will evaluate them.

RJIF's strategic objectives

Similar to earlier funds, the government established RJIF to deliver social and economic outcomes in regional Victoria, including enhancing job opportunities, improving the economic environment and increasing the liveability of regional Victoria. RJIF's three programs have individual objectives which align with the fund's overarching objectives.

RDV developed a business case for RJIF, outlining its objectives, how it would divide the funds between the programs and how it would evaluate the outcomes of the fund. However, as the government had already committed to a fund, the business case did not consider different options to achieve its goals. As RDV has been unable to evaluate its former funds, the business case could not draw on an evidence base that linked RJIF's design with successful practices identified in the previous funds.

Needs analysis

Program logic design begins with a needs analysis—identifying who should benefit from the program in order to best achieve the program's objectives. When establishing RJIF, RDV did not conduct such an analysis, which would rely on inputs such as population and jobs data, to identify areas or project types that would produce the greatest positive impact. A thorough needs analysis would enable the effective targeting of RJIF's finite resources.

A needs analysis is particularly important considering the challenge RJIF has in creating a statewide impact. The relatively small size of the fund and the many issues RJIF seeks to address, such as jobs and population growth in regional areas, mean it is unlikely that RDV can attribute changes to state-level measures, like unemployment, directly to RJIF. The large number and different types of projects supported through RJIF increases the complexity of this attribution. Therefore, targeting grants to areas of identified need, and evaluating those interventions collectively, may enable RJIF to demonstrate real impact, for example in a particular location or in a specific occupation of high need.

Resources available to RDV

Of the $500 million allocated to RJIF, $354 million was committed before it began considering grant applications from the public, meaning RDV was unable to target the full $500 million to activities that best met its strategic objectives.

The pre-committed funds included funding for RDV and the delivery of election or budget commitments the government allocated to RDV to deliver through RJIF. For example, the government made an election commitment of $5.2 million to Ararat Rural City Council to redevelop its art precinct and decided to fund this project from RJIF. Only $146 million over four years was fully available to RDV to allocate to applications for RJIF grants.

Figure 3A shows the allocation of RJIF funds.

Figure 3A
RJIF fund allocation to 31 December 2018

Source: VAGO, based on RDV data.

What outputs has RJIF delivered?

RDV reporting indicates that at the end of RJIF it will have supported a wide range of infrastructure projects, including airport, stadium, and streetscape upgrades and social improvement grants. RJIF has also reported it has contributed to the creation of more than 7 000 jobs. RDV currently reports on RJIF activity and outputs, as shown in Figure 3B.

Figure 3B
RJIF activity and outputs at 31 December 2018

Source: VAGO, based on RDV data.

RDV requires grant recipients to identify the outputs they will deliver and includes them in the grant agreement. Payment of funds is contingent on the organisation completing milestones and submitting evidence of agreed milestones to RDV.

For infrastructure projects, milestones link to the completion of the agreed project which RDV can readily verify. For jobs fund projects, RDV links payments to the delivery of job-creating projects—for example expanding a factory—and the establishment of the jobs the organisation agrees to create. RDV's verification involves various methods including receiving statutory declarations and independent payroll verification, depending on the size of the grant.

From a reporting perspective, RJIF jobs-creation projects' outputs are standardised, which allows RDV to aggregate and evaluate them. For social engagement projects, however, the agreed outputs are difficult to measure. These measures can be specific to a project—for example, increasing participation at an arts precinct—so it may be challenging to aggregate outputs across the fund, or to draw conclusions based on a small sample size.

Evaluating RJIF's outcomes

As discussed, RDV has been unable to show the outcomes of previous funds, so it is essential it can demonstrate the value of RJIF and establish an evidence base for future fund design.

RDV has established a framework of documents that outline RJIF's objectives, outputs and outcomes and how it plans to evaluate RJIF. We refer to the collection of documents as the 'evaluation framework'. It includes:

• the Implementation and Outcomes Framework, which outlines the key objectives of the fund, how RDV will assess grant applications and refers to the Monitoring and Evaluation Framework
• output and outcome measures and performance indicators, and notes the use of a CBA approach to evaluate the RJIF, but does not directly link the indicators to the fund's five objectives
• the Monitoring and Evaluation Framework, which outlines the method and time lines for RJIF's evaluation. This document is a single PowerPoint slide and does not provide detailed evaluation methodologies.

RJIF's business case promised a detailed evaluation methodology, which the government accepted as part of agreeing to the fund's business case. It included:

• a program logic model to inform the evaluation design
• assessing outcomes at state and local levels
• undertaking evaluation across short, medium and long-term time frames
• developing a data plan to support the evaluation.

RDV's evaluation planning to date does not contain this level of detail. Although RDV plans to evaluate RJIF's outcomes at the state level, there is no clear plan of how it would do so at the local level, or its methods for collecting data to enable an aggregation of results across the whole fund. RDV's evaluation plan also does not detail how RDV will evaluate the fund's short, medium or long-term outcomes.

Figure 3C shows the gaps in the existing evaluation framework compared to the business plan.

Figure 3C
Comparison of RJIF evaluation methodology noted in the business case with RJIF's current evaluation plan

RJIF business case evaluation plan	VAGO's analysis of RJIF's evaluation framework
A program logic model (incorporating an investment logic model) which sets out the goals, objectives and targeted actions of RJIF and demonstrates how specific activities contribute to short, medium and long-term program outcomes.	The evaluation framework does not fully implement a program logic model. It does not identify how grants contribute to the fund's objectives or clearly link the fund's objectives to the measures RDV has included in its evaluation framework. There are no separate short, medium or long-term outcomes.
A set of key performance measures derived from the program logic model, which will assist in measuring the progress of RJIF in meeting its goals and objectives.	The framework identifies high-level measures and a plan to use CBAs to evaluate the fund. However, RDV has not linked these to fund objectives. As a result, some objectives do not have measures or have measures that are only loosely attributable to the fund. While RDV plan to use CBA analysis to fill in these gaps, the documentation lacks enough detail to provide assurance that this approach will be successful.
A program map that provides a snapshot of the spread of programs within RJIF and demonstrates alignment between specific activities, key performance measures and RJIF outcomes.	The framework does not have a program map linking outcomes, activities and measures. At best, RDV's reporting shows allocation at the LGA level and on a per person basis. However, this report does not link funded projects to the specific objectives and measures outlined in the evaluation framework.
A data collection, reporting and storage plan to ensure RJIF reporting and evaluation occurs in a structured, consistent and systematic way.	The framework does not have a data management plan. RDV therefore risks having incomplete, inconsistent and inaccurate data that will prevent it from evaluating RJIF.

Source: VAGO, based on RJIF's business plan developed by RDV.

RDV refined the objectives highlighted in the business case into five specific objectives in RJIF's guidelines. RDV intends to measure RJIF's outcomes against those objectives in two ways—assessing its impact on high-level indicators and undertaking CBAs of individual grants that it will aggregate at the fund, program and stream levels.

RDV's evaluation framework identifies 14 high-level outcome measures it will use to evaluate RJIF against its five objectives. However, the evaluation framework does not align the indicators and the objectives.

We have aligned these in Figure 3D and, in doing so, have identified gaps and attribution issues that would lead to an incomplete evaluation. We have grouped objectives two and three together because they both focus on improving business conditions and RDV could assess them using the same indicators.

Figure 3D
Measuring RJIF's outcomes by high-level indicators

From RDV evaluation framework				VAGO analysis
Objective	Indicator	Data source	Attribution to RJIF	Comment
Create jobs of the future and diversify the regional employment base.	Number of direct, indirect, construction and retained jobs	Review jobs figures after completion of project	High	These are relevant indicators that identify the impact of RJIF. Information sources exist that will enable RDV to evaluate these indicators. However, RDV has not developed a methodology for its CBA implementation.
	Number of jobs created in high‑growth and emerging sectors	Analysis of jobs created in RJIF	High
	Unemployment rate in regional Victoria	Government data	Low
	Change in average regional wage	Government data	Low
	Average Benefit: Cost ratio at the fund, program and stream levels	RDV cost-benefit analysis	High
Improve productivity and enhance the long‑term competitiveness of regional Victoria through innovation and a transition to new growth opportunities. Create the conditions for business growth by enhancing workforce skills, providing enabling economic infrastructure and facilitating expansion into new markets.	Change in Gross Value Add in regional Victoria	Government data	Low	These are relevant indicators that identify the impact of RJIF on most components of the objectives. However, it is not clear how RDV will evaluate competitiveness or enhanced workforce skills using these measures. Information sources exist that will enable RDV to evaluate these indicators. However, RDV has not developed a methodology for its CBA implementation.
	Change in business diversity in regional Victoria	Government data	Low
	Change in turnover of RJIF‑supported businesses	RJIF-funded businesses' data	Medium
	Change in export sales of RJIF‑supported businesses	RJIF-funded businesses' data	High
	Level of investment in RJIF‑supported projects	RJIF-funded businesses' data	High
	Change in annual visitor numbers	Government data	Low
	Change in total number of visitor nights	Government data	Low
	Average Benefit: Cost ratio at the fund, program and stream levels	RDV cost-benefit analysis	High
Improve the liveability of our cities, centres and towns to attract and retain families and young people to live and work.	Change in net migration	Government data	Low	As there is only a limited attribution between the objective and the indicators, RDV may be unable to conclude whether it has achieved this objective.
	Change in life satisfaction index	Government data	Low
	Change in community wellbeing index	Government data	Low
Enhance community capacity through collaboration, leadership development and regional planning.	No indicators	No indicators	–	RDV will be unable to evaluate this objective given the lack of measures.

Source: VAGO, based on RDV's evaluation framework.

Challenges to attribution

Attributing outcomes to RJIF is challenging as its funding is small in comparison to the regional population and economy—RJIF's average annual spend of $125 million represents just 0.16 per cent of regional Victoria's GRP for 2018. External factors, such as migration levels and Australia's overall economic performance, also influence outcomes.

As such, RDV's state-level measures against these outcomes can have limited attribution to RJIF, leaving RDV at risk that it will be unable to effectively evaluate achievement. RDV recognises this issue and advise that CBAs will instead provide more meaningful information for evaluating RJIF's achievements.

RDV's evaluation framework identifies that the CBAs should enable it to:

• assess the relative value of the benefits and costs of RJIF, wherever possible
• assess whether RJIF investment has been in the right areas
• inform future decision-making in investment and program development.

However, RDV has not developed a policy to ensure it applies a consistent CBA methodology—for example, evaluation criteria and data used—in its grant evaluation framework. Consequently, it cannot be sure that it will be able to aggregate the grants' outcomes to provide fund-level results.

Further, despite its stated reliance on CBAs to evaluate RJIF, RDV itself has not undertaken test-CBA evaluations of completed grants to assess the effectiveness of its approach. Creative Victoria received RJIF funds for the Regional Arts Program, a program that also required evaluation under DTF's 'lapsing program' requirements. As part of that evaluation, the contractors Creative Victoria hired completed CBAs of the seven individual projects within the Regional Arts Program. However, RDV does not have access to the methodology, assumptions made, or data used by the consultant in those CBAs. As such, RDV cannot replicate the method across other similar RJIF-funded projects and may not be able to aggregate these CBA results with those for other projects.

Lapsing program—a program where government provided funding for a specified period and which will conclude by the end of the financial year.

Measuring outputs

RDV plans to measure 13 outputs of RJIF, which we have grouped into three categories:

• process measures—number of grants approved, total funds distributed, number of successfully completed projects, percentage of cases that do not proceed to application, percentage of cases withdrawn or rejected, client satisfaction survey
• value for money—total funds leveraged by RJIF, funding ratio
• distribution—per person funding by LGA, percentage of Regional Partnership projects funded through RJIF, total funds to regional cities, total funds to rural councils, number of LGAs that received a grant.

However, these measures are incomplete because they do not contain outputs against RJIF's productivity objectives or the efficiency criteria RDV has established to assess value for money.

The outputs also lack targets to determine whether RDV has been successful in delivering RJIF. For example, RDV will measure the value of grants to each LGA, but does not have a target to show what amount represents a good outcome. At a more strategic level, many of the output measures also fail to measure performance. For example, counting the number of grants approved will not enable RDV to determine whether many small grants or fewer large grants are better for regional Victoria.

Evaluation risk mitigation

RDV has been unable to evaluate previous funds due to deficiencies in their administration and particularly data collection. RDV is aware of this issue and is taking steps to mitigate the risk of being unable to evaluate RJIF, but several risks remain. Figure 3E describes the key mitigation activities devised by RDV to reduce the risk of being unable to evaluate RJIF's outcomes.

Figure 3E
RJIF evaluation risk mitigation approach

Item	RDV description	VAGO analysis
Data probes	Testing/sampling grants to determine whether they have measurable objectives and baseline data and if RDV has collected the necessary data during project delivery	If managed effectively, the data probes should assure RDV that it will have enough, aggregable data to undertake a fund-level analysis. However, several key issues remain that can limit the probes in achieving their intended goals: RDV is not defining the data it requires from grant recipients, so similar projects may collect different data RDV has no control over the quality of data collected RDV hasn't linked the data collected to the fund's objectives to show how it is relevant to assessing RJIF's overall outcomes.
Project profiles	Detailed examinations of individual projects	These should provide case-study and qualitative analysis of grants but does not guarantee a state-level evaluation.
Evaluation Readiness Review	Scheduled near the conclusion of RJIF to test whether RDV has collected enough data and information prior to the commissioning of RJIF's overall evaluation	While the review is of benefit, the timing assumes that RDV has successfully implemented RJIF's evaluation strategy and collected the required data. If the review finds that the fund is not ready for evaluation, it is not clear how RDV would resolve this issue after the fund has concluded. A clearly articulated data plan would have mitigated this risk.

Source: VAGO, based on RDV information.

Strategic evaluation of RJIF

RDV's evaluation plan focuses on the delivery of RJIF's outputs and outcomes and does not consider its overall strategy or structure. RJIF's evaluation could provide evidence for future fund design by considering:

• whether a grant scheme is the best way to achieve outcomes in regional Victoria as opposed to alternate methods, such as direct investment in projects or ongoing programs
• whether an allocative or competitive grant program is the best design choice
• which strategic approach to distributing funds achieves the best outcomes—for example, equal distribution or targeting disadvantaged areas
• what grant size achieves the best outcomes—for example, a few large or many small grants
• which outputs best link to the outcomes the fund wants to achieve, such as job creation or infrastructure investment
• the scale of investment needed to influence outcomes in regional Victoria.

Answering these questions is fundamental to inform future investment supporting regional Victoria's development.

3.4 Using data to plan grant distribution

RJIF's business case highlights that 'Regional Victoria has a number of pockets of entrenched disadvantage, with a high demand for services' and that 'this place‑based disadvantage reduces the growth potential of regions'. In providing this advice, RDV recognised that its interventions needed to address specific local issues and that it needed to undertake evidence-based decision-making to prioritise investment.

The government consequently approved RJIF with the objectives of improving job, economic and social outcomes in regional Victoria. RJIF's objectives and guidelines do not explicitly identify reducing social or economic disadvantage as a goal. However, targeting resources to areas of disadvantage would support the overall objective.

RDV uses its networks, Regional Partnerships and officers' experience to understand the challenges facing regional Victoria, giving it context to assess grant applications. Using data analysis would bring an additional perspective and assist RDV to target funds to areas where they could make the most impact. Linking RDV's data with social and economic indicators—and assessing change over time—could form a key part of the RJIF's formal evaluation.

To test whether RDV has linked RJIF's grants with local needs, we compared the distribution of grants to datasets focused on jobs and social outcomes. This type of analysis is available to RDV and aligns with the 2018 Victorian Government's A Data Reform Strategy for the Victorian Public Service. The strategy advocates harnessing data to improve government decision-making, including linking datasets to understand issues affecting citizens and the economy.

We tested RJIF's distribution of grants against:

• population
• unemployment rates and job growth
• indexes of relative socio-economic advantage and disadvantage.

Population

RDV has distributed at least one RJIF grant to each of the 48 eligible LGAs. However, the spread of RJIF grants is uneven geographically and on a per person basis.

Of the funds distributed to 31 December 2018, 29.7 per cent went to Ballarat and Geelong LGAs. Election and budget commitments drove most of the allocation to these LGAs—81 per cent for Geelong and 91 per cent for Ballarat.

Figure 3F shows the distribution of grants across the regional LGAs.

Figure 3F
RJIF grant allocation by local government area

Source: VAGO, based on RDV data.

This contrasts with RGF, which RDV distributed more evenly—only six LGAs have received more than $10 million in RJIF funds, compared to 16 LGAs for RGF.

Figure 3G compares the total value of grants received against LGA population for the two funds and shows that small LGAs received higher levels of funding under RGF.

Figure 3G
RGF and RJIF funds by local government area

Source: VAGO, based on RDV and the Australian Bureau of Statistics (ABS) data.

One way to measure RJIF distribution is to compare the funds per person that each LGA received. We found that the per person rate ranged from $15 for Surf Coast to $2 101 for Northern Grampians LGAs. RDV now includes data on the per person rate of RJIF distribution in its quarterly report to government, which it did not include for earlier funds, which increases its ability to oversee the equity of RJIF distribution around the state. This is a positive improvement because it gives important context to decision-makers about overall RJIF distribution. RDV does not make this information publicly available.

Unemployment rates and job growth

We found no consistent relationship between RJIF distribution and unemployment or job growth, despite a RJIF outcome measure to lower unemployment. Figure 3H shows funding distribution per person for LGAs with the highest and lowest average job growth rates. There is no clear link between job growth and RJIF distribution—for example, Mount Alexander and Loddon have each received similar per person funding despite significantly different job growth figures.

Figure 3H
Job growth and RJIF distribution

LGA	Average job growth 2007–08 to 2016–17 (%)	RJIF funding per person
Lowest job growth
Yarriambiack	–6.5	$147
Loddon	–4.9	$209
West Wimmera	–4.7	$172
Buloke	–4.3	$623
Towong	–3.4	$208
Highest job growth
Macedon Ranges	2.4	$44
Greater Geelong	2.4	$309
Mount Alexander	1.9	$233
Ballarat	1.9	$805
Surf Coast	1.7	$15
Median of all LGAs	–0.2	$109

Note: The RJIF funding per person is based on an average population from 2007–17.

Relative socio-economic advantage and disadvantage refers to people's access to material and social resources and their ability to take part in society. The indicators used to calculate it include income, education, employment, occupation and housing.

Source: VAGO, based on RDV and ABS data.

Socio-Economic Indexes for Areas

We measured RJIF distribution against Socio-Economic Indexes for Areas (SEIFA) data, which is an ABS measure of relative socio‑economic advantage and disadvantage. It is a useful tool that can help governments to direct funding and services where they are most needed.

Figure 3I shows the five most and least disadvantaged regional LGAs and the per capita RJIF funding they have received.

 

Figure 3I
SEIFA and RJIF distribution

LGA	SEIFA disadvantage rank (statewide)	RJIF funding per person (average population 2007–17)	Total RJIF funding to 31 December 2018
Most disadvantaged
Central Goldfields	1	$344	$4 403 666
Latrobe	4	$331	$24 414 194
Mildura	5	$153	$8 116 945
Northern Grampians	6	$2 101	$24 902 500
Yarriambiack	7	$147	$1 032 700
Least disadvantaged
Moyne	58	$121	$1 988 284
Golden Plains	61	$28	$566 145
Macedon Ranges	71	$44	$1 988 284
Queenscliffe	75	$1082	$3 240 000
Surf Coast	76	$15	$414 500
Median of all LGAs		$109

Source: VAGO, based on RDV and ABS data.

This data shows that mostly RJIF funds were distributed to LGAs with greater need, but not consistently. For example, Yarriambiack is the seventh most disadvantaged LGA and has only received $147 per person, which is comparable to Moyne's $121 per person, despite Moyne being the 58th least disadvantaged LGA in Victoria.

These analyses of population, unemployment and disadvantage are some of the ways in which RDV could give evidence-based advice to government on effectively investing in regional Victoria and achieving the outcomes of grant programs like RJIF.

Back to top

We have consulted with DJPR, and we considered their views when reaching our audit conclusions. As required by section 16(3) of the Audit Act 1994, we gave a draft copy of this report to those agencies and asked for their submissions and comments. We also provided a copy of the report to DPC.

Responsibility for the accuracy, fairness and balance of those comments rests solely with the agency head.

Responses were received as follows:

• DJPR

RESPONSE provided by the Acting Secretary, DJPR

 

 

Back to top

Figure B1
VAGO assessment of RDV's implementation of our 2015 audit recommendations

Recommendation	Assessment
That DJPR increases awareness of the initiatives to ensure the best possible projects are funded	Fully implemented RDV supplies public information about the fund through its website, offices, forums and Regional Partnerships. RDV's client satisfaction results reflect this outcome.
That DJPR implements robust governance arrangements, including an effective role for assessment committees in funding decisions	Partially implemented RDV have a RJIF management committee to monitor the fund's implementation. It uses a combination of internal and external committees to evaluate grants depending on the program. RDV has a monitoring and reporting framework stipulating the timeliness, content and recipients of its reports to enable oversight of RJIF's implementation. However, the data in these reports can be inaccurate. RDV has showed commitment to implementing the 2015 audit recommendations, by utilising an oversight committee and reporting regime. In assessing grants, RDV has used undocumented policies to apply weightings to its assessment. This increases the risk of inconsistency and reduces transparency in decision-making. RDV have not kept enough grant documentation to enable transparency of decision making and managerial oversight.
That DJPR develops and documents detailed pre-application funding processes that are transparent and demonstrate that the best available applications are funded	Partially implemented RDV has tracked pre-application cases in GEMS. This enables it to report on potential opportunities prior to it receiving a formal application. However, we found that as RDV inconsistently captures information about its pre‑application engagement it cannot demonstrate that it is advancing the 'best available' grant applications.
That DJPR implements identified risk management actions and plans	Substantially implemented RDV has developed a range of reporting that includes grant, program and strategic risks. While RDV's reports identify risks they require greater detail of the mitigation activities, including who is responsible and when RDV will implement the changes. RDV plan to review its risk management framework by 30 June 2019.
That DJPR implements robust monitoring and reporting systems and processes	Partially implemented RDV has developed an appropriate monitoring and reporting framework that contains the necessary types of information required for the management of RJIF. However, significant issues with data quality undermine the reliability of the reporting.
That DJPR develops a robust evaluation framework including relevant and appropriate performance measures, benchmarks, targets and baseline data	Partially implemented RDV has established an evaluation framework for RJIF, however, it does not fully meet the recommendation as it lacks benchmarks and targets, does not document baseline data for identified indicators, and is missing measures for some of the fund's objectives.
That DJPR undertakes regular reviews and evaluation activities to demonstrate that grant initiatives are meeting their goals and objectives	Partially implemented RDV includes outputs in grant agreements and monitors project implementation. It also reports internally and to the Minister on the funds administered, projects approved and outputs produced (e.g. jobs created). RDV was unable to evaluate the RGF or PVGF—its attempt in 2016 to comprehensively review RGF was unsuccessful. RDV is preparing to evaluate RJIF after its completion, but advises it is too early to evaluate the fund's outcomes. However, grant recipients had completed $60 million worth of RJIF projects by 30 June 2018—12 per cent of the fund. RDV did not include an interim evaluation in its plan. It therefore cannot report on outcomes to date and did not use the opportunity to test its evaluation approach on this sample.
That DJPR demonstrates the achievement of objectives by: monitoring and reporting on all outcomes of the initiative developing guidelines, in line with better practice, for making appropriate adjustments to outcomes prior to reporting accurately and fairly reporting outcomes attributable to the initiative	Partially implemented RDV has developed a monitoring and reporting framework and uses GEMS to report on outputs and outcomes. RDV has also established an evaluation framework and plan to report on the outcomes of the fund after its conclusion. RDV's evaluation plan includes high-level indicators and their attribution to RJIF to assess its success. However, there are gaps in the framework that may leave RDV unable to evaluate its performance against all the fund's objectives. RDV has not developed guidelines for adjusting data prior to reporting. RDV continues to attribute all jobs created by a project to RJIF regardless of the relative value of the grant to the overall project cost.
That DJPR demonstrates the effective use of public money by: developing robust guidelines and frameworks to assess value for money establishing and effectively implementing benchmarks and targets for all programs providing advice to government on value-for-money thresholds	Substantially implemented RDV has developed a framework for assessing value for money and provides advice to government on the fund's and grants' value-for-money outcomes. Its planned cost-benefit-analysis evaluations should support this approach, however, RDV have not provided their methodology to us for examination. RDV has not developed benchmarks or targets against which to assess a grant's value-for-money outcome.

Note: Recommendations were made to the then Department of Economic Development, Jobs, Transport and Resources, which transitioned to DJPR on 1 January 2019.

Source: VAGO, based on RDV information.

Back to top

Figure C1
Value of RJIF (to 31 December 2018) and RGF grants by LGA

LGA	RJIF value ($)	RGF value ($)	Total value ($)
Alpine	1 099 645	7 902 860	9 002 505
Ararat	7 582 621	7 072 382	14 655 003
Ballarat	79 170 871	15 340 500	94 511 371
Bass Coast	2 180 000	5 202 591	7 382 591
Baw Baw	5 402 535	11 496 813	16 899 348
Benalla	2 112 750	4 309 075	6 421 825
Borough of Queenscliffe	3 240 000	3 473 320	6 713 320
Buloke	3 983 100	3 745 620	7 728 720
Campaspe	3 883 120	8 061 307	11 944 427
Cardinia	–	3 909 264	3 909 264
Casey	–	42 150	42 150
Central Goldfields	4 403 666	7 646 086	12 049 752
Colac-Otway	2 450 000	4 714 656	7 164 656
Corangamite	880 821	4 774 383	5 655 204
East Gippsland	1 383 770	10 548 900	11 932 670
Gannawarra	565 000	4 898 960	5 463 960
Glenelg	2 204 000	6 759 631	8 963 631
Golden Plains	556 145	15 924 072	16 480 217
Greater Bendigo	18 923 000	15 912 800	34 835 800
Greater Geelong	69 523 209	33 016 910	102 540 119
Greater Shepparton	13 022 766	10 900 237	23 923 003
Hepburn	1 868 769	4 356 019	6 224 788
Hindmarsh	815 000	4 707 698	5 522 698
Horsham	3 362 688	10 418 891	13 781 579
Indigo	1 029 666	3 396 875	4 426 541
Latrobe	24 414 194	19 505 069	43 919 263
Loddon	1 580 000	4 488 495	6 068 495
Macedon Ranges	2 315 975	5 043 460	7 359 435
Manningham	2 315 975	86 200	2 402 175
Mansfield	343 299	3 724 075	4 067 374
Melton	–	91 700	91 700
Mildura	8 116 945	29 193 812	37 310 757
Mitchell	3 130 000	8 783 487	11 913 487
Moira	1 999 230	7 790 800	9 790 030
Moorabool	455 000	4 290 900	4 745 900
Mornington Peninsula	–	26 785	26 785
Mount Alexander	4 264 967	2 972 349	7 237 316
Moyne	1 988 284	6 650 942	8 639 226
Murrindindi	591 890	4 060 197	4 652 087
Nillumbik	–	162 900	162 900
Northern Grampians	24 902 500	4 456 334	29 358 834
Port Phillip	–	18 000	18 000
Pyrenees	1 522 578	15 376 662	16 899 240
South Gippsland	1 932 500	19 657 490	21 589 990
Southern Grampians	3 404 770	3 200 605	6 605 375
Statewide	80 291 326	122 960 000	203 251 326
Strathbogie	1 592 280	3 952 867	5 545 147
Surf Coast	414 500	9 141 206	9 555 706
Swan Hill	3 793 690	10 677 137	14 470 827
Towong	1 242 399	4 422 275	5 664 674
Wangaratta	5 516 626	7 809 975	13 326 601
Warrnambool	8 220 000	10 009 800	18 229 800
Wellington	5 994 000	21 937 990	27 931 990
West Wimmera	710 000	2 230 371	2 940 371
Whittlesea	–	58 315	58 315
Wodonga	6 459 962	11 621 895	18 081 857
Yarra Ranges	–	8 079 980	8 079 980
Yarriambiack	1 032 700	2 877 324	3 910 024

Note: RDV advises some RGF grants were provided to non-rural councils in response to the 2009 bushfires by a ministerial exemption.

Source: VAGO, based on RDV data.

Back to top

This dashboard is an interactive tool that provides information about the distribution of Regional Development Victoria’s (RDV) Regional Infrastructure Development Fund (RJIF) and Regional Growth Fund (RGF) grants to each regional local government area (LGA). It also provides a snapshot of economic and social indicators for those LGAs.

We have taken the grant data from RDV’s database and made it public for the first time through this audit and dashboard.

The economic and social indicator data comes from the Australian Bureau of Statistics (ABS).

The dashboard has three pages:

1. The first page displays the amount of RJIF and RGF grants given to each LGA. Each of the 48 regional LGAs received at least one RJIF grant, and RDV advises that some additional LGAs received grants under the RGF.
2. The second page provides a snapshot of economic and social indicators for the 48 regional LGAs. The page provides information about each LGA’s employment, population and Socio-Economic Indexes for Areas (SEIFA) results. Further data is available from the ABS and RDV.
3. The third page displays funding for each LGA compared to their average population (2007–17).

Dashboard instructions

 

General instructions for all pages
The dashboard is easiest to navigate in the full screen view. Click the button at the bottom right of the window.
To access a detailed view of each chart, hover over the chart and click the ‘Focus Mode’ button at the top right of the window.
To return to the dashboard from ‘Focus Mode’, click ‘Back to Report’ at the top left of the window.
To share the link to the dashboard via email, LinkedIn, Facebook or Twitter, click the ‘Share’ button at the bottom right of the window.

Back to top