Implementation of the Strengthening Community Organisations Action Plan

Auditor-General's comments

John Doyle Auditor-General

Audit team Andrew Evans—Acting Sector Director Rosy Andaloro—Team Leader Sophie Fisher—Analyst Chris Sheard—Engagement Quality Control Reviewer

Community sector or not-for-profit organisations play an important role in ensuring Victoria’s economic and social prosperity. They are major providers of services such as health, education, aged care, and sport and recreational activities. The Office for the Community Sector (OCS) was established in 2008 as part of the then government’s action plan to strengthen community organisations.

The Victorian Government’s Action Plan: Strengthening Community Organisations was a four year $13.8 million plan developed to strengthen the capacity and long-term sustainability of the community sector in Victoria.

The action plan had two objectives:

• to ensure that, in its direct interactions with community and not-for-profit organisations, government acts in ways that are simple, easy to navigate and designed to optimise value to the community
• to create an environment for community organisations to invest in their own capabilities and long term sustainability.

OCS was responsible for leading implementation of the action plan and coordinating whole-of-government activities. In this audit, I assessed the effectiveness and efficiency of the implementation of the action plan—essentially, how was it implemented, what it achieved and what aspects of implementation could be improved and applied to future activities.

My overall conclusion is that OCS has done a good job in delivering on the action plan and is able to demonstrate achievement of the main outputs. This includes the development of a Common Funding Agreement, a Standard Chart of Accounts and the Not for Profit Compliance Support Centre. These achievements are significant given the size of the office and the challenges of working with a large and diverse sector and across government. In recognising the valuable role OCS had in connecting community organisations and coordinating activities across government, I have identified areas in planning and implementation where a more structured approach would benefit future programs activities.

There were weaknesses with some aspects of planning, implementation and evaluation. The implementation plan should have included a well-developed risk, monitoring and reporting, and evaluation framework, describing the project outcomes and how these were intended to be measured and reported. The absence of these means OCS cannot demonstrate that the intended outcomes of the action plan were fully achieved or that the plan was implemented efficiently and effectively. OCS should improve its project planning and management practices to increase the likelihood of successful implementation of future initiatives and programs.

I acknowledge and thank the staff at the Office for the Community Sector for their assistance and cooperation during this audit and I look forward to receiving updates on their progress in implementing the recommendations.

John Doyle

Auditor-General

October 16 2013

Conclusions

OCS has made good progress in implementing the action plan and delivering its main outputs. It played a valuable role in connecting community organisations and coordinating activities across government, which assisted with sector engagement and delivering specific actions under the plan.

While the majority of actions and outputs under the plan were completed, there were weaknesses in some aspects of planning, implementation and evaluation. This means OCS cannot demonstrate that the intended outcomes of the action plan were fully achieved or that the plan was implemented efficiently and effectively. OCS should improve its project planning and management practices to increase the likelihood that its future initiatives and programs will be successfully implemented.

Findings

Major achievements

OCS delivered a substantial portion of the 25 actions in the plan. Across the 25 actions, there were 113 sub-actions, which included delivering specific outputs and monitoring and reporting activities. The audit assessed the extent of completion of the sub-actions as well as the views of community sector stakeholders involved. OCS can demonstrate eight of the 25 main actions have been completed in full, with 13 substantially completed, and three partially completed. Key achievements include:

• reforms to legislation affecting community organisations
• establishing the Not for Profit Compliance Support Centre
• developing a Common Funding Agreement
• developing a Standard Chart of Accounts for the sector
• developing a Workforce Capability Framework
• funding 12 community foundations.

Sector engagement

A key focus of the action plan was to strengthen the capacity of the community sector to improve its long-term sustainability and to make it easier for community organisations to deal with government. OCS established a sector reference group and specific reference groups with representatives from community organisations. These groups were integral to facilitating sector input and involvement in delivering specific actions under the plan.

Cross-government coordination

Effective consultation and coordination across departments was essential as some actions, such as the development of the Common Funding Agreement, directly affected several departments and required their collaboration for successful implementation. OCS established a number of interdepartmental working groups to oversee implementation of specific actions. The working group arrangements were successful in sharing information across departments and facilitating the delivery of actions requiring cross-government coordination and activity.

Implementation planning

OCS developed an implementation plan for the action plan but this did not adequately address time frames, resources, responsibilities, risks, monitoring, evaluation and reporting. OCS' sector engagement and cross-government coordination activities essentially drove implementation of the action plan. However, the lack of comprehensive implementation planning means that it is unclear how progress was to be monitored and reported, how outcomes would be evaluated and how emerging lessons would be captured and used to enhance the effectiveness of the action plan and future activities.

Governance arrangements

OCS established governance arrangements including reference and working groups to help implement the plan. While these groups were effective, there were shortcomings in the clarity of roles, documenting actions and reporting across the groups.

Risk management

OCS identified two high-level risks as part of developing the implementation plan but it did not develop a detailed risk management plan. Therefore, it is not clear how risks were identified or whether they were purposefully managed during the plan's implementation.

Monitoring and reporting

OCS used a combination of written reports and verbal briefings to monitor progress and report to stakeholders and government. However, this was not part of a structured and coherent approach to monitoring and reporting. OCS did not develop appropriate targets to measure the extent to which intended outcomes had been achieved. OCS did effectively manage the allocation and monitoring of grants provided to community foundations.

Evaluation

OCS did not develop or apply an appropriate framework to evaluate the outcomes of the action plan. As a result, it has never assessed the action plan's overall effectiveness and has not adequately evaluated it's emerging impacts or the achievement of outcomes. Therefore it cannot demonstrate that the action plan fully achieved its intended objectives and outcomes.

Recommendations

The Office for the Community Sector should apply the lessons learned in implementing The Victorian Government's Action Plan: Strengthening Community Organisations to its ongoing activities and any future programs by:

1. developing implementation plans that comprehensively address objectives and outcomes, roles and responsibilities, time frames and resourcing
2. developing a robust monitoring, evaluation and reporting framework supported by relevant and appropriate performance measures and targets
3. undertaking evaluations at the conclusion of programs with a focus on demonstrating the achievement of objectives and outcomes.
4. developing and applying a sound risk management framework that clearly documents how identified risks will be assessed, prioritised and managed.

Submissions and comments received

In addition to progressive engagement during the course of the audit, in accordance with section 16(3) of the Audit Act 1994, a copy of this report was provided to the Department of Human Services for submissions or comments.

Agency views have been considered in reaching our audit conclusions and are represented to the extent relevant and warranted in preparing this report. Their full section 16(3) submissions and comments are included in Appendix C.

1.1 The role of community organisations

There are around 140 000 community organisations in Victoria delivering approximately $3 billion in government programs funded through grants or service payments. They range from small organisations, relying predominantly on volunteers, to large organisations accounting for a large part of the service economy.

Community organisations are a significant provider of health, education, aged care and social services, as well as sport and recreational activities. They are also significant employers—making up 8 per cent of national employment—and contribute $43 billion to Australia's gross domestic product. Community organisations play an important role in enhancing liveability, and promoting social inclusion and community life. A growing number of people are relying on community organisations for the delivery of important government programs.

Community sector and not-for-profit organisations have a variety of legal structures including cooperatives, Indigenous corporations, gift funds and trusts. Most are incorporated associations or companies limited by guarantee.

1.1.1 Policy context

The 2006–07 government initiative Reducing the Regulatory Burden committed Victoria to reduce the red tape imposed by regulation on the business and not-for-profit sectors.

The aims of the initiative were to:

• cut the existing administrative burden of regulation by 15 per cent over three years and 25 per cent over five years
• ensure the administrative burden of any new regulation is met by an offsetting simplification in the same or a related area
• undertake a program of reviews to identify the actions necessary to reduce compliance burdens.

The then government's A Fairer Victoria strategy acknowledged the contribution of community organisations in building stronger communities. It included a commitment from the Victorian Government to work with the Victorian Council of Social Service and other community sector, philanthropic and business leaders to improve the capacity of the community sector and to plan for the key challenges of sustainability and growth.

In 2007 the government commissioned the State Services Authority's (SSA) Review of Not-for-Profit Regulation and the Stronger Community Organisations Project (SCOP). Both of these projects examined the challenges facing the sector, including its future role, capacity, and relationship with government. Figure 1A summarises the aims and recommendations of these reviews. In response to the recommendations of SCOP and the SSA review, the then government developed The Victorian Government's Action Plan: Strengthening Community Organisations.

Figure 1A

Summary of the two government-commissioned reviews

Source: Victorian Auditor-General's Office.

1.2 The Victorian Government's Action Plan: Strengthening Community Organisations

1.2.1 Overview of the action plan

Released in 2008, the four-year $13.8 million action plan was developed to strengthen the capacity and long-term sustainability of the community sector in Victoria. The action plan had two objectives:

• 'to ensure that, in its direct interactions with community and not-for-profit organisations, government acts in ways that are simple, easy to navigate and designed to optimise value to the community
• to create an environment for community organisations to invest in their own capabilities and long-term sustainability'.

To meet these objectives the action plan outlined a set of 25 actions under five themes:

• reducing regulatory burden (actions 1–11)
• building the capacity of community organisations (actions 12–16)
• supporting innovation and growth (actions 17–18)
• enhancing the role of community organisations in local community life (actions 19–22)
• improving cross-sector coordination (actions 23–25).

A complete list and summary of actions is contained in Appendix A. Figure 1B shows the distribution of funding across the action plan.

Figure 1B

Distribution of funding

Source: Victorian Auditor-General's Office based on Budget Paper 3, Service Delivery 2008–09.

1.2.2 Office for the Community Sector

The Office for the Community Sector (OCS) was established in 2008, in the then Department of Planning and Community Development, to lead the coordination and implementation of policy priorities affecting the community sector across the whole of government. In implementing the action plan, OCS worked across government departments and agencies such as Consumer Affairs Victoria and the Department of Human Services, and with organisations within the community sector including sports, arts, welfare and children's services organisations.

OCS was initially funded for three years under the action plan, from 2008–09 to 2010–11. A further $4.6 million was provided to fund the office from 2011–12 to 2014–15 to continue to drive whole-of-government and community sector reforms, including further red tape reductions and easing regulatory compliance to:

• reduce the unnecessary duplication of service standards
• streamline reporting and data collection
• develop a Single Funding Agreement—one agreement between government departments and a funded community organisation covering multiple grants from different departments
• expand opportunities for community organisations to access broader funding sources including business, philanthropy and the social finance market
• support and promote new ways for community organisations to work and deliver services
• improve community organisations' governance practices and skill level
• work across government to develop the community sector workforce
• assess and respond to Commonwealth community sector reform proposals.

OCS currently has two key objectives:

• driving cross-government activity that reduces unnecessary accountability and compliance burdens
• strengthening the capacity and capability of community organisations.

1.3 Audit objective and scope

The audit assessed the effectiveness and efficiency of the implementation of The Victorian Government's Action Plan: Strengthening Community Organisations.

To address the objective, the audit examined:

• whether the action plan was implemented effectively and efficiently, and achieved its intended outputs and outcomes
• the effectiveness of the ongoing activities of OCS.

The audit examined OCS' role in implementing the action plan, including its role in leading the coordination and implementation of the action plan, and the extent to which current activities are achieving intended outcomes.

1.4 Method and cost

The audit method included interviewing staff from OCS, examining documentary evidence on the plan's implementation and surveying community organisations that were involved in the plan's implementation.

To seek stakeholder views on the plan's implementation, an online survey was directed at community organisations involved in reference and working group activities related to the action plan. The survey included a set of questions about the development of the plan and its results—both outputs and outcomes. Survey questions allowed for rating scales as well as free text and open-ended comments.

Initially, the survey was pilot tested with four participants. Their feedback helped to refine questions asked of the broader participant group. An email sent to 125 stakeholders involved in the action plan invited them to participate in the online survey. Twenty-nine people responded to the online survey between 11 and 19 June 2013. The results of the four pilot tests were incorporated in the final results.

The audit was conducted in accordance with section 15 of the Audit Act 1994 and Australian Auditing and Assurance Standards. Pursuant to section 20(3) of the Audit Act 1994, any persons named in this report are not the subject of adverse comment or opinion.

The cost of the audit was $300 000.

1.5 Structure of the report

The report is structured as follows:

• Part 2 examines the action plan's achievements.
• Part 3 identifies lessons for program implementation and future activities of OCS.

2.1 Introduction

The Office for the Community Sector (OCS) was responsible for implementing The Victorian Government's Action Plan: Strengthening Community Organisations and leading the coordination and implementation of policy priorities affecting the sector across government. A number of other departments were responsible for delivering actions or sub-actions, including the former Department of Planning and Community Development, Consumer Affairs Victoria within the Department of Justice, and the Department of Human Services.

Across the action plan's 25 actions, there were 113 sub-actions, which included delivering specific outputs as well as monitoring and evaluation activities. This Part examines progress and major achievements in implementing the plan, as well as OCS' approach to sector engagement and cross-government coordination.

2.2 Conclusion

Good progress has been made in implementing the action plan and delivering its main outputs. This included establishing OCS and the delivery of key outputs such as the Common Funding Agreement (CFA), legislative reform and funding of community foundations. These are significant achievements given the diversity of the sector, the broad nature of the plan, the resources available and the challenges of coordinating activities across government.

OCS played a valuable role in connecting community organisations and coordinating activities across government. Establishment of a sector reference group and working groups with representatives from community organisations and government departments assisted with sector engagement and the coordination and delivery of specific actions under the plan. The majority of community organisations surveyed by VAGO viewed the OCS' work in a positive light and acknowledged its important role in providing a link between the sector and government, and in coordinating government activity.

2.3 Progress in implementing the action plan

Our assessment of the action plan considered the completion of actions and sub-actions as well as the views of community sector stakeholders involved in implementing the action plan. Figure 2A summarises the status of the action plan.

Figure 2A

Summary of status of the plan's actions

Source: Victorian Auditor-General's Office.

OCS can demonstrate eight actions have been completed in full, with 13 substantially completed, three partially completed and one not progressed.

Some of the incomplete actions have outstanding sub-actions that are related to recent Commonwealth Government reforms that have occurred since the action plan was developed. For example actions 3, 6 and 11—aimed at reducing the regulatory burden through legislative reforms—included sub-actions relating to working with the sector and the Commonwealth to develop a national regulatory framework and seeking national agreement of fundraising legislation between jurisdictions. OCS has fulfilled its obligations under the action plan relating to these actions. However, the completion of these is now tied to decisions and reforms being progressed through the Council of Australian Governments' agenda, which is beyond the control of OCS.

Other actions have incomplete sub-actions relating to activities that have not been undertaken. For example, action 23—establishing the Office for the Community Sector—has a component relating to reviewing the role and impact of OCS and this has not occurred.

Although OCS advised that some activity has occurred in relation to action 21—attracting more support to local sport and recreational associations—this could not be verified.

A summary of our assessment of the status of the plan's actions is included in Appendix B and some of the key achievements are outlined in Figure 2B.

Figure 2B

Implementation achievements

Source: Victorian Auditor-General's Office.

Survey data indicates that 61 per cent of respondents thought implementation of the plan delivered outputs extremely or very well. Respondents also commented on the usefulness of outputs achieved:

I was 'impressed with the professional conduct of the office. They have run good projects and consultations and clear and useful outputs. We have made use of lots of these'.

'Workforce Capability Framework was developed and taken up by organisations.'

'The program of encouraging community foundations to raise funds to match the Government's initiative with a matching grant has been very positive for the sector.'

'The challenge grant built our community fund and also built our confidence in fundraising. Community awareness and support increased as a result of the challenge grant.'

2.4 Sector engagement and cross-government consultation

OCS' role was to lead the coordination and implementation of the action plan across the whole of government. To achieve this, OCS expected to employ a staff of nine full-time equivalents. However, this did not eventuate due to departmental budget and restructuring constraints. Staff peaked at around 4.5 full-time equivalents, although external parties were engaged to assist with delivery of some actions or parts of actions. Sector engagement and cross-government consultation underpinned much of the plan's implementation.

2.4.1 Approach to sector engagement

A key focus of the action plan was strengthening the capacity of the community sector to improve its long-term sustainability and to make it easier for community organisations to deal with government. It was therefore critical to effectively engage the sector in its implementation to facilitate sector input and to increase the likelihood of success and take-up of actions.

OCS undertook a sector scan to identify key stakeholders from a cross-section of the community sector. Following this, a sector reference group consisting of representatives from different community organisations and peak bodies was established. The group's terms of reference were to:

• provide advice to OCS on the implementation of the government's action plan
• provide advice on sector representation on the other reference groups formed to support the implementation of the individual actions in the plan
• assist to strengthen a collaborative and partnership approach to the sector and government working together
• provide advice on future directions for the government's commitment to developing and supporting NFP organisations in Victoria.

The group met 11 times during the implementation of the action plan and continues to meet to provide advice to OCS on future directions in developing and supporting community sector organisations in Victoria. Working groups with community sector representatives were also established by OCS, and these are discussed in Section 2.4.3.

OCS also established an annual NFP summit as part of action 24—to facilitate dialogue and exchange between the sectors. This statewide event was first held in 2009, and provided an opportunity for government to engage with the NFP sector while facilitating networking among attending organisations. The summit explored a different theme each year through workshops, presentations, exhibitions and networking. Forums are still held annually.

78 per cent of survey respondents rated the OCS' performance in involving the relevant stakeholders as excellent or good. Comments made by respondents also indicate OCS effectively engaged the sector:

I valued 'the enthusiasm of OCS ... the engagement with and listening to the sector ... and the fact that our concerns were heard'.

'There was good community engagement during the projects and ongoing engagement. It was not tokenistic. It was for the sector and by the sector. There was an understanding of our needs and results were delivered to meet those needs.'

2.4.2 Cross-government coordination

Effective consultation and coordination across departments was essential, as some actions, such as the development of the CFA, directly affected other departments and required their collaboration for successful implementation.

OCS established and chaired several interdepartmental working groups to oversee the implementation of specific actions that required cross-government coordination. For example, an interdepartmental working group was established to oversee the implementation of actions relating to Theme 1—Reducing the regulatory burden and streamlining interaction with government. This group had representatives from the Department of Planning and Community Development (through OCS), as well as from the departments of Justice (through Consumer Affairs Victoria), Human Services, Education and Early Childhood Development, Premier and Cabinet, and Treasury and Finance.

The working group arrangements were successful in sharing information across departments and delivering actions requiring cross-government coordination. They also assisted in building relationships between government and the community sector through OCS.

2.4.3 Framework for stakeholder engagement and cross-government coordination

The sector reference group and the various other reference and working groups established to help implement the action plan are illustrated in Figure 2C.

Figure 2C

Arrangements to implement the action plan

Note: Pale blue groups comprised of both government and community sector representatives.

Source: Victorian Auditor-General's Office.

As well as the sector reference group, specific groups were established to deliver particular actions and, in some cases, more than one action. Several actions, such as those relating to service agreement consistency—action 9—had more than one group to assist in implementing different parts of each output. Joint meetings were also held between community sector representatives and government representatives to deliver specific actions.

The arrangements to develop the CFA included an interdepartmental committee with members from across government to oversee the project. Two cross-government working groups were established to develop the agreement template and review systems and processes. OCS also consulted an illustrative projects group consisting of representatives from community organisations on the development and implementation of the CFA.

Figure 2D demonstrates the structure in place for the delivery of the CFA relating to action 9.

Figure 2D

Arrangements for the common funding agreement

Source: Victorian Auditor-General's Office.

The establishment of the sector reference group and working groups was useful because it meant that OCS frequently had access to a range of views and advice on delivery of the plan. Representatives of the community sector who were surveyed viewed the governance of the reference groups as positive. They commented on the benefit of discussions between sectors and the presence of government agencies, as well as the benefits for the community sector in having OCS work across government departments.

Around 76 per cent of survey respondents viewed the groups as extremely or very effective. Some respondents also provided comments around the frameworks established by OCS:

'OCS' performance was leading practice; as a whole OCS has demonstrated strong leadership and engagement with the community sector and government. Its ability to effectively use existing government expertise and materials has been limited only by the time frames required to convince other government departments to take the work of the community sector seriously.'

'What worked well: the reference group was high performing and achieved some good outcomes for the sector. OCS was successful in championing the community sector in government. It initiated some whole of government projects. They were very successful and we were able to provide some good input...'

In fulfilling its role, OCS demonstrated an adaptive approach to implementation by focusing on cross-government and community sector engagement and consultation, and getting the right people together to drive implementation of the action plan. Comments made by survey respondents on outcomes of the plan and what worked well also demonstrate this. There are opportunities to leverage off this position:

'OCS represents an important step forward in progressing government sector collaboration and the long-term vision for the sector.'

'Government understanding of sector is much clearer and relationship is stronger. Sector more confident about approaching government.'

3.1 Introduction

Effective and efficient program implementation is central to the successful delivery of plans and programs and to demonstrate the achievement of intended objectives and outcomes. Robust frameworks including appropriate governance, risk, monitoring, reporting and evaluation arrangements are important to demonstrate the effective use of public funds and to effectively implement and deliver government policy.

This Part examines the planning and implementation of The Victorian Government's Action Plan: Strengthening Community Organisations, evaluation of outcomes and how lessons learned can be applied to the future activities of the Office for the Community Sector (OCS). Key areas of implementation covered are planning, risk management, monitoring and reporting, evaluation and measuring outcomes.

3.2 Conclusion

There were weaknesses in some aspects of planning and implementation of the action plan. Implementation plans lacked sufficient detail on how the action plan would be implemented and contained inadequate information on key aspects such as risk management and monitoring, evaluation and reporting. Although the majority of actions and outputs under the action plan were completed, OCS did not plan for, develop and apply an appropriate evaluation and performance monitoring framework.

It is important that OCS improves its planning and establishes effective monitoring, evaluation and reporting systems and processes. This will assist with demonstrating the achievement of its intended objectives and outcomes and increase the likelihood that its future initiatives and programs will be successfully implemented.

3.3 Lessons for program implementation

Planning lays the foundation for the effective delivery of programs and projects. Adopting a structured approach to program planning and implementation increases the likelihood of success.

Implementation planning should address key elements such as governance, roles and responsibilities, resources, risk management, performance measures, monitoring and evaluation. Our assessment of the action plan's implementation and of survey responses from community organisations involved in implementing the plan indicate that there are opportunities for improvement in the way OCS plans and manages its activities and programs.

3.3.1 Implementation planning

Planning how an initiative will be implemented should address elements such as governance, time frames, milestones, roles and responsibilities and resourcing. Better practice implementation plans are realistic, flexible and have sufficient detail.

Implementation plans should:

• show a clear alignment between policy objectives and implementation, as well as make clear the long-term and short-term outcomes
• reflect adequate consideration of key risks and potential barriers to implementation throughout the entire implementation process—not just at the beginning—and how they will be dealt with
• provide a map of how an initiative will be implemented, covering time frames, roles and responsibilities of all those involved in implementation, resources—including funding and human resources—monitoring, reporting and evaluation requirements.

Planning for the action plan's implementation

An implementation plan was developed in June 2008, but this did not sufficiently address time frames, resources, responsibilities and risks, or monitoring, evaluation and reporting requirements. Consequently, it is difficult to assess whether the action plan was implemented efficiently and effectively.

OCS' sector engagement and cross-government coordination activities essentially drove implementation of the action plan. However, the lack of detailed implementation planning means that it is unclear how and when the plan was intended to be implemented, how progress was intended to be monitored and reported, how outcomes would be evaluated, and how emerging lessons would be captured and used to enhance the effectiveness of the action plan and future activities.

3.3.2 Governance arrangements

Governance is the set of responsibilities, policies and procedures which provide strategic direction, so that objectives can be achieved and resources can be deployed responsibly and effectively. A governance framework should consider the roles, responsibilities and accountabilities of those involved, the rules and procedures for decision-making, and the integration of the project governance arrangement within an agency's broader corporate governance framework.

Sound governance arrangements are fundamental to the success of program and policy implementation. The nature and complexity of governance arrangements should reflect the size and scope of the initiative. These factors will influence the degree to which senior managers need to be involved, and the formality of the arrangements.

Governance arrangements for implementing the action plan

OCS developed a framework to implement and monitor the action plan, and for it's own operation. Reference and working groups with responsibilities for particular actions were established to help deliver the plan. While these groups were generally effective, there were some shortcomings in role clarity, documentation and record keeping, and performance monitoring and reporting.

Terms of reference were developed for the sector reference group, in consultation with the group members. These clearly described the group's primary role to advise OCS on the implementation of the action plan, including sector representation on the other reference groups formed to support the individual actions in the plan.

For 10 of the working groups there was evidence that terms of reference were developed or that the group's role was at least discussed. The roles of the remaining five groups were not documented and it was not evident for any of these how decisions were supposed to be recorded, followed through and progress reported.

Our survey results indicate that 60 per cent of respondents thought their role was either extremely clear or very clear. However, some respondents commented about the lack of clarity in the terms of reference and that the working groups had too many members:

'It was difficult to see how any views were incorporated in the work. They may have been but it wasn't obvious.'

'The reference group was very "fluid", i.e. no terms of reference, clear and consistent membership or clear agenda. I withdrew after the first couple of meetings.'

Other respondents commented that the governance arrangements were a positive aspect of the action plan's implementation:

'Inter-sectorial discussions, breaking down "silos" and talking with people who interact with government agencies in different ways ... the range of organisations involved ... and the cooperation and follow through that happened' was a positive.'

While there may be a need for flexibility in how each group approached diverse actions, terms of reference for all groups should have been developed outlining, as a minimum:

• the role of the group and individual members
• procedures for recording information, such as formal minutes or notes and who should receive them
• how often the group would meet
• reporting procedures, such as documenting agreed actions and who is responsible.

Members of community organisations were not funded to participate in reference or working groups. Some survey respondents commented on the impact of this. OCS also recognised this was something it needed to consider for future programs:

'The plan produced great resources but it is up to you to get involved. We are a quite small organisation. An HR group in the sector got together about the capability stuff but there was no momentum from OCS. There were no implementation dollars to use the resources ...'

'There is a real problem around the viability of ongoing NFP engagement with a process like this. Engagement doesn't exist without funding. It takes a lot of time to attend meetings and read documents before and after meetings but it can't keep going on without some payment. None of the organisations involved received any funding for their input. It was all done at our expense.'

I find it 'sometimes difficult to keep linked into the reference group's actions and intentions given all my other workload and immediate demands.'

3.3.3 Risk management

Risk management involves the identification, assessment and prioritisation of risks, followed by their active monitoring or management, with the aim of reducing the likelihood of them occurring, or their impact, should they materialise. Effective risk management reduces the likelihood that unforeseen circumstances will jeopardise the achievement of objectives.

A risk management framework should be based on a clear understanding of the objectives and allow for processes that:

• identify and assess key strategic, operational and financial risks
• ensure appropriate responses are determined, and provide assurance that these are effective
• allow the monitoring of risks and public reporting of the effectiveness of the risk management system.

Initiatives that involve cross-portfolio collaboration face increased complexity of risk management, so it is important to ensure that there is a common understanding of the risks associated with shared implementation. This requires a clear and agreed identification of who carries which risks. This applies not only to identifying and assigning responsibility for risk, but also to actively managing, monitoring and reporting throughout implementation. Responsibilities should be clear and documented where organisations outside government are involved in risk management.

As implementation is usually a staged process, risks should be reviewed and actively managed at key stages to take into account changing circumstances through the various phases of implementation. Appropriate responses to emerging problems will be more manageable where robust contingency plans have already been developed as part of the risk management strategy.

Risk management under the action plan

The implementation plan developed by OCS identified two high-level risks—failure to establish OCS by the first quarter 2008–09 and failure to engage broadly with the community sector. Considering the complexities involved in working with a diverse sector and across government, it is likely that OCS dealt with many issues and risks as they arose. However, in the absence of a documented risk management framework, it is not clear whether risks were purposefully managed.

As part of this audit, OCS identified a number of lessons from implementing the action plan including:

• getting the right tactical approach—establishing clearly explained change management processes early in the project and balancing the use of high-level interdepartmental committees, reference and working groups to maintain buy-in and get the right people around the table so the project is a success
• working constructively with the sector—engaging the sector to make them see the benefits of participating, considering whether the engagement of the sector—and other departments—needs to be funded, and building and maintaining the perception within the sector that OCS is able to maintain broad and high-level focus on reform rather than on the details of service delivery.

These lessons indicate OCS saw aspects of engagement with the community sector as risks to implementation. Had a systematic approach to risk management been taken earlier, these risks may have been identified, and appropriate strategies put in place to manage them in the initial stages of implementation.

3.3.4 Monitoring and reporting

Monitoring and reporting supports effective program implementation and oversight. It underpins key activities, such as evaluation, by facilitating the collection of data required to measure and demonstrate the achievement of outputs as well as outcomes. It also allows stakeholders to assess progress, identify and address problems, and review the ongoing relevance of an initiative.

Monitoring and reporting practices should include the provision of timely and reliable status reports throughout implementation so that any risks to achievements are identified and dealt with. Where the implementation of initiatives is multi-staged, it is particularly important that the concerns or lessons from each stage are addressed and resolved before implementation progresses.

Performance measures and targets

Measuring performance is fundamental to accountability. It provides information on how effectively and efficiently agencies deliver programs, and how economically they use public funds. Effective performance measurement requires well expressed objectives that clearly articulate the outcomes to be achieved and appropriate measures that demonstrate whether objectives and outcomes are being met.

Monitoring and reporting of the action plan

Working groups were the main mechanism used to monitor the progress of the action plan. OCS sat on each of the working groups and led progress reporting, preparing a combination of written reports and verbal briefings to stakeholders and government.

Written reports covered progress against actions, and working group minutes show that verbal briefings provided to stakeholders also covered this. However, this was not part of a structured and coherent approach to monitoring and reporting, nor was it supported by performance measures to enable an assessment of the effectiveness and efficiency of the action plan's implementation.

Progress reports generally provided an update on overall progress but did not:

• consistently report against all the plan's actions
• cover actions to the level of detail in the government's action plan, often providing a catch-all summary without covering the specific sub-actions of the published plan
• raise implementation issues or challenges together with details of how OCS planned to address these.

Survey respondents also indicated the need to improve monitoring and reporting of the action plan:

'Plan outputs and outcomes were very clear in their definition. But they were hard to measure and sometimes the reference group would get side-tracked.'

There is a 'need for annual published updates on implementation.'

As part of the audit, OCS identified a number of lessons related to monitoring and reporting, including the need to:

• develop an outcomes-based framework in order to secure increased social investment and reduce the reporting burden on community organisations
• build effective communications channels and provide regular, easy to understand information in a form that can be easily used to brief upwards.

This indicates that OCS recognised there were aspects of monitoring and reporting which require improvement for future programs.

Monitoring and reporting of grant recipients

Almost 40 per cent, or $5.3 million, of the action plan's $13.8 million in direct funding was allocated to action 20—a stronger role for community foundations. This action involved providing grants of up to $300 000 to 12 community foundations that:

• were held in perpetuity by the local community to provide for locally determined needs into the future
• supported initiatives that build relationships between business, philanthropy and the local community
• ensured that the community foundation had strong, locally connected governance
• developed the profile and capacity of the community foundation
• assisted communities of disadvantage within the area covered by the foundation.

The foundations approved for funding were provided an initial $100 000 grant. If they matched this amount within 12 months, and met performance measures, they received another $200 000. It is important that monitoring and reporting of individual grant recipients provides assurance that outputs are delivered as required.

Funding agreements with approved grant recipients should:

• contain appropriate reporting requirements that enable monitoring of the progress of projects and the use of funds
• require clearly defined outputs to be delivered by the recipients
• require evidence that agreed milestones and/or outputs have been met before payment is made.

To assess the effectiveness of OCS' management of grants we examined a sample of grant recipient project files. OCS managed grant funding effectively in terms of:

• documenting obligations in funding agreements
• monitoring that funding obligations were met prior to the disbursement of funds
• reporting on payments and achievements.

3.3.5 Evaluation

Evaluation planning is fundamental for effective performance measurement and for demonstrating the achievement of objectives and outcomes. An evaluation plan should establish the management practices and systems required for an effective evaluation including:

• clarification of evaluation questions, activities, roles, appropriate methods and ethics requirements
• ongoing data sources, systems and practices needed during the life of the program
• evaluation resource requirements in terms of budget and staff
• time lines for evaluation activities
• communicating results to relevant stakeholders.

A well designed evaluation should also demonstrate the link or contribution that program outcomes are expected to make to higher level departmental outcomes.

Evaluations should also aim to identify lessons that may help in the future to:

• improve policy design implementation and decision making
• help resource allocation
• enhance accountability in terms of assessing what outcomes were achieved
• promote organisational learning and good practice.

Evaluation planning for the action plan

OCS did not develop or apply an appropriate framework for evaluating the outcomes of the action plan. The action plan's overall effectiveness was never assessed and consequently OCS cannot demonstrate whether intended objectives and outcomes were achieved. This was a significant weakness in the implementation of the action plan.

OCS did not establish methods for identifying and collecting data to monitor performance against the action plan. Relevant and appropriate measures and targets were not developed to measure the extent to which intended outcomes were achieved. As a result, the impact of the actions and outputs is unknown. There was, however, an interim evaluation and evaluations of some specific actions, which are discussed below.

The action plan interim evaluation

In February 2010, OCS engaged an external party to review the progress of the implementation of the action plan. The evaluation assessed value for money achieved as a result of the 25 actions in the plan in terms of:

• appropriateness—of reform objectives in meeting stakeholder needs and the extent to which higher level community and government priorities are addressed.The interim evaluation found OCS played a critical role in implementing the action plan, demonstrating the required versatility and understanding of government and the community sector. Stakeholder feedback from the review also indicated strong support for the government's commitment to strengthening community sector capability
• effectiveness—the extent to which objectives have been met and stakeholder satisfaction has been achieved.The interim evaluation found significant progress was made in achieving actions set out in the action plan. Stakeholders highlighted the cross-sector and cross-government collaboration on work to date as positive
• efficiency—by which inputs are converted to outputs and return on investment is achieved.The evaluation's estimates of efficiency were illustrative only, due to its limited scope and the challenges in measuring efficiency in the community sector. Based on indicative measures of the sector's economic contribution, the government's return on investment would appear to be justified.

While the interim evaluation covered the progress of actions under the plan, the lack of quantitative information meant that impacts and outcomes could not be assessed. The review identified the following limitations:

• Not all actions were completed.
• It was difficult to measure the achievement of objectives and stakeholder satisfaction without established performance indicators and measures.
• The assessment of appropriateness and effectiveness relied on qualitative data and provided no meaningful information on the impact of activities.
• Indicative estimates were used to measure efficiency of the action plan due to the lack of reliable data.

Despite these limitations, the review found an important factor in OCS' success was its focus on partnering with and gaining the acceptance of the community sector and government.

Advice from the then government on the interim evaluation recommended that a more robust and comprehensive evaluation be submitted to the relevant Cabinet committee prior to the end of 2010 in order to inform any future decisions about the action plan. OCS advised that due to the change in government following the 2010 election, and limited resources, it focused on reframing and refining its work to better align with the new government's policies rather than completing the final evaluation.

Evaluation of specific actions

OCS evaluated parts of five of the 25 actions under the action plan.

The following outputs were evaluated:

• the provision of grants to community foundations (action 20)
• the Standard Chart of Accounts (action 10)
• the Community Sector Workforce Capability Framework (action 13)
• the Community Enterprise Catalyst (action 18)
• the Volunteering Portal (action 15).

While none of these evaluations adequately assessed the extent to which the outputs contributed to the achievement of associated outcomes, the post-implementation reviews of actions 10, 13 and 15 provided some useful information:

• they demonstrated the efficiency outcomes in terms of prospective administrative savings in for community organisations that adopted the Standard Chart of Accounts
• they showed that the Workforce Capability Framework could deliver wider economic benefits to community organisations that adopted it, although this information was high level
• the review of the Volunteering Portal looked at the extent to which the objectives of the portal had been met as well as areas for improvement.

Overall, OCS did not adequately evaluate the emerging impacts or the achievement of outcomes under the action plan, nor did it adequately prepare to do so by developing and then applying a structured and integrated approach to evaluation during its implementation planning stage. This means OCS did not undertake the necessary work to understand and track the longer-term benefits of the action plan.

Recommendations

The Office for the Community Sector should apply the lessons learned in implementing The Victorian Government's Action Plan: Strengthening Community Organisations to its ongoing activities and any future programs by:

1. developing implementation plans that comprehensively address objectives and outcomes, roles and responsibilities, time frames and resourcing
2. developing a robust monitoring, evaluation and reporting framework supported by relevant and appropriate performance measures and targets
3. undertaking evaluations at the conclusion of programs with a focus on demonstrating the achievement of objectives and outcomes
4. developing and applying a sound risk management framework that clearly documents how identified risks will be assessed, prioritised and managed.

Response provided by the Sectretary, Department of Human Services – continued