Public Participation in Government Decision-Making

Conclusion

When managed well, public participation activities in Victoria are demonstrably making a valuable contribution to government decision-making.

However, Victoria has no overarching public participation framework to drive activities and promote consistency of practice. Consequently the quality and extent of public participation to inform decision-making varies across the public sector.

Findings

Public participation as a strategic priority

In its role as a central agency, DPC has not developed a whole-of-government strategic public participation framework or statement to help establish a consistent understanding of public participation responsibilities and priorities. Nor has it developed a more detailed whole-of-government framework or statement to provide agencies with clear directives on the importance and broad public participation expectations for the Victorian public service (VPS).

In contrast, other jurisdictions—the Australian Capital Territory, New South Wales, Queensland, South Australia and Tasmania—all have a current whole‑of‑government framework for public participation. The lack of a Victorian framework contributes to inconsistency in activities and terminology across the VPS—for example, the term 'co‑design' is used differently across departments.

Despite the absence of an overarching framework, DPC has led by example in its public participation activities. DPC's commitment is evident in its recent work on the whole-of-government digital consultation platform and the Family Violence Service Reform.

Policies and support in agencies

All three audited agencies have shown leadership by establishing public participation as a priority. It is referenced in key strategic and business plans, partly as a result of using public participation for key projects over the past 12 months. This helps to ensure these agencies remain focused on public participation activities.

Policy framework

Although all three agencies note the importance of public participation in high-level strategic documents, DHHS and DELWP have not yet developed a comprehensive public participation policy or framework. Consequently, these agencies have no documented expectations for their public participation activities. We found that this contributed to these agencies only partially fulfilling their public participation purpose.

Tools to support public participation

Since the beginning of 2016, all three agencies have developed tools to support staff—including guidance, training and access to expertise—although there is variation in the quality of the tools and in the approaches the three agencies have taken.

The LXRA and DHHS guidance material is comprehensive and focused on better‑practice engagement. DELWP has provided guidance only for specific program‑related engagement activities. It plans to develop more comprehensive guidance once it has completed work to investigate requirements for engagement skills and knowledge for different departmental roles.

All three agencies are in the early stages of developing training to build staff understanding of, and capability in, public participation methodologies. DHHS has made most progress in developing and delivering a comprehensive training program.

The three agencies have appropriate access to public participation expertise. DELWP has public participation staff in central, regional and district levels of the agency. DHHS has a smaller centralised team that works with different sections of the agency as required. It is currently developing a 'community of practice' to strengthen peer support and reach across the agency. LXRA has public participation staff in each level crossing removal project, and also engages external public participation expertise.

The three agencies have begun to establish reporting and monitoring systems, although reporting has been narrowly focused on the number and type of public participation activities. The three agencies need to better plan how they will collect relevant data, such as stakeholder satisfaction data, to enable them to assess the effectiveness of public participation activities and identify opportunities for improvement.

Public participation in practice

The three public participation plans we examined stated their purpose, rationale and limitations, although these were not always clearly understood by stakeholders. This was because of inconsistent or ambiguous messaging and agencies' failure to sufficiently take into account the background and context of particular projects. For example, the level crossing removal project did not sufficiently consider the impact of a previous failed rail crossing removal proposal in its public participation plan, and had no strategy for addressing this risk with stakeholders.

Timing

Public participation activities, and the time needed to conduct them effectively, do not always correspond with government processes. The level crossing removal program conducted its public participation activities as planned. The mental health plan and the bushfire management plan only partially fulfilled their public participation purpose because there was not enough time available. Both agencies originally intended to but decided not to seek stakeholder comments on their draft plans.

Incomplete consultation processes have had a negative impact on public participation outcomes. This was evident for the mental health plan where stakeholders expressed disappointment and disillusionment with the public participation process, which damaged relationships with DHHS.

An important part of public participation is to give feedback to stakeholders on how their input was used. None of the three agencies gave feedback on how they used input from public participation activities. However, LXRA's feedback to stakeholders has improved, with the next phase of the level crossing removal project featuring explicit feedback to stakeholders.

Evaluation

There are opportunities for all agencies to improve their approach to evaluating public participation activities. DHHS did not undertake any evaluation of the mental health plan public participation activities. DELWP and LXRA need to improve both the comprehensiveness of their planning and their assessment of how effectively their public participation activities align with key better practice principles.

Agencies should have a comprehensive evaluation plan early in the public participation process. Only DELWP had a documented, comprehensive evaluation plan at the start of its public participation activities. LXRA did not plan an evaluation, but did undertake one.

Evaluations should assess public participation activities against key better practice features. DELWP and LXRA evaluated some but not all key better practice features. They also assessed agency public participation processes and identified achievements and areas for improvement.

There is scope for both agencies' evaluations to be more in line with better practice. DELWP and LXRA did not explicitly evaluate whether the public participation activities met their purpose and fulfilled their promise to engage effectively. In addition, DELWP and LXRA evaluations did not collect data on stakeholder views about their public participation processes and activities.

Recommendations

We recommend that the Department of Premier and Cabinet:

1. collaborate with departments and agencies to develop a whole-of-government framework to provide the Victorian public sector with guidance and parameters for public participation—this framework should:

• provide guidance about public participation principles and align with better practice
• ensure there are clear definitions of public participation terminology and unambiguous language
• provide agencies with guidance about governance and oversight, capability development, access to expertise and monitoring, and evaluation mechanisms
  
  (see Section 2.2.1)

2. develop its own internal framework and guidance for public participation (see Section 2.2.1).

We recommend that the Department of Health and Human Services and the Department of Environment, Land, Water and Planning:

3. develop a comprehensive public participation policy or framework that establishes:

• an operating model for public participation in each agency, including the required resources, governance and oversight arrangements
• the mandate, priorities, responsibilities and mechanisms for collaboration for public participation (see Section 3.2.1)

4. provide their staff with guidance about governance and oversight of public participation activities, information on capability development, access to public participation expertise and monitoring, and evaluation mechanisms (see Section 3.2.1).

We recommend that the Department of Health and Human Services, the Department of Environment, Land, Water and Planning, and the Level Crossing Removal Authority:

5. establish a single source of information to allow for monitoring and reporting of public participation activities across the whole agency (see Section 2.3.5)
6. evaluate and review the effectiveness of their public participation training programs (see Section 3.3.2)
7. clarify and be consistent in all messaging regarding the purpose of public participation (see Section 4.4.1)
8. provide feedback to stakeholders about how their input has been used (see Section 4.4.2)
9. where there are externally imposed time constraints, plan realistic public participation activities to suit the time available (see Section 4.5.1).

Responses to recommendations

We have consulted with the Department of Premier and Cabinet, the Victorian Public Sector Commission, the Department of Environment, Land, Water and Planning, the Department of Health and Human Services and the Level Crossing Removal Authority, and we considered their views when reaching our audit conclusions. As required by section 16(3) of the Audit Act 1994, we gave a draft copy of this report to those agencies and asked for their submissions or comments.

The following is a summary of those responses. The full responses are included in Appendix A.

All agencies responded. The Department of Premier and Cabinet, the Department of Environment, Land, Water and Planning, the Department of Health and Human Services and the Level Crossing Removal Authority accepted our recommendations, and provided detailed action plans and time frames for implementing them.

1.1 What is public participation?

The International Association for Public Participation (IAP2) defines public participation as the involvement of those affected by a decision in the decision-making process. Public participation encompasses activities ranging from simply informing people about government's activities to delegating decision-making to the public.

As shown in Figure 1A, the IAP2 public participation spectrum includes five levels of public participation, from the lowest level (Inform) to the highest level (Empower). The different levels reflect different objectives and depths of public participation.

'Co‑design' is an emerging public participation methodology used by a number of Victorian agencies. Within the IAP2 public participation spectrum, this sits at the more intensive level of the public participation spectrum, between Collaborate and Empower.

Figure 1A

IAP2 public participation spectrum

Source: VAGO, based on IAP2.

According to IAP2, better practice public participation:

• is based on the belief that those who are affected by a decision have a right to be involved in the decision-making process
• includes the promise that the public's contribution will influence the decision
• promotes sustainable decisions by recognising and communicating the needs and interests of all participants, including decision-makers
• seeks out and facilitates the involvement of those potentially affected by or interested in a decision
• seeks input from participants in designing how they participate
• provides participants with the information they need to participate in a meaningful way
• communicates to participants how their input affected the decision.

In line with these principles, the Organisation for Economic Co-operation and Development (OECD) recommends that governments embed public participation as part of their core business, by providing:

• strong leadership and commitment
• coordination of public participation across and within government agencies
• adequate financial, human and technical resources
• appropriate guidance and training
• a supportive and accountable organisational culture.

In this report, when we refer to 'public participation' we use it in line with the IAP2 definition. Public participation is often used interchangeably with other comparable terms such as 'community consultation' and 'stakeholder engagement'.

It is important to recognise that public participation is not the same as public relations or communications, which have the objectives of maintaining a favorable public image, managing public expectations and communicating government decisions and actions.

1.2 Relevant legislation

1.3 Why this audit is important

1.4 What this audit examined and how

2.1 Conclusion

The Department of Premier and Cabinet (DPC) has demonstrated leadership by example at a whole-of-government level by actively using public participation in its recent activities.

However, DPC has not developed a strategic public participation framework or guidance for the Victorian public sector. Despite this, the agencies we audited showed strong leadership in establishing public participation as an agency priority.

2.2 Public participation as a whole-of-government priority

Strong leadership from central agencies is essential for consistent and effective public participation activities across the government.

The Public Sector Reform Agenda, launched by the Special Minister of State in September 2016, includes key messages about the importance of public participation as a whole-of-government priority. It identifies 'co-design' and 'citizen-centric services' as critical to good public administration.

We assessed the extent to which DPC and the Victorian Public Sector Commission (VPSC) have demonstrated leadership in establishing public participation as a priority for the Victorian public sector.

2.2.1 Whole-of-government strategy

At a whole-of-government level, a strategy for public participation is important to provide agencies with high-level guidance, principles to follow, and a clear statement of their obligations. DPC is responsible for leading and communicating public participation across the public sector as a whole‑of-government priority.

DPC's Corporate Plan 2015–19 emphasises the importance of public participation. It identifies 'engaged citizens' as its second objective, and focuses on empowering citizens to participate in policy-making and service design.

Two recent initiatives show DPC's efforts to establish public participation as a whole‑of‑government priority:

• DPC's whole-of-government online consultation platform, Engage Victoria, aims to make public participation more accessible. DPC also plans to use it to provide guidance on planning and delivering broader public participation activities to achieve consistency and efficiency across government agencies.
• DPC is overseeing the implementation of the Family Violence and Service Reform Strategy, based on co-design principles, and has established a Co‑Design, Consultation and Communications team with responsibility for building co-design and engagement capability into DPC's work on this project.

DPC's work in public participation needs to be evaluated to ensure its approach is in line with better practice, and it identifies key lessons and improvements.

Need for a whole-of-government strategic framework

Although DPC has included public participation in key strategic documents and shown leadership in specific initiatives, it has not developed a strategic framework or statement for public participation across the whole of government.

Without such a framework or statement, there is no consistent understanding of public participation responsibilities, priorities or practices. Interviews with DPC staff revealed confusion about public participation and inconsistent practices within agencies. This is particularly concerning at a time when new public participation priorities and methodologies are emerging, such as digital public participation and co-design.

VPSC has not provided any statement or guidance on the importance of public participation for the Victorian public sector, despite its statutory objective to strengthen public sector capability in this area. Public participation is not one of its strategic priorities and its effort is constrained by its current priorities and lack of resources.

Whole-of-government frameworks in other jurisdictions

With the exception of the Northern Territory and Western Australia, all other Australian jurisdictions have a current whole-of-government strategic framework for public participation.

Our analysis of state public participation frameworks found that all:

• highlight public participation as a government priority
• provide guidance and resources to inform better practice
• identify the policy and legislative drivers for public participation.

Four of these whole-of-government strategic frameworks for public participation include an online engagement platform.

2.2.2 Monitoring at a whole-of-government level

There is no data available on the quality or effectiveness of public participation within the Victorian public sector. DPC and VPSC do not monitor their own public participation activities or those of other government agencies.

Monitoring public participation activities would allow the public sector to evaluate its activities, identify areas for improvement and design more effective processes.

In the absence of government data on the adequacy of public participation, we analysed our findings on public participation from VAGO audits between 2013 and 2016. In total, 30 per cent of the 84 performance audits we conducted during this period included negative findings about public participation in the Victorian public sector.

Figure 2A breaks down the types of negative public participation findings identified in VAGO audits between 2013 and 2016.

Figure 2A

Types of negative public participation findings in VAGO performance audits, 2013–16

Note: A single audit may have included more than one negative public participation finding.

Source: VAGO.

This analysis shows the particular aspects of public participation that DPC needs to address as a priority, and demonstrates that agencies need greater direction. DPC also needs to improve its oversight and monitoring of agencies' public participation activities.

2.3 Public participation as an agency priority

Strong leadership and governance within portfolio agencies is fundamental to establishing public participation as a priority throughout the Victorian public sector.

We assessed the extent to which the Department of Environment, Land, Water and Planning (DELWP), the Department of Health and Human Services (DHHS) and the Level Crossing Removal Project (LXRA) have established public participation as an agency priority. We also looked at whether they had established reporting and monitoring systems and how effective they were.

DELWP, DHHS and LXRA have all shown strong leadership in establishing public participation as an agency priority, through key strategic and business plans. All three agencies have high-level, multi-tiered governance committees that oversee their public participation activities. Although these agencies have begun to establish reporting and monitoring systems, there are opportunities to improve the way they collect and analyse data about the performance of their public participation activities.

2.3.1 Key policies, reports and legislation

Recent policy changes have emphasised the importance of public participation for all of the audited agencies. DELWP and DHHS have strengthened their existing focus on public participation, and LXRA has continued to prioritise it. Figure 2B shows key policy changes since 2010 that have had implications for public participation in the audited agencies.

Figure 2B

Key policy changes and their implications for public participation

Source: VAGO.

As a result of policy changes, all three agencies are in transition, and it is still too soon to determine the effectiveness of the improvements they have put in place to better support public participation activities.

2.3.2 Strategic and operational plans

It is important that agencies embed public participation into their strategic planning. This helps to set the agency's direction and use of resources, and guide the activities it undertakes.

Public participation in strategic plans

All three agencies have included public participation in their strategic or corporate plans and have demonstrated their commitment to public participation by providing resources and establishing effective systems.

Within their plans, each agency focuses on a different aspect of public participation:

• DELWP emphasises 'collaboration and service excellence' by putting the 'community first'
• DHHS emphasises 'co-design and engagement'
• LXRA emphasises 'stakeholder engagement'.

The agencies vary in how they define public participation and in their chosen methodologies and priorities.

DHHS's planning documents focus on co‑design, which is positioned at the higher end of the International Association for Public Participation (IAP2) Spectrum, between Collaborate and Empower.

At DELWP and LXRA, public participation experts work across the strategic and operational levels of the agency to ensure that staff have access to their expertise.

All three agencies have allocated significant resources and contracted experts in public participation to support their activities, build staff capability and establish effective agency systems. This demonstrates the agencies' commitment to public participation.

Public participation in operational plans

Although their approaches vary, it is positive that DELWP, DHHS, and LXRA have all incorporated public participation into their operational planning. It is too early to tell how well the three agencies have incorporated public participation into their operations through their operational plans.

DELWP and DHHS require divisional business plans to include public participation as a priority and report on progress every six months. They also require executive performance plans to identify priority actions for public participation.

DHHS has developed a leadership charter that commits individual executives and the board to 'collaborate with partners and stakeholders to support person-centred innovation'. DHHS plans to report on its leadership charter every six months.

DELWP has introduced a community charter to clarify its approach for agency staff and for members of the public. Under the charter, DELWP encourages divisions to identify public participation activities, and include progress on these activities and the lessons learnt as standing agenda items in all team meetings. To support the implementation of the community charter, DELWP has delivered community service and interaction training sessions to over 2 000 employees (over 95 per cent of its staff).

Both DELWP and DHHS have also engaged extensively with their staff to emphasise the importance of public participation in their work, including in key messages and in regular communication between ministers, executive leadership and staff.

LXRA has developed detailed public participation plans and governance systems. Its redeveloped strategic communications strategy and its Workplan 2016–17 identify clear objectives for public participation. They also recognise the risk of poor public participation and identify key actions, including a comprehensive methodology for overseeing and delivering public participation programs.

2.3.3 Governance

DELWP, DHHS and LXRA all have high-level governance committees that oversee the effectiveness of public participation activities and identify opportunities for improvement. All three agencies have recently strengthened their strategic oversight of public participation activities by establishing further governance committees at multiple levels of the agency, beginning with the executive leadership level. It is too early to determine the effectiveness of these governance structures.

Because of the size and dispersed structure of DELWP and DHHS, both agencies face challenges in ensuring their governance systems are effective. Recognising this, both agencies have begun work to improve their governance, although there is room for further improvement.

At DELWP, the agency's public participation oversight subcommittee is part of the Forest, Fire and Regions Group (FFRG). As the regional directorate that delivers services across Victoria for the entire DELWP portfolio, FFRG was given leadership and implementation responsibility for all of DELWP's public participation activities. As a result of this structure, DELWP faces the challenge of ensuring that all parts of the agency take responsibility for public participation. DELWP's senior executive team has recognised this challenge and is considering forming an additional department-wide subcommittee that reports directly to the senior executive team on public participation activities.

At DHHS, public participation is overseen by the Strategic Engagement Group (SEG), a subcommittee of the Integrated Community Services Reform Subcommittee. The SEG's purpose is to ensure that:

• co-design and engagement activities are coordinated and aligned with each other and DHHS's broader reform agenda
• stakeholders, including DHHS staff, receive consistent messages.

The SEG is a way for DHHS to oversee public participation across priority reform areas. Although in its early days, its mandate is confined to better coordinating public participation rather than monitoring the performance of public participation activities, analysing lessons or proposing improvements to processes or systems.

During its first year of operation, LXRA had under-developed governance systems. Since then, it has progressively strengthened its governance committees and reporting mechanisms for public participation, with a focus on monitoring stakeholder engagement and identifying improvements to systems across the agency. The roles and responsibilities of these committees are still being developed and need further clarification.

2.3.4 Reporting and monitoring

Across all three agencies, monitoring and reporting mechanisms are under-developed. DELWP and LXRA are taking action to improve monitoring and data collection to help them better understand the effectiveness of their public participation activities.

DELWP used the results from its 2015 Customer Stakeholder Intelligence Survey to establish benchmarks for stakeholder satisfaction with DELWP's activities. This benchmark allows DELWP to compare its performance to that of other government and non-government agencies. DELWP has not yet fully realised the benefits of this research.

DELWP is also in the early stages of developing new data collection mechanisms, including measures for the effectiveness of community engagement. The FFRG is developing an engagement monitoring, evaluation and reporting framework. DELWP expects that these tools will be adopted across the whole agency over time. The framework will enable DELWP to test and refine its public participation model.

Key issues for DELWP to consider include:

• ensuring public participation methods, tools and lessons are applicable and relevant across the agency—DELWP staff raised this concern during our interviews
• ensuring information about participant satisfaction is collected consistently, rather than relying on self-reporting from district staff
• providing training and instruction so that DELWP staff are able to collect consistent, accurate and useful information for monitoring and evaluation activities.

LXRA has periodically reviewed and monitored its public participation activities informally. Internally, it reports on its community engagement activities weekly and monthly, and produces regular progress reports for the public. More recently it has established monitoring and reporting requirements within its community engagement strategy.

2.3.5 Information management systems

LXRA has used an information system to capture stakeholder feedback on its level crossing removal project. It allowed the project team to compile feedback from residents on what was considered important in their local community.

Both DELWP and DHHS are in the early stages of developing information systems to manage feedback and information about stakeholder and public participation activities, to help them coordinate activities and improve performance. These systems should enable the agencies to use community input across programs and to capture and share lessons, which should help them to continue improving their systems.

DELWP and DHHS are working to overcome some issues they have encountered in the development of information management systems for public participation, including:

• ensuring the systems are suitable for collecting and analysing the information
• developing detailed guidance and building capability across the agency to ensure staff know what information to record and how
• managing issues arising from the collection and recording of private information from citizens for use for a non-specified purpose
• allocating the appropriate resourcing required in the development of a system and its ongoing maintenance and support.

3.1 Conclusion

The extent and quality of DELWP's, DHHS's and LXRA's agency-wide policies and support for staff varies. LXRA and DHHS have comprehensive guidance, whereas DELWP does not. LXRA has an overarching framework, but DHHS and DELWP do not.

All agencies have taken different approaches to training, mentoring and providing support. These inconsistencies in practice are partly caused by the absence of a whole-of-government framework.

3.2 Policies and frameworks

Clear direction to build consistent understanding and expectations across agencies is necessary to ensure high-quality public participation practice throughout the Victorian public sector.

We assessed the extent to which DELWP, DHHS, and LXRA have agency‑wide policies and frameworks in place to establish a consistent understanding of public participation and agency responsibilities and requirements for their staff.

LXRA has developed a comprehensive framework for public participation. DHHS and DELWP do not have an overarching framework. A comprehensive framework is important to clarify responsibilities across both agencies.

3.2.1 Agencies' policies and frameworks

DELWP, DHHS and LXRA have noted the importance of public participation in key high-level strategic and corporate documents. However, these are high-level documents which indicate importance but do not clarify agencies' expectations.

LXRA has recently developed a comprehensive engagement and communication framework that provides clear messages about agency requirements, expectations, roles and responsibilities. This framework incorporates information gathered during 12 months of public participation activities and will shape LXRA's future approach.

DHHS has not developed a comprehensive public participation policy or framework to underpin their high-level strategic documents. Guidance material provides some information, but it is focused more on giving instructions on public participation and co‑design to practitioners. DHHS is currently developing a stakeholder engagement guide to provide comprehensive guidance on conducting public participation activities.

Key staff in DHHS have indicated in interviews that there is confusion about public participation, including lack of clarity about the emphasis and use of co-design techniques and how they relate to other public participation priorities. The stakeholder engagement guide and other recently released guidance material may address some of this confusion, although DHHS still needs to develop a clearer statement about its agency-wide approach to public participation.

DELWP does not have an overarching public participation strategy or framework. The Forest, Fire and Regions Group (FFRG) has developed a community partnership engagement strategy to guide its work, but this is not applicable to other DELWP portfolio areas and intentionally does not reference public participation concepts or provide detail, but rather uses broad descriptive language. Although this strategy is positive in flagging the importance of public participation and the range of forms public participation may take, it does not provide a clear conceptual framework across the full breadth of DELWP's functions and public participation responsibilities.

FFRG has advised us that it has intentionally not included what is often perceived to be complex public participation information in order to build a stronger recognition across DELWP that public participation is everyone's business and is fundamental to all customer and community interactions. Not providing a roadmap approach to public participation creates a risk that key aspects of public participation may be overlooked.

3.3 Tools to support public participation

Well-designed tools support consistent practice and staff capability in public participation methodologies.

We assessed the extent to which DELWP, DHHS and LXRA provide sufficient guidance, training and access to public participation expertise to support consistent and efficient approaches to public engagement.

All three agencies have developed tools to support public participation. However, as the agencies have recently developed these tools, or are still to fully implement them, it is too early to assess whether they effectively support public participation.

3.3.1 Guidance for staff

It is positive that all three agencies have developed guidance for staff, although DELWP has not yet rolled out its resources and should do so as a priority. Both LXRA and DHHS have developed comprehensive guidance, but it is too early to tell whether it is being used by staff.

LXRA recently developed a communications and engagement strategy, including a handbook and planning template, which outline requirements and provide instructions, tools and templates. These resources provide a strong foundation to support consistent approaches to engagement across LXRA projects, but it is too early to evaluate how well staff are applying them.

DHHS is currently developing a comprehensive stakeholder engagement guide. This guide will emphasise the role of engagement in DHHS work, the process needed to run a successful public participation activity, and a guide to DHHS's use of co-design methods.

FFRG has developed the Community Engagement Capability and Capacity Framework, although it is not clear when and how DELWP will roll it out. It focuses on training. Comprehensive information is also available in DELWP's engagement planning workbook and tools, developed by the former Department of Sustainability and Environment in 2004 and republished in 2015. However, DELWP advises that there is poor awareness and use of this resource and that its comprehensiveness sends a signal that engagement is complex and requires specialist expertise. Given these concerns, it is important that DELWP develops an overarching framework with supporting resources and training so that there is a consistent approach.

3.3.2 Training

Training in public participation methodologies is another key support to build staff capability. All agencies are heavily investing in public participation training, although it is too early to assess how effective this training is.

DHHS has invested significant resources in training. The training uses a model based on practical skills-based training rather than classroom-style delivery, and at this stage is focused on developing co-design capability. DHHS will also use its stakeholder engagement guide, once it is completed, as well as expert support from specialist public participation staff and champions, and its ongoing work to establish a 'community of practice'—a peer support network across the agency.

This work demonstrates that DHHS has a comprehensive approach to the challenging objective of building capability across such a large and dispersed agency. It is critical that DHHS monitor and evaluate its training and resources regularly.

DELWP has shown its commitment to public participation by providing introductory training to 95 per cent of its staff (over 2 000 people). The aim is to develop agency‑wide basic public participation skills in order to deliver on the promise in DELWP's Community Charter. This training has now become part of the DELWP orientation program.

DELWP is seeking to consolidate this training with more in-depth or tailored training programs depending on staff members' roles. FFRG, which has primary responsibility for community interaction, is the first group to be offered this training.

LXRA has provided specialised training, such as resilience training, as needed. LXRA has also engaged external consultants to create training on the implementation of its handbook and specialist communications training. The aim is to make this training available to engagement staff across all projects. Although this is a positive step, the provision of training to staff has been slow, given that LXRA has been in operation since May 2015.

3.3.3 Access to public participation expertise

Another support to build staff capability is access to mentoring from internal and external experts. All three agencies have taken a different approach, and all are effective in ensuring staff have access to public participation expertise.

We assessed the extent to which DELWP, DHHS and LXRA provide access to specialist public participation expertise and provide opportunities to share lessons and build capability in public participation methodologies.

LXRA has public participation staff located in each project team. This model is effective given LXRA's role and the large number of projects it is managing. LXRA supports its staff with external public participation expertise as required.

DELWP has a Community Partnership Engagement Support team, composed of 11 full-time-equivalent staff—with six fixed-term staff to provide engagement support and advice related to forestry and fire planning.

Reviews of documents and interviews show that the teams in both agencies inform direction setting, develop guides and tools, deliver advice across their agencies and provide intensive support on key public participation activities. Both specialist teams locate their own staff within operational teams to provide intensive support for a specific time period. This is an effective way to build capability, self-sufficiency and strong networks across the agency.

DELWP has a longstanding community of practice focused on engagement, but it is unclear how it fits in DELWP's new public participation structure, which aims to support peer learning and information exchange to inform system improvements.

DHHS is in the early stages of establishing a community of practice to build capability and foster shared learning, through both online and face-to-face training. DHHS's approach of providing broad general training and supporting co-design pilot projects should also contribute to building expertise in public participation throughout the agency.

4.1 Conclusion

Conducting an effective public participation program is difficult and complex. All three agencies were generally effective in their public participation activities, and the programs met most of their objectives. There are opportunities for all agencies to learn from their activities and identify opportunities for improvement. DHHS and DELWP need to better plan their activities to fit within available time frames.

4.2 Scope of our assessment

The three public participation programs we looked at are located within different agencies and sectors, and have different purposes. This means that some features of these public participation programs are not directly comparable, but they do help us to identify key achievements and common challenges in the public participation process.

We measured effectiveness by assessing how well the programs achieved their public participation objectives and the extent and effectiveness of stakeholder consultation. We did not assess the resulting outcomes or stakeholders' satisfaction with these outcomes.

IAP2 better practice states that good public participation should include a clear purpose and defined deliverables, and use appropriate public participation methods to fulfil this purpose. It should also involve those who are affected by the decision in the decision-making process.

Clarifying the purpose of public participation means that stakeholders have a clear understanding of the impact they are able to have, and agencies can effectively manage the public's expectations. All three agencies could have clarified the purpose of their public participation better.

For two of the programs—the mental health plan and the bushfire management plan—the purpose of public participation was to seek stakeholder input into the development of strategic plans. The third program—the level crossing removal project—had a broad‑ranging purpose to inform and consult stakeholders about the implementation processes and key design and construction features of the program.

All three programs used a wide range of public participation activities to inform and consult relevant stakeholders, agencies and communities. Only one program—the level crossing removal project—fulfilled its purpose and undertook all of its activities as planned. More detailed information on each program's purpose, activities and stakeholders can be found in Appendix B.

4.3 Planning public participation activities

To be effective, public participation activities must have:

• clearly defined objectives and scope
• an appropriate methodology that reflects understanding of affected stakeholders and their needs
• a plan for identifying and mitigating risk
• appropriate resources and time lines
• plans for review and evaluation.

We examined the extent to which the audited agencies' public participation activities were comprehensively planned, and in line with requirements and documented public participation objectives and methodology.

All three programs had public participation strategies and plans for their public participation activities, and the plans included most components. There are opportunities for all three agencies to improve the comprehensiveness of their planning documentation.

4.3.1 Objectives and scope

DELWP developed a comprehensive engagement plan for the bushfire management plan that included clear objectives and clarification of elements that were negotiable and non-negotiable.

For the mental health plan, the purpose for the public participation activities was unclear. DHHS developed three primary public participation documents. Across these documents, the purpose of the public participation and how input from stakeholders would inform the plan's development was ambiguous. At various points, it was described as aiming to inform, consult, collaborate and involve. These terms imply different degrees of public participation, decision-making power and ownership, which confused stakeholders.

LXRA developed an overarching communications and stakeholder engagement strategy for public participation during phases one and two of the level crossing removal project, with additional public participation plans for particular activities. These included clear objectives focused on both the purpose of the public participation and the quality of the process.

4.3.2 Methodology

The methodology was mostly planned well across the three programs, with some room for improvement. The three agencies did not adequately document their risk mitigation strategies and resourcing.

All three programs demonstrated sufficient planning of their approaches to public participation, with an appropriate balance of traditional and innovative activities, and clearly delineated outcomes and time lines for each component. In some cases, the programs used innovative activities that established new ways to engage with stakeholders.

All three programs identified a diverse range of stakeholders as the focus of the public participation. However, analysis of stakeholders' levels of interest, influence and need varied. The bushfire management plan included a comprehensive list of key stakeholders and categorised them according to interest and influence, and then identified appropriate public participation activities to maximise their participation.

All three programs identified risks or issues associated with public participation, but the agencies' mitigation strategies to manage these issues or risks varied. Agencies need to improve their planning for risk mitigation. Careful planning for risks at the outset could have contributed to better outcomes during implementation of the agencies' public participation programs.

Resource planning for the three programs was incomplete, with little clarity within each plan to determine whether resources, staffing numbers and capability would be sufficient.

For the level crossing removal project, there appeared to be appropriate resources and staffing allocated for public participation activities, using a combination of LXRA staff and external contractors and consultants. However, LXRA had insufficient documentation of budgets for activities, staffing resources and skills across internal and external staff.

Although time lines were outlined in the public participation plans for each program, staff across the three agencies gave consistent feedback that effective public participation takes time and that time lines were a constant challenge.

4.4 Managing stakeholders

Stakeholders need to be fully informed throughout the public participation activity to ensure they have a good understanding of the reasons for seeking their feedback. This includes:

• informing stakeholders about the purpose and scope of public participation and ensuring they understand them
• providing relevant and timely information and resources to stakeholders to enable them to give meaningful input
• informing stakeholders of viewpoints given during public participation activities and how this information was used to inform decision-making
• giving stakeholders an opportunity to provide feedback on the public participation process, to help assess which aspects worked and which could be improved.

We examined the extent to which the selected programs provided appropriate information to stakeholders to clarify the purpose of the public participation activities, enable informed input, and to share the input received from stakeholders and how it was used. We also examined the extent to which the programs collected stakeholder feedback on satisfaction with the public participation process.

One of the challenges facing the level crossing removal project and the bushfire management plan was the difficulty of engaging stakeholders on issues that were strategic but not currently urgent to stakeholders.

The level crossing removal project and the mental health plan engaged groups that can be hard to reach, such as culturally and linguistically diverse groups and regional stakeholders. The bushfire management plan did not. DELWP has acknowledged it did not adequately engage the program's diverse stakeholders.

4.4.1 Clarifying the purpose for stakeholders

All three programs provided information to stakeholders, such as discussion papers, fact sheets, newsletters and stakeholder briefings. This information identified the role of stakeholders in the decision-making process, the scope of public participation activities and the extent of their influence on decision-making.

For the bushfire management plan, the purpose of public participation activities was sufficiently clear and unambiguous.

Stakeholders for the mental health plan and the level crossing removal project did not fully understand the purpose and limitations of the public participation activities, despite being given information about them. Reasons for this lack of understanding included inconsistent or ambiguous messaging, and agencies' failure to sufficiently take into account the previous history and context of the issue, which was likely to influence how stakeholders interpreted the purpose of the public participation.

Community expectations created by a previous failed rail crossing removal proposal resulted in stakeholders not fully understanding or accepting the purpose of public participation activities, particularly the first ones conducted.

4.4.2 Reporting back to stakeholders

Reporting back to stakeholders is important to ensure that they understand how their feedback has been used. It also helps develop stakeholders' ownership of the outcomes. When reporting back to stakeholders, the three programs provided different levels of information.

For the mental health plan and the level crossing removal project, the community perceived the information they received about their input to be insufficient and not well distributed. For the mental health plan, this contributed to stakeholder dissatisfaction and a perception that stakeholder input was not clearly evident. As a result, stakeholders were confused about the purpose of the public participation and its limitations.

The engagement team for the bushfire management plan advised us that they had reported general feedback to the community reference group but they had not provided information about how input had been used. They acknowledged this was an area that could be improved in the future.

DHHS did not clearly report how community input had been used to shape the mental health plan. Engagement staff reported that feedback to stakeholders was not as extensive as planned due to tight time lines and changes to the format of final public participation activities. This appears to have contributed to the disappointment among participants, particularly consumers and carers.

The level crossing removal project provided detailed information about the input they had received from stakeholders and made a good effort to advise stakeholders how this input had informed decisions. However, the agency could have provided more information about how community input had been used in the competitive design selection process. LXRA advised us that this information could not be shared due to a commercial-in-confidence selection process. Although this is a real and reasonable constraint, it could have been better managed by LXRA explaining these constraints to stakeholders.

4.4.3 Stakeholder satisfaction

Across the three programs, stakeholder feedback about satisfaction with the public participation process has only been collected for a small number of activities. Where stakeholder feedback was collected, it was generally positive, although—as already discussed—there was some dissatisfaction with the lack of feedback about how input had been used.

The level crossing removal project and bushfire management plan both collected participant satisfaction feedback for some public participation activities, as outlined in Figure 4B. For these activities, participants were satisfied with the process and opportunity to provide input.

Figure 4B

Stakeholder feedback collected on public participation activities

Source: VAGO.

For the mental health plan, DHHS did not actively collect formal feedback from stakeholders about their satisfaction with the process, despite it being a component of the program's public participation plan. However, based on discussions with DHHS staff and our meetings with a limited number of key public participation participants, stakeholders appear to be largely very satisfied with the public participation process and activities, but they were dissatisfied with information provided about how their input was used.

4.5 Timeliness of public participation activities

It is essential that agencies build in sufficient and reasonable time lines for meaningful public participation. Rushed public participation processes can lead to scepticism among participants and create the risk that the program will not deliver its objectives.

We examined the extent to which the selected public participation programs allowed enough time to gain informed stakeholder input.

Despite tight time frames, the level crossing removal project fulfilled its purpose and undertook its activities as planned. The mental health plan and the bushfire management plan were unable to follow through on their public participation activities due to short time lines. This reduced the effectiveness of their programs. These agencies need to better plan their public participation activities to fit within available time frames.

Mental health plan

Public participation activities for the mental health plan were delayed by several months, which reduced the time available for consultation. Production of the discussion papers, and the ministerial and departmental approval processes, exceeded the planned time frames. As a result, despite being planned for May and June 2015, public participation activities occurred in August and September. This was very close to the release date of the final plan in November 2015.

This delay limited DHHS's consultation of and feedback to stakeholders, and contributed to some disappointment among consumer and carer stakeholders—which is important given the previous history of poor consultation with the sector.

Bushfire management plan

The bushfire management plan also experienced challenges in the timing of public participation activities, which it identified in the project evaluation report. Staff advised that they did not undertake all planned public participation activities due to time constraints. Internal delays in approval processes also resulted in a delay in the community feedback workshops on the draft bushfire management plan. This compromised the value of the input collected in these workshops, as they occurred too late for the input to be taken into account before the plan was finalised.

4.6 Evaluating and improving public participation

All public participation activities should be evaluated to confirm that they achieved the public participation purpose and identified key lessons and areas for future improvement.

Failing to evaluate public participation means agencies have no clear understanding about whether they have achieved their objectives or successfully engaged with stakeholders. They are also unable to address any weaknesses and learn from their experiences in future public participation processes.

We examined the extent to which the three programs have demonstrated effectiveness in achieving public participation objectives and informing improvements.

All three agencies need to improve their approach to evaluation to ensure their agency can effectively learn from its public participation practices and improve public participation.

The bushfire management plan and level crossing removal project conducted evaluations of their public participation activities, although there is scope for both agencies to improve their approach. For the mental health plan, DHHS did not evaluate the public participation processes, outcomes or stakeholder satisfaction.

Bushfire management plan

The bushfire management plan evaluated the introduction of risk-based bushfire management planning, focusing on the impact, effectiveness and achievements of the project, and on learning and continuous improvement. The evaluation included questions focused on public participation, including both the process and outcomes.

The evaluation was planned before the program began and involved a detailed matrix to measure effectiveness. Although this approach to evaluation is encouraging, it could be improved by broadening the data collected, making public participation a specific focus within the evaluation, and including assessment of features of good public participation practices.

DELWP's Forest, Fire and Regions Group is currently working to review and improve the evaluation process of strategic bushfire management public participation activities. This should help DELWP to conduct more informative assessment of the effectiveness and quality of public participation.

Level crossing removal project

LXRA conducted a range of informal and mid-term reviews of the public participation activities that were part of the level crossing removal project. LXRA also recently completed a formal external review.

Both the reviews and evaluation relied on the views of a small number of staff from LXRA and project partners. LXRA needs to also routinely collect stakeholder feedback about their public participation experiences and their satisfaction with public participation opportunities. LXRA recently recognised this in its newly developed Community Communications and Engagement Strategy and Handbook.

LXRA has adopted and consolidated recommendations from these reviews in the Frankston level crossing removal project, and in enhanced governance, guidance and reporting systems.

RESPONSE provided by the Secretary, Department of Premier and Cabinet

RESPONSE provided by the Commissioner, Victorian Public Sector Commission

RESPONSE provided by the Secretary, Department of Environment, Land, Water and Planning

RESPONSE provided by the Secretary, Department of Health and Human Services

RESPONSE provided by the Chief Executive Officer, Level Crossing Removal Authority