Clinical Governance: Health Services

Do health services' systems and processes assure quality and safe care?

Video presentation

Video transcript

Key facts

Note: * The People Matter Survey is an annual public sector survey. ** We excluded Djerriwarrh Health Services as there were only five serious incidents during our sample period. *** The nature of recommended actions Djerriwarrh Health Services had to undertake were relatively simple.  

What we found and recommend

We consulted with the audited agencies and considered their views when reaching our conclusions. The agencies’ full responses are in Appendix A. 

Establishing and embedding clinical governance frameworks

The Victorian Clinical Governance Framework (VCGF) sets the Victorian Government's expectations for the systems and processes that public health services need to deliver safe and high-quality healthcare. The VCGF requires each health service to establish its own clinical governance framework to support its staff to work towards common goals. While accounting for local needs, these individual frameworks must comply with the VCGF. 

Melbourne Health (MH) and Peninsula Health (PH) have met this requirement. Both health services have developed clinical governance frameworks that comply with the VCGF and have embedded them through initiatives to translate their expectations into practice. Through these efforts, MH and PH are reinforcing a consistent message to their staff and keeping them focused on achieving high-quality care. 

In contrast, while Ballarat Health Services (BHS) started developing its framework in October 2019, it only completed it in January 2021—three and a half years later than expected. It has only recently started promoting and implementing it.

Djerriwarrh Health Services' (DjHS) framework does not fully comply with the VCGF because it does not identify underlying priorities to help it achieve its framework goals, such as activities to ensure a visible and engaged executive leadership. To date, DjHS has not implemented its framework; it has not actively promoted or embedded it in its operations to drive quality improvement activities. 

As a result, unlike MH and PH staff, staff we interviewed at BHS and DjHS did not have a good understanding of their organisation’s clinical governance framework or the priorities and expectations it contains.

While the Department of Health (DH) requires health services to comply with the VCGF as part of its Policy and Funding Guidelines, it does not assess this requirement.

Recommendations about establishing and embedding clinical governance frameworks

Establishing and supporting a positive patient safety culture

Health services with a positive patient safety culture are more likely to detect clinical risks early, which allows them to act and prevent avoidable harm to patients. A health service has a positive patient safety culture when staff:

• are safe from bullying and harassment and do not fear reprisal or retribution when they speak up about personal or patient safety concerns
• are confident to speak up to their peers and managers about personal or patient safety concerns and are confident that management will act
• actively engage in activities that maintain or increase their focus on safe and high quality care.

Only MH and PH have established a positive patient safety culture.

People Matter Survey

The Victorian Public Sector Commission’s annual People Matter Survey (PMS) captures staff perceptions of the patient safety culture at their workplace and how safe they feel to speak up. At MH and PH, staff perceptions have improved since 2016, which was when the Victorian Government released the Targeting Zero: Supporting the Victorian hospital system to eliminate avoidable harm and strengthen quality of care report (Targeting Zero). This means that staff at these health services are now more likely to report incidents, understand the importance of quality improvement activities and participate in them to reduce the risk of patient harm.

By contrast, BHS and DjHS have not improved their relevant PMS results since 2016. DjHS’s results for bullying and how safe staff feel to speak up have deteriorated. 

All four health services, especially DjHS, scored low results in:

• staff feeling safe from reprisal if they report improper conduct
• staff confidence in the integrity of investigations into safety issues.

This suggests that the audited health services need to be more transparent about their investigation processes and address staff concerns about reprisal to strengthen their patient safety cultures.

Culture initiatives

MH and PH have a comprehensive suite of initiatives to build and maintain a positive patient safety culture. They directly support their staff to speak up and provide multiple avenues for them to do so. For instance, MH trains its staff to use a communication tool that uses a stepped approach to raising and escalating concerns with colleagues. MH and PH also promote and reward desired values and behaviours, and MH has evaluated its initiatives to identify barriers and enablers to patient safety.

BHS and DjHS have initiatives to increase staff awareness on patient safety and set expected values and behaviours. However, they lack initiatives to develop their staff’s skills and confidence to speak up. This is concerning especially for DjHS, whose PMS results indicate that its staff do not feel safe to do so.

Recommendations about establishing and supporting a positive patient safety culture

Identifying and responding to quality and safety risks

Gaps in how health service boards monitor quality and safety 

Health services identify quality and safety risks by monitoring clinical incidents and their quality and safety performance indicators. A health service should prepare regular reports for its board so the board can assure DH, the Minister for Health and its local community that it is providing high-quality and safe care.

The boards at all four audited health services receive regular reports on incidents and quality and safety performance indicators. However, there are gaps in these reports that limit each board’s ability to assure that their health service is promptly identifying and addressing quality and safety risks and areas of underperformance. Specifically:

Recommendations about gaps in how health service boards monitor quality and safety 

Analysing and responding to performance indicators

To comprehensively oversee its quality and safety performance, a health service board must:

• monitor sufficient and relevant performance indicators
• understand when results represent a genuine change in performance 
• know that its executives are acting when performance does not meet targets. 

Statement of priorities

At a minimum, health services should monitor their quality and safety performance by tracking it against the mandatory key performance indicators (KPIs) in their statement of priorities (SOP) to have a sufficient view of safety and quality. Beyond their SOP, health services should also monitor KPIs that are relevant to their clinical governance framework goals to assess if they are achieving them. 

Of the four audited health services, only PH monitors all of its SOP quality and safety indicators in its routine internal board reporting. MH monitors most of its relevant SOP indicators and BHS and DjHS have significant gaps. Specifically:

See Appendix F for a full list of SOP KPIs that each audited health service monitors.

BHS and DjHS's boards rely on quarterly ‘Monitor’ reports from the Victorian Agency for Health Information (VAHI) to review their performance against most of the SOP quality and safety KPIs that are not reported in their bimonthly and monthly KPI board reports respectively. The frequency of this reporting may be appropriate for some indicators, particularly when considering the resources required to generate additional reporting. However, monitoring the large majority of SOP indicators only quarterly limits these health service boards’ ability to identify and address any underperforming areas in a more timely way. 

Since feedback from the audit, DjHS has expanded its monthly activity performance report and now includes most of the unreported SOP KPIs.

Additional indicators

Beyond their SOP indicators, all health service boards regularly monitor additional KPIs in their internal quality and safety KPI reports that they identify as a priority (see Appendix G for a full list of these KPIs). Some common additional KPIs include:

• timeliness in providing hospital discharge summaries
• common clinical incidents, such as pressure injuries, falls and medication errors
• unplanned readmissions for specific community groups or medical conditions.

For MH and PH, who already monitor most of their SOP indicators, these additional KPIs mean their boards have a comprehensive view of their quality and safety performance. Both health services have also aligned their internal quality and safety KPI reports to their clinical governance frameworks to allow them to track their progress against specific clinical governance goals. 

In contrast, while BHS and DjHS’s boards monitor their additional KPIs, these KPIs do not address the gaps in their SOP KPI monitoring, which is the minimum monitoring requirement set by DH. This means that BHS and DjHS’s boards are not monitoring sufficient KPIs to have a comprehensive view of service quality and safety.

Additionally, BHS and DjHS do not group and report on KPIs against their local clinical governance framework goals, unlike MH and PH. This means that BHS and DjHS’s boards cannot easily assess if they are meeting their clinical governance goals. 

Identifying and investigating poor performance 

Health services need to promptly identify quality and safety risks so they can swiftly address them and prevent harm. To do this, health service boards need comprehensive reports that present clear analysis of quality and safety performance, risks and actionable insights. In particular, these reports need to include long-term trend analysis and reasons for underperformance.

Only two of the four audited health services are doing this:

Recommendations about analysing and responding to performance indicators

Investigating and responding to clinical incidents

Incident management and open disclosure policies

All audited health services have incident management policies that provide staff with clear information on their processes for reporting, investigating and responding to incidents.

However, BHS and DjHS’s policies do not clearly emphasise the support available to staff involved in incidents. DjHS's policy also does not:

• stress that staff must act immediately to contain risks and treat harm
• clearly indicate time frames for completing in-depth case reviews (IDCRs) and Safer Care Victoria's (SCV) recommendations and action plan document. 

All audited health services have open disclosure policies and procedures that are broadly consistent with the Australian Commission on Safety and Quality in Health Care's (ACSQHC) Australian Open Disclosure Framework. However, only MH routinely monitors its open disclosure process to ensure each case is promptly and appropriately actioned.

Delays in completing serious incident investigations

All audited health services complete sentinel event investigations within SCV's time frames. However, none of them consistently complete investigations into other serious incidents within timeframes set in their own policies. This means that these health services are not consistently identifying causal and contributing factors early enough, which creates a risk that similar serious incidents will recur. All health services stated that these investigations are delayed because they lack the required staff capacity and capability. 

Recommendation about completing serious incident investigations

No regular thematic analyses of incidents

Health services should undertake regular, service-wide thematic analyses of all serious (ISR 1 and ISR 2) and less serious (ISR 3 and ISR 4) incidents to determine common themes or clusters relating to:

• clinical divisions or units
• common causal or contributing factors.

Not all health services undertake regular analyses of serious incidents. Only MH analyses serious incidents to identify any underlying themes every six months. While BHS analyses serious incidents monthly to identify themes, its analysis does not go into enough detail to identify common contributing factors. For instance, in February 2020, BHS identified that 70 per cent of medication errors involved its ‘administration process’ but did not further examine specific issues in its process for administering medication. BHS's individual monthly analyses also do not identify if common contributing factors recur across multiple months. 

PH and DjHS analyse serious incidents for common themes on an ad-hoc basis. For DjHS, this also depends on it having sufficient data available for analysis. 

None of the four audited health services undertake regular thematic analyses of less serious incidents. Consequently, they risk not identifying clusters of incidents or common factors underlying individual incidents.

Recommendations about thematic analyses of incidents

Delays in completing serious incident recommendations

If a health service identifies a risk through a serious incident investigation, it needs to quickly address it to prevent the incident from recurring. 

Overdue recommendations

Of the audited health services, only MH had no overdue recommendations from its serious incident investigations. PH, BHS and DjHS are not implementing their recommendations within their own specified time frames. 

The reasons for PH and DjHS’s delays in implementing recommendations are unclear. Concerningly, at April 2020 during our audit conduct, the majority (70 per cent) of BHS’s recommendations were overdue, which put patients at risk of known and avoidable harm. BHS advised us that these delays are due to resourcing and skill deficiencies in its centre for safety and innovation team, which it has been working to build and address. 

Time taken to implement recommendations

MH and PH were comparable in the typical amount of time they took to implement their recommendations—ten and six months respectively. Broadly, both health services took longer to implement more complex recommendations, which is reasonable. 

In contrast, BHS typically took longer to implement its recommendations (16 months) and there is no distinct pattern as to why some recommendations took longer than others. This further indicates that BHS is not acting early enough to prevent future harm to patients. 

DjHS took the least amount of time to implement its recommendations—about two months—but many of its actions were to mitigate basic risks, such as keeping walkways clear, rather than more comprehensive actions to address root causes.

Recommendation about implementing serious incident recommendations

Assessing the impact of implemented recommendations

Following a serious incident investigation, a health service should identify how it will assess if its actions are effectively preventing similar patient harm. 

None of the four audited health services consistently identify appropriate measures to assess the effectiveness of their implemented actions. Of the 16 serious incident action plans we assessed across these health services, only two identify measures to assess if a causal or contributing factor recurs.

Two factors contribute to the audited health services’ lack of effective monitoring:

Recommendations about assessing the impact of implemented recommendations

Back to top

Health services provide care in complex and high-pressure environments where avoidable harm to patients can occur. Effective clinical governance cultures, systems and processes minimise this risk and reduce the potential for harm.

A health service's clinical governance framework describes the activities it will undertake to minimise harm and maximise the quality of patient care. Health services must meet national and state standards for clinical governance.

1.1 What is clinical governance?

According to ACSQHC, clinical governance refers to systems and processes that maintain and improve the reliability, safety and quality of healthcare provided to patients. Strong clinical governance results in healthcare that is safe, effective, patient centred and continuously improving. 

Victorian health services must meet national and state standards for clinical governance. This includes the VCGF, which SCV developed in response to Targeting Zero's recommendations. This review primarily focused on how the then Department of Health and Human Services (DHHS) was managing, overseeing and monitoring quality and safety across the health system. Where relevant, it also briefly examined health services' clinical governance management, oversight and monitoring.

Targeting Zero's recommendations included establishing new agencies (see Section 1.3) and systems to support more effective clinical governance, including frameworks and projects to improve the practical capability of Victorian health services. 

This report 

This is the first of two performance audit reports that follow up on the sector's progress since the Targeting Zero review. The review recommended that we assess the sector's progress in implementing recommendations in 2020. This report examines health services' clinical governance systems and processes, with a particular focus on actions taken at the board and executive levels. The second report examines DH's oversight of clinical governance across the health system. 

1.2 Clinical governance standards and expectations

National Safety and Quality Health Service Standards

All Australian health services must be accredited against the National Safety and Quality Health Service Standards (NSQHS Standards) to operate. We outline them in Figure 1A. 

FIGURE 1A: NSQHS Standards

Source: VAGO, adapted from ACSQHC's NSQHS Standards (second edition), 2017.

Standard 1 specifically relates to clinical governance and requires health services to 'implement a clinical governance framework that ensures that patients and consumers receive safe and high-quality health care'. Health services must use their local clinical governance framework when implementing policies and procedures, managing risks and identifying training requirements for other standards. 

Victorian Clinical Governance Framework

Under DH's Policy and Funding Guidelines, all Victorian health services must comply with the VCGF, which requires them to:

• establish a clinical governance framework that complies with the VCGF 
• implement their framework by: 
  • socialising it with their staff 
  • using it 
  • improving it.

The VCGF sets out expectations regarding best-practice clinical governance for Victorian public health services. It describes the principles of effective clinical governance, which Figure 1B outlines, and identifies five domains required for implementing these principles, which Figure 1C shows.

FIGURE 1B: The VCGF’s clinical governance principles 

Source: the VCGF.

1.3 Structure of Victoria's health system 

Department of Health

DH manages the Victorian public health system. It oversees and monitors the state's health services. According to the Victorian health services Performance Monitoring Framework 2019–20 (Performance Monitoring Framework), DH is responsible for:

• partnering with health services to identify and address performance concerns early and effectively
• supporting or intervening to ensure long-term and sustained performance improvement
• making use of available data and intelligence to maximise the depth and breadth of information used to assess health services’ performance
• enhancing health service boards’ skills and capabilities in clinical governance and other required information to ensure high-quality and safe care.

DH sets the rules for all Victorian health services through the Policy and Funding Guidelines. Annual service agreements, or SOPs, outline the Minister for Health's key performance expectations, targets and funding for public health services. DH monitors health services' performance against these expectations and targets using the Performance Monitoring Framework. 

Victorian Agency for Health Information

VAHI is a business unit of DH that analyses and shares information across the health system. It does this by identifying measures of patient care and outcomes and using them for public reporting, oversight and clinical improvement. Its key functions include: 

• collecting, analysing and sharing data so the community is better informed about health services and health services receive better information about their performance 
• providing health service boards, executives and clinicians with the information they need to best serve their communities and provide better, safer care 
• providing patients and carers with meaningful and useful information about care in their local community 
• improving researchers’ access to data to create evidence that informs the provision of better, safer care.

VAHI provides regular quality and safety reports to public health services, DH and SCV. 

Safer Care Victoria

SCV is an administrative office of DH and is responsible for leading quality and safety improvements in the state's health system. SCV’s key functions include:

• supporting health services to prioritise and improve the safety and quality of patient care by, for instance, developing and providing best-practice resources
• implementing targeted improvement projects across the health service system
• providing independent advice and support to health services to help them respond to and address serious quality and safety concerns
• reviewing health services’ performance, along with DH, to investigate and improve patient safety and quality of care
• monitoring sentinel events reported by health services, as well as the quality of health services’ investigations and how successful their actions are to prevent similar events recurring
• undertaking reviews of systemic safety issues to identify areas for local and system-level improvement. 

SCV partners with VAHI to monitor and review individual health services’ performance data and advises health services and DH about areas for improvement.

Public health services

Under the Health Services Act 1988, the Minister for Health appoints independent boards for health services, except for denominational and privately owned public hospitals. Boards must have effective and accountable risk management systems. This includes systems and processes to monitor and improve the quality, safety and effectiveness of the health services provided.

The Performance Monitoring Framework requires public hospitals and health services to: 

• partner with DH and other agencies to improve performance at an individual and system-wide level
• promptly report any emerging risks or potential performance issues, including immediate action taken, to DH 
• establish and maintain a culture of safety and performance improvement
• submit data and other information, including information about implementing agreed action plans and status update reports, in an accurate and timely way
• collaborate with other health services and system partners to maintain and improve performance and meet community needs.

The VCGF also details the roles and responsibilities of health service boards, chief executive officers (CEO), executives, clinical leaders, managers and staff.

1.4 Audited health services

This audit examined four public health services, which Figure 1D shows. These public health services differ in size, location and clinical capacity. This means they have different local contexts to consider as they adapt their local clinical governance frameworks to the VCGF.

Djerriwarrh Health Services' operating environment

DjHS has been operating under an administrator since October 2015. Under the Health Services Act 1998, the administrator has and may exercise all of the board’s powers and is subject to all of the board’s duties. Given this, we refer to the administrator as 'the board' in this report.

Since its serious patient safety issues in 2013 and 2014, DjHS has been operating in a significantly challenging environment due to:

• multiple senior executive changes, including its CEO, between 2015 and mid-2018
• lengthy proceedings with the Australian Health Practitioner Regulation Agency and the Victorian Civil and Administrative Tribunal 
• the December 2019 announcement that it will potentially voluntarily amalgamate with Western Health, and the subsequent consultations as part of the potential amalgamation process.

Back to top

Conclusion

Not all audited health services have embedded their clinical governance frameworks in their organisations. While their frameworks are generally consistent with the VCGF, only MH and PH use their frameworks to identify specific local quality and safety priorities, raise staff awareness and drive changes in organisational practices. 

Our comparison of health services' progress in embedding clinical governance shows the difference between having a document and applying it in practice. Over four years since Targeting Zero, all health services should be 'living' their local clinical governance frameworks, but this is not yet the case. 

2.1 Establishing local clinical governance frameworks 

Two of the four audited health services—BHS and DjHS—are not yet meeting DH’s requirement to have an established and embedded local clinical governance framework that complies with the VCGF. 

A health service’s local clinical governance framework complies with the VCGF when: 

• its definition of high-quality care is consistent with the VCGF
• it includes activity domains that are consistent with the VCGF and has corresponding priorities and activities in each domain.

The VCGF defines high-quality care as:

• safe—'avoidable harm during delivery of care is eliminated'
• effective—'appropriate and integrated care is delivered in the right way at the right time, with the right outcomes for each consumer'
• person centred—'people’s values, beliefs and their specific contexts and situations guide the delivery of care and organisational planning. The health service is focused on building meaningful partnerships with consumers to enable and facilitate active and effective participation'.

As Figure 1C shows, the VCGF identifies five underlying activity domains that health services require to achieve high-quality care. It also recommends a range of activities for each domain. Using the VCGF as a guide, health services should:

• reflect on and identify their own priority activities for each VCGF domain
• articulate these activities as priorities in their local clinical governance framework.

Figure 2A shows the progress of each audited health service in meeting these requirements. 

FIGURE 2A: Audited health services' progress in implementing clinical governance framework requirements as at June 2020

Source: VAGO.

Three of the audited health services chose to expand their definition of high-quality care, with MH including 'timely care' and PH and BHS including 'connected care' in their frameworks. These additions prompt these health services to prioritise and monitor these aspects of safe and quality care.

Only MH and PH have identified priority activities within each of their activity domains to target their improvement efforts. BHS only identified its priorities in January 2021, and these priorities were still to be approved by BHS's quality care committee (board subcommittee) in early 2021. 

DjHS has not identified priority activities in its Quality and Safety Framework. Instead, DjHS relies on a range of other policies and plans to operationalise clinical governance, such as its strategic and operational plans and Safe Practice Framework. Not having this information in a consolidated document makes it difficult for DjHS to identify gaps and for staff to easily understand the range of activities needed to provide safe, effective and person-centred care. 

Catalysts and challenges 

We found that the audited health services' progress in implementing the VCGF has been affected by:

• leadership, including leadership changes and if leaders champion patient safety and align their local clinical governance framework with the VCGF 
• organisational culture, particularly the extent to which health services' executives and managers address bullying and harassment, occupational violence and aggression, and prioritise staff’s physical and psychological safety.

A major challenge for health services in implementing their local clinical governance frameworks has been staff lacking awareness and knowledge about:

• the importance of clinical governance
• how they and their work unit should contribute to delivering high-quality care to patients. 

As Figure 2B shows, each of the four health services have experienced different catalysts and challenges associated with implementing the VCGF and establishing and implementing their own clinical governance frameworks.

FIGURE 2B: Catalysts and challenges associated with implementing the VCGF and local clinical governance frameworks 
 

Source: VAGO.

2.2 Embedding clinical governance frameworks

Clear staff roles and responsibilities

To meaningfully implement a clinical governance framework, a health service needs to express how the framework translates to staff roles and responsibilities. This enables staff to have a clear understanding about how they contribute to providing high-quality care.

All four audited health services' local clinical governance frameworks identify staff roles and responsibilities for achieving high-quality care that broadly align with the VCGF. Health services have also formalised their expectations of staff by adding quality and safety roles and responsibilities to staff position descriptions, which Figure 2C shows. When health services identify defined roles and responsibilities, staff have a clear and consistent understanding of their part in providing high-quality care. They can also be held accountable if their behaviour does not align with stated expectations. 

FIGURE 2C: Examples of how audited health services have included quality and safety expectations in staff position descriptions

Source: VAGO.

Building expectations into day-to-day activities

As figures 2D, 2E and 2F show, MH and PH have implemented extensive initiatives to translate their local clinical governance frameworks into practice. These initiatives collectively reinforce a consistent message to staff across the health services and maintain their focus on achieving high-quality care. 

In contrast, BHS and DjHS are yet to demonstrate the impact of their clinical governance frameworks on achieving high quality care. This is because they have implemented few initiatives to embed their frameworks. We note BHS and DjHS's initiatives in Figure 3D in Chapter 3.

FIGURE 2D: Common MH and PH initiatives to embed their clinical governance frameworks

Source: VAGO.

FIGURE 2E: Case study: MH's approach to embedding its local clinical governance framework

Note: Parking lot is an area for staff to ‘park’ ideas to consider later. 
Source: VAGO, based on information from MH. Image supplied by MH.

FIGURE 2F: Case study: PH’s approach to embedding its local clinical governance framework

Note: CRP refers to community rehabilitation program. COPM refers to the Canadian occupational performance measure. FIM refers to functional independence measure. SDP refers to the stroke detours program.
Source: VAGO, based on information from PH. Image supplied by PH.

Back to top

Conclusion

MH and PH have made greater improvements to their patient safety cultures since Targeting Zero, than BHS and DjHS. They have done this by embedding their clinical governance frameworks in their organisations and supporting staff to actively uphold patient safety. 

In contrast, BHS and DjHS have not implemented an appropriate mix of quality and safety initiatives. In particular, they lack actions to build their staff’s confidence to speak up about safety concerns. By not prioritising and engaging in this work, these health services are not doing enough to ensure patient safety. 

3.1 What is patient safety culture?

If a health service has a positive patient safety culture:

• staff feel safe to speak up when they have concerns about patient safety 
• the health service is committed to learning from errors
• the health service responds to warning signs early and avoids catastrophic incidents. 

DH and health services annually assess patient safety culture through the PMS (this survey did not occur in 2020 due to the coronavirus (COVID-19) pandemic). The survey provides insights into how health service staff perceive their wellbeing in the workplace and the patient safety climate at their health service. Factors that threaten staff wellbeing, including a lack of psychological safety, contribute to a poor patient safety culture.

Three interrelated dimensions contribute to an ongoing positive patient safety culture:

To promote a positive patient safety culture, health services need to implement initiatives that contribute to these three dimensions. Health services should also evaluate their initiatives to assess if they are achieving the intended outcomes. 

3.2 Patient safety culture at audited health services

All Victorian health services should have strengthened their patient safety culture following Targeting Zero as part of the sector's renewed focus on improving care. 

We analysed the audited health services' results for selected PMS questions between 2016 and 2019 on:

• staff experience of bullying
• staff confidence in speaking up
• patient safety climate.

We looked for changes in the results between 2016 and 2019 and analysed the data to assess if these changes were statistically significant at a 95 per cent confidence interval. We also looked at the raw results to identify areas where further improvements are necessary.

Improvements since Targeting Zero

Of the four audited health services, only MH has consistently improved its staff’s perceptions of patient safety culture across the PMS measures since 2016. While PH’s results have declined for a minority of patient safety culture measures, it has improved its performance overall. This means that since 2016, MH and PH staff:

• better understand the importance of quality improvement activities
• are more likely to participate in these activities to reduce the risk of patient harm
• are more likely to report incidents. 

In contrast, BHS and DjHS have not made substantial improvements since 2016, and for DjHS, some results have deteriorated. 

Figure 3A summarises these changes between 2016 and 2019. 

Areas for further improvement

The PMS data in Figure 3B shows that the percentage of staff who have experienced bullying at MH, PH and BHS has declined since 2016. All four health services also have fair to strong results regarding:

• how confident staff feel about reporting concerns to their manager
• staff feeling encouraged by colleagues to report concerns
• how management responds to and handles safety issues.

However, DjHS has seen an increase in staff reporting that they have experienced bullying. All four health services, but especially DjHS, also have significant room to improve:

• staff feeling safe from reprisal if they report improper conduct
• staff confidence in the integrity of investigations into safety issues.

This indicates that the audited health services can do more to transparently demonstrate the quality of their investigation processes to staff and address their concerns about reprisal, which the low results strongly suggest is occurring. Unless directly addressed, these factors will continue to act as barriers to these health services developing and maintaining a strong patient safety culture. We discuss the persistent patient safety culture issues at DjHS in more detail in Figure 3C. 

FIGURE 3B: The four audited health services’ PMS performance results for 2016 and 2019

Note: Green = improved, orange = improved between 2016 and 2018 then declined in 2019, red = declined and black = no statistically significant change.
Source: VAGO, based on PMS data between 2016 and 2019. 

FIGURE 3C: Case study: DjHS’s challenges in creating a positive patient safety culture

Source: VAGO, using PMS data.

3.3 Initiatives to create a positive patient safety culture

To promote a positive patient safety culture, health services need to implement initiatives that increase their staff’s psychological safety, confidence to speak up and engagement. 

As Figure 3D shows, MH and PH have implemented an extensive mix of initiatives that address these three dimensions. This means that staff at these health services are more likely to be vigilant in detecting and reporting patient safety issues. 

However, MH is the only audited health service that has evaluated its initiatives to understand barriers and enablers to fostering a positive patient safety culture. Figure 3E explains how MH identified ways to improve one of its initiatives. PH advised us that it will be evaluating its initiatives to assess their impact on raising staff awareness of bullying and harassment. 

BHS and DjHS have not developed or implemented sufficient initiatives across all three dimensions. This is because they have not developed and/or implemented a comprehensive local clinical governance framework. Without a comprehensive mix of initiatives, BHS and DjHS are not doing enough to promote a positive patient safety culture. 

Figure 3D provides an overview of the audited health services’ current initiatives categorised by the three dimensions. Appendix D contains further details about these initiatives.

FIGURE 3D: Audited health services’ initiatives for each patient safety culture dimension

Note: Comparisons should not be made between health services having or missing specific initiatives because each health service's circumstances are different. Instead, we assessed if health services have appropriate initiatives against the patient safety culture dimensions.
Source: VAGO.

The initiatives BHS has implemented to date have focused on increasing its staff’s psychological safety. BHS has only recently started to actively engage staff in quality and safety activities. Figure 3D shows that BHS has implemented a pathway for staff to raise concerns confidentially. However, it has not implemented initiatives to equip staff with the skills and confidence to speak up about safety concerns with their peers and managers. BHS advised us that its workplace safety committee will consider possible actions to address this.

DjHS's initiatives focus on increasing staff awareness of patient safety in wards. Like BHS, DjHS has not implemented initiatives to increase its staff’s skills and confidence in speaking up. This is concerning given that DjHS's PMS survey results show that the percentage of staff who feel safe to speak up has declined and DjHS does not have any initiatives in place to address this. 

DjHS advised us that while it had developed a ‘safe to speak up’ program, it did not fully implement it due to competing demands associated with:

• a restructure of its people and culture team
• the potential voluntary amalgamation with a larger health service
• the COVID-19 pandemic. 

DjHS noted that instead, its health and safety representatives and managers promote the message that it is safe to speak up through various formal and informal meetings with staff. Nonetheless, DjHS recognises that it needs to take a different approach to improve its PMS results. 

DjHS also advised us that it is revising its safe to speak up program, which will include staff focus groups to identify the root causes of its cultural issues.

Figure 3E highlights MH's use and review of its weCare system as an example of how a health service can assess the effectiveness of its cultural initiatives.

FIGURE 3E: Case study: how MH evaluated one of its safety culture initiatives

Source: VAGO, based on information from MH.

Back to top

Conclusion

While health services act when they identify underperformance or emerging risks, they do not consistently identify and respond to quality and safety risks in a timely way. Significant delays in completing serious incident investigations and resulting actions to address underlying issues mean that patients remain at risk of known avoidable harm for too long. 

Health service boards, especially at the audited regional and rural health services, are not consistently monitoring enough information across incidents and KPIs to have a comprehensive view of quality and safety risks. 

4.1 Clinical incidents

A clinical incident is an event or circumstance that results in unintended or unnecessary harm to a patient. 

DH requires health services to classify all clinical incidents according to the degree of harm that occurred to the patient using the ISR scale, which Figure 4A shows. Sentinel events are a subset of ISR 1 incidents because they are the most serious incidents that are wholly preventable and involve death or serious harm.

FIGURE 4A: ISR categories of clinical incidents 

Source: SCV.

4.2 Identifying and monitoring quality and safety risks

While there are multiple layers of governance within a health service, the board and its quality and safety subcommittee are accountable for assuring DH, the Minister for Health and the community about their health service’s quality and safety. 

Board quality and safety subcommittees: 

• include board members
• provide their meeting minutes to the board
• can escalate risks and issues to the board. 

As such, throughout this section we refer to both boards and board quality and safety subcommittees as 'boards'. 

To have effective oversight, boards should:

• receive regular reports on clinical incidents and quality and safety KPIs
• understand current and emerging risks
• ensure actions occur to address underperformance and clinical risks. 

At the four audited health services, the boards' oversight of quality and safety risks is not consistently adequate. There are various gaps at each audited health service. Overall, there are more significant deficiencies in oversight at BHS and DjHS relating to incident investigations, implementing recommendations and holding executives accountable for underperformance against KPIs. None of the four boards receive regular thematic reports on less serious incidents to check for underlying systemic issues. 

Monitoring incident investigations and responses

Health service boards need assurance that their executives are quickly identifying and addressing issues that have caused avoidable serious incidents (mainly ISR 1 and ISR 2 incidents) to minimise the chances of them recurring. For less serious incidents (mainly ISR 3 and ISR 4 incidents) that may occur more frequently, they should receive regular thematic reports that identify any common contributing factors and actions taken to address them. 

None of the audited health service boards receive comprehensive information on incidents and responses to them. In particular, none of the health services undertake systematic analyses of less serious incidents to gauge emerging risks. Figure 4B shows these gaps. 

FIGURE 4B: What board incident reports contain at the audited health services

Source: VAGO.

Serious incident investigations 

MH’s board

MH's board regularly oversees the status of serious incidents but inadequately oversees how the health service implements actions to address them. 

MH's oversight of actions is limited because it only reports on the overall proportion of completed recommendations. There is no rule or requirement for when MH needs to account for overdue recommendations. While MH had no overdue recommendations to report during the course of this audit, MH should ensure it has a clear process to do so to avoid a potential gap in its board’s oversight.

PH’s board

PH’s board regularly oversees the status of serious incident investigations and actions to address risks. However, its oversight is inadequate in the following ways:

• PH's reports do not include overdue investigations and recommendations for incidents that occurred outside of the reporting period. This means that the board does not have a full account of overdue investigations and recommendations. 
• PH does not account for why investigations or recommendations are overdue and what rectifying actions are in place. Without this information, the board cannot be assured that the executive is taking steps to minimise delays in completing overdue investigations and recommendations. PH has advised us that it will address this gap.

BHS’s board

While BHS’s board does not regularly oversee the status of serious incident investigations, it does regularly oversee the status of actions to address risks. However, its oversight is not adequate. BHS's reports on implementing actions identify overdue actions but do not consistently account for why they are overdue and what actions are needed for improvement. Hence, the board does not know if these delays are reasonable. 

DjHS’s board

DjHS does not provide its board with clear and consistent information on the status of its investigations and recommendations. DjHS provides an ISR 1 and ISR 2 summary report to its board at each meeting. While the intent of this report is to inform the board of serious incidents that occurred during the reporting months, it only sometimes includes the status of investigations and implementation of actions for some incidents. Hence, the board does not receive consistent and clear information on whether there are overdue investigations and actions. 

Monitoring quality and safety KPIs

To have a comprehensive view of quality and safety performance and risks across a health service, in addition to monitoring incidents, the board needs to:

• assess quality and safety performance against appropriate KPIs as part of its regular board KPI suite, including:
  • at a minimum, the mandatory quality and safety SOP KPIs
  • any additional KPIs it sees as necessary to meet obligations under the Health Services Act 1988
• receive reports that: 
  • clearly identify areas of significant underperformance
  • include reasons contributing to underperformance
  • include information on actions taken to address underperformance and assure that these actions have been effective
  • preferably group and report KPIs against the health service's clinical governance framework goals to enable clear assessment of its progress.

As Figure 4C shows, not all audited health service boards receive sufficient information to adequately understand quality and safety performance. 
 

FIGURE 4C: What audited health services report to their boards about quality and safety performance

Source: VAGO.

Monitoring SOP quality and safety-related KPIs

Only PH's board monitors all of its SOP quality and safety KPIs as part of its monthly and bimonthly KPI reporting suite. MH's board monitors almost all of its SOP KPIs. Except for minor gaps in MH’s reporting, the boards and executives at PH and MH therefore have a good understanding of performance issues.

However, BHS and DjHS's boards only monitor 19 per cent and 9 per cent (respectively) of their SOP quality and safety KPIs as part of their monthly or bimonthly KPI reporting suite. 

There is no distinct pattern across the four health services for which SOP KPIs they do not monitor (see Appendix F for full list of SOP KPIs that each health service monitors). Specifically:

Unreported SOP KPIs

MH reports on its KPI for elective surgery wait times in a separate quarterly SOP report to the board, and therefore excludes it from its regular board KPI reporting suite to reduce duplication. However, the less frequent reporting may reduce the board’s opportunity to address any issues in a timely way. 

Both BHS and DjHS rely on VAHI’s quarterly Monitor reports to review their performance for the SOP KPIs excluded from their regular bimonthly and monthly reporting suite, respectively. This means there is a risk that both boards do not become aware of underperformance and take action in a timely manner.

BHS advised us that clinical directorates monitor the 26 SOP KPIs omitted from its board reports that are relevant to their clinical area in their respective quality and governance committee meetings. These committees can escalate issues to BHS’s board quality subcommittee through the patient safety and innovation committee, which is BHS's peak executive quality committee. However, we did not see evidence of this escalation in the meeting minutes we reviewed across a four-month period (2 committee reporting cycles). There is also a two to three month lag in escalating issues through this process.

DjHS advised us that as a small health service, recreating and analysing the 10 SOP KPIs omitted from its regular internal reporting is not a priority given other pressing matters. 

Despite the reasons described above, BHS and DjHS's reliance on VAHI's Monitor reports to inform their boards on SOP KPIs is problematic because they only receive them quarterly. This is not frequent enough to allow these boards to identify and address underperforming SOP KPIs within the annual agreement period. 

Since feedback from this audit, DjHS has expanded its monthly activity performance report and now includes most of the unreported SOP KPIs.

Monitoring other quality and safety KPIs

Beyond their SOP KPIs, all health service boards regularly monitor additional KPIs they identify as priorities (see Appendix G for a full list of these KPIs). Some common KPIs cover:

• timeliness in providing hospital discharge summaries
• common clinical incidents, such as pressure injuries, falls and medication errors
• unplanned readmissions for specific community groups or medical conditions.

For MH and PH, who already monitor most of their SOP KPIs, these additional KPIs mean their boards have a comprehensive view of quality and safety performance. Both health services have also grouped their SOP and additional quality and safety KPIs to align with the goals in their clinical governance frameworks for their internal reporting. This allows the boards and internal stakeholders to understand performance results against organisational quality and safety goals.

In contrast, BHS and DjHS’s boards do not sufficiently monitor their KPIs to have a comprehensive view of service quality and safety. As mentioned above, BHS and DjHS only review 19 per cent and 9 per cent (respectively) of all mandatory SOP KPIs. While both boards monitor additional KPIs they identify as priorities, these KPIs do not address the gaps in their SOP monitoring, which is the minimum set by DH. 

Additionally, unlike MH and PH, BHS and DjHS do not group and report on their KPIs against their local clinical governance framework goals. This means that the boards at these health services cannot easily assess if they are meeting their clinical governance goals. 

Identifying and accounting for poor performance

Health services need to clearly highlight any areas of underperformance and emerging areas of performance risks to their boards so they can make further queries and informed decisions on actions needed for improvement.

Not all health services have a comprehensive approach to identifying and accounting for underperformance to their boards, which Figure 4D shows.

We reviewed board quality and safety subcommittee performance reports for the four audited health services between July 2019 to February 2020 to assess if they: 

• identify changes in performance 
• detect deteriorating performance early using clearly established criteria
• summarise their achievements against benchmarks or targets
• provide adequate accounts for poor performance
• compare and analyse performance across relevant patient cohorts and/or service streams to identify areas of strengths and weaknesses across the health service. 

Figure 4D outlines the outcomes of our analysis.

FIGURE 4D: Does reporting to audited health services' boards show underperformance and emerging trends?

Source: VAGO.

Identifying performance issues

All audited health services clearly identify poor performance using colour codes to bring them to the board's attention in their KPI reports. However, only MH takes a statistical approach to identifying significant performance changes over time for every board report. This means that MH’s board knows when real performance changes occur, as opposed to insignificant levels of change associated with expected variation. This allows the board to focus its attention on genuine issues. 

Figure 4E highlights why MH's statistical approach to identifying significant emerging trends using run charts is better practice for discovering current or emerging performance issues. 

FIGURE 4E: Case study: MH's approach to identifying significant performance variation using run charts

Source: VAGO. Run chart image provided by MH.

Areas for improvement 

PH's approach to identifying emerging risks is adequate overall but can be improved. PH has a low threshold for initiating investigations into poor-performing KPIs (not meeting the target for two consecutive months). These investigations, which we discuss further below, include reviewing longer-term data to identify any concerning trends and issues. Because of the low threshold, PH is still be able to identify emerging risks early. Nonetheless, we note that PH has a strong data analytics capability and could take a more statistical approach to identify emerging trends in its performance data.

BHS's approach to identifying emerging risks is inadequate. It lacks trended performance information, which means it may not identify emerging performance risks. BHS now includes sparkline plots to identify trends over the last three months in its board KPI reports. However, it is unlikely that three months' worth of data is sufficient to show a reliable trend.

None of the audited health services' board reports provide a comprehensive view of whether particular areas of a health service are underperforming. MH, PH and DjHS’s boards only receive a breakdown by campus for selected indicators. None of the audited health services consistently examine KPIs that are performing below target by campus and clinical unit to identify and promote learning between those that are performing above expectations and those that are underperforming. 

We note that DjHS only improved its data collection, analysis and reporting to its board in the last three years. It introduced monthly activity performance reports in 2018 and a DjHS dashboard in February 2020. DjHS advised us that its improvements since 2018 are from a ‘zero base’, where reporting was previously a manual and fragmented process.

Despite this work, gaps remain. Given Targeting Zero’s imperative, DjHS has not acted fast enough or taken sufficient action to improve its performance analysis and reporting. 

Accounting for performance issues

MH and PH investigate and provide clear accounts of underperformance to assure their boards about steps taken to improve performance. Specifically, PH provides detailed 'in-focus' analyses, in addition to its board KPI report, that discuss reasons for underperformance and actions to address them. 

In contrast, BHS and DjHS's regular board reports on quality and safety do not consistently provide: 

• detailed explanations of underperformance
• actions taken to address performance issues 
• information on who the responsible executive is to address the matter. 

This means that BHS and DjHS’s boards cannot ensure that their health services are addressing quality and safety issues. For example:

• BHS's board KPI reports have an indicator to measure the percentage of patients who have comprehensive medication assessments (also known as best possible medication history) documented at admission. In October 2019, BHS achieved 57 per cent against its target of greater than 80 per cent for this KPI. The only comment reported against this result was that the national average is 62 per cent. 
• In its KPI reports, DjHS provides details on medication errors and reasons for why they occurred, such as drug charts being read incorrectly, when this KPI does not meet its target. However, DjHS does not provide a similar level of explanation for its underperforming KPIs on falls at Grant Lodge. 

DjHS only established criteria for when executives need to provide an action plan to the board to address KPI underperformance in April 2020. Our review of a sample action plan showed that it focused on actions taken to improve performance, rather than first identifying the factors that contributed to underperformance. This means the actions taken may not address the root cause. 

4.3 Responding to clinical incidents

All Victorian health services must meet SCV's Policy: Adverse patient safety events, which sets out health services' responsibilities in relation to clinical incidents. According to this policy, health services must:

• have clear incident management polices 
• openly disclose with patients and carers when harm has occurred 
• investigate all incidents
• identify and implement actions 
• monitor the impact of actions. 

Figure 4F sets out these steps. 

Incident management policy

As Figure 4G shows, all audited health services have incident management policies and procedures that provide staff with clear processes for reporting, investigating and responding to incidents. However, BHS and DjHS could further improve theirs.

We assessed these health services' incident management policies against expectations and principles set out in SCV's Policy: Adverse patient safety events and the NSQHS Standard 1.11 under incident management systems and open disclosure. 

FIGURE 4G: Assessment of audited health services' incident management policies and procedures

Note: ~While DjHS’s policy does not include time frames to complete IDCRs and SCV’s risk reduction action plan, its document templates do.
Source: VAGO.

Following feedback from this audit, DjHS is updating its incident management policy to address the gaps shown in Figure 4G. 

All audited health services also have policies and procedures that detail the required reviews for incidents of different severity levels:

• All ISR 1 incidents, including all sentinel events, require comprehensive root cause analysis (RCA) followed by a detailed report that specifies the full range of causal and contributing factors that led to an incident.
• All ISR 2 incidents require an IDCR and a report that identifies any breakdowns in care processes and how these affected an incident’s occurrence. 
• All ISR 3 and 4 incidents are subject to reviews by senior clinical staff in the relevant units. 

During this audit, we notified BHS that its ‘adverse event/incident management non clinical protocol’ did not specify time frames for completing IDCRs. BHS promptly addressed our feedback and is now able to assess the timeliness of IDCRs and reports to improve quality and safety. 

Openly disclosing clinical incidents

Patients and their carers may only become aware of a clinical incident when health service staff openly disclose the event to them. All Victorian health services are obligated under Victoria's Charter of Human Rights and Responsibilities Act 2006 and the NSQHS Standards to have a formal process for openly disclosing clinical incidents.

Open disclosure involves a meeting in which clinical staff provide patients and their carers with an:

• expression of regret or apology that an event occurred
• explanation of the event and its impact on the patient and their carer/s
• invitation for the patient to account for their experience of the event
• outline of the intended steps to manage any consequences of the event
• outline of what will be done to prevent the event from recurring.

ACSQHC's Australian Open Disclosure Framework sets out minimum standards for conducting open disclosures. 

As Figure 4H shows, all audited health services have open disclosure policies and/or procedures that are broadly consistent with the Australian Open Disclosure Framework. This supports a consistent approach for staff undertaking open disclosures. While DjHS has a minor gap in its open disclosure policy, which concerns the principle of 'good governance', it is addressing this. 

Across all four audited health services, only MH routinely monitors its open disclosure process to ensure each case is promptly and appropriately actioned. The other audited health services have gaps in monitoring their open disclosure processes. 

PH staff can record open disclosures in two separate locations—the Victorian Health Incident Management System, the patient’s medical record or both. As a result, there is not one consistent record of open disclosures that have occurred, and PH’s board and executives cannot easily monitor or assess compliance. PH advised us that its executives are aware if open disclosures have occurred for ISR 1 incidents because this is discussed during the initial incident review meeting for these incidents. 

BHS does not have a process for monitoring open disclosures. Its clinical governance committee is aware of this issue and is considering changes to its incident management system to record staff compliance. 

DjHS staff use the Victorian Health Incident Management System to record open disclosures. However, its board and executives do not routinely monitor relevant data to ensure that staff are undertaking open disclosures as required by its open disclosure policy.

FIGURE 4H: Assessment of health services’ open disclosure policies and/or procedures

Note: ~The policy or procedure generally references a principle but does not provide a specific description of the expected activity or behaviour.
*DjHS has updated its open disclosure policy to include this principle and anticipates that the board will endorse this policy in late May 2021.
Source: VAGO.

Investigating clinical incidents

Health services need to investigate clinical incidents in a timely manner to quickly identify causal and contributing factors, learn from the incidents and implement improvements to prevent future avoidable harm. 

Completing timely serious incident investigations

In 2019–20, all audited health services performed better than the statewide average (40 per cent) for reporting sentinel events to SCV within 30 days. For example, MH and PH reported almost all sentinel events within 30 days (eight out of nine events for MH and 12 out of 14 events for PH). 

While these results are comparatively better than other health services, it is still concerning that health services, including the audited health services, are not always meeting requirements for reporting sentinel events in a timely way. 

Aside from sentinel events, DH has not set specific time frames for completing investigations into other incidents. Health services should set their own clear time frames for investigating ISR 1 and 2 incidents to ensure they respond to patient safety risks in a timely way. As Figure 4I shows, the four audited health services have set varying periods for completing RCA and IDCRs.

FIGURE 4I: Audited health services’ time frames for investigating serious incidents 

Source: VAGO.

Despite health services setting their own time frames for completing serious incident investigations, none of the audited health services were consistently meeting them. This risks health services not identifying causal factors of incidents early enough to prevent similar harm from recurring. 

As Figure 4J shows, we assessed the median proportion of overdue serious incident investigations at MH, PH and BHS. We excluded DjHS from this analysis because there were only five serious incidents reported during our sample period of board papers between October 2019 and February 2020. Nonetheless, we note that three out of five ongoing investigations at DjHS were overdue. For MH, PH and BHS, the median proportion of overdue serious incidents investigations ranged from 17 per cent at MH to 82 per cent at BHS over respective sampling periods. 

These three audited health services attributed delays in serious incident investigations to a lack of staff capacity and capability. 

MH also advised us that it aims to meet reporting time frames, but this can be difficult to achieve and sometimes depends on appropriate clinicians and consumers being available.

PH advised us that it prioritises new incidents and focuses on the quality of investigations rather than meeting internal reporting time frames. While its focus on quality is understandable, we did not find evidence of PH staff’s obtaining approval to extend timelines for quality purposes or assessing the risks of delays. 

Analysing incident themes

Health services should undertake regular and service-wide thematic analyses of all serious (ISR 1 and ISR 2) and less serious (ISR 3 and ISR 4) incidents to determine common themes or clusters relating to:

• clinical division or units
• causal or contributing factors.

As Figure 4K shows, not all of the audited health services do this. Consequently, they risk not identifying clusters of incidents or factors underlying incidents. Figure 4K shows the frequency of thematic analyses at each audited health service. 

FIGURE 4K: Frequency of thematic analyses of serious and less serious incidents across the whole organisation at the four audited health services

Source: VAGO.

MH advised us that it will examine regular thematic analyses of less serious incidents after implementing DH's Victorian Health Incident Management System Local Solution. 

While PH has not conducted regular thematic analyses, we have seen evidence that it has undertaken this type of analysis on request. For instance, following a serious incident review committee request, PH reviewed its medication error data over a three-month period and analysed it by clinical craft group and 'reason for the error'. PH could improve its thematic analysis of incidents through its ‘Incident Explorer’ dashboard. 

BHS analyses its serious incidents monthly by key incident types, such as falls and medication errors, within clinical directorates and by clinical units to identify if there are any concerning themes and trends. However, these analyses are not detailed enough to identify common contributing factors. For instance, in February 2020, BHS found that 70 per cent of medication errors involved an ‘administration process’ but did not further explain what specific issues led to the errors. BHS's individual monthly analyses also do not identify if common contributing factors recur across multiple months. 

DjHS advised us that as a small health service, thematic analyses are only possible when there is sufficient data. We note some examples of DjHS conducting thematic analyses in response to multiple clinical incidents to determine common underlying causes, such as analyses of postpartum haemorrhage, medication errors and falls.

Figure 4K also shows that none of the audited health services undertake regular thematic analyses of less serious incidents to identify potential themes across the health service. 

For BHS, while its risk and compliance team review all incidents across the health service to identify themes, there is no documented, standard approach to do this so staff may be using inconsistent methods. 

For MH and PH, thematic analyses primarily occur at the local unit or ward level. PH’s analyses also occur on an ad-hoc basis by its NSQHS Standard committees. However, similar to BHS, MH and PH do not have clear guidelines for local managers to ensure that they adopt a consistent approach. Hence, there is a risk that they may miss early warning signs. 

PH does analyse themes for specific less serious incidents that are part of its regular KPI suite, such as patient identification and wrong site or procedure. It also investigates contributing factors as part of its variance-reporting process. However, PH does not complete these analyses regularly over standard time periods and across similar incident types to identify and track issues. 

Implementing actions to prevent future incidents

Health services should implement recommended actions from incident investigations in a timely manner to prevent events recurring. For serious incidents, we expect health services to implement recommendations as soon as possible after finalising the RCA or IDCR. 

The amount of time it takes for a health service to implement actions depends on the action’s nature and complexity. Actions such as updating current procedures or providing staff training are quicker to complete compared to more complex actions, such as purchasing new medical equipment or coordinating intricate multidisciplinary protocols across an entire health service. 

We examined: 

• the proportion of overdue actions at each audited health service
• how quickly they implemented actions between February 2019 and February 2020. 

As Figure 4L shows, only MH completed all actions within its time frames and consistently took prompt action to address serious patient safety issues. PH and DjHS did not implement actions within their own estimated time frames and could not provide clear reasons for delays. 

BHS had not implemented the majority (70 per cent) of its recommended actions as of April 2020, which allows risks to patient safety to remain. BHS advised us that delays were due to a resourcing and skills deficit in its centre for safety and innovation team and it had been working to build capacity in 2020.

FIGURE 4L: Audited health services’ delays in completing serious incident recommendations

Note: The time periods differ between health services due to the staged nature of our audit conduct work and the availability of health service reports.
Source: VAGO.

Figure 4M shows the median amount of time that health services took to complete actions across three or four reporting periods between February 2019 and February 2020. 

MH and PH implemented recommended actions within approximately six to 10 months from the date of the original incident. We found this reasonable given the complexity of the recommended actions. For example:

• MH changed ward-level fittings and fixtures and conducted staff education within four to seven months.
• PH purchased new equipment and introduced new staff positions within 10 months. 

However, we note that PH took at least 14 months longer than its median time taken to implement recommended actions for 10 incidents. PH was unable to provide reasons for the delays.

BHS took longer than the other health services—between 20 to 26 months—to implement a quarter of all its recommended actions. BHS could not provide reasons for these delays due to a lack of documentation and there were no obvious patterns associated with the types of actions. 

DjHS took the least amount of time to take recommended actions, but many of these were immediate actions to mitigate basic risks (such as keeping walkways clear) rather than more comprehensive actions to address root causes.

Assessing the effectiveness of actions to prevent incidents

Health services should monitor the effectiveness of actions they implement to ensure they are addressing underlying causes of serious incidents and preventing future harm. Health services should use better-practice approaches to monitor the effectiveness of their actions, including: 

• identifying the intended result to be assessed
• setting a target 
• specifying the sample group and sampling period 
• identifying how data will be collected.

None of the four audited health services have consistently identified appropriate measures to assess the effectiveness of implemented actions. Out of a sample of 16 serious incident action plans across the four health services, we were only able to identify two instances where the health service set measures to specifically assess the impact of the actions taken to prevent the same incident from recurring, which Figure 4N shows. 

FIGURE 4N: Instances of appropriate measures that assess the impact of actions from 16 serious incident action plans 

Source: VAGO.

Two factors contribute to the audited health services' lack of effective monitoring:

• While SCV's RCA recommendations and action plan template requires health services to document an 'outcome measure' (which evaluates the impact of an intervention), it does not define what this is and how it differs from an output measure (which might, for example, just note that the planned action was completed).
• IDCR templates designed and used by MH, PH and DjHS do not require staff to identify outcome measures. 

While BHS requires its staff to specify measures to assess the impact of recommended actions, it does not provide guidance on how to identify appropriate measures.

We have included examples of better measures that the audited health services could consider using in Appendix E.

Back to top

We have consulted with BHS, DjHS, MH, PH and DH, and we considered their views when reaching our audit conclusions. As required by the Audit Act 1994, we gave a draft copy of this report, or relevant extracts, to those agencies and asked for their submissions and comments. 

Responsibility for the accuracy, fairness and balance of those comments rests solely with the agency head.

Response provided by the Chair, BHS

Response provided by the Secretary, DH

Response provided by the Chief Executive/Chief Financial Officer, DjHS

Response provided by the Chief Executive, MH

Response provided by the Chair, PH

Back to top

Back to top

Our methods

As part of the audit we:

• audited four health services, including reviewing their:
  • clinical governance frameworks
  • policies and procedures on quality and safety
  • documents on their patient safety culture improvement initiatives
  • board reports on incidents and KPIs
• interviewed various staff across each health service from board members and executives through to clinical directors and nurse unit managers
• visited clinical wards at all health services except PH due to the COVID 19 pandemic.

We selected the four health services as a representative spread of health services by location and size. 

We undertook the majority of audit conduct work from January 2020 to April 2020. The COVID-19 pandemic resulted in some delay to the completion of the audit.

We conducted our audit in accordance with the Audit Act 1994 and ASAE 3500 Performance Engagements. We complied with the independence and other relevant ethical requirements related to assurance engagements.

We also provided a copy of the report to the Department of Premier and Cabinet and the Department of Treasury and Finance.

Back to top

We briefly discuss the audited health services' initiatives to promote a positive patient safety culture below. PH's placemats and MH's improvement huddles and improvement noticeboards are excluded here as these are covered in Section 2.2.

MH's initiatives

• weCare system: provides an anonymous avenue for staff to raise concerns about colleagues' behaviour.
• Progressive communication tool: for staff to communicate and escalate concerns about patient safety.
• ‘Good catch’ awards: recognises staff for intervening and speaking up about patient safety.
• ‘You made a difference’ awards: recognises and celebrates staff and volunteers for building on MH's safety culture by living MH's values in their work.
• Safety champions: staff are role models for peers on speaking up for MH's values and use the progressive communication model to communicate and escalate concerns about patient safety.
• Safety huddles: routine, structured short clinical unit briefings designed to give nurses and other caregivers opportunities to stay informed, review events, and make and share plans to ensure well-coordinated safe care.
• Independent facilitator: an informal, independent, neutral and confidential resource for staff to raise concerns, build capacity to speak up and seek support on conflict resolution.
• Leadership program: to develop leadership capabilities in areas such as self awareness and developing others and leading in a complex work environment. 

PH's initiatives

• Values integration program: staff workshops that assert expected and acceptable behaviours.
• Internal human resources advisory line, and an external and independent whistleblower alert service: to encourage staff to report any matters that may adversely affect the organisation, its employees or other persons.
• Safer care consultants: provides advice and support to clinical and operations leaders of all clinical areas on patient safety, continuous improvement and risk management.
• Daily operating system meetings: to discuss various operational aspects of the health service and quality and safety issues, such as clinical incidents that needed escalation in the previous 24 hours. There are three tiers of daily operating system meetings—tier 0 occurs at the local unit or ward area, tier 1 occurs between managers and their director, and tier 2 occurs between the CEO, executive directors, clinical and operational directors and all department directors. 
• ‘Ask the chief executive’ portal: online portal for staff to raise concerns directly with the chief executive anonymously and in confidence.
• ‘Know better be better’ campaign: DH and WorkSafe Victoria's bullying and harassment awareness campaign that has been rebranded and incorporated as part of PH's ongoing ‘Safe culture’s campaign to improve staff wellbeing. 

BHS's initiatives

• ‘Above and below the line’ behaviours initiative: across all levels of BHS from the board and executive down to departments, each staff group identifies a range of acceptable and not-acceptable behaviours.
• Values-based training: BHS's expectations on staff behaviour and professional conduct.
• Confidential feedback: an email address monitored by the CEO's office where staff may raise concerns. 
• ‘BHS together’ values awards: recognises and rewards staff who exhibit behaviours that align with the health service's values.
• Ward noticeboards on safe, effective, person-centred and connected care: raises staff awareness on what high-quality care means and how staff are contributing to these goals of care.
• Daily operating system meetings: daily operational initiative that cascades from the local department level up to the executive level to discuss current and emerging patient safety issues and incidents.
• Safety crosses at some wards: staff use a colour-coding system on a calendar to record when and what type of clinical incidents have occurred.
• ‘Clinical incident champions’: ward-based roles that provide leadership and assist ward staff with education on various incidents, such as falls and blood management.

DjHS initiatives

• Management development program: builds the capability of managers in a range of areas including having difficult conversations and driving change in teams.
• Annual staff awards: from 2020, DjHS shifted the focus of these awards from years of service to performance that exceeds expectations over a range of areas that demonstrate quality and commitment to the health service spanning:
  • improvement and innovation
  • patient-centred care and safety
  • diversity and inclusion
  • clinical excellence
  • commitment to DjHS and the community.
• Safety crosses: at each ward or clinical unit staff use a colour-coding system on a calendar to record when and what type of clinical incidents have occurred.
• Local ‘Know how we are doing’ noticeboards: raises staff and consumer awareness on incidents related to specific NSQHS Standards, such as medication safety and comprehensive care, using safety crosses and also notes what staff are doing to improve and how consumers can play a part in preventing incidents.
• Local area safety huddles: local huddles to engage staff on information and data presented on safety crosses and local ‘know how we are doing’ noticeboards.

Back to top

Figure E1 shows a selection of measures out of a sample of 16 serious incident action plans we reviewed across the four audited health services compared to better practice examples. The selection captures a spread of incident and measure types from the sampled action plans. There were two incidents where health services used relevant measures to assess the impact of implemented actions. We denote these measures with a ‘^’. 

Click the link below to open a PDF copy of Figure E1: Reported and proposed outcome measures for selected serious incidents.

 

Click here to download Figure E1: Reported and proposed outcome measures for selected serious incidents.

Back to top

Click the link below to open a PDF copy of Appendix F: Audited health service monitoring of SOP indicators for 2019–20.

 

Click here to download Appendix F: Audited health service monitoring of SOP indicators for 2019–20.

Back to top

Click the link below to open a PDF copy of Appendix G: Additional quality and safety performance indicators.

 

Click here to download Appendix G: Additional quality and safety performance indicators.

Back to top