ICT Strategic Planning in the Health Sector

Conclusion

Since late 2015, the department has been improving its leadership and support for public health services, including through the release of the November 2016 Digitising Health Strategy. The department's recent initiatives show that it is committed to better engaging the sector and planning more effectively for integrated clinical ICT systems.

However, further work is required from the department to fully and effectively implement the Framework. Strategic planning at a local level also requires strengthening.

The department has not completed all of the actions it committed to in the 2014 Framework, with four of its 16 actions requiring further work. It also has not effectively monitored its activities through a governance body. There is therefore limited assurance that the department's actions and the overall Framework have had the desired impact.

The department's investment decisions have not been informed by a comprehensive understanding of clinical ICT maturity or an ICT assets inventory for the sector. The department has not developed eligibility and selection criteria for health ICT projects to be funded through the Health Projects Fund. It has also not required assessment of benefits realisation as a specific condition of funding for health ICT projects.

At the local level, two of the audited health services have weaknesses in their ICT strategic planning processes. The four health services' ICT strategic plans do not refer to the Framework. We note that the department wrote to all public health services in late November 2016 informing them that local ICT strategies need to refer to Digitising Health.

Overall, this means there is little assurance that government funding is being directed to addressing the most critical health ICT needs, or that statewide interoperability of clinical ICT systems will be achieved.

Findings

Department of Health and Human Services

Alignment with the government's ICT strategy

The strategic themes and initiatives in the Framework and Digitising Health are largely in line with the four priorities of the government's ICT strategy—digital opportunity, information and data reform, technology reform, and capability uplift.

Progress with actions in the Framework

The Framework included 16 actions for the department. When completed effectively, these actions should provide a sound basis for the department to implement its statewide health ICT strategy.

There has been no governance body to oversee the implementation of the 16 action items and no system in place to evaluate whether the Framework is achieving its objectives.

Two of the 16 actions are ongoing tasks with no set completion date. The remaining 14 action items had completion dates between September 2014 and June 2015. The department has implemented 10 of these actions. The other four actions require further work.

Support for ICT investment decision-making

The department has yet to develop a framework to evaluate the effectiveness of the Digital Health Strategic Advisory Committee and the Health Technology Solutions Client Advisory Committee in supporting health services to make decisions about ICT investments.

Health ICT project assurance framework

The department has set up a health ICT project assurance framework (the assurance framework) for projects funded through the ICT Innovation Fund, and for other projects of $1 million or more that the department funds or co-funds. The aim is to ensure that:

• project outcomes are achieved within time lines and budget, as indicated in the project business case
• risks and issues are identified promptly and remedial actions are taken whererequired.

Applying the assurance framework using a threshold based on the project's cost does not adequately recognise the potentially complex nature of health ICT projects of less than $1 million.

The assurance framework is also not supported by a guidance document that clarifies how assurance requirements can be applied to projects considered low cost or low risk.

Capability and maturity of health sector ICT

Despite the recommendation in our 2013 report, the department has not completed its assessment of the capability and maturity of the health sector's ICT applications and infrastructure. The assessments undertaken to date are not comprehensive and do not produce consistent assessment results—a particular health service may be rated in different assessments as having different maturity levels.

The department told us that it would carry out a comprehensive assessment after an external review of the ICT maturity assessment model it intends to use for this purpose.

The department lacks a comprehensive inventory of key ICT assets at health services. There is limited assurance that government funding is directed to addressing the most critical health ICT needs, and is appropriately distributed so that health services can achieve the minimum clinical ICT system maturity by a designated time.

Engaging consumers

In June 2015, the department developed a five-level consumer engagement maturity model for health services. The lowest maturity level (Capability 1) means that a health service has identified the need to provide consumer access to their health information as a priority. The highest maturity level (Capability 5) means that a health service adopts and leads national consumer engagement processes for ICT solutions across the system, and has strong participation in national health ICT consumer engagement initiatives.

The Framework committed the department to preparing an options paper by February 2015 on how health services could work to improve their maturity level.

The department is unable to provide evidence that it has prepared the options paper or confirm whether it has assessed health services using the maturity model.

Reporting on benefits realisation

The department's assurance framework includes a step on reviewing benefits realisation. However, assessing and reporting on benefits realisation is not a specific funding condition in the department's funding agreements for health ICT projects. Without reporting on benefits realisation, the department cannot know whether it is investing effectively in health ICT.

Health Projects Fund eligibility and selection criteria

The department has not prepared and documented clear project eligibility and selection criteria to enable it to transparently, accountably and effectively choose projects to fund. The department told us that it would use the strategic themes in Digitising Health and its 2016–17 business plan to help choose projects to fund in 2017–18.

Strategic planning in the selected health services

Both BHS and EH have effective health ICT strategic planning processes. MH and NH need to improve elements of their ICT strategic planning processes.

Melbourne Health

MH's ICT strategic plan and other relevant documents did not include information on how MH prepared its strategic plan. We were unable to assess the extent of stakeholder engagement, and there was no review and monitoring framework for the implementation of MH's ICT strategic plan.

Northern Health

At NH, the change in strategic direction for clinical ICT—from a 'best-of-breed' model to integrated clinical ICT system—was not based on a robust analysis of options. Choosing a best-of-breed model means sourcing the best available component for each part of the system, rather than choosing an integrated system from one provider, which may have individual components that are not the best available.

Although the change of strategic direction was approved in March 2016, NH has yet to release a new ICT strategic plan to guide the introduction of what will be a significant investment in clinical ICT.

Alignment of local ICT strategic plans with the Framework

None of the four health services refers to the Framework in its ICT strategic plan. The health services told us that this was due to the department's strategic direction for health ICT being unclear at the time. Despite this, there is general alignment between the health services' ICT strategic plans and the Framework's 10 priority strategic initiatives.

Clinical ICT systems at the selected health services

Except for at HealthSMART sites, such as EH, funding limitations and local or regional needs, rather than a statewide focus of integration and interoperability, have driven how the health services have invested in their clinical ICT systems.

If this continues, the department cannot be assured that statewide interoperability will be achieved.

Improved leadership, support and engagement from the department should help to reduce the risk and ensure that local investments and statewide health ICT strategic direction become better aligned.

Sharing innovative approaches

To best realise the benefits of innovation, the health sector needs to share knowledge effectively. Establishing a collaboration and knowledge-sharing environment is one of the 16 action items under the Framework.

The department informed us that GovDex—the government's secure knowledge-sharing collaboration tool—and the sector's Health Chief Information Officer forums are used to share and exchange information.

Health services can share knowledge more effectively. The four health services have either introduced, or are considering a 'virtual desktop' solution to reduce the costs of maintaining up-to-date software and operating systems. Virtual desktop technology is a central data centre and set of applications that authorised users can access from any computer.

Each of the four health services was unaware of the virtual desktop technology at the other services. They did not share knowledge of this innovative approach or any lessons learned from its implementation.

Recommendations

We recommend that the Department of Health and Human Services:

1. develop and implement a framework to evaluate its new health sector committees to ensure that they better engage with and support health services (see Section 2.3.1)
2. require a risk assessment for health ICT projects it fully funds or co-funds that are worth less than $1 million, to determine whether such projects will be subject to the health ICT project assurance framework (see Section 2.3.1)
3. develop and communicate a guidance document on the use of the health ICT project assurance framework, to improve transparency for health services and make more consistent the application of the assurance framework. In particular, the guidance document should articulate how assurance requirements of the framework can be applied to projects considered low cost or low risk (see Section 2.3.1)
4. complete its assessment of health ICT application and infrastructure capability and maturity at health services (see Sections 2.3.1 and 2.5.4)
5. prepare a comprehensive inventory of key health ICT hardware and software in the sector (see Section 2.5.4)
6. prepare and communicate to health services an options paper to increase engagement with consumers (see Section 2.3.1)
7. assess, in conjunction with health services, the maturity level of the approach to consumer engagement at the health services (see Section 2.3.1)
8. ensure that an appropriate governance committee oversee the completion of action items from the Statewide Health ICT Strategic Framework that we have identified as requiring further work (see Section 2.3.2)
9. set up an evaluation framework to assess the effectiveness of completed action items under the Statewide Health ICT Strategic Framework (see Section 2.3.2)
10. set up an evaluation framework to assess the effectiveness of Digitising Health: How information and communications technology will enable person‑centred health and wellbeing within Victoria (see Section 2.3.2)
11. make assessment and reporting of benefit realisation a funding condition for health ICT projects the department fully funds or co-funds (see Section 2.5)
12. prepare and document clear project eligibility and selection criteria for allocating funding under the Health Projects Fund (see Section 2.5.3)
13. work with the sector to ensure that knowledge of innovative approaches in health ICT is shared effectively (see Section 3.6).

We recommend that Melbourne Health:

14. review and document its ICT strategic planning process to ensure that:

• it identifies and consults relevant stakeholders to determine health ICT vision and goals and a roadmap to achieve them
• it clearly defines the review and monitoring framework for overall implementation (see Section 3.3.2).

We recommend that Northern Health:

15. review its decision to implement the HealthSMART clinical ICT system by undertaking an analysis of options (see Section 3.3.5)
16. prepare an ICT strategic plan that sets out its direction for clinical ICT (see Section 3.3.5).

We recommend that the four health services:

17. review their ICT strategic plans to ensure they are in line with Digitising Health: How information and communications technology will enable person‑centred health and wellbeing within Victoria (see Section 2.4).

We recommend that other public health services consider whether these recommendations apply to them and take appropriate action.

Responses to recommendations

Throughout our audit, we consulted the Department of Health and Human Services, the Department of Premier and Cabinet and the four audited health services—Ballarat Health Services, Eastern Health, Melbourne Health and Northern Health—and we considered their views when reaching our conclusions. As required by section 16(3) of the Audit Act 1994, we gave a draft copy of this report to the Department of Health and Human Services, the Department of Premier and Cabinet and the four health services, and asked for their submissions and comments.

Below, we summarise those responses. Appendix A contains the full responses.

All agencies responded. The Department of Premier and Cabinet welcomed the recommendations, and the Department of Health and Human Services and the four audited health services accepted the recommendations directed to them. The Department of Health and Human Services also provided a detailed action plan on how it intends to implement the recommendations.

1.1 Overview of ICT in the health sector

The Department of Health and Human Services (the department) is responsible for developing sector-wide ICT standards, programs and guidance. The department aims to ensure the sector's clinical ICT initiatives are consistent with the state's health ICT strategy.

Victoria's public health services are responsible for defining and implementing their local health ICT strategy and related risk-management activities. Under the Health Services Act 1988 (the Act), a health service's board oversees and manages each health service to ensure that it complies with the requirements of the Act and meets organisational objectives.

Health services' chief executives are responsible for advising their respective boards, the Minister for Health, and the department of any significant decisions, issues or risks that may affect the health service.

The department is responsible for:

• allocating state recurrent and project-specific funding
• sector-wide performance monitoring and accountability
• strategic asset management
• strategic planning at the state level.

1.1.1 The HealthSMART program

In 2003, the government introduced the $323 million HealthSMART program as part of a strategy to modernise and replace the public health sector's ICT systems. The objective of HealthSMART was to deliver standardised approaches to ICT systems and provide the foundation for a clinical ICT system for public health services.

The mandatory participation policy in place under HealthSMART required health services to either procure a state-mandated clinical system if they wished to receive state funding, or seek an exemption from the then Department of Health (DH).

By June 2012, the funds allocated for HealthSMART had been spent, and the clinical system program component had been delivered to only four of the planned 19 health services.

1.2 Previous audits and reviews

1.2.1 VAGO's 2013 report

In our 2013 report, Clinical ICT Systems in the Victorian Public Health Sector, we stated that HealthSMART failed because of poor planning and inadequate understanding of the complex requirements for designing and implementing clinical ICT systems. DH had significantly underestimated the cost, time and change management required to successfully deliver the project.

Our 2013 report also noted the significant benefits of electronic medical records (EMR), which allow the public health sector to effectively and efficiently share information about patients. Complete, accurate and timely information at the point of care can minimise waste from unnecessary and duplicated clinical tests by providing clinicians with relevant medical data when needed. Sharing information about patients digitally requires health services to integrate their ICT systems.

The 2013 report included recommendations that DH:

• prepare a comprehensive strategic plan for the ongoing development of EMR or clinical ICT systems in the public health sector
• carry out a comprehensive and standards-based assessment of clinical ICT system functionalities across the Victorian public health sector.

1.2.2 Ministerial reviews

After the end of HealthSMART, the 2013 report of the Ministerial Review of Victorian Health Sector Information and Communication Technology:

• highlighted the absence of a statewide health ICT plan, and recommended that one be developed
• recommended that the mandate of the participation policy under HealthSMART be removed.

DH accepted both recommendations.

After the discovery of a cluster of avoidable perinatal deaths at Djerriwarrh Health Services during 2013 and 2014, the Minister for Health asked the department to review hospital safety and quality assurance. In October 2016, the report Targeting Zero: Supporting the Victorian hospital system to eliminate avoidable harm and strengthen quality of care found that the department had not effectively overseen hospitals and was unable to assure that public hospitals provide consistently safe, high-quality care.

The 2016 report stated that clinicians, hospitals and the department lacked ready access to complete and comprehensive information about patients, and about the performance of clinicians and hospitals.

Although data is collected, the use of paper-based rather than electronic record systems and the absence of a unique patient identifier make it difficult to analyse patient information to support clinical decision-making, monitor clinicians' and hospitals' performance and identify opportunities for improvement.

The report's recommendations confirmed the need for integrated clinical ICT systems to allow health services to share information about patients and increase the safety and quality of care that hospitals provide.

1.3 Health ICT strategies

Statewide Health ICT Strategic Framework

In late October 2014, the department introduced the Statewide Health ICT Strategic Framework (the Framework). The Framework was intended to help health services prepare local ICT strategic plans and align key outcomes throughout the sector.

The Framework set out a vision of 'Connected knowledge to ensure your best possible health' to be realised through three stages of change:

• Stage 1 Digitise and build foundations—move from manual to digital processes and create the ability to analyse and share information across the system.
• Stage 2 Connect the system—move towards a consumer-centric system that is connected, shares information and takes advantage of different care delivery models.
• Stage 3 Empower through knowledge—move towards predictive, personalised and preventive health care that is driven by knowledge.

Strategic themes and priority strategic initiatives are listed in the Framework to help health services to prepare their health ICT strategic plans in support of the Framework's overall vision, as summarised in Figure 1A.

The objective of the Framework was to provide a high-level document to guide how the health sector designed, adopted and invested in ICT. It was not intended to be a detailed strategic plan or a roadmap for achieving the government's health ICT strategic direction. The Framework did not commit to available funding.

Following the change of government in November 2014, the department republished the Framework in January 2015 with no change in content.

Figure 1A

The Framework's themes and initiatives

^(a) = Priority Strategic Initiatives.

Source: VAGO, based on the Framework.

Digitising Health

On 24 November 2016, the Minister for Health approved a new strategy, Digitising Health: How information and communications technology will enable person-centred health and wellbeing within Victoria (Digitising Health) Digitising Health acknowledges the critical role of digital technologies in improving how healthcare is delivered. The implementation is supported by a business plan for each of three financial years, beginning with the 2016–17 business plan, finalised in January 2017.

Digitising Health's strategic themes, guiding principles and components are largely consistent with the Framework. Figure 1B shows how Digitising Health's strategic themes relate to the Framework's three stages of change.

Figure 1B

How Digitising Health aligns with the Framework

Source: VAGO.

The Framework does not include information on the investment required or the funding available to deliver its strategic initiatives.

In contrast, Digitising Health identifies:

• 10 health ICT projects priorities for 2016–17, totalling $10 million
• funds available for 2017–18 ($10 million) and 2018–19 ($20 million) for health services and the department to deliver digital technology projects.

The total investment of $40 million will be funded through the Health Projects Fund (see Section 2.5.3).

The government's ICT strategy

In May 2016, the Special Minister of State published the Information Technology Strategy: Victorian Government 2016–2020 (the government's ICT strategy). The government's ICT strategy provides direction on whole-of-government information management and technology for the next five years and is to be reviewed every year. It supports the changes in technology required to focus public sector reform on value and effectiveness.

The government's ICT strategy sets out the government's direction in:

• information and data reform—improved sharing of information and data to deal more effectively with complex matters and social issues, such as family violence
• digital opportunity—better digital platforms to give Victorians access to everyday services
• new technology—more use of off-the-shelf ICT systems that are shared across government, with new cloud-based platforms to further improve productivity
• better capability—improving public service capability for successful project delivery on time, within budget and specification, using partnerships with experts.

The government's ICT strategy also includes the order of consideration for ICT investment, from sharing existing solutions to building 'bespoke' systems, as shown in Figure 1C. The strategy covers all government departments. Other agencies are to consult their portfolio departments to determine whether they are included.

Figure 1C

Planning for investment in ICT systems

Source: The government's ICT strategy.

1.4 Funding for health ICT

Public health services receive recurrent funds for activity‑based services (output)—but not for health ICT investment and maintenance—from a national funding pool of Commonwealth and state funding.

With no recurrent funding stream for health ICT capital and maintenance costs, health services source funding from:

• the Budget and Expenditure Review Committee State Budget process, either for a major ICT reform project or as a component of a capital redevelopment proposal
• the department, in the form of grants or loans
• internal health services'resources or fundraising activities
• the Commonwealth Government, in the form of grants.

In practice, public health services rely largely on external funding for clinical ICT investment because of funding constraints.

Our report Public Hospitals: 2015–16 Snapshot noted that public hospitals generated a net deficit of $115.9 million in the year ended 30 June 2016 and $135.8 million in the year ended 30 June 2015.

1.4.1 Departmental grants

After the HealthSMART program ended, the department gave grants to health services for their health ICT needs through the following funding programs:

• $50 million in redirected departmental funds for the Victorian Innovation, eHealth and Communications Technology Fund (ICT Innovation Fund)—$37.5 million for 2013–14 and $12.5 million for 2014–15
• $10 million in approved state government funding for the 2015–16 Clinical Service Hardware Replacement program.

In 2012, the ICT Innovation Fund was established to support innovation in the health system and ICT projects, including system and software upgrades and installations. Funding was allocated using set criteria and expressions of interest.

The Clinical Service Hardware Replacement program was targeted at the highest priority and at-risk clinical service hardware in the public health sector. The 2015–16 State Budget allocated $10 million to the program. Grants were allocated to health services using set criteria and expressions of interest.

In 2016, the department set up a third program—the Health Projects Fund—to fund the digital technology projects for 2016–17, 2017–18 and 2018–19 that were identified in Digitising Health. Internal departmental funding of $40 million was allocated for this program. For 2016–17, the department chose to fund 10 health ICT projects through the Health Projects Fund, as outlined in Figure 1D.

Figure 1D

Health Projects Fund—ICT projects for 2016–17

Source: VAGO, based on Digitising Health.

1.5 Structure of the department

The department appointed a Health Chief Information Officer (Health CIO) in late 2015 to provide collaborative ICT leadership in the health sector and in the department.

The Health CIO leads the department's Digital Health branch (formerly the Health Technology branch). In 2016, following a review of the branch structure, the department created three new business units within the branch and renamed three existing units. The key objective was to create a structure that better reflects the sector and the health program areas, and supports them in delivering the intended outcomes of Digitising Health. Figure 1E shows the new structure of the Digitising Health branch.

Figure 1E

Digitising Health branch

Source: VAGO, based on Digital Health Business Plan 2016–17.

The new business units are Governance and Reporting, Sector Projects and Sector Planning.

The Sector Solutions business unit now provides technical support for clinical ICT systems beyond the systems mandated by the HealthSMART program. This important and positive change will help ensure alignment between local ICT investments and the state's health ICT strategic direction.

1.6 Services profile of the audited health services

The four health services we looked at provide a range of services at several sites, as shown in Figure 1F.

Figure 1F

Services profile of the audited health services

Note: Statement of Priorities is an annual accountability agreement between public health services and the Minister for Health. It outlines the key performance expectations, targets and funding for the year and government service priorities.

Source: VAGO.

1.7 Why this audit is important

The public sector's investments in ICT have generally been costly and poorly managed. In 2015, in Digital Dashboard: Status Review of ICT Projects and Initiatives, we reported that public entities were not able to assure Parliament and Victorians that investments in ICT had resulted in enough public value to justify the significant spending.

Strategic health ICT planning is essential for the public health sector to deliver efficient and high-quality care to Victorians and to help ensure that scarce funding is appropriately allocated. With 86 public health services, Victoria needs a coordinated approach to clinical ICT systems development and investment so that it can achieve systems integration.

1.8 What this audit examined and how

To assess whether the public health sector's ICT strategic planning is effective, we looked at whether:

• the Framework is consistent with the broader strategic objectives and priorities of the government's ICT strategy
• the department is appropriately implementing the Framework
• the department is providing clear leadership and direction to inform ICT strategic planning by health services, particularly for clinical ICT systems
• ICT strategic planning at the health services level is effective, particularly for clinical ICT systems.

We looked at the department and four selected health services:

• Ballarat Health Services
• Eastern Health
• Northern Health
• Melbourne Health.

We carried out this audit in accordance with section 15 of the Audit Act 1994 and the Australian Auditing and Assurance Standards. The total cost of the audit was $620 000.

1.9 Report structure

The remainder of this report is structured as follows:

• Part 2 discusses the department's statewide planning for ICT systems in the health sector.
• Part 3 discusses strategic planning of ICT systems at the four health services.

2.1 Conclusion

The department has not appropriately implemented the Framework. Implementation of actions committed under the Framework is incomplete and not properly monitored by a governance body. No evaluation framework is in place to assess the effectiveness of the Framework and the completed actions.

The department's investment decisions have not been informed by a comprehensive understanding of clinical ICT maturity or an inventory of key ICT assets in the sector. As a result, there is limited assurance that scarce government funding has been directed to address the most critical health ICT needs or has been appropriately distributed.

The department has shown that it is committed to better engage with health services, and plan for more coordinated clinical ICT systems development within the health sector.

2.2 Aligning health ICT strategies

We compared the strategic themes of the Framework and Digitising Health with the objectives and priority actions of the government's ICT strategy, and found that their themes and priorities are largely aligned, as shown in Figure 2A.

Figure 2A

Health ICT strategic themes, initiatives, objectives and priority actions

Source: VAGO.

2.3 Implementing the Framework

The department published the Framework in October 2014. It contained a list of activities for 2014–15, comprising 16 actions that focused on:

• strengthening the governance of the sector to enable accountable and strategic health ICT investment decisions
• clarifying and improving ICT capability, underpinned by a minimum baseline capability for the sector
• developing key resources and actions to help achieve the minimum baseline ICT capability more quickly, including setting up benefits realisation assessments and health ICT project assurance frameworks
• putting consumer needs at the forefront of health ICT planning and delivery.

The department told us that it chose the 16 action items to further the objectives of increasing the capability of health services to implement and manage clinical ICT systems and increasing health ICT integration and interoperability across the sector. Fourteen of the 16 action items had due dates between September 2014 and June 2015, and two were correctly identified as ongoing tasks.

We find that these 16 action items correspond to weaknesses we identified in our 2013 performance audit Clinical ICT systems in the Victorian Public Health Sector, including in governance arrangements, skills of ICT staff, project assurance, benefits realisation assessments, and inventory of health ICT capability. When completed, these actions should help the department to effectively implement its statewide health ICT strategy.

Figure 2B shows our assessment of the status of the 16 action items.

Figure 2B

Status of the Framework's 2014–15 action plan

Note:

● Completed, ● Further work required.

Source: VAGO.

2.3.1 Further work required

Of the 14 action items with due dates, we assessed 10 as complete and four as requiring further work.

Sector oversight for ICT development—Action item 1

This action item aimed to better position the department to engage with and support health services in their ICT investment decision-making. To help this partnership, the department established two committees in 2016:

• Digital Health Strategic Advisory Committee—comprising senior officers from the department and health services. The committee is responsible for endorsing health ICT strategies and priorities, overseeing sector strategic technology projects, sharing lessons from previous health sector ICT projects, and identifying opportunities for future investment in technology.
• Health Technology Solutions Client Advisory Committee—comprising senior officers from the department and health services who are clients of the department's Sector Solutions business unit, in its Digitising Health branch. The committee provides formal and regular engagement with the health sector on strategic ICT issues and oversees the performance of the business unit in delivering technology services. This committee replaced the former Health Shared Services Council.

The department has not established arrangements to periodically evaluate how and to what extent these new advisory committees are better engaging and supporting health services. The department needs to regularly evaluate these governance committees to ensure they are effective.

Project assurance model—Action item 3

Under the Framework, the department is responsible for:

• overseeing and assuring the quality of health ICT, including monitoring ICT projects that are publicly funded and others considered to be high risk
• reporting on the performance and benefits delivered through ICT, including sharing information and lessons, and monitoring benefits realisation.

The Framework included an action item for the department to develop a project assurance framework for health ICT projects to help ensure that:

• project outcomes are achieved within the time lines and budget specified in the project business case
• risks and issues are identified promptly, and remedial actions are taken when required.

In early 2016, the department developed and implemented a health ICT project assurance framework (see Appendix B). It outlines assurance activities to be conducted at key project milestones from feasibility and planning to completion of the project. These milestones align with the government's gateway review process.

Scope of the project assurance framework

The ICT project assurance framework applies to:

• ICT projects that the department fully funds or co-funds for at least $1 million
• all projects funded through the ICT Innovation Fund (see section 2.5.1).

The board of each health service is responsible for assuring ICT projects that the health service fully funds.

Applying a project cost threshold as part of the assurance framework does not take into account the potentially complex nature of health ICT projects of less than $1 million. Rather than excluding all projects of less than $1 million from assurance processes, the department should apply a risk assessment approach to determine whether these projects should also be subject to the project assurance framework.

Project assurance framework guidance document

The department informed us that it applies the ICT project assurance framework to all projects valued at $1 million or more, although assurance requirements may be modified for projects it considers to be low cost and low risk.

However, the department has no documented guidance on how the assurance requirements should be applied to low cost and low risk projects. The department needs to develop a guidance document to improve transparency for health services and improve consistency in how the project assurance framework is applied.

Assessing current sector ICT capability and maturity—Action item 9

This action item's objective is for health sector ICT application and infrastructure capability and maturity to be assessed using agreed maturity assessment models.

The department has begun to assess some health services using a widely accepted health ICT application and infrastructure capability and maturity model.

At the time of the audit, the department had conducted assessments at:

• 14 metropolitan health services
• seven regional hospitals
• five regional health alliances.

Effective completion of this work is fundamental for establishing a sound basis for the department to prioritise investments in health ICT. The department's records and our discussions with departmental staff confirmed that the assessments conducted so far have been neither detailed nor comprehensive. Not all public health services have been assessed. The assessment results for some rural alliances are aggregated, rather than being presented at health-service level, and the department is unable to confirm which health services are included in the aggregated results.

The assessments we saw are also inconsistent—the same health service can be identified as having different maturity levels in different assessments. The department told us that it would assess all health services after an external review of the ICT maturity assessment model in 2017.

Options for increasing consumer engagement—Action item 15

This action item aims to increase Victorians' access to their digital health information and enable them to schedule consultations online. Specifically, it requires the department to develop options on how health services can better allow consumers to access their health information, such as through consumer portals. The objective is to enable Victorians to be more involved in managing their health care.

The due date for this action under the Framework was February 2015. The department was unable to provide evidence that it had prepared an options paper and communicated it to health services.

The department has developed a consumer engagement maturity model for health services, as outlined in Figure 2C, although it cannot confirm that health services have been assessed against this model.

Figure 2C

Consumer access and engagement maturity model

Source: VAGO, based on information from the Department of Health and Human Services.

2.3.2 Governance and evaluation

The department has no governance arrangement in place to oversee how the Framework's action items are implemented or to acquit their completion.

The department told us that the previous Health ICT Ministerial Advisory Council did not meet after the change of government in November 2014. The Digital Health Strategic Advisory Committee established in 2016 has focused on preparing and implementing the new health ICT strategy, Digitising Health.

The department acknowledges that it could have better tracked work on the action items and linked any incomplete items with deliverables under the new strategy.

Through the 16 action items, the department has made significant progress laying a foundation for stronger governance in the sector and working towards achieving the state's health ICT objectives.

However, the department lacks an evaluation framework to assess the effectiveness of completed action items and the Framework as a whole.

2.4 Developing a new health ICT strategy

The Framework set the vision for an integrated Victorian healthcare system. Its objective is to provide guidance and direction to health services to develop their local ICT strategic plans and operational priorities.

In January 2015, the department republished the Framework on its website with no change in its content. The department told us that, towards the end of 2015, it began work on a new health ICT strategy to align with the current government's Health 2040 discussion paper and to identify priority health ICT investments.

The Minister for Health approved the new strategy, Digitising Health, near the end of our audit, in November 2016.

The department's Health Chief Information Officer advised all public health services on 29 November 2016 that they must refer to Digitising Health when developing local ICT strategies and business cases for ICT projects.

Impact on health services' ICT strategic planning

The ICT strategic plans of the four audited health services do not refer to the Framework. These were finalised at the following times:

• Ballarat Health Services (BHS)—July 2014
• Northern Health (NH)—September 2014
• Eastern Health (EH)—April 2015
• Melbourne Health (MH)—August 2015.

EH and MH completed their health ICT strategic plans after the re-release of the Framework in January 2015. EH advised that, although it considered the Framework when drafting its ICT strategy, it decided not to make reference to the Framework because the health sector was anticipating a new health ICT strategy at that time.

BHS's and NH's ICT strategic plans were finalised before the Framework's first publication in late October 2014. NH's plan was not reviewed and updated to refer to the Framework. BHS advised that it is currently drafting its 2017–2022 ICT strategic plan, which will reference Digitising Health.

We found some misalignment between the themes in BHS's ICT strategic plan and the Framework's strategic initiatives. We discuss this further in Section 3.5.

2.5 Department-managed health ICT funding

Since the HealthSMART program was wound up in 2012, the department has managed three other ICT funding programs for the health sector. The department is yet to make benefits realisation a specific funding condition. It also has not prepared eligibility and selection criteria for projects that receive money for health ICT projects funded as part of Digitising Health.

2.5.1 ICT Innovation Fund

In 2012, after the end of the HealthSMART program, the department redirected $100 million of its internal budget to provide health ICT funding for three identified categories:

• refreshing critical infrastructure
• developing EMR
• innovation.

Due to government budget-saving levies in place at the time, the department reduced the available funding to $50 million. In June 2014, the department received 74 expressions of interest for the ICT Innovation Fund, totalling $258.8 million in requested grants. The department allocated the first tranche of $37.5 million in 2014–15 and the remaining $12.5 million in 2015–16 for EMR systems at NH and another metropolitan health service.

The department's health ICT project assurance framework (see Appendix B) incorporates a step on reviewing benefits realisation. However, benefits realisation assessment is not included as a funding condition in the department's funding agreement for the ICT Innovation Fund.

Although the funding agreement requires recipients to provide a post-implementation report, they are not specifically required to report on how well they achieved the intended outcomes. This means that the department cannot be assured that its investments were effective.

The department told us that future funding agreements would explicitly require reporting on benefit realisation as a funding condition.

2.5.2 Clinical Service Hardware Replacement program

The department received $10 million funding for the Clinical Service Hardware Replacement program in the 2015–16 State Budget. A further $10 million was allocated for the program in the 2016–17 State Budget. The objective of the program is to replace obsolete ICT infrastructure that is used directly in patient care and is most at risk of failure.

In 2015, health services were invited to submit a list of their health ICT hardware assets and identify those that were either obsolete or at high risk of failure, with the potential to affect the availability and quality of clinical services.

On the recommendation of the department, in September 2015 the Minister for Health approved the allocation of $10 million to selected health services under the program. The department reimbursed health services when they submitted invoices for asset replacement.

2.5.3 Health Projects Fund

Early in 2016, the department redirected $40 million of its internal budget to the Health Projects Fund. The objective was to fund priority health ICT projects, in line with Digitising Health, for three years from 2016≠–17 to 2018–19. Ten million dollars is available for each of the first two financial years and $20 million for the third year.

To allocate the $10 million funding for 2016–17, the department considered previously submitted, but unsuccessful, applications for the ICT Innovation Fund and chose projects based on:

• urgency of need
• readiness of the health service to undertake the project
• potential impact and benefit
• return on investment
• alignment with the strategic themes of the Digitising Health strategy—then in draft form.

For the 2017–18 funding round, the department told us that it will use the strategic themes in Digitising Health and its 2016–17 business plan to help choose projects to fund.

It is unclear how the department will apply the strategic themes, and whether certain strategic themes will have greater weighting than others. It is important that the department prepare and document clear project eligibility and selection criteria, so that its decisions on project funding are transparent, accountable and effective.

Funding agreements for projects funded through the Health Projects Fund also do not include benefits realisation as a funding condition.

2.5.4 Allocating health ICT funding

As discussed in Section 2.3.1, the department's decision-making processes on health ICT investments are not yet underpinned by a comprehensive assessment of the maturity level of the clinical ICT systems at each Victorian public health service. The department also lacks a comprehensive inventory of key health ICT hardware and software. Without this fundamental information, there is limited assurance that the department has:

• directed government funding to address the most critical health ICT needs
• distributed the funding so that all health services will achieve the minimum clinical ICT system maturity level by the designated time.

The department told us that it has begun to prepare a roadmap for replacing obsolete key health ICT systems and applications, such as EMR and patient administration systems, but was not able to provide us with a draft of the roadmap.

2.6 Managing cyber security

Cyber security is one of the focus areas of the government's ICT strategy. Like other sectors, the health sector must manage cyber security issues and associated risks. A recent example is the Qbot infection of MH's network in January 2016, which affected its computers and pathology systems.

Clinical services technology within health services is highly interdependent, and system failure arising from cyber security attacks, ageing ICT assets or other causes can pose risks in several clinical areas. Figure 2D lists examples of such risks.

Figure 2D

Example of clinical risks arising from system failure

Source: VAGO, based on the Department of Health and Human Services Clinical Services Technology Replacement Program, Full Business Case, December 2015.

The department has acknowledged that a sector-wide approach is needed to address common cyber-security threats. Digitising Health identifies cyber security as one of the critical factors in achieving the desired outcomes of the government's ICT strategy.

In May 2016, the department directed health services to complete a cyber health check baseline assessment. This is an important initiative to ensure that robust processes and controls are in place at health services to prevent, detect and respond to cyber security attacks.

The cyber health check baseline assessment found that health services are at varying levels of maturity, and all health services need to work to achieve minimum cyber security standards.

After the assessment, the department prepared a set of cyber security minimum baseline requirements, comprising 72 cyber security controls. A working group of representatives from the health services and the department has been set up to plan how the health sector will attain the maturity required to protect the quality and safety of clinical care from cyber breaches.

We commend the department's work in improving cyber security in the health sector, and will consider an audit in the future to provide feedback on the effectiveness of recent activities in mitigating cyber security risks.

3.1 Conclusion

Both BHS and EH have effective health ICT strategic planning processes. In contrast, MH and NH need to improve aspects of their planning processes. We found gaps in the alignment of the themes of BHS's ICT strategic plan and the strategic initiatives of the Statewide Health ICT Framework (the Framework).

In the past, the department had provided little support to MH, NH and BHS when planning for their health ICT systems. These three health services told us that they have received more support since the department appointed a Health Chief Information Officer (Health CIO) in late 2015. This should help ensure better alignment between local ICT investments and statewide health ICT strategic direction.

3.2 Key stages of strategic planning

We assessed ICT strategic planning processes at the four health services against the following key stages of strategic planning:

• Stage 1—Develop and endorse a strategic plan
• Stage 2—Implement the endorsed strategic plan
• Stage 3—Evaluate the endorsed strategic plan
• Stage 4—Confirm the continued validity and relevance of the endorsed strategic plan.

Figure 3A defines the key components of each stage.

Figure 3A

Key stages of strategic planning

Source: VAGO.

In Section 3.3, we discuss what we found at the four health services, focusing on the weaknesses we identified.

3.3 Health services ICT strategic planning

Figure 3B shows that the four health services have in place most of the elements of the four stages of strategic planning.

Figure 3B

ICT strategic planning process in the health services

^(a)These assessments are based on the planning process for NH's 2014 ICT strategic plan. NH does not have a current ICT strategic plan, following a change in strategic direction in 2016 (see Section 3.3.4).

Source: VAGO.

3.3.1 Stage 1—Develop and endorse a strategic plan

BHS, EH and NH have all of the key components for Stage 1 in place. MH has most of the components in place.

Melbourne Health's planning process

MH was unable to show us sufficient evidence that it has a robust process for preparing its ICT strategic plan.

MH prepared an ICT strategic plan for 2015–20. Supporting documents indicate that MH's executive approved this plan in 2015, although the documents we saw do not state the date of approval.

The strategic plan identifies four priority areas:

• innovation, transformation and collaboration
• patient, consumer and workforce
• technology
• governance and sustainability.

For each of the four priorities, the ICT strategic plan includes deliverables, related projects and expected project outcomes. It includes a list of potential projects and their status, and identifies project owners. At the time of our audit, MH's two most significant ICT projects are:

• the electronic medical record (EMR) clinical ICT system, in collaboration with the Royal Women's Hospital and Peter MacCallum Cancer Centre
• the Pathology Laboratory Information System replacement project.

MH's ICT strategic plan and other documents we saw did not give us enough background information to understand how MH prepared its strategic plan. MH told us that executives and other relevant stakeholders took part in ICT strategic planning but we saw no records of stakeholder engagement activities.

MH did not define clearly how it will review and monitor progress of the ICT strategic plan implementation against the four priority areas. In practice, progress monitoring occurs at project level.

We note that MH has adequate processes for tracking the progress of key ICT projects, including gateway review processes for the two significant ICT projects mentioned above. Progress monitoring at project level may not provide enough assurance that ICT projects continue to align with the overall ICT strategic direction, especially when there are variations in project scope.

3.3.2 Stage 2—Implement the endorsed strategic plan

All four health services have in place key components to ensure that they implement their ICT strategic plan effectively. They have assigned roles and responsibilities for implementation and mechanisms in place to monitor progress. The evidence of a strong emphasis on ICT project governance also shows that all four health services are committed to implementing projects effectively.

3.3.3 Stage 3—Evaluate the endorsed strategic plan

A health service must evaluate whether a strategic plan is on track to meet intended objectives and realise expected benefits, so that it can assess whether it needs to make any changes to the strategic plan.

Evaluating and analysing benefit realisation form part of the four health services' strategic planning. Governance mechanisms in place at EH, BHS and NH appropriately require benefits realisation analysis. MH carries out benefits realisation analysis in its gateway reviews.

3.3.4 Stage 4—Confirm the continued validity and relevance of the endorsed strategic plan

BHS, EH and MH showed us evidence that they carried out Stage 4 as part of their ICT strategic planning process. Although NH showed us evidence of updating a previous ICT strategic plan based on outcomes from Stage 3, it was unable to show robust analysis to support a recent change to their ICT strategic direction.

Northern Health's change of strategic direction

In 2014, NH's board approved an ICT strategic plan for 2015–20, which set out how NH would achieve its strategic objectives. The plan stated clearly that NH would pursue a 'best-of-breed' system—sourcing the best available component for each part of the system, rather than choosing an integrated system from one provider, which may have individual components that are not the best available.

In March 2016, the board revised this ICT strategic direction and approved a move to adopting the integrated clinical ICT system that was previously mandated under HealthSMART.

Board meeting minutes listed several advantages in moving to the HealthSMART system. These advantages included:

• an existing statewide licence
• clinical processes and workflows already in place
• potential clinical ICT integration with another HealthSMART site in northern Melbourne.

The minutes also noted that the department encouraged and approved the change in direction.

Although adopting the HealthSMART clinical ICT system brings benefits, NH could not show that it properly analysed the advantages and disadvantages of other available options. This means that the board cannot be assured that the decision will deliver the best outcome for NH.

Northern Health's strategic plan

Although NH's change of strategic direction was approved in March 2016, it has yet to release a new ICT strategic plan to guide the adoption of what will be a significant investment in clinical ICT. During the audit, NH told us that it was preparing a new strategic plan.

3.4 Achieving integration and interoperability

One of the Framework's objectives is to achieve clinical ICT integration and interoperability within each public health service and throughout the sector. Interoperability of clinical ICT systems will help ensure that health services can care for patients using all relevant clinical information, irrespective of where previous care was given.

Achieving this objective requires coordinated planning and implementation of compatible clinical ICT systems throughout the health sector. The department plays a critical role as systems manager to set, and help health services to meet, statewide standards that will enable interoperability.

3.4.1 Status of clinical ICT systems

In our 2013 audit, Clinical ICT Systems in the Victorian Public Health Sector, we found considerable variation in the nature and extent of clinical ICT systems in use in the public health sector. This remains true.

Apart from at EH, local or regional needs rather than a statewide focus have driven how the other three health services have prepared their clinical ICT systems. This is not surprising, since decision-making on ICT systems development and implementation has been devolved to each health service, and they operate within tight fiscal constraints. Another notable contributing factor is the department's historically poor leadership, and inadequate guidance and support for ICT strategic planning at health services.

Ballarat Health Services

BHS initially introduced a scanned medical record system in December 2009. This provided BHS with digital capability for, among others, pathology and radiology results reporting, medical and nursing handover, admission and discharge medication management, and discharge summaries.

This system has since evolved into a digital medical record (DMR) and is BHS's core application for managing clinical information. Together with the Grampians Rural Health Alliance, BHS is expanding its DMR to establish a shared regional DMR for the Grampians region.

Despite these changes, BHS's core medical record documentation and clinical processes remain largely manual and paper-based. BHS is now aiming to incrementally transition to an EMR system for the efficient delivery of health care. During the audit, BHS submitted a concept brief for an EMR to the department.

Eastern Health

Of the four health services, EH is most advanced in implementing its clinical ICT system, which has been designed to enable statewide interoperability. This is because EH became a HealthSMART site in 2003, and implemented the state-approved clinical ICT system.

Northern Health

NH's previous ICT strategic plan, which was approved in 2014, proposed a best‑of‑breed solution. The plan identified the need for a foundation stage to prepare for a more mature clinical ICT system.

Due to the change in strategic direction, NH has completed only the highest priority activities of the foundation stage—improving ICT connectivity by upgrading desktop computers, preparing a mobile device strategy and putting in place a standard operating environment. NH has also begun to electronically scan paper medical records as part of its implementation of an integrated EMR system.

Melbourne Health

In 2003, the department chose MH as one of the health services to introduce the HealthSMART program, but this did not go ahead because funding ran out. MH continues to use paper medical records and has lower clinical ICT capability than the other three health services. It is the last major teaching hospital in Victoria that uses solely paper records. A total of 1.28 million medical record movements were recorded in MH's tracking system in 2011, requiring 49 full-time-equivalent staff to manage.

In September 2016, MH began electronic scanning of medical records. This is intended as an interim solution ahead of the planned EMR implementation with Peter MacCallum Cancer Centre and the Royal Women's Hospital.

3.4.2 Engagement and support

Senior ICT staff at BHS, MH and NH reported that the department had historically provided limited support during their ICT strategic planning.

EH has worked better with the department because:

• key members of the EH ICT management team worked on the Framework
• the department actively helped to govern the introduction of HealthSMART at EH.

The other three health services told us that the department provided more engagement and support after August 2015, when it appointed a Health CIO, who takes part in:

• health service meetings
• governance committees for major ICT initiatives at the health services
• other health sector forums.

This active engagement will help to achieve sector-wide integration and interoperability.

3.5 Aligning with government strategic priorities

In Part 2, we noted that the four health services do not refer to the Framework. To identify how their ICT strategic plans align with the Framework, we compared the themes of the BHS, EH and MH ICT strategic plans with the strategic priorities of the Framework. NH lacks a current ICT strategic plan.

BHS's ICT strategic plan reflects the intent of seven of the Framework's 10 priority strategic initiatives, as shown in Figure 3C. EH's ICT strategic plan is fully aligned with the intent of the priority strategic initiatives of the Framework. This is not surprising, considering it is a HealthSMART site that has had, and continues to receive, strong engagement and support from the department. MH's ICT strategic plan is also aligned with the intent of the Framework's priority strategic initiatives.

Figure 3C

Health services' ICT strategic plans—alignment with the Framework

Note: ✔ Alignment observed, ✘ Alignment not observed.

Source: VAGO.

Despite the identified gaps, all four health services are strengthening their clinical ICT systems to increase the use of data for better decision-making, and to improve patient safety and efficiency in clinical delivery.

3.6 Sharing innovative approaches to ICT

When demand for funding exceeds available funding, innovative thinking can result in increased efficiencies and cost reductions to enable delivery within the available budget.

During our review of the four health services, we found that EH and NH have implemented a 'virtual desktop' solution to reduce the costs associated with maintaining up-to-date software and operating systems on individual desktops. BHS has also adopted a virtual desktop solution, which will be phased in from June 2017. At the time of our audit, MH was considering similar options.

Virtual desktop technology is a central data centre and set of applications that authorised users can access from any computer. As well as reducing costs, this technology offers health services four main benefits:

• ICT managers can centrally control the applications available to users and prevent the use of unapproved applications
• every user has an individual login, so all activity can be tracked, creating an audit trail and providing robust data security
• start-up times are much faster, giving users quick access to systems
• clinicians can use their own devices to access the system from any location.

Virtual desktop technology may help health services save money. In the case of NH, for example, the three-year projected cost to replace all of the computers running on unsupported operating systems was $2.4 million. The virtual desktop solution was implemented for about $1.2 million—a saving of $1.2 million, or 50 per cent.

However, some health services do not find virtual desktop technology to be cost‑effective. MH has determined that the technology is too expensive for its purposes, and is considering other options.

It is encouraging to see that the four health services are considering or implementing ICT solutions that best use limited funding, although they did not share knowledge of their chosen solutions and were implementing them in isolation. Benefits of innovation will be maximised when a culture and mechanism for sharing innovative approaches and ideas, and lessons from their implementation, are embedded in the sector.

One of the 16 action items from the Framework is for the department to establish a collaborative, knowledge-sharing environment. The department informed us that GovDex, the government's secure knowledge-sharing collaboration tool, and the Health CIO forums are used to share and exchange information with the public health sector. However, knowledge-sharing appears not to have been fully embedded.

RESPONSE provided by the Secretary, Department of Health and Human Services

RESPONSE provided by the Secretary, Department of Premier and Cabinet

RESPONSE provided by the Chief Executive Officer, Ballarat Health Services

RESPONSE provided by the Board Chair, Eastern Health

RESPONSE provided by the Executive Director, Corporate and Information Services, Melbourne Health

RESPONSE provided by the Chief Executive, Northern Health