Service Victoria—Digital Delivery of Government Services

Has digital delivery of government services improved customer experiences and reduced costs? 

Video presentation

Video transcript

Key facts

What we found and recommend

We consulted with the audited agencies and considered their views when reaching our conclusions. The agencies’ full responses are in Appendix A. 

Transaction reform 

Most people want to be able to complete transactions, such as pay bills, apply for services or update their details, anywhere and at any time. 

The Department of Premier and Cabinet (DPC) estimated that the Victorian Government conducted over 55 million transactions with customers in 2015, with only 1 per cent being fully digital. Most transactions were face-to-face, or via phone or mail. 

Recognising that such transactions are costly to government and often not preferred by customers, DPC and the former Department of State Development, Business and Innovation commissioned external consultants to examine the case for reform.

DPC stated in its 2015 Improving the efficiency and effectiveness of Victorian Government transactional services full business case (the Reform Program business case) that it cost government $461 million per year to deliver transaction services across six departments and seven major agencies. It predicted that without changes, by 2026, this would increase to $713 million per year. 

DPC, in its Reform Program business case, recommended: 

• creating a new service unit to establish digital capability
• optimising a subset of transactions
• developing capabilities for transaction delivery that can be used across Victorian Government agencies. 

The Victorian Government allocated $15 million for planning this new service unit—now known as Service Victoria (SV)—in the 2015–16 State Budget and a further $81 million for its development in the 2016–17 State Budget. As at 30 June 2020, SV has cost the Victorian Government $156.9 million.  

The implementation of SV was complex. It involved multiple interrelated projects, information and communications technology (ICT) and whole-of-government collaboration. 

The first three years

The Reform Program business case sets out three delivery 'horizons'. DPC anticipated that in the first three years, or Horizon 1, SV would deliver the scope shown in Figure A.

Figure A: Delivery of Horizon 1 scope (2015–18)

Source: VAGO.

The Reform Program business case

The primary purpose of a business case is to provide the government with enough information to make an informed investment decision. DPC's Reform Program business case clearly showed a need for change but lacked sufficient detail. 

Narrow range of options

DPC's Reform Program business case referenced alternative solutions interstate and overseas, but it did not explore these as project options. 

DPC explored four project options. All options were similar and included high-value transactions. The options only differed on the number of services that SV would deliver and whether SV would also offer a complaints portal and customer service strategy. DPC limited its cost–benefit analysis to these options. 

DPC did not adequately explore alternatives such as outsourcing to the market or delivering lower-risk transactions to first establish SV's capability.

Estimation of benefits

DPC outlined in its Reform Program business case that SV would drive benefits through transaction reform. Its benefits included: 

• improving customer satisfaction and reducing the time it takes for customers to transact with government
• saving the Victorian Government $61 million per year from the improved productivity of government departments and agencies
• increasing the effectiveness of government policy and regulations by making it simple and easy for people to transact with government. 

However, achieving these benefits relied on SV:

• optimising a set of existing high volume and high cost to government transactions such as VicRoads licensing services, Victoria Police applications, and the then Land Victoria property services
• being able to deliver and measure improvements in compliance, even though this was not well scoped in the Reform Program business case.

For example, DPC predicted that SV would deliver increased revenue because more people would obtain the required licence or pay their bills if simple and fast digital options were available. DPC suggested that the new agency would measure this by the reduction in unpaid fines. 

However, at the time DPC developed the Reform Program business case, the government was already considering an alternative fines ICT solution. Now known as Fines Victoria, this began operation in December 2017. SV did not onboard fines transactions, nor did it have a way to measure whether people's use of SV resulted in fewer unpaid fines. As such, this was unlikely to be a real or attainable benefit.

Reliance on assumptions and estimates

There were inherent problems with some assumptions that DPC used to build its business case and estimate the value of SV's benefits to government. DPC's Reform Program business case assumed that:

• the identified stakeholders would onboard with SV without a requirement for them to do so
• SV would be able to achieve the desired shift from in-person, post, and phone transactions to digital interactions.

In addition, because not all departments and agencies had reliable transaction data, DPC relied on estimates of the cost, volume and type of transactions undertaken.

Lack of key details 

DPC's Reform Program business case did not contain the full details and documentation recommended by the Department of Treasury and Finance’s (DTF) business case guidelines, such as a detailed project plan, procurement strategy or stakeholder management plan, funding model, governance framework or a detailed risk management strategy.

Further, DPC did not include in its business case a comparative assessment of what it would cost agencies to implement their own digital solutions. Nor did it include several items that it claimed would be explored as the program progressed, such as whether SV would have a physical presence or baselines for customer satisfaction.

The government considered the business case in 2015. While it recognised that the business case lacked key details, it noted that there was policy merit in the initiative and approved SV for inclusion in the 2015–16 State Budget with:

• an initial commitment of $15 million for planning—including the development of a detailed Program Implementation Plan 
• further funding (up to $122 million), which the government would hold in contingency, with release subject to DPC meeting project milestones determined in the Program Implementation Plan.

Failure to revise benefits and costs

As a complex ICT project that required the collaboration of many agencies, SV was subject to the high value high risk (HVHR) assurance process. However, despite Gateway reviews recommending that SV update the business case with revised benefits and costs, neither DPC nor SV made these revisions. 

When SV developed Program Implementation Plans, these did not detail and quantify the program benefits or address changes to underlying assumptions even after significant changes and government policy decisions had occurred. 

This reduced:

• transparency in the program
• the ability for government to track SV's progress against its original vision, which would have assisted government to:
  • decide whether SV remained a viable option
  • take action to improve project benefits (such as assess whether the program should be mandatory).

SV’s intended transactional benefits

The Reform Program business case predicted that SV would deliver a net financial benefit of $61 million per year. While SV would achieve almost $8 million per year in benefits through savings in payments and streamlining physical service centres, most of SV's financial value was though providing cheaper and more efficient transactions. 

In 2019–20, SV reported that it achieved $6 million in transactional benefits. This is a significant shortfall from the Reform Program business case’s annual transaction benefit estimate of $53 million. The shortfall is because SV did not deliver the volumes or types of transactions that DPC outlined in its Reform Program business case. 

This transactional benefit also does not reflect the true transactional savings to government. Rather, it reflects the savings to the agency from transferring the transactions to SV. For most transactions, it does not capture the ongoing costs of the transactions such as agency staff time or other indirect costs. 

In 2015, DPC did not have formal agreements with any of its key stakeholders to deliver their transactions. While it had engaged with these stakeholders, it had not 'locked in' the transaction delivery via a contract or formal agreement. SV’s success relies on agencies using it. One of DPC's risk mitigation strategies was that it would not implement SV without a mandated vision. This did not occur. The government chose not to mandate the use of SV and it remains optional for agencies to use.

The subsequent failure to onboard two bundles of transactions that the business case had anticipated has significantly impacted SV’s benefits realisation. These are:

• land registration and title services, with $23.4 million in projected benefits per year
• the full scope of VicRoads transactions, with $17.6 million in projected benefits per year.

Land registration and title service transactions

DPC's Reform Program business case included transactions managed by the then Land Victoria. This was a tenuous assumption because at the time of the Reform Program business case, the then Land Victoria had already begun to deliver some digital transactions via a national digital property platform and had recently transferred to a different department.

In May 2017, the Victorian Government announced a scoping study of land titles registry functions. In 2018, it announced that it would commercialise the land titles and registry functions of Land Use Victoria. 

SV removed this transaction type and related benefits from its annual financial benefit in July 2017.

VicRoads transactions

DPC expected SV to deliver all licence and registration transactions for VicRoads. However:

• SV does not deliver any VicRoads licence transactions
• at December 2020, SV was only delivering 3 per cent of VicRoads’ online 12-month vehicle registration renewals and 53 per cent of registration checks. 

Over the last five years, VicRoads has developed its own digital capacity and now delivers several digital transactions, including online registration checks and renewals, licence address changes, appointment bookings, notifications of vehicle transfers and an option to have a central VicRoads account. 

Several factors have contributed to VicRoads' decision not to onboard all transactions with SV. Some of these factors included the development of its own transactions and technology, its uncertainty about SV’s ability to deliver at scale, and the future of the VicRoads registration and licensing division.

Duplicated systems

In its Reform Program business case, DPC highlighted that SV would drive reform, consistency, and the delivery of common capabilities across government, including payments and identity management systems. 

DPC advised that over the medium to longer term, the government could decommission high-cost legacy systems and save money. However, agencies such as VicRoads, Ambulance Victoria and Working with Children Check Victoria still run their own digital transactions. This has meant that the Victorian Government has missed the opportunity to realise benefits from one centralised platform. Until departments and agencies decommission legacy systems, the government will continue to pay for multiple, fragmented IT systems and their associated costs.

Overall transaction volume

SV is only delivering a small fraction of its intended transaction volume. DPC estimated that by the end of Horizon 1 (June 2018), SV would be delivering approximately 11 million transactions per year. During the 2019–20 financial year (the end of Horizon 2), SV delivered 814 282 transactions. While this is an increase on previous years, it is a significant shortfall from the business case estimates and makes up only 1.3 per cent of all Victorian Government transactions. 

These low transaction numbers are due to: 

• the types of transactions that SV has onboarded
• the low percentage of digital transactions that agencies have directed to SV for some transactions.

COVID-19 related transactions

Between July to December 2020, SV introduced five transactions related to COVID-19, including border permits and exemptions, QR check-ins and poster generations, and regional travel vouchers.

SV's ability to quickly stand up these transactions demonstrates the ability of SV to build and utilise its technology. For example, SV set up online applications for regional travel vouchers in 2.5 days.

The introduction of the COVID-19 transactions has increased SV’s transaction volume to 2.57 million transactions in six months. However:

• QR scans are largely automated and are low value on a cost-per-transaction basis
• SV counts each individual QR poster generation, which may not be a true indication of the use and value of the service
• these transactions are likely temporary.

SV’s predicted volume

SV states that it is on track to deliver 6.7 million transactions (worth $20 million per year) by the end of its first 10 years, in 2025. It has not included COVID-19 transactions as a long-term benefit. 

SV’s predicted volume is still lower than Horizon 1's initial scope of 11 million transactions by 2018 (worth $53 million). SV’s ability to deliver its projected volume is dependent on it onboarding new services and delivering an increased proportion of the transaction volume for the services it currently delivers. SV has a pipeline of transactions that it is developing, but it is too early to tell whether it will deliver the scale of transactions anticipated.

Stakeholder engagement

As it is not mandatory for agencies to use SV, it is essential to SV's success that it can engage and earn the trust of government agencies. This includes understanding agencies' needs and retaining their commitment. 

Barriers to onboarding agencies

During the development of the Reform Program business case, various stakeholders supported the proposed SV. However, by the end of its first three years of operation, SV had still not onboarded many of the forecast transactions. Factors leading to this include: 

DPC outlined in its Reform Program business case that SV would benefit from being close to central government. As such, SV was established within DPC, which was ultimately responsible for implementing SV and ensuring its success. However, we saw limited evidence of DPC encouraging agencies to use SV prior to 2020, when it advocated for SV to have a role in the reform of the fines system. 

Stakeholder feedback

Almost all of the agencies that use SV stated that SV's capabilities have grown. However, SV does not have a mechanism to collect formal feedback from agencies, such as an annual survey. Rather, it meets with agencies during, prior to and after it has onboarded transactions. This is a missed opportunity for SV to identify issues from a stakeholder’s perspective and track its stakeholder engagement performance over time. 

Recommendations about stakeholder engagement

SV’s cost per transaction

Due to its low transaction volume, SV is costly on a per-transaction basis. In 2019–20, it cost SV $46.54 to deliver each digital transaction. As shown in Figure B, this was more expensive than face-to-face transactions. 

SV's cost per transaction is based on SV’s budget appropriation for 2019–20 divided by the number of transactions it delivered that year. As SV delivers more transactions, this cost will decrease as shown in Figure C. However, there are limitations to this calculation. It does not reveal SV's direct or indirect costs, nor does it reflect the complexity of the transaction. Further, if SV delivers one high volume, low complexity transaction (such as QR scans), this will skew the transaction cost despite the transaction requiring limited customer interaction. 

Figure C: Impact of COVID-19 transactions on SV's cost per transaction 

Source: VAGO from SV data.

SV's annual financial benefit

In 2015, DPC estimated that SV would deliver an annual financial benefit to the Victorian Government of $61 million per year. In January 2018, SV revised its benefit realisation framework and calculation of its annual financial benefit. It: 

• revised down its target to $30 million per year to reflect the removal of Land Victoria transactions
• added several benefits not explored in the Reform Program business case such as avoided costs, re-use benefits, intellectual property, payment savings and time returned to customer.

SV reported that it achieved this annual benefit for both 2018–19 ($75 million) and 2019–20 ($45 million). However, SV's actual benefit to government is likely to be less than SV reported. Across the two years, SV has claimed: 

• $50 million in benefits from reusing its technology to deliver transactions
• $52 million in avoiding costs to agencies through avoiding a more expensive market solution and procurement costs. Most of this benefit was from Solar Homes transactions (SV claimed $48 million for these transactions). 

As Figure D shows, there are limitations with these benefits. 

Figure D: Added SV benefits and their limitations

Source: VAGO.

Recommendation about SV's annual financial benefit

Reporting and oversight

DPC is the lead agency for the Victorian Government's Information Technology Strategy 2016–2020. A key priority of this strategy is ensuring that Victorians can interact digitally with government services in a way that is useful, easy, and always available. 

Key to this strategy was the need for DPC to drive increased digital uptake by Victorian departments and agencies. DPC's Reform Program business case outlined that SV would define the strategy, standards and policies that must be adhered to by departments and agencies in the delivery of transactional services. 

This model has changed over time. SV sets customer service standards and identity verification standards, but only for the services that it delivers. It does not define whole-of-government transactional strategy or provide a consultancy service for agencies.

DPC has published a series of best-practice guides that aim to make digitisation easier for agencies to understand and implement. It also suggests that agencies assess how they are performing by measuring:

• user (customer) satisfaction 
• time to completion
• transaction completion rate
• cost per transaction
• digital take-up, which refers to how many customers are using the service digitally compared to in-person, via phone or mail.

DPC does not know how many transactions exist across government, what they cost and whether they are digital as agencies do not report to DPC or publicly on these measures. While DPC commissioned work to estimate this in 2015, neither SV nor DPC has updated this data. This reduces the ability of government departments and agencies to:

• make data-driven decisions about how to improve their services
• compare data across multiple government services
• be open and transparent to the public about the performance of their services.

While SV reports on these measurements internally, it does not publicly report on its cost to deliver transactions or volume of services. SV's only public performance measure is customer satisfaction, which DPC includes in its annual report and the Victorian Government Budget Paper No. 3 (BP3) reporting. While customer satisfaction is an important aspect of SV's service delivery, it is also important whether Victorians use SV and whether SV is reducing the costs to government to process transactions. 

Recommendations about reporting and oversight

SV's customer satisfaction results

SV achieved its customer satisfaction target of 95 per cent in 2018–19 and 2019–20.  

However, it is difficult to compare SV across like services as there is no standard customer satisfaction measure used for public sector agencies in Victoria and other states. Each department or agency chooses its own metric. This can vary from CSAT (a customer satisfaction methodology) to Net Promoter Score, Customer Effort Score, or their own customer satisfaction metric. 

SV has used its own approach to measure customer satisfaction. This approach has been consistent and relies on statistically reliable data. However, it only captures feedback from customers who have finalised their transaction with SV. This means it potentially excludes customers who did not complete their transaction due to dissatisfaction with the service. 

Recommendation about customer satisfaction

Compliance with government regulations and policies

The Reform Program business case highlighted that the transaction reform would increase peoples' compliance with Victorian Government regulations and policies due to more consistent, accessible, and simplified processes. 

SV cannot show that it has achieved this benefit. SV's measures do not directly relate to SV's actions or specific policies and regulations, as shown in Figure E.

Figure E: Performance measurement and tracking

Source: VAGO.

Recommendation about government regulations and policies

Baselines used by SV

SV measures several key performance indicators (KPI) relative to the transaction baseline of the agency that uses SV. These measures include:

• reduced cost of transactions
• completion rate
• improvement in customer satisfaction
• reduced effort to undertake a transaction.

Agencies do not always have reliable data on their transactions. To address this, SV undertook its own research and used this to set baselines for measuring its performance against customer satisfaction, transaction cost, time to completion and completion rate. However, SV did not do this for all transactions. For the transactions that it did obtain baselines for, SV's approach to establish these baselines was unreliable due to low sample sizes. 

Recommendations about baselines

Back to top

Population growth and changing consumer expectations are placing increasing pressure on the cost, timeliness and quality of government services. Digitising transactions can help to address this. 

The Victorian Government launched SV in 2015 to transform the way that government agencies deliver transactions to citizens, improve customer satisfaction, and reduce costs.

1.1 Victoria’s need for digital services

The demand to access services online is growing rapidly, as is people’s expectations of the range of services they should be able to access anytime and anywhere. 

In 2015, Victorians completed around 55 million state government transactions by mail, phone, online or face-to-face. Considering the state’s population growth since, and assuming government transactions have grown in proportion, we estimate that Victorians completed over 62 million transactions in 2020. Transactions include: 

• making a payment, such as vehicle registration or topping up a myki (public transport) card 
• applying for a document, licence, or government program, such as a fishing licence, birth certificate, or the Solar Homes rebate 
• renewing or changing personal information, such as updating personal details for a driver’s licence or a Working with Children Check (WWCC). 

While some Victorian Government transactions are fully available online, others require customers to fill in paper applications or visit an office. It can also be difficult for customers to find information about services because the Victorian Government has hundreds of phone hotlines and websites. This is inefficient and costly to both consumers and the government.

The 2014 Improving the efficiency and effectiveness of Victorian Government transactional services Preliminary business case (the Preliminary business case) and the Reform Program business case, which were commissioned by DPC and the former Department of State Development, Business and Innovation, highlighted the need for the Victorian Government to reform the way it delivers transactions.

These business cases found that the government had not embraced modern technology and was lagging behind other jurisdictions. Figure 1A illustrates the state of digital service delivery in Victoria in 2015. 

The Reform Program business case also highlighted that, in 2013, Victoria’s customer satisfaction and ease of completing transaction rates were low compared to other jurisdictions.

1.2 The Victorian Transactions Reform Program

In 2015, DPC recommended that the government establish a new service unit, which is now known as SV. This aimed to address three primary transaction delivery challenges, as Figure 1B shows. 

FIGURE 1B: The Victorian Government’s three primary transaction delivery challenges 

Source: VAGO, from the Preliminary business case and the Reform Program business case.

Establishing SV

The vision DPC expressed in its Reform Program business case—to establish digital transactions across Victorian Government agencies—was consistent with the Victorian Government ICT Strategy 2014–15. This strategy had objectives to implement new digital and mobile channels for Victorians, standardise systems, improve productivity and increase the government’s capability to innovate. 

In its Reform Program business case, DPC predicted that SV would deliver three key benefits: 

• improved customer satisfaction and productivity (reducing the amount of time it takes for customers to transact with the government) 
• increased compliance with regulation (increased effectiveness of government policy and regulation) 
• increased government productivity (cost savings). 

DPC highlighted in its Reform Program business case that to achieve these benefits, the government must:

• transform its approach to transactional service delivery
• make it simpler, faster, and more convenient for people to access government services. 

As Figure 1C shows, the Reform Program business case outlined that SV would be delivered in three key stages, or ‛horizons’. 

1.3 Timeline

1.4 Service Victoria 

SV's project management approach

The Victorian Transactions Reform Program (Reform Program) was complex. It involved establishing a new unit that would provide a whole-of-government approach to transactional service delivery, including the development and implementation of a technology solution. 

SV aimed to address past failures with government ICT projects, such as delays and budget blowouts, by blending an Agile and waterfall project management approach.  

The waterfall method is a more traditional project management method. Projects that follow this approach outline their specifications before they start and deliver them at predetermined stages. This approach often does not involve the end user until late in the project. Changing requirements at that late stage can be expensive and time consuming. This can contribute to projects not being delivered on time, within budget or at all.

SV therefore took an Agile and iterative approach to progressively define its program and technology requirements and build capability. It then used a waterfall model of defined program phases designed to provide checkpoints and governance at key stages. Figure 1E shows the components of these methodologies.

In addition to these methodologies, SV also used the lean start-up methodology in developing its product, with the core intent of maximising customer value while minimising costs. It involves challenging processes and the underlying thinking to determine whether it can be more efficient.    

SV today

On 1 July 2018, SV became an administrative office of DPC. The Service Victoria Act 2018 outlines SV’s functions. This Act:

• enables SV to deliver government services
• provides a regulatory framework for SV to provide identity verification functions
• allows agencies to transfer their functions to SV. 

Customers can transact with SV via its website or a mobile app. Figure 1F shows what SV’s website and app look like.

Customers have the choice to transact as a guest or create an account if they want SV to store their information for future use. The SV app also allows customers to store digital licences. For now, SV has only released digital fishing licences.

SV provides three types of services: 

• end-to-end transactions, where a customer can complete the entire transaction via the SV website or app
• links to other services, where a customer clicks a link on SV’s website and is redirected to the relevant agency’s website
• white-label transactions—these are capabilities that SV has built for an agency but does not host on its website. For example, the identity verification component of a WorkSafe Victoria licence. 

We discuss the transactions that SV offers in Chapter 2. 

1.5 DPC’s role and responsibilities

DPC is responsible for developing and leading whole-of-Victorian-Government policies, including the Information Technology Strategy 2016–2020. This strategy’s key deliverables include providing better digital transactions and implementing SV. 

SV was initially approved as a project within DPC so it could be established quickly, rely on existing supporting policies and systems and be close to the centre of government and decision-making. DPC was responsible for the successful delivery of SV.

Program and project governance

The Senior Responsible Officer led SV's operation, management, and implementation. They were responsible for recommending program level changes or changes that represent a material departure from the business case to DPC's Secretary and the then Special Minister of State. 

SV had over 220 deliverables for the program. Many of these were interrelated. Rather than having individual project boards to oversee each key project, SV used an Agile approach to delivery and conducted frequent cycles of build–test–learn phases. It then used one project control board to oversee the delivery of the outcomes. 

Audit and Risk Management Committees

DPC's Audit and Risk Management Committee's (ARMC) purpose is to provide independent assurance and advice to DPC's Secretary on the effectiveness of the department's finance management, performance, compliance and risk management functions. 

In 2015, the DPC ARMC formed the SV subcommittee to assess and make recommendations on SV's governance and monitor risk. This subcommittee was disbanded in 2016 and replaced by a Program Control Board (PCB). Appendix I outlines the role of these committees and period of time they were operational. 

Current governance arrangements

In 2018, SV became an administrative office of DPC and gained powers and responsibilities under the Service Victoria Act 2018. SV's Chief Executive Officer reports directly to the Minister for Government Services for functions exercised under the Service Victoria Act 2018 and reports to DPC's Secretary on the general conduct of SV. This included the establishment of an internal audit function to provide assurance to SV, the DPC Secretary and the ARMC about SV’s operations. SV must follow DPC’s standards, policies, and governance frameworks.

SV's Board of Management (BoM) is responsible for overseeing SV’s operations and functions. It establishes SV’s strategic direction and provides assurance over SV's benefits realisation. The BoM is made up of SV’s Chief Executive Officer, Chief Information Officer, three executive directors and two directors. 

Figure 1G outlines SV’s governance arrangements.

Back to top

Conclusion

SV has delivered good customer satisfaction results for the limited services it offers. However, it has realised only a small proportion of the benefits expected in the Reform Program business case. 

In 2019–20, SV achieved transactional benefits of $6 million—well below the $53 million per year that the Reform Program business case predicted. It has also not achieved measurable improvement of people following government regulations and policy, as SV is only conducting about 1 per cent of government transactions. 

2.1 Realisation of the Reform Program’s expected outcomes

Now in its fifth year of operations (or Horizon 3 of the program), SV has delivered key components of the Horizon 1 scope. However, because of the limited range and type of transactions moved onto its online platform, SV has not achieved many of the expected outcomes.

Program scope and five-year vision

In its Reform Program business case, DPC established the scope for SV’s first three years (or Horizon 1) of transaction reform and set its longer-term vision as shown in Figure 2A.

FIGURE 2A: Program scope and SV's progress 

Note: SV was allocated three years of funding in the 2020–21 State Budget to continue its operations and partner with local governments to assist with streamlining business licence processing.
Source: VAGO.

SV has also only fully achieved two out of 10 specific outcomes outlined in the Reform Program business case and expected to be delivered in the first three years. 

SV's ability to achieve these outcomes was impeded by:

• changes in the program scope such as changed transaction mix and the decision not to proceed with a whole-of-government complaints system
• government decisions, such as not to proceed with a storefront or physical presence
• poor stakeholder engagement and agency decisions not to use SV resulting in fewer transactions onboarded than expected.

Figure 2B summarises the status of these outcomes. 

FIGURE 2B: Horizon 1 Reform Program outcomes and status

Note: Appendix D outlines SV’s achievement of Horizon 1 outcomes in further detail. 
Source: VAGO analysis of the Reform Program business case and SV data.

2.2 Delivering the intended transaction volume and mix

In its Reform Program business case, DPC anticipated SV would deliver a net present value (NPV) of $97 million over 10 years and that in Horizon 1, SV would:

• establish both an online and physical (storefront) presence
• deliver the top 41 per cent of all government transactions by cost—approximately 11 million transactions each year.

SV did not achieve this transaction target, and so is not on track to deliver the anticipated NPV.

Customer accessibility: online versus storefront

DPC’s Reform Program business case outlined that SV would to be accessible both online and via storefront locations. DPC flagged that establishing an in-person presence would require further exploration and development of a separate business case. The government did not approve this later business case. As a result, SV adjusted its focus to digital-only transactions and has not achieved the added benefits that a physical presence would bring. 

Transaction scope and mix

By the end of Horizon 1, SV had delivered only one of the 14 transactions originally within scope of the Reform Program business case—VicRoads end-to-end registration renewals, although this was not exclusive. By 2019–20 it had added WWCC transactions.

DPC's estimates of SV benefits relied on SV onboarding particular transaction bundles, many of which were high volume and costly for the government to deliver via existing channels. Two of the most important bundles were for Land Victoria and VicRoads. However, SV was not able to fully onboard either of these:

There are several reasons why SV did not deliver the original transaction mix, including that it was not mandatory for agencies to use SV and many decided to delay, defer or not onboard at all with SV. We discuss this further in Chapter 3. A detailed examination of transaction delivery is also shown in Appendix E.

Revising the transaction mix

SV is now aiming to deliver a positive NPV of $25.17 million over 10 years. This relies on it adapting the transaction mix and delivering additional benefits not listed in the original Reform Program business case. We discuss these new benefits further in Section 3.3.

To prove its capability, SV revised the transactions that it would onboard in 2018. These changes were listed in its Program Implementation Plans.

The types of transactions SV onboards influence the ultimate volume that it can deliver. For example, fewer people will register an 'absence from residence' with Victoria Police than would apply for a criminal history check. 

Figure 2C shows the number of services that SV has delivered over time and how this compares to estimates in the Reform Program business case. 

FIGURE 2C: SV's transaction volume (business case estimates and after changes to the transaction mix) 

Note: ^(a) The figures are based on Reform Program business case estimates prepared in 2015 with the transactions to be delivered at the end of Horizon 1 (i.e. by mid-2018).
^(b) 2017–18 was SV's beta (testing) phase.
^(c) DPC conducted an exercise to estimate transaction volume in 2015. The figures between 2017 and 2020 are VAGO estimates based on the impact of population growth. 
Source: VAGO analysis of SV data.

Other impacts on SV's transaction capacity and timing

SV's governing legislation, the Service Victoria Act 2018, did not come into force until 1 July 2018, almost two years after Cabinet gave in principle approval for the legislative model to transfer customer service functions to SV. The delay in passing this legislation contributed to delays in SV delivering transactions. 

Delivered SV transactions

SV offers a range of end-to-end transactions and links on its website to other external services. As Figure 2D shows, as at June 2020, SV had 17 end-to-end transactions available on its website and was the primary provider of seven of these. 

FIGURE 2D: Transactions offered by SV 2019–20

Note: The three Ambulance Victoria transactions are not currently available on SV’s website as Ambulance Victoria is updating its technical systems.
Note: We list the channel mix for services that are only available via SV as 99 per cent. This allows for a small volume of transactions that customers may complete outside of the agency options provided. 
Note: The percentage of transactions delivered by SV are based on SV data that forecasts the 2019–20 total agency volume.
Note: * these agencies have agreed to 'ramp up' or transfer 100 per cent of their digital transactions to SV.
Source: VAGO analysis of SV data.

Between 1 July and 31 December 2020, SV delivered five customer-facing transactions in response to the COVID-19 pandemic. It generated QR code posters, QR check-ins, border permits and exemptions, and the regional travel voucher scheme. 

SV's quick establishment of these transactions shows its ability to build on and utilise its technology. For example, SV established online applications for regional travel vouchers in 2.5 days.

This has resulted in significant growth in the volume of transactions delivered, with 2.57 million transactions delivered in six months. However, the longevity of services delivered in response to COVID-19 and their financial value is not yet clear. 

SV does not have a clear definition of a how it counts a service or a transaction. For example, SV counts each generation of a QR poster as a transaction. A business may print multiple posters to display at their premises; however, this should count as one transaction as only one QR code is generated. SV also includes non-customer-facing transactions (such as kangaroo tag scans and verification of solar installation scans) in its transaction count but does not count transactions for other services such as 'white label' transactions.

White-label transactions

Agencies can also use SV to deliver part of a transaction or service. These transactions are hosted on an agency website but use SV technology (such as online identity verification) for part of its transaction. SV calls these ‘white-label transactions’. As at 31 December 2020, SV delivered white-label services for:

• Residential Tenancies Bond Authority
• the Department of Environment, Land, Water and Planning (for its Melbourne Strategic Assessment program)
• WorkSafe Victoria.

SV does not count the number of transactions delivered for white-label services in its transaction volume.

Factors affecting channel mix

SV needs more customers to transact digitally for it to realise its expected cost benefits. Several factors can affect the decision of a customer to transact digitally, such as:

• the complexity of the transaction
• alternatives offered by agencies (such as their own online services as well as face to-face, phone or mail)
• customer preferences.

If SV was the sole provider of all transactions for the 17 established end-to-end services it offered in 2019-20, it would have processed approximately 14 million transactions per year. SV processed about 6 per cent of this number in 2019–20. 

Agency onboarding and referrals

When onboarding with SV, an agency that already has a digital channel may decide to automatically redirect a percentage of its customers to SV’s portal. 

The rate at which agencies have elected to redirect (or 'ramp up') has varied:

• Some agencies, such as Solar Victoria and the Department of Jobs, Precincts and Regions, have redirected 100 per cent of digital transactions to SV.
• Others, such as WWCC Victoria, have progressively increased their redirection rate. 
• VicRoads has been slow to ramp up. It was not until June 2020 that VicRoads agreed to:
  • increase its rate of referral of transactions delivered by SV (renew and check registration)
  • put the SV referral button on relevant VicRoads webpages. This is a button with SV branding that directs a customer to the transaction on SV's website.

In cases where SV is not the sole provider of the digital transaction (such as VicRoads, Ambulance Victoria and WWCC transactions), agencies run their own ICT systems, websites, customer service centres and phone lines. This means they are not realising savings through rationalisation of technology and system efficiencies. We discuss this further in Chapter 3.

2.3 SV transaction cost

Non-digital transactions are more expensive to provide than digital transactions. In 2014, DPC engaged a consultant to estimate how much transactions were costing the Victorian Government and what percentage of people were completing transactions digitally. 

Around the same time, Deloitte Access Economics looked at the economic benefits of digitising customer transaction services for federal and state government departments. While this work was not specific to Victoria, it suggested that the savings from moving transactions online could be even more pronounced than the Reform Program business case estimated, with digital transactions dropping to 40 cents per transaction. We reflect the business case estimates in Figure 2E. 

FIGURE 2E: Benchmark costs per transaction and channel mix (2015)

Source: Reform Program business case. 

SV calculates transaction costs by dividing its total running cost by the volume of transactions delivered. The marginal cost of onboarding new services is low. As SV increases its volume, its per unit transaction cost will decrease, assuming that its variable costs do not scale up with volume. 

However, there are limitations to this calculation. It does not reflect SV's direct or indirect operational costs, nor does it reflect the complexity of the transaction. Further, if SV delivers one simple, high-volume transaction (such as QR scans), this will skew the transaction cost despite the transaction requiring very limited customer interaction.

As shown in Figure 2F, in 2019–20 SV was expensive compared to similar services. 

Between 1 July 2020 and 31 December 2020, SV introduced COVID-19 transactions. Including these transactions significantly changes the cost per transaction amount as shown in Figure 2G. 

FIGURE 2G: Impact of COVID-19 transactions on SV's cost per transaction

Source: VAGO, based on SV data.

2.4 Customer benefits

DPC's Reform Program business case highlighted that digital reform would make it quicker and easier for customers to undertake transactions and improve customer satisfaction. 

SV has consistently achieved high customer satisfaction results for the transactions it delivers. 

Measuring customer satisfaction

Customer feedback provides important insights into SV’s products and services and helps it identify any problems.

DPC’s Measure how content performs—digital guide describes what better practice for digital services involves. It states that:

• at a minimum, a service should invite customers to give feedback once they reach the page that confirms a successful transaction
• a more robust approach is to capture data on people who abandon a form part way through the process. 

The Victorian Government does not have a prescribed approach for measuring customer satisfaction. This means that agencies can choose their preferred methodology to calculate it.

Customer satisfaction is SV‘s only BP3 measure, and it has set itself a high target of 95 per cent satisfaction. SV reported that it achieved its target for both 2018–19 (97 per cent) and 2019–20 (96 per cent). Between 1 July and 31 December 2020, SV reported that it achieved 96.7 per cent satisfaction.

SV does not ask all customers for their feedback but uses a sample to determine customer satisfaction with its service. When an SV customer completes a transaction, they may be offered an option to provide feedback on a five-point scale from ‛great experience‘ to ‛I'm not happy‘. SV then calculates customer satisfaction by using the median responses of the 3, 4 and 5 ratings.

Figure 2H compares SV’s customer satisfaction rate to that of other digital transactions and services.

Since its establishment, SV has used the same metric and methodology for measuring customer satisfaction. It uses this approach so it can compare its results with other similar jurisdictions. SV counts people who gave a score of 3 (OK, I guess) as satisfied. 

SV's calculation approach can impact the overall customer satisfaction ratings:

While using the CSAT approach would result in a lower customer satisfaction score, this measure is usually also associated with a lower target (75 to 80 per cent).

Other methods of gauging customer satisfaction

In addition to its externally reported customer satisfaction metric, SV also: 

• internally monitors scores of only 4s and 5s, which it calls its ‛happiness score’
• analyses written feedback from customers using sentiment analysis. 

SV's sentiment analysis suggests that the most commonly used word for customers who leave written feedback is ‛easy’. 

SV also reviews and monitors:

• error messages received by customers
• anonymised screen recordings of real customer visits to understand customer behaviour and how customers move their cursors across the screen
• the number of customer-initiated interactions, including live webchat, email, or online feedback forms
• all rating scores of 1. These are: 
  • opened as cases and assigned to a customer experience officer to review and resolve if possible
  • reported to the Chief Customer Officer.
• customer feedback and suggestions for continuous improvement shared by staff via Yammer, SV's internal chat system. This provides visibility of complaints across the organisation and allows staff to suggest solutions. 

The following case study (Figure 2I) illustrates the impact that proactively reviewing issues can have.

FIGURE 2I: Case study—VicRoads customer satisfaction with registration renewals

Source: VAGO analysis from SV data.

Gaps in assessing customer satisfaction

SV’s customer satisfaction assessment does not survey customers who end a transaction before completing it. This means that SV is likely excluding some dissatisfied customers from its survey and does not receive their reason for not completing the transaction. 

This is supported by the feedback received on SV's app, available via Google Play and the Apple App Store. 

Between January 2019 and December 2020, 87 people had reviewed SV’s app with a weighted average of 2.7 stars. 

As shown in the examples in Figure 2J, most reviews relate to the app’s lack of services or the information it can store. SV does not respond to these reviews.

FIGURE 2J: Customer reviews of SV's app in Google Play and the Apple App Store

Source: VAGO, from Google Play and Apple App reviews.

Customers can also provide feedback via SV's website. There is a general ‛Give feedback’ form at the bottom of its home page where people can get in touch, send a compliment, or make a complaint. 

During 2019–20, 10 647 customers used the website form to contact SV. Of these, 12 per cent of people clicked the 'make a complaint' option. A review of a sample of this data shows that customers also left comments indicating their dissatisfaction in the ‘get in touch’ category. If these customers did not complete their transactions, this feedback would not be included in SV’s overall customer satisfaction score.

2.5 SV's performance against agency baselines

In its Reform Program business case, DPC anticipated that it would be able to obtain performance baselines for existing Victorian Government transactions. However, at the time of the business case, not all agencies and departments had clear baseline data. Performance reporting against digital transactions across the Victorian Government is still not strong. 

Performance reporting at a whole-of-government level

DPC's best practice guidelines highlight the importance of measuring performance against benchmarks.

Despite this, the Victorian Government does not have a platform for agencies to report on their performance against its digital standards. In particular:

These limitations hinder the Victorian Government’s initiatives to drive digital reform because agencies cannot compare their performance against clear and reliable benchmarks.

SV's actions to address gaps in agency data

Between 2017 and 2019, SV conducted some research to address this gap, but not for all transactions and often not with reliable sample sizes. A low sample size increases the risk that the true result may vary from the reported figure.

This lack of reliable data is problematic as several of SV's KPIs and measures compare its performance against an agency baseline. These include:

• time returned to customer
• improvement to customer satisfaction by transaction
• completion rate
• reduced transaction cost.

To address this lack of data, SV has updated its operating agreement template to ensure that agencies either provide baseline data or give SV access to their transaction data when onboarding future transactions.

Figure 2K shows SV's reported achievements against these KPIs together with limitations we found with the methodology. 

FIGURE 2K: SV's KPIs, achievement and limitations

Note: We summarise SV’s performance against all of its KPIs in Appendix F. 
Source: VAGO.

Failure to implement a cost attribution methodology

VAGO's 2015 Delivering Services to Citizens and Consumers via Devices of Personal Choice: Phase 1—Interim Report highlighted that at that time, the public service did not have a suitable method to determine the costs of delivering service transactions that included both direct and indirect costs. It recommended that DPC develop a method for departments to use. DPC allocated this task to SV. 

SV developed a cost attribution method in its 2015 Program Implementation Plan that did attempt to include indirect costs. However, it is not using this method to calculate the cost of transactions, nor is there evidence that other Victorian Government agencies use this method. 

2.6 Public compliance with regulation and policy

DPC highlighted that people may be more likely to comply with government policies and regulations if the process is quick and easy. For example, as customers can now purchase and renew digital fishing licences via a mobile device, it is easier for people to obtain them, meaning more may do so.

However, SV cannot show that it has improved people's compliance with regulation and policy. This is difficult to measure. Further, neither the business case nor SV tracks compliance of specific policies or whether digital reform impacts this directly. 

Figure 2L looks at DPC’s measurements outlined in the Reform Program business case together with SV’s changes and the relevant limitations. 

FIGURE 2L: Measurements used to assess whether SV has improved compliance with regulations and polices

Source: VAGO.

2.7 SV’s future projected benefits

Since its establishment, SV has increased its service offering each year, achieving a growth rate of 100 per cent of the total volume of transactions delivered as compared to the 2018–19 financial year. However, this is still only a fraction of the Victorian Government’s total transactions, which we estimate to be approximate 63 million transactions per year.

SV’s transaction growth between June 2018 and January 2020 was low. However, since then two key agencies have increased their transaction redirection rates to 100 per cent: VicRoads in June 2020 and WWCC Victoria in September 2020 (for its WWCC renewals and teacher notification transactions). Further, the introduction of COVID-19 services is predicted to significantly increase transaction volume in the short-term.

As Figure 2M shows, SV’s transaction delivery is expected to become more economical due to new services and increased volume in 2020–21 and in subsequent years.

SV estimates that by 2024, it will cost $5.21 to deliver each transaction. This prediction is based on SV onboarding more transactions, attracting more agencies and increasing the ramp up rate of its current transactions. 

Back to top

Conclusion

DPC and SV have delivered a technology solution that is repeatable and reusable. However, DPC's implementation of SV did not focus on the delivery of financial benefits. This lack of focus, along with poor reporting and stakeholder management by DPC and SV, has resulted in lower-than-expected delivery of benefits.

The Reform Program business case was inadequate for a project of this complexity. It lacked key details such as governance arrangements, a detailed procurement strategy or project plans. The business case also included several risky assumptions, such as that certain agencies would use SV to deliver key transactions. DPC did not fully explore these risks to inform the investment decision. Further, when key changes to the program occurred, DPC did not amend the business case. This reduced the transparency of benefits delivery and the information available to key decision-makers. 

3.1 The business case for SV

A business case gives decision-makers information to help them to decide whether to invest in a project or not. It should provide decision-makers with the confidence that:

• the strategic justification for the investment is valid
• they are selecting the right investment option
• the agency can deliver the investment as planned.

The strategic justification for SV

As discussed in Chapter 1, DPC's Reform Program business case clearly outlined the need to improve transaction delivery in Victoria. This aligned with the government’s Victorian Government ICT Strategy 2014–15.

DPC, through the Reform Program business case, looked at options to improve transaction delivery. Together with key stakeholders, it developed an investment logic map that identified seven strategic interventions to address the problem. 

As shown in Figure 3A, DPC then grouped these interventions into four strategic options before assessing what the preferred options should be. 

DPC recommended Option 4 following a high-level initial assessment of the benefit, costs, and risks of each option. It determined that this option would maximise benefits from the digital initiatives that agencies already had in progress, build future capabilities, drive cost-efficiencies, and improve customer satisfaction.

However, the business case did not provide: 

• detailed costs of alternative solutions, such as the government using and building on existing platforms or outsourcing the technology or transactions to a market provider
• an overall risk ranking for each strategic option
• an overall cost figure. Instead, it ranked costs for each strategic option as low, medium, or high without a clear rationale for this assessment. 

The Gate 1 review, conducted in 2014, also noted that there were conflicting views on the relative priority between two key drivers of the program: efficiency and improved service delivery. Clarity around these drivers is important as they informed SV's focus and establishment. 

Once DPC identified a preferred strategic option, it explored a range of project options. These reflect what the investment would look like in practice and at a more granular level.

Reform Program project options

There are often many ways in which government can implement a project. However, as Figure 3B shows, DPC, through the Reform Program business case, only considered a narrow range of options. Its project options:

• all involved establishing a new service unit and optimising a set of transactions that were high volume or high cost
• differed from each other only in relation to the volume of services and whether the new service unit would have a service/contact centre or complaints function
• did not cost the use of an off-the-shelf or market solution, although the business case did note that the new service unit should explore myGov as an option for the identity verification project
• did not include a comparative assessment of what it would cost agencies to implement their own digital solutions
• did not consider whether it should have tested its technology solution with smaller lower-risk transactions during Horizon 1. Instead, each focused on the high-risk approach of optimising bundles of transactions that represented the government’s top 41 per cent of transactions by value. 

FIGURE 3B: Project options presented in the Reform Program business case 

Note: The Department of Justice was subject to machinery of government changes in 2015 and 2019 and is now known as the Department of Justice and Community Safety
Source: VAGO, from the Reform Program business case.

Selection of SV as the recommended solution

DPC's Reform Program business case included analysis of the benefits, costs, and risks of each of the four project options and recommended that Option 2 b would deliver the greatest financial and socio-economic benefits. 

The expected advantages and benefits of Option 2 b—establishment of a service unit now known as SV—included the: 

• highest ongoing annual benefit ($61 million) and 10-year NPV ($97 million)
• greatest impact on customer satisfaction, including that it would take customers the least effort to undertake a transaction and that it would reduce red tape
• power to drive change across the Victorian Government and facilitate collaboration and cooperation across departments and agencies
• capacity to improve the Victorian Government’s image and reputation around customer service and transaction delivery.

This recommendation heavily relied on DPC's calculation of the financial costs and benefits of reform. We found limitations with the analysis undertaken. 

Financial benefits of reform

To understand the cost and benefits of transaction reform, the Victorian Government needed to clearly understand the volume, type, and cost of transactions undertaken. However, this data was not readily available at the time (2014–15) because:

• departments and agencies had inadequate performance monitoring and data collection for digital initiatives
• it was difficult for departments and agencies to consistently map and estimate the types, volumes, and costs of their service transactions
• there was no suitable cost attribution method that could be used across the public sector to determine the cost components of delivering digital transactions.

Development of the transactions catalogue

To address the data limitations, DPC commissioned an external consultant to estimate how many transactions people complete with the Victorian Government each year, their cost, the channel used and their physical footprint. This was known as the transactions catalogue.

DPC used information from six departments and seven agencies including VicRoads, the State Revenue Office, Victoria Police and Public Transport Victoria. However, few departments and agencies were able to supply complete datasets or the required cost data. As a result, the consultant extrapolated transaction costs and volumes based on assumptions. 

DPC relied on these figures to estimate the base case position (the situation if the government did not invest in change) and the transaction benefits the investment would bring.  

Limitations with calculation of benefits

DPC used the transactions catalogue to help quantify the benefits of reform. It stated in the Reform Program business case that these figures were verified with stakeholders. However, there remained the risk that these costs and benefits were incorrect or would change with time. Despite this, neither DPC nor SV revisited the calculations. 

The Gate 2 review in 2015 recommended that DPC gain agreement from key stakeholders about:

• the accuracy of benefit calculation
• who was responsible for delivering benefits. 

The Gate 2 review team also recommended that DPC articulate a timeline for realisation of benefits. This was not clear in the Reform Program business case. 

Stakeholders expressed concern that the Reform Program business case was double counting benefits against agency initiatives that were already in progress or planned. While DPC did not include benefits from funded and implemented projects, it did include benefits from projects that were at an early stage of development. It also did not clearly cost these benefits. 

This increased the risk that DPC had overestimated the benefits of the program.  

DPC stated in its Reform Program business case that some of its benefit targets were ‛bold’. However, it believed they were achievable. The Reform Program involved multiple stakeholders, interdependent projects, and new technology solutions. Variations in any of these factors, changes in policy or consumer behaviour may impact the delivery of costs and benefits. 

Where investments involve uncertainty, DTF recommends that the investor incorporate flexibility into the investment option and undertake a detailed real options analysis. DPC did not do this. It instead completed a traditional cost–benefit analysis that required practitioners to value costs and benefits that were not known with certainty.

DPC did, however, conduct an external cost review. 

External cost review

In 2015, DPC engaged a contractor to review the Reform Program business case's costs and benefits. This review found that the: 

• assumptions underpinning the business case’s benefits were sound
• the business case’s cost estimates (supplemented with a supporting methodology document) were traceable, reasonable, sound, and complete.

The review also noted that DPC could make improvements, including:

• further quantifying the shift to digital channels, which allows for process improvements and transaction optimisation
• improving the detail and accuracy of its cost estimates.

The review recommended that DPC undertake a sensitivity analysis of the Reform Program’s overall costs, benefits, and major risks. 

Sensitivity analysis

A sensitivity analysis helps decision-makers understand the impact a range of variables have on a given outcome. While there is no strict guidance on how to conduct a sensitivity analysis, the analysis typically isolates variables (such as volume) and then records the impact this has on the outcome (both positive and negative). 

DPC engaged a contractor to conduct a sensitivity analysis in March 2015. This looked at two alternative scenarios but did not clearly isolate variables that may impact on the outcome. DPC explored the impact of:

• delays in the project but increased benefits through payments optimisation
• a slowed ramp up of VicRoads and Land Victoria transactions with the substitution of two new transactions. 

The contractor's analysis found that under either of these scenarios, the Victorian Government would still achieve similar financial benefits to those outlined in the Reform Program business case. 

However, the analysis did not provide decision-makers with all the information needed:

DPC's business case attributed $23.4 million in expected savings to Land Victoria transactions and $13.5 million to new vehicle registrations and VicRoads’ licensing transactions. As highlighted in the Gate 5 equivalent review, removing Land Victoria transactions from benefits realisations changed SV from an investment that would produce a return of $97 million over 10 years, to one that would cost the government money. This risk was not clearly costed and reported early in the implementation of SV.

3.2 Delivering SV

DPC's Reform Program business case stated that some details, such as benefits, costs and projects, would undergo further development as the program progressed. This meant that its scope, cost, and schedule estimates were only indicative. The Reform Program business case did not include the documents that DTF’s guidelines require, including a:

• clear governance framework
• procurement strategy
• rigorous risk management plan
• funding model 
• stakeholder management plan
• detailed project plan.

The use of an iterative approach to program delivery, while designed to maximise project efficiency and ensure continuous improvement, does not mean that key documents should be omitted. This is supported by the Gate 2 review, which highlighted a need for further development of the Reform Program business case to inform immediate government decision-making. 

The decision to invest in SV

The government recognised that the recommended reform was high risk, and that the Reform Program business case: 

• had an ambitious project schedule
• needed to establish long-term commitment across departments and agencies to be successful
• was based on assumptions about costs and benefits. 

However, rather than revise the Reform Program business case, in 2015 the government approved $15 million of funding for further planning of SV and an additional $121.1 million to be held by DTF for SV. 

This further funding was to be released in stages, contingent on SV successfully delivering program milestones and obtaining the Treasurer’s approval. In approving the funds, the government:

• asked the then Special Minister of State to report back in late 2015 with a detailed program implementation plan before seeking approval to release funding for the next stage of works
• noted that the Program Implementation Plan would be subject to the HVHR assurance process.

SV’s program implementation

SV's project management model is a blend of both the Agile and waterfall methodologies, which we discuss in Chapter 1. In February 2016, the Treasurer and the then Special Minister of State approved SV’s first Program Implementation Plan. SV revised this Program Implementation Plan on two subsequent occasions and provided this to the Program Control Board and DTF. We do not have evidence that the later plans were reviewed by the Treasurer or Special Minister of State, however, they were subject to review as part of the HVHR assurance process. Each of these Program Implementation Plans progressively defined SV's model and deliverables. 

The Gate 3 (equivalent) review in 2016 highlighted that while the project approach of using Agile, waterfall and Lean project methodologies was sensible, the program team was inexperienced in their use. It also stated that this was the first application of these approaches to a program of this complexity within the Victorian Government. 

This approach also meant that SV’s scope was still only developed at a high level almost 12 months into its delivery. SV undertook an iterative approach to its delivery, which meant that it progressively defined requirements across interrelated workstreams such as customer need, developing the new service unit and delivering the technology. SV stated that the program implementation plans were ‛as complete as possible’ for each stage of development. 

This progressive approach allowed SV to adapt to change, but it also meant that: 

• SV and DPC focused on delivering the current stage of the program and a specified product, not ensuring high-value transactions were onboarded and would deliver the intended benefits.
• The decision to invest in SV was based on a vision that has changed over time. For example, DPC, in the Reform Program business case, envisaged that SV would set standards and help agencies drive digital reform. SV amended its business focus in mid-2016 to focus on competitive reform, disruption of current agency business models, and the provision of new assets to government. 
• There was no clear line of sight of the changes to the Reform Program business case and their impact.

The business case lays the blueprint for the whole investment lifecycle. Over time, SV and DPC amended SV's scope, the transactions it would deliver, its targets and its vision. 

DTF Investment Lifecycle and High Value High Risk Guidelines—Business case states that if there are any material changes that affect the business case, the business case should be updated. Despite recommendations in Gateway reviews, DPC and SV did not update the business case, nor did they use DTF's Investment Management Standard to evaluate how well SV had delivered the expected benefits. Had they done so, it may have helped government and SV make decisions about whether to continue with the program, amend it or invest further to maximise benefit realisation.  

3.3 Benefits and performance measures

As discussed in Chapter 2, DPC's Reform Program business case listed three key benefits that SV would achieve. DTF guidelines outline that benefits should be real in nature, attainable and be a direct consequence of the proposed investment. Not all of SV’s intended benefits met these guidelines. 

In particular, the stated benefit of increased compliance with regulation was not:

• clearly linked to the problems outlined in the business case (poor customer satisfaction, costly and fragmented services and the risk that the government may fail to meet increasing demand)
• attainable—there was no evidence that simplifying and digitising transactions would improve compliance and regulation. 

Key performance indicators

DTF guidelines outline that a business case should include KPIs that are specific, measurable, achievable, relevant, and timely. DPC's Reform Program business case did not meet this requirement. 

DPC recognised this lack of detail and, in the business case, stated that:

'[d]ue to the complex and ongoing nature of the project, the identified benefits will be generated at differing times in the future and therefore some KPIs and measures can be more readily defined and measured than others’.

In 2018, SV revised its KPIs and targets. Despite this, the following limitations remain:

• The benefit of increased compliance with regulation still cannot be effectively assessed through the measurements, and associated KPIs are inadequate to address this.   
• The KPI measurements and targets are SV-centric, and do not reflect SV's performance in delivering the government-wide change envisaged by the Reform Program business case. 
• SV does not have reliable baselines for all transactions, meaning it cannot accurately measure whether it has improved outcomes (see Section 2.5). 

SV's annual financial benefit

One of SV's anticipated benefits is improved productivity of government transactions. SV measures this through its 'annual benefit to government' KPI, which it introduced in January 2018. 

DPC's Reform Program business case predicted that SV would deliver an annual financial benefit of $61 million. However, in 2018, SV: 

• revised the annual benefit figure down to $30.3 million, reflecting the removal of Land Victoria transactions and a lowered cost target
• added several benefits that the Reform Program business case did not outline. 

These additional benefits were outlined in the program close report and are listed in SV’s benefit realisation framework. 

SV reports its achievement of benefits to its BoM, the DPC Secretary and the Minister for Government Services. SV reported that it achieved its total annual benefit in 
2018–19 and 2019–20. However, this is based on the additional benefits SV introduced in 2018. While an internal audit found that SV had a reasonable basis for reporting benefits across categories, we found limitations to these benefits, as illustrated in Figure 3C.

FIGURE 3C: Savings claimed by SV for additional benefits (2018–19 and 2019–20) and the limitations to these benefits

Note: *These figures do not add up due to rounding
Source: VAGO.

Most of the additional benefits that SV states it has achieved have been generated from a small number of transactions. For example, between 2017 and 2020, $65 million of SV’s reported annual benefit was from avoided costs and re-use benefits for Solar Homes transactions alone. We outline these in Appendix G. 

SV is not a commercial enterprise, and most services delivered to date have been free or low cost to agencies. This provides agencies who are unable to afford a market solution with an alternative option to digitise its transactions. 

This is one of the benefits of a centralised digital service. However, it does not mean that the government is saving money by using SV, as neither DPC nor SV:

• measure the 'unavoided' costs of SV (the cost of agencies not using SV and instead developing their own systems)
• calculate the cost of agencies running their own parallel systems. 

The fragmentation and duplication of systems was one of the problems SV was intended to address. There is no evidence that SV has done this.

Total financial benefit reported by SV

DPC predicted that most of the financial benefits SV would deliver would be transactional in nature. As Figure 3D shows, SV has not achieved these transactional benefits. 

FIGURE 3D: SV’s reported annual benefit by financial year

Note: ^(a) Other benefits include savings from streamlining service centres, avoided agency costs, re-use benefits, intellectual property benefits, time returned to customer benefits and payment benefits. 
Note: *This figure does not add up due to rounding
Source: VAGO analysis of SV data.

The annual benefit is also far short of its potential benefits of between $120 to $150 million per year that DPC predicted SV could achieve by year five if SV captured all Victorian Government transactions.

3.4 Governance and performance reporting

SV did not submit a new business case or clearly re-baseline its costs and benefits when it became clear that it would not deliver the original scope of transactions identified in the Reform Program business case. Instead, SV reported its implementation through:

• Program Implementation Plans
• end of stage reports and program close reports provided to DPC's Secretary, the then Special Minister of State, and audit committees.

As outlined in Chapter 1, SV had a mix of program and project governance. It was also subject to oversight through DPC's ARMC and gateway reviews.

Audit committees and Project Control Boards

Audit committees play a key accountability role in an organisation’s governance framework. While SV's management team is ultimately responsible for its operations, the audit committee independently reviews and provides assurance on key areas such as governance and risk. 

As outlined in Appendix H, DPC’s ARMC has overseen SV's performance since 2015, and had a dedicated SV subcommittee until May 2017. 

In September 2017, a separate PCB was established to ensure that SV: 

• was on target to deliver its agreed outcomes and expected benefits
• had appropriate risk management plans, which were followed
• optimised its cost value and delivered the program in a timely manner.

Since 2018, SV has also had its own internal audit function, which provides assurance to SV, the DPC Secretary and the ARMC. 

The ARMC and PCB’s early meeting minutes show that these committees focused on establishing governance and ensuring that the program's technical component was delivered. It also noted that the original project timeframes had slipped and that these were replaced by new milestones in the Program Implementation Plans. 

SV did not begin realising benefits until its third year (2017–18) and even then, only with a small amount of transactions in a testing phase. However, it was clear that the benefits were at risk earlier in the program. 

In 2016, the PCB raised questions on benefit realisation and: 

• approved SV's proposal to provide a paper on how benefits would be re baselined
• requested further information to understand stakeholder reporting. 

The PCB did not adequately address SV's risk of achievement of benefits and particularly whether it remained economically viable with a changed transaction mix. It also did not recommend other options (such as further influence by DPC) to improve stakeholder engagement and onboarding. 

Reporting during the implementation phase

In 2016, the ARMC subcommittee and PCB recognised the need for SV to provide clear reporting on the benefits and challenges that were in focus. However, the PCB still had concerns about the level of detail reported to it and measurability of SV’s benefits in June 2017. The PCB also highlighted the need for re-baselining benefits and linking them back to the business case. 

SV completed a program close report when its implementation phase concluded in June 2018, which the PCB reviewed, and which was provided to DPC's Secretary and the Special Minister for State. This report did not provide a clear comparison of the benefits SV had delivered against the expected benefits outlined in the Reform Program business case. 

SV's current performance management framework

In 2018, SV became an administrative office and gained powers and responsibilities under the Service Victoria Act 2018. This means that SV's CEO reports to DPC's Secretary about the general conduct and management of SV, and to the Minister for Government Services for the exercise of its functions under the Service Victoria Act 2018. 

SV is not required to report publicly on its annual benefit, volume of services and transactions or cost to deliver transactions. Its only public reporting measure is customer satisfaction. This reduces transparency of SV’s performance.

SV’s benefits realisation plan sets out KPIs and measurement metrics for each of its identified benefits. SV monitors and reports on these benefits through a benefits realisation register. This is an Excel spreadsheet that tracks SV’s KPIs against its actual values for each financial year. SV uses the register to inform its BoM about significant deviations from planned targets and timelines. The register also supports decision making and risk management processes.

SV does not have an annual plan for its performance. Instead, it uses a scaled Agile planning framework. As Figure 3E shows, this framework incorporates six-monthly and quarterly plans with monthly, fortnightly and daily reviews. 

FIGURE 3E: SV’s performance management approach 

Note: ^(a) Kanban boards are an Agile project management tool used to show work status, optimise workflow and highlight issues. SV uses a virtual board called JIRA. 
Source: VAGO, based on SV’s reporting framework.

SV's Board of Management

As outlined in Chapter 1, SV's BoM is responsible for overseeing SV’s operations and functions. The Gate 6 review noted that SV's governance structure is fit for purpose and highly focused on delivering benefits. We saw evidence of regular reviews of the program benefits and opportunities to onboard new agencies. However, given the lack of mandate to use SV, its approach is focused on smaller agencies commencing new digital transactions rather than the high-volume and high-value transactions originally intended by DPC.  

3.5 Risk management by DPC and SV

DPC outlined the top 14 risks to the program in its Reform Program business case. 

In the Reform Program business case, DPC presented, discussed, and assessed 32 risks. While it outlined mitigating actions for the top 14 risks, it did not include a rigorous risk management plan. SV’s initial program implementation plan in 2015 introduced its risk management plan and risk register. 

Top risks listed in the Reform Program business case included a lack of buy-in from agencies and that government leaders would not mandate the program. It also included a risk mitigation strategy that the project would not continue without a mandated vision. This did not occur.

One of the key risks of this program was the complexity and knowledge needed to deliver a project of this nature, particularly given the use of Agile and Lean methodologies. Gateway reviews found that the project team lacked a detailed knowledge of these methodologies, particularly in the implementation stage.

SV embedded its Agile and iterative approach in its risk management approach. It outlined risks in the most detail for the phase that the project was in. It provided less detail about risks associated with its later stages. This reduced DPC and SV's ability to clearly predict future risks and articulate plans to mitigate them in a timely way. 

Figure 3F shows how SV’s core and significant risks changed as the program progressed.

The 2018 Program Assurance Review (Gate 5 equivalent review) found that SV had an appropriate and active risk management approach, including for its implementation phase. SV’s current risk management framework, DPC’s risk management framework and the Victorian Government Risk Management Framework require agencies to outline the steps they will take to identify, monitor and treat risks. However, SV's risk register had several gaps. 

SV's 2018 program close report suggested that only one strategic risk remained in October 2018—achieving the benefits targets. There were no stakeholder risks listed in SV’s enterprise risk summary despite this clearly being an ongoing risk to the SV.

DPC’s risk register also did not include a specific risk about SV until June 2019, which Figure 3G shows.

FIGURE 3G: DPC’s risk register—controls and treatment actions for SV risk

Note: ^(a) DPC added this treatment action to its register in 2020.
Source: VAGO, from DPC’s risk register.

Since SV's establishment, the Gateway reviews and SV’s risk management plans have identified its benefits realisation and stakeholder management as key risks. 

Benefits realisation

The Reform Program business case highlighted the risk that SV may not realise its expected benefits. SV included this in its risk management plans and implemented treatment plans to address it, such as:

• identifying alternative sources of benefits and a decision-making framework for agreeing to future transactions
• developing a benefits realisation plan and early warning measures to identify benefits that were not being realised.

However, despite early recommendations in its Gateway and Program Assurance reviews, SV did not make agreements with stakeholders to lock them in at an early stage. It also did not confirm its scope changes or outline how they would impact its benefits by amending the business case.

SV did not develop a revised business case to recalculate benefits given the altered project scope and advise government of the results. 

SV has taken a cautious approach to benefit realisation. Its rationale has been to prove its product prior to delivering at scale. This is designed to prevent typical ICT project failures where early high expectations are not met. This has meant that SV does not have a strong public profile. The documentation that agencies provide to customers concerning transaction options has limited SV branding. For example, VicRoads registration renewal notices do not list SV as a payment option. Feedback from stakeholder agencies supports the view that SV’s brand is ‛underdone’, with customers often regarding SV as an extension of the agency and not visiting SV as the first portal to find transactions. 

Gate 6 identified this lack of brand recognition within the community as a risk. Yet SV does not have a marketing or brand recognition strategy.

Stakeholder management

The volume of services that SV onboards and the number of transactions it delivers is crucial to it delivering its expected benefits. This requires agency commitment and buy-in, particularly in situations where it is not mandatory to use the service.

SV had high-level stakeholder support during its initial development. However, this did not translate to the scope or volume of transactions it onboarded.

In other jurisdictions, digital transactional services have developed at different paces and with different models. For example, Service NSW was announced in 2012 and leveraged existing technology and services, such as Transport for NSW and Births, Deaths and Marriages NSW. By its second year it was delivering 13.5 million transactions, but its development and overhaul of dedicated end-to-end transactions progressed over time.  

myGov is another example. While it is not mandatory for agencies to use myGov, it almost doubled its expected number of customers by its third year due to key large agencies such as Centrelink and the Australian Taxation Office coming on board. However, the Australian National Audit Office found that myGov’s effectiveness was hampered by government services not joining it. 

Gateway reviews between 2014 and 2018 identified concerns from stakeholders about SV’s benefits realisation, delivery model and achievability. While SV developed a stakeholder engagement plan in 2015, this was only a high-level strategic document. It did not outline how agencies would be signed up to SV or specific communication strategies. 

Initially, SV focused on engaging with VicRoads because its transactions were in Horizon 1's scope. It set up a VicRoads transition board, which allowed a dedicated forum of senior stakeholders to consider strategies for transitioning services to SV.

Despite this, VicRoads has not onboarded the scope or scale of transactions that the Reform Program business case predicted. As shown in Appendix E, SV did not revise the transactions that it would deliver until 2018. It has also not delivered all of these revised transactions. Issues that have impacted SV’s ability to onboard transactions included:

One of the Information Technology Strategy 2016–2020’s priorities is to re-use technology where possible. SV provides an opportunity to deliver on this, but it requires clear engagement and buy-in from agencies. 

After onboarding Solar Homes transactions in 2019, SV focused on onboarding new services in addition to high-volume legacy services.

Identifying potential transactions

SV has developed a process for identifying suitable transactions and liaising with the relevant agencies. 

SV uses an Excel spreadsheet to identify transactions that have a high value to both SV and the agency and are suitable for use on its platform. This spreadsheet lists:

• most Victorian Government transactions and the agency involved
• each transaction’s annual volume (where available)
• if the transaction includes identity verification or payment options
• whether the service needs physical or digital evidence of transaction (for example a WWCC card or fishing licence) 
• any key decisions or actions SV has made or requires.

SV then uses this list to develop a pipeline of opportunities that it prioritises and reviews every six months.

Once SV identifies a transaction or service, it follows its service improvement framework to transition from identifying opportunities to delivering transactions. This is a process where SV progressively obtains more information, such as the benefits, technical options, funding and timing before submitting a proposal to the agency, entering into a memorandum of understanding (MoU) and building the transaction. 

SV’s Excel spreadsheet contains 10 separate Victorian Government transactions that are high volume and involve identity verification (equating to almost 3.4 million transactions per year). Of these 10 services, SV has:

• delivered one (new WWCC applications)
• a further two on its delivery roadmap (Residential Tenancies Bond Authority and WorkSafe Victoria licensing applications)
• six that are awaiting a government decision (all VicRoads transactions)
• one that SV cannot deliver due to a decision by the agency (land tax payments and amendments). 

There are several other high-volume transactions, such as Fines Victoria, myki and local government transactions, that SV is not delivering. 

SV also relies on DPC to identify opportunities for delivering services. However, we saw limited evidence of DPC encouraging agencies to use SV prior to 2020, when it advocated for SV to have a role in the reform of the fines system. While it is an agency or department's decision to use SV, there is a benefit in DPC driving the government priority of reusing technology and services where possible. SV stated that it also:

• checks if departments’ contracts for ICT systems are about to expire (if they are, SV approaches them to discuss opportunities for onboarding their transactions)
• maintains contact with DTF to check which agencies are requesting investment in digital technologies and uses this to focus engagement discussions
• monitors government agencies’ organisational charts. Changes to executive or leadership structures can create new approaches for digital delivery. If staff change, then SV approaches the new staff to present its offering and revisits any opportunities to work with the agency. 

However, decisions to invest in SV should be based on evidence of value and be driven by the Victorian Government strategy. This requires clear senior executive buy in and sponsorship to ensure that Victoria moves away from the fragmented and duplicated transaction delivery model first identified in 2015. 

Memorandums of understanding 

Stakeholder feedback 

We spoke to four agencies that have services with SV and three that have either used an alternative service or not onboarded transactions with SV. Stakeholders advised us that their decision not to onboard with SV was influenced by various factors including the readiness of SV at the time, internal policy decisions and the availability of other suitable technology (such as myGov). 

Agencies that use SV gave the following feedback:

Stakeholders told us that many agencies want to test SV’s service before ramping up their transactions. They also said that they need to have confidence in SV’s ability to deliver a transaction before increasing the redirection rate. 

While SV sets up an engagement process and meets regularly with stakeholders when it onboards a service, it does not have a method for agencies to provide formal feedback, such as through an annual survey. This is a missed opportunity for SV to identify issues from a stakeholder’s perspective and track its performance. 

Performance reporting provided to agencies 

We reviewed a sample of reports that SV provides to stakeholder agencies. SV uses Google Analytics to give agencies information on SV's performance in relation to agency transactions. The reports outline the:

• number of customer visits to the transaction page
• number of transactions started and completed
• median completion time
• customer satisfaction score for each transaction.

SV's monthly performance reports to agencies do not include information about the KPIs outlined in five MoUs, such as incidents or its platform’s service availability. While meeting minutes suggest that SV and the agency discuss incidents, it should also form part of regular reporting to agencies. 

Solar Homes raised opportunities for SV to improve its reporting in August 2019 such as including actual performance against the service level agreements and severity one incidents. However, SV has not addressed this yet.

The broader stakeholder feedback that we obtained suggests that SV’s lack of structured project reporting during the development phase can make it difficult for agencies to ensure governance over new projects. One stakeholder opted to engage with an external contractor to bridge this project management gap. 

SV’s ability to market its services, onboard and retain new transactions and ensure faith in its delivery is central to its success and whether it is a viable service in the long term. 

Back to top

We have consulted with DPC and SV, and we considered their views when reaching our audit conclusions. As required by the Audit Act 1994, we gave a draft copy of this report, or relevant extracts, to those agencies and asked for their submissions and comments. 

Responsibility for the accuracy, fairness and balance of those comments rests solely with the agency head.

Response provided by the Secretary, DPC

Response provided by the Chief Executive Officer, SV

Back to top

Back to top

Our methods

As part of the audit we:

• inspected and reviewed relevant documents, including:
  • legislation, policies and guidance
  • business cases and gateway reviews
  • business and strategic plans and frameworks
  • internal audits
  • performance data and reporting
• reviewed and analysed SV’s performance data, including (but not limited to) its transaction volume, cost, and customer satisfaction 
• interviewed relevant DPC and SV staff 
• consulted with agency stakeholders who have used SV’s platform or have run their own digital initiatives.

We conducted our audit in accordance with the Audit Act 1994 and ASAE 3500 Performance Engagements. We complied with the independence and other relevant ethical requirements related to assurance engagements. We also provided a copy of the report to DTF. Unless otherwise indicated, any persons named in this report are not the subject of adverse comment or opinion.

Back to top

FIGURE D1: Horizon 1 expected outcomes

Source: VAGO analysis from SV data and Reform Program business case.

Back to top

SV did not deliver all original transactions outlined in the Reform Program business case. Figure E1 outlines the changes to the transactions over time and whether these have been delivered. 

FIGURE E1: SV's transactional delivery

Note: The potential volume includes digital and non-digital transactions and is based on 2014-15 data in the Reform Program business case
Source: VAGO.

Back to top

FIGURE F1: Summary of SV's performance against its KPIs

Source: VAGO. 

Back to top

Figure G1 shows SV’s reported avoided costs and re-use benefits by transaction type for 2018–19 and 2019–20.

Figure G2 examines SV's performance against its transaction time baselines.

Figure G3 shows SV’s reported performance for customer satisfaction against baselines and Figure G4 shows SV's completion rates on a per transaction basis. 

Back to top

As the government classified the Reform Program as HVHR, DTF (in consultation with DPC) arranged six Gateway reviews of SV. Figure H1 provides more information about these reviews. 

FIGURE H1: Gateway reviews of SV

Source: VAGO, from Gateway reviews of SV.

Back to top

FIGURE I1: Oversight committees during implementation (2015–18)

Note: ^(a) PCB’s first meeting occurred on 28 September 2016. However, it did not establish its terms of reference until 1 May 2017. 
Source: VAGO.

Back to top