Follow up of Collections Management in Cultural Agencies

Conclusions

Creative Victoria, the National Gallery of Victoria (NGV), Museum Victoria and Public Record Office Victoria (PROV) have demonstrated progress towards addressing the recommendations. However, the size and diversity of collections, and limited resources mean that the recommendations have not been fully completed. The nature of the collections means that addressing the recommendations will likely take considerable time.

Creative Victoria has been more purposeful in leading action to address systemic issues in facilitating both a long-term and interim storage solution. While it agreed to coordinate a plan to help agencies address the documentation backlogs caused by legacy data, Creative Victoria has not done this yet. Given the number of agencies and collections involved, addressing this recommendation will enable a more strategic approach.

The audit committees for these agencies are tracking recommendations, but are not assessing the impact of actions to address recommendations from external audits. As a result neither the agencies nor the audit committees can be assured that the effort put into addressing recommendations is effective.

Findings

Leadership and oversight of arts agencies

Creative Victoria has taken steps to advocate for the necessary resources to increase storage capacity, consistent with our 2012 recommendation. It has secured funding to address short-term storage issues and has continued to pursue long-term collection storage solutions.

Creative Victoria has taken steps to address the scope and rigour of performance benchmarking and measurement, but has not acted on the recommendation to coordinate a plan to address legacy data issues. This gap needs to be addressed.

Accessibility of collections

Museum Victoria, NGV and PROV have developed strategies to guide digitisation activity, and have established oversight arrangements to regularly monitor and report on progress. The extent to which agencies track and report the total investment of staff and other resources into digitisation activity, and the level of access to online collection material, varies.

Both Museum Victoria and NGV have undertaken targeted visitation analysis to better inform them about the impact of their exhibitions.

Collections management

Museum Victoria, NGV and PROV have modified processes to improve information about the condition of their collections.

In addition, these agencies are working towards improving collection stocktakes and deaccessioning, which should provide them with information about the care and management requirements of their collections, and also alleviate some storage constraints.

Conservators at work on John Herbert's Moses bringing down the Tables of the Law c.1872–78. Photo: Predrag Cancar.

Audit committee oversight

Agencies report to their audit committees, boards and council of trustees on progress in addressing accepted performance audit recommendations. However, audit committees are not performing their function of assessing the impact of actions to address external recommendations. As a consequence, neither the agencies nor the audit committees can be assured that the effort being put into addressing the issues underlying the recommendations is effective.

Audit method and cost

The audit was conducted under section 15 of the Audit Act 1994, and was performed in accordance with the Australian Auditing and Assurance Standards. Pursuant to section 20(3) of the Audit Act 1994, unless otherwise indicated any persons named in this report are not the subject of adverse comment or opinion.

Total cost of the audit was $119 000.

Submissions and comments received

We have professionally engaged with Creative Victoria, Museum Victoria, the National Gallery of Victoria and the Public Record Office Victoria throughout the course of the audit. In accordance with section 16(3) of the Audit Act 1994, we provided a copy of this report to those agencies and requested their submissions or comments.

We have considered those views in reaching our audit conclusions and have represented them to the extent relevant and warranted. Their full section 16(3) submissions and comments are included in Appendix A.

1.1 Introduction

Victoria's state collections have natural history, scientific, historical, artistic and cultural significance and economic value, and require effective management to make them available to current and future generations. As at March 2015, there were around 38.7 million individual collection items valued at over $5 billion.

In October 2012, VAGO tabled an audit of Collections Management in Cultural Agencies. We examined whether state collections were adequately managed, including whether:

• agencies holding key collections had adequate collection management policies, systems and practices and could demonstrate performance against relevant statutory obligations
• Creative Victoria—previously Arts Victoria—adequately supported the delivery of the relevant government priorities, and oversaw the agencies in relation to collections management.

The audit examined the collection management policies and practices of:

• Museum Victoria
• National Gallery of Victoria (NGV)
• Public Record Office Victoria (PROV).

As a result of machinery-of-government changes effective from 1 January 2015, Creative Victoria has been relocated to the Department of Economic Development, Jobs, Transport & Resources. PROV has remained at the Department of Premier and Cabinet.

The 2012 audit made four recommendations covering accessibility of collections, collections management and Arts Victoria's leadership role in addressing systemic issues with the management of state collections.

This follow-up audit examined the progress that Creative Victoria and the three agencies have made in implementing actions to address the recommendations from the 2012 audit.

Figure 1A provides an overview of whether the agencies have begun to address the recommendations from the 2012 audit.

Figure 1A

Status of recommendations

Source: Victorian Auditor-General's Office.

1.2 Leadership and oversight of arts agencies

The 2012 audit found that Creative Victoria needed to be more purposeful in overseeing and supporting agencies to facilitate their delivery of government priorities. It should be driving solutions to long-standing systemic collection management challenges.

Figure 1B

Recommendation 1

1.2.1 Advocacy for collection storage

In 2012, VAGO found that the agencies' collections storage facilities were at, or near, capacity. A site was purchased in December 2005 that could be used for a purpose‑built integrated facility as a long-term storage solution. While a business case for an integrated storage facility was developed, there was no commitment from the government.

The audit recommended that Creative Victoria should more purposefully lead action to address systemic issues with the management of the state collections by more rigorously advocating for the necessary resources to increase collection storage capacity.

In 2015, collections storage remains an issue. Data provided by Creative Victoria indicates that as at March 2015, NGV's storage capacity was fully utilised and was at 95 per cent or over at Museum Victoria and PROV.

However, Creative Victoria has taken steps to address this recommendation.

In 2012–13, funding of $15.2 million was secured to address short-term storage issues—including overcrowding and inadequately stored collections. Around $14 million of these funds was distributed to Museum Victoria and NGV, and used to implement the collections' interim storage project, due to be completed in June 2016. The agencies and Creative Victoria advise that while this project is providing some relief to storage issues, it is not a long-term solution.

Collection of stuffed animals in storage at the Museum Victoria collection facility. Photo courtesy of Museum Victoria / Photographer: John Broomfield.

Creative Victoria has continued to pursue collection storage solutions and it is a current priority in its Agencies and Infrastructure Branch Business Plan 2014–15. However, it has not developed a clear strategy to guide implementation of its priorities.

Creative Victoria is leading the Collection Storage Victoria project. This includes:

• facilitating collaboration between agencies to develop a revised business case in 2015
• establishing collection storage as a Creative Victoria priority project
• briefing the new Minister for Creative Industries
• dedicating an internal working group to progress the project.

Without funding the project will not be realised, which may compromise the future management, care and preservation of collections.

1.2.2 Collaboration

In 2012, VAGO found that Creative Victoria had facilitated some collaboration between agencies, but had limited success in producing tangible results. We also found that government priority areas for 2006–07 and 2007–08 had not been implemented. These priority areas included storage, digitisation programs, access, and the measurement and comparison of the cost of managing and storing collections.

The audit recommended that Creative Victoria should more purposefully lead action to address systemic issues with the management of the state collections by more assertively facilitating collaboration between its portfolio agencies.

Creative Victoria has taken action to address this recommendation. For example, it has:

• continued to be involved in the Arts Agencies Collections Working Group (AACWG)—which facilitates information sharing across agencies—and contributed funding towards a collection risk and management workshop, organised by the AACWG, and another workshop focusing on digitisation that is being planned for the second half of 2015
• established a common repository for key collections data that is accessible to all agencies and expanded the type of data collected from agencies to include digitisation status, plans and policies developed, and more definitive cost categories to separate staff costs and storage collections management costs
• continued to be involved in an arts agencies chief information officers' forum, which identifies potential for collaboration of information and communication technology—a review of minutes indicates that in December 2012 this forum was asked to advise the Department of Premier and Cabinet's efficiency review on collaboration and sharing efficiencies.

Creative Victoria is also chairing a national digital technologies working group established under the National Arts and Culture Accord. The working group focuses on aligning digitisation activities in public collections to facilitate searching and access to collections across institutions. This has involved a national survey of galleries, libraries, archives and museums to identify opportunities for national government‑owned collecting institutions to open data and align digital search capability.

1.2.3 Performance measurement

In 2012, VAGO found that Creative Victoria had not acted with sufficient purpose to address longstanding inconsistencies in measuring performance and collection management costs across the agencies.

In addition, Creative Victoria's publicly reported performance information on agency collections storage meeting industry standard was not based on robust evidence. It was introduced without a rigorous, documented methodology and agencies based this reported figure on professional judgment.

VAGO recommended Creative Victoria should more purposefully lead action to address systemic issues with the management of the state collections by improving the rigour and scope of performance measurement and benchmarking.

In doing so, we expected performance measurement to include reporting on the:

• proportion of website visitors accessing collection-related content
• number of incidents of significant damage or deterioration of collection items
• proportion of collection with recorded condition information
• number of collection items subjected to conservation treatment
• collection management costs as a percentage of agency operating budget, as opposed to reporting collection management and storage costs.

Creative Victoria has taken steps to address this recommendation. It has:

• developed a new Budget Paper 3 qualitative measure for collection storage based on a significantly more rigorous risk-based assessment of individual storage facilities
• established a common repository of key collections data accessible to all agencies, enabling a live snapshot of agency performance
• made adjustments to the types of information it collects, including developing a consistent cost methodology for storage collection costs
• continued to produce an overview of the State Collections Report, which compares agencies across similar categories—this report is not developed annually, however, Creative Victoria intends to do this biennially from 2015.

There is scope to further improve the quality of key performance indicators in the service level agreements to provide greater insights into agency collection management performance. An assessment of the agency service level agreements for 2013–16 found no change to key performance indicators. Creative Victoria should address this.

1.2.4 Coordinate a plan to address legacy data

In 2012, VAGO found that commitments to develop common solutions to address data backlogs had not been implemented. Further, there was no evidence agencies had shared approaches, models and resources for tackling collection legacy data issues.

VAGO recommended Creative Victoria should more purposefully lead action to address systemic issues with the management of the state collections by actively coordinating a plan to address legacy data issues.

Creative Victoria has not acted on this recommendation and has advised that agencies face very different challenges in relation to legacy data and other digitisation issues. As a result, strategies to address these are unique to each agency and are driven by analysis of risk and resources required. This is consistent with its response to the Responses to 2012–13 Performance Audit Recommendations self‑attestation report, even though Creative Victoria indicated the response is complete.

Given the complexity of the issue, Creative Victoria should still develop a plan and overarching guidance to enable a strategic approach to the issues across multiple agencies and to support them in addressing this.

1.3 Accessibility of collections

1.3.1 Digitisation strategy development and access to online collection material

In 2012, VAGO noted that extending access to state collections through digital media was a key government priority area for the arts portfolio between 2006–07 and 2011–12. However, we found that a low proportion of collections were ever on public display. While agencies had made progress in increasing this form of access to their collections, digitisation activity had not been guided by adequate strategies.

Figure 1C

Recommendation 2

Agencies have made clear progress in actioning this recommendation by developing strategies to guide digitisation activity, having oversight arrangements in place to regularly monitor and report on progress of digitisation activity, and tracking the level of collections accessible online.

However, the extent to which agencies track and report the total investment of staff and other resources into digitisation activity, and the level of access to online collection material varies.

NGV approved its digitisation project strategy in July 2014. This strategy guides digitisation activities and sets time frames for completion for specific collection areas. Progress is monitored through weekly reporting to senior management and monthly reporting to executive management. As at June 2015, NGV reported that 29 per cent of its collection is available online in fully digitised form.

As the project is funded by private benefactors, time frames for completion are determined based on available funding for a particular area. Cost estimates in the project plan identify a funding shortfall that may impact project delivery if additional funds are not secured.

PROV has had a digitisation policy in place since 2011 and has developed its digitisation strategy for 2014–17. The strategy identifies oversight arrangements and criteria for prioritising digitisation of records. Digitisation projects are funded either through collaborations with community organisations or internally.

Digitisation activity is tracked in terms of hours spent on digitisation projects and access to online material. As at June 2015, PROV had 0.81 per cent of its available shelf metreage digitised and accessible online.

Museum Victoria has not yet finalised its overarching digitisation strategy or plan, but has identified digitisation as a priority in its strategic and business plan. Museum Victoria advised that both documents are expected to be formally endorsed in the second half of 2015. Digitisation activity has still progressed—with 31 per cent of the collection available online in digitised form as at June 2015.

While the previous audit recommended that agencies should track resources required for digitisation activity, Museum Victoria does not do this because digitisation activities form part of collection staff roles. We consider this approach adequate.

1.3.2 Blockbuster exhibitions

In 2012, VAGO found that Museum Victoria and NGV analysed and reported on audience numbers and financial outcomes associated with blockbuster exhibitions, but undertook limited analysis of their impact on the numbers of visitors to the state collections.

Figure 1D

Recommendation 3

Both NGV and Museum Victoria have taken action to address this recommendation. The agencies have undertaken specific audience research to better understand motivations and attractors for visitation to support both blockbuster and permanent exhibition planning.

NGV has a dedicated visitation analysis program in order to understand the motivation and behaviour of its visitors. Part of this involves surveying its audience as they exit NGV. It has also modified its visitor survey to include questions about what other parts of the gallery they plan to, or have, visited that day. Information collected from these surveys is analysed and a report is produced. NGV also collects attendance data that shows an increase in visitation to the permanent collection on exhibition days compared to non-exhibition days for 2013 and 2014.

Similarly, in 2014 Museum Victoria commissioned research to refresh its understanding of its audience and to support future planning. It also surveys general museum visitors biannually and surveys each touring hall exhibition. This allows Museum Victoria to collect visitation data—including frequency, length and purpose of visit. Museum Victoria also collects attendance data for both general museum and exhibition admissions. The information collected suggests that in 2013 and 2014 there had been an increase in visitation to the permanent exhibitions.

1.4 Collections management

Collections management includes activities from acquisition, recording and cataloguing, storage, preservation, display and access, through to disposal. In 2012, VAGO found that collections were well managed at the individual item level, but there were systemic challenges that needed to be addressed.

Figure 1E

Recommendation 4

1.4.1 Condition information

In 2012, VAGO found that the extent to which the agencies had a documented policy and strategy driving conservation activity varied, as did the resources assigned to these activities. Conservation was largely driven by the need to access items in each collection and was often associated with preparing items for exhibition, loan or other use.

Condition surveys to identify risks and issues requiring attention were done on a limited basis. The lack of comprehensive condition information on the entire collection created risks that there were unknown conservation issues.

VAGO recommended that NGV, Museum Victoria and PROV gather comprehensive condition information to better inform the allocation of conservation resources.

The agencies have all acted to address this recommendation to varying extents. All agencies face unique challenges in relation to the size, diversity and resources available to enable this to occur in a timely way for their entire collections.

Museum Victoria has a Collection Risk Assessment and Management (CRAM) initiative in place. The initiative started in September 2011 and is a preventative conservation program designed to determine the condition of the collection against 10 agents of deterioration.

CRAM serves to better inform the allocation of conservation and other collection care resources, and manage the quality of the collection. It is overseen by a project team and results are reported to executive management and the Finance, audit and risk committee. Museum Victoria also has an annual program of condition surveys of parts of the collection.

At the start of this follow-up audit, NGV had not yet developed a plan to guide the capture of condition information for the entire collection in the collection management system. The need for such a plan is highlighted by resourcing constraints and the size and differing conservation needs of the collection. During the conduct of this audit, NGV documented its approach to capturing the ongoing recording of condition information of items.

Conservation of Yorta Yorta possum skin cloak with Elders: Conservation museum staff: Bunjilaka, Melbourne Museum, 2013. Photo courtesy of Museum Victoria / Photographer: John Patten.

NGV has reviewed and modified its procedures to assess and document the condition when acquiring new items. It records condition information in its collection management system—including conservation treatment and staff hours. NGV advises that it also plans to record this information for items requested for loan.

There is no change to the methodology for recording the condition information for the remainder of the collection. This is based on collection use and movement.

PROV has developed a policy to guide its conservation activity. In 2014 a targeted survey of the PROV collection identified opportunities to collect more detailed information about the condition of items, and an action plan was developed based on this. However, PROV advised that due to the small budget for conservation, activity is largely driven by records that are in most demand and at risk.

PROV tracks the total number of units in its collection with current condition information. However, a detailed condition report is only prepared for exhibition items.

1.4.2 Stocktake

Systematically sighting and verifying collection items as part of a regular stocktake is an important process in the overall care, management, custody and stewardship of a collection.

Our 2012 audit found that stocktakes were not standard practice across the agencies. NGV was the only agency with a stocktake policy. Museum Victoria had a pilot collection auditing program that equated to a partial collection stocktake. PROV had no stocktake policy, but had implemented projects to improve storage arrangements and enable them to test the accuracy of information in the collection management system on the location of the objects.

VAGO recommended that agencies implement active collection stocktake programs. All agencies are working toward implementing this recommendation.

NGV revised its collection stocktake policy in February 2013 and then completed a stocktake that started in 2005. This identified there was considerable work required to reconcile findings and address legacy data issues. To address this, a stocktake methodology to support the revised policy was developed and approved in February 2014.

The methodology uses a risk-based approach to reconciling findings and resolving location tracking issues. The project is expected to be completed in December 2018.

Following the completion of its pilot audit program in 2012, Museum Victoria implemented an ongoing five-year stocktake program from September 2012. This is supported by a collection audit plan and a procedure for collection auditing and managing missing collection items.

The audit plan identifies 19 discreet audits over five years—including one whole‑of‑collection audit and varying audit types such as sampling, full, location and random audits. Audit results are reported to the executive management team and the Finance, audit and risk committee.

PROV started an ongoing stocktake program in November 2012. Based on the size of its collection and available resourcing, PROV has estimated an entire stocktake will not be completed until 2021. It has advised that rather than a targeted stocktake program, it has integrated stocktaking activity into collection management activities each time collection items are accessed.

A stocktake briefing paper details the methodology and process to be followed. It is not a full stocktake as it occurs on a row by row basis and, while each item on the shelf is checked, the units within each item are not.

Photo courtesy of PROV.

1.4.3 Deaccessioning

Deaccessioning is the formal removal of an item from a state collection. A deaccessioning review program is important given storage space pressures, costs and the need to ensure the collections remain relevant. In 2012 we found that agencies could not demonstrate they were consistently reviewing collection material for deaccessioning. This created further pressure on collection storage.

VAGO recommended that agencies purposefully review collections for deaccessioning opportunities. Since 2012 all agencies have programs to actively review collections for deaccessioning opportunities.

While NGV's Deaccession and Disposal Policy is not scheduled for completion until December 2015, it has established a process of regular monitoring and review by senior management to progress possible deaccessioning opportunities identified since 2010.

All items are now tracked along the various stages of the process through regular meetings with responsible staff from each curatorial department. This has resulted in NGV having 414 collection items in various stages of the deaccessioning process, with a further 201 works awaiting formal approval.

Museum Victoria has an active deaccessioning program guided by a policy that sets out applicable criteria for deaccessioning. It is working towards a strategic approach to deaccessioning with all collection plans—including a section to identify acquisitions and deaccessioning opportunities where relevant. Deaccessioning also occurs on a needs or reactive basis, where an item presents an occupational health and safety issue or the costs of managing the item outweigh the benefits of keeping it. These items are identified as part of the CRAM initiative and condition assessments. Between 2012–13 and 2014–15 Museum Victoria deaccessioned 1 250 items, 60 lots and 6 625 coins.

Various modes of historical transport and printing presses, Museum Victoria collection facility. Photo courtesy of Museum Victoria / Photographer: John Broomfield.

Museum Victoria also has an active repatriation program whereby it is committed to returning ancestral remains, associated burial goods and sacred objects to traditional/rightful owners. Between 2012–13 and 2014–15, Museum Victoria has repatriated 132 ancestral remains, two grave goods and one burial container.

PROV completed a disposal opportunities project in 2013–14. Its final report contained a survey of its Victorian archives centre collection, made up of over six linear kilometres of records. As a result, it disposed of 600 linear metres. PROV has also established a working group that meets every two months to monitor the progress of analysing, prioritising and progressing further disposal activities. Assessment of deaccessioning opportunities is now a regular component of collection management activities. In 2014–15 it disposed of over 446 linear metres.

1.4.4 Policies and procedures

Comprehensive organisational policies establish a clear framework, expectations and accountabilities for sound management of legislative and other responsibilities. In 2012, VAGO found none of the agencies had a comprehensive framework of policies, as there were common gaps in the documented collection management procedures of each agency including storage, conservation and deaccessioning procedures.

Collection management policies should cover:

• collection development through acquisition and other methods
• documentation to an appropriate standard so that the collection can be properly managed and made accessible
• storage to an appropriate standard
• conservation and access
• planned response to emergency/disaster events
• deaccessioning and disposal of items considered unsuitable for retention.

VAGO recommended the agencies should give greater priority to addressing the gaps in the policies and procedures guiding collection management activities.

Since 2012, all agencies have made progress toward addressing the gaps in their policies and procedures. While not all policies have been developed, agencies have plans in place to address this.

NGV established a Collection Management Policy Working Group in February 2013. Its purpose is to review existing policies and conduct a mapping and benchmarking exercise against peer institutions to identify priority areas for policy development and review.

NGV has reviewed the existing Collections and Loans Policy and developed an Acquisition and Accession Policy in August 2014. It expects to complete the Deaccession and Disposal Policy by the end of 2015. It is developing or reviewing other policy areas progressively according to an established and monitored time frame.

Museum Victoria revised key collection policies in 2013 and established four new collection policies, including a new Collection Care and Preservation Policy, to address the gaps in collection management activities. Its digitisation strategy and plan is scheduled to be endorsed and finalised in the second half of 2015.

Guided by its collection management strategy 2013–15, PROV identified five areas—storage, use, documentation, access and preservation—where internal policies required development or review. PROV has demonstrated that it has a framework in place to guide its collection management practices.

However, it is not clear if the policies are final and approved. PROV has advised that its policies are working documents and are reviewed as part of business planning. Version control is not always reflected as a result of this, and incorporating this process would enhance transparency.

1.5 Audit committee oversight

Agency audit committees are required to monitor audit recommendations and enact planned responses through Standing Direction 2.6(f) under the Financial Management Act 1994. This includes requirements for audit committees to:

• recommend how the responsible body should act on advice received from external audit
• monitor actions taken by management to resolve issues raised by external audit
• monitor whether accepted recommendations of the external auditors are adopted and addressed by management on a timely basis
• review the impact of actions taken by management intended to resolve issues.

To enable audit committees to perform these functions, agencies should have systems and processes to support them to monitor progress on implementation of accepted recommendations and to assess their impact.

Creative Victoria, Museum Victoria and NGV report to their audit committees, boards and council of trustees on progress in addressing accepted performance audit recommendations. PROV's reporting arrangements post machinery-of-government changes were clarified in May 2015. PROV now reports to the Department of Premier and Cabinet's audit and risk committee.

While audit committees are monitoring the status and progress with implementing recommendations, they are not performing their function of assessing the impact of actions to address external recommendations. This is in part because of the lack of supporting frameworks and also because of a lack of awareness of the Standing Direction 2.6(f) requirement to review the impact of actions intended to resolve issues.

As a consequence, neither the agencies nor the audit committees can be assured that the effort being put into addressing the issues underlying the recommendations are effective. This needs to be addressed.