Crime Data

Crime statistics are one measure the community uses to judge our relative safety and security. They show the rate of recorded crime across local government areas or regions, and can highlight trends such as increases or decreases in types of crimes and how and where they are committed. The government uses crime statistics as an indicator of the effectiveness of criminal justice policies, and Victoria Police uses crime statistics to help determine its resourcing needs.

Given the importance of crime statistics in managing public safety and the high level of public attention and scrutiny they receive, it is important that they accurately reflect reported crime. Past reviews have revealed the misleading use of crime statistics and some cases that were recorded as resolved inappropriately.

Victoria Police collects, reviews and records information on crimes—and the victims and alleged offenders involved—in its Law Enforcement Assistance Program (LEAP) database. The Crime Statistics Agency (CSA), part of the Department of Justice and Regulation, converts data extracted from LEAP into statistics which it publishes on its website quarterly.

In this audit, we examined whether crime data is reliable and effectively used for decision-making. We focused on the completeness, accuracy and timeliness of Victoria Police data and the methodology used by CSA to analyse and publish crime statistics. We made nine recommendations for Victoria Police.

Back to top

Ordered to be published

VICTORIAN GOVERNMENT PRINTER September 2018

PP No 436, Session 2014–18

Back to top

ABS	Australian Bureau of Statistics
ANZSOC	Australian and New Zealand Standard Offence Classification
BP3	Victorian Budget Paper 3: Service Delivery
CompStat	Compare Statistics
CSA	Crime Statistics Agency
CSU	Corporate Statistics Unit
DJR	Department of Justice and Regulation
DQU	Data Quality Unit
ePDR	Electronic Patrol Duty Return
ERP	estimated resident population
ESTA	Emergency Services Telecommunications Authority
IT	information technology
ITGC	information technology general control
LEAP	Law Enforcement Assistance Program
LEDR Mk2	LEAP Electronic Data Recorder Mark 2
MDT	mobile data terminal
MNI	Master Name Index
NCRS	National Crime Recording Standard
NOI	National Offence Index
OLR	Online Reporting
OPI	Office of Police Integrity
PAL	Police Assistance Line
PSA	police service area
RSD	Records Services Division
SOCIT	Sexual Offences and Child Abuse Investigation Team
VAGO	Victorian Auditor-General's Office
VPM	Victoria Police Manual
WIR	Workplace Inspection Report

Back to top

Crime statistics are one measure the community uses to judge our relative safety and security. They show the rate of recorded crime across local government areas or regions and can highlight trends such as increases or decreases in types of crimes, and how and where they are committed.

The government and the community also use crime statistics as one of several indicators to assess the effectiveness of criminal justice policies. Victoria Police uses crime statistics to help determine its resourcing needs.

Member—any sworn member of Victoria Police.

Given the importance of crime statistics in managing public safety and the high level of public attention and scrutiny they receive, it is important that they accurately reflect reported crime. The public also needs to be able to access crime statistics and understand what the results represent. Crime statistics that are reliable and robust are a useful tool for making evidence-based decisions, for developing justice policies, and for informing resourcing of police and other emergency services.

Offence—any criminal act or omission by a person or organisation for which a penalty could be imposed by the Victorian legal system.

Victoria Police is responsible for collecting, reviewing and recording information on criminal offences. Members use the Law Enforcement Assistance Program (LEAP) database to record reported crimes as part of an incident—a distinct event that can be made up of one or many sub-incidents or offences occurring at the same time and location. An incident can include multiple victims and alleged offenders. Members are required to record complete information about the incidents reported to them, including details of all offences, victims and offenders involved.

Victoria Police provides raw crime data to the Crime Statistics Agency (CSA). CSA is part of the Department of Justice and Regulation (DJR) and is responsible for publishing Victoria's crime statistics.

In this audit, we assessed whether crime data is reliable and effectively used for decision-making. We focused on the completeness, accuracy and timeliness of Victoria Police data and the methodology used by CSA to analyse and publish crime statistics.

Conclusion

Victoria Police accurately records the number of offences that have originated from divisional van reports in LEAP . Its members by and large correctly report on jobs dispatched from the Triple Zero hotline, and the information in LEAP is a reasonably materially complete record of reported crime.

Victoria Police's measures included in the Victorian Budget Paper No. 3: Service Delivery (BP3 ) accurately reflect the level of crime recorded in LEAP and Victoria Police's progress in addressing it. We did not detect any manipulation of crime data or cases falsely recorded as resolved.

However, risks to reliable reporting remain:

• Victoria Police has done little to improve its members' understanding of prima facie reporting since 2013 and does not know if members comply fully with the Victorian Crime Recording Standards.
• Members often fail to record sufficient detail in their divisional van reports, and van technology makes it difficult to track job outcomes, so there is a risk of important detail being lost between an offence being detected and recorded in LEAP.
• Victoria Police has no clear targets for timely recording of incidents and has data quality issues from incorrectly recorded date fields in LEAP records. This means Victoria Police cannot measure how quickly its members create reports after they are aware of incidents. Without a benchmark, there is little incentive for members to report in a reasonable time frame, so members may be unable to access crime information when they need it.
• Victoria Police can improve the usefulness of data available for internal purposes and decision-makers if members consistently record specific location information and the country of birth of victims.
• Supervising sergeants can play a stronger role in enforcing Victorian Crime Recording Standards.

The CSA methodology for transforming Victoria Police's raw crime data into published crime statistics is transparent and produces a reliable set of statistics for the benefit of policymakers and the wider community. The methodology is also publicly available, so the public can understand how to interpret the statistics.

Findings

Recording crime data

Prima facie reporting

Victoria Police can improve the consistency and reliability of prima facie reporting by coaching junior members and more closely monitoring reports before approving them.

The Victorian Crime Recording Standards are based on the principles of the National Crime Recording Standard (NCRS ), which Victoria Police adopted in 2009. The principles include the requirement to report a crime if, on the face of it, it appears that a crime has occurred. This is known as prima facie reporting, and its introduction was a change in approach for Victoria Police. Previously, members were accustomed to investigating an incident before deciding whether to record it in LEAP.

Prima facie crime recording—the requirement to report a crime if, on the face of it, it appears that a crime has occurred, as distinct from investigating further to determine if it occurred before reporting.

The Australian Bureau of Statistics (ABS) does not include reporting on assaults in Victoria in its national dataset because it states that Victoria Police does not comply with prima facie reporting. This makes Victorian data inconsistent with that of most other Australian jurisdictions. A 2013 Victoria Police internal review, 'Assessment of the Collection Recording and Reporting of Crime Data', found that 61 per cent of members would not report a minor assault, which contravenes the Victorian Crime Recording Standards. Victoria Police has not reviewed the progress of prima facie reporting since 2013.

Prima facie reporting should result in more information about offending, such as minor assaults, being captured on LEAP. If members do not consistently comply with prima facie reporting, it is unlikely that this is occurring. Recording family violence incidents has additional requirements—members must record criminal matters such as assault but also verbal disputes and other psychological and controlling behaviours that are considered civil matters. This additional reporting provides members with more comprehensive information about this type of offending and can be used to better assess risk.

Four of the 14 constables we interviewed during this audit spoke of investigating an incident further before reporting it, even if, on the face of it, a crime had occurred. This approach does not support the intent of prima facie reporting, which is to ensure that crime data in LEAP is complete and that this valuable information is quickly accessible without the need for further investigation.

Completeness of reporting

In the sample we examined, almost all reported crimes received by members in divisional vans and captured in Electronic Patrol Duty Returns (ePDR) from 1 July to 30 September 2017 were satisfactorily recorded in LEAP (96.1 per cent) or had valid reasons for not being recorded. Our analysis of unexplained omissions detected no systematic under-reporting of crime.

Electronic Patrol Duty Returns—capture information from callers to Triple Zero operators about crime, generated from information transmitted to mobile data terminals in divisional vans. They include essential information about the reported crime and the member's actions in response, as well as speed checks, breath tests and licence plate checks.

Completeness of demographic data

Collecting demographic data allows police and policymakers to identify emerging trends such as whether groups of people are increasingly becoming victims of crime. It is not mandatory for members to ask a person's gender or nationality, but they are expected to record as much relevant detail as possible. Asking about Indigenous status is mandatory, but it is possible to leave the field blank or unknown in reports.

We looked at how completely and consistently members record demographic information and found that information on offenders was more complete than information on victims. Country-of-birth data was complete in 27.9 per cent of victim records and was consistently recorded across different LEAP tables and reports 25.8 per cent of the time compared with offender records, which were 92.7 per cent complete and 78.9 per cent consistent.

Victoria Police has made great improvements in gender data and has reduced the number of gender fields marked 'unknown' in offender data from 339 in December 2017 to 37 in March 2018. Data about Indigenous status was missing for only 4.2 per cent of offender records but was missing for 41.2 per cent of victim records. Victoria Police indicated that asking these questions can be awkward for members—but without this information it cannot fully understand who is most affected by crime.

Quality assurance of LEAP data

Victoria Police's central Records Services Division (RSD) conducts checks of LEAP data guided by its data quality framework. The framework guides how RSD's data operators review LEAP reports for high-risk errors such as:

• incorrectly selecting or updating name information
• incorrectly entering member or station details
• failing to add most recent address or contact details
• incorrectly adding offender details or not adding an offender
• not linking charges to the correct sub-incident (the specific offence within the crime event).

Other checks involve running data analysis queries across the entire LEAP dataset to identify missing and incomplete data and other issues that prevent LEAP meeting the Victorian Crime Recording Standards.

This checking approach works well considering the complexity of LEAP and the manual processes it requires.

Accuracy of recording

Some high-volume crimes, such as theft, do not have a high clearance rate and members continue to get reminders to keep investigating as long as it remains unsolved. In such circumstances, there is a risk that members will artificially clear cases to improve the clearance rate—the time it takes to clear or solve crimes. Changing an open case (which would be categorised as 'unsolved') to a status like 'complaint withdrawn' changes it from unsolved to cleared crime.

We looked at the outcome of a selection of cases with the status 'complaint withdrawn'. We found that the narrative description did not always clearly indicate why the complaint was withdrawn, but we found no evidence of manipulation in the selection of cases we reviewed.

Another risk is that a serious offence, such as aggravated burglary, is not recorded accurately and downgraded to the less serious offence of theft. Such inaccuracies could mislead the community about crimes and provide a false picture of police success in addressing crime.

We assessed whether there had been shifts from more serious offences to less serious ones from 2016 to 2017. The shifts we observed involved small numbers, and we found no patterns in the data that would indicate intentional downgrading.

To test whether common errors might exist in LEAP, we looked at the expected sequence of dates. For example, a correct sequence might show a home burglary that occurred on 30 April being reported to police on 1 May, with police creating a LEAP record on 2 May and resolving the crime on 1 June. If the sequence of dates showed the crime resolved on 1 April but reported on 1 May, it would indicate an error. In checking for such errors, we found that for intervention orders it was standard practice for members to record the date the order was 'resolved' as occurring before the offence was committed. LEAP is not set up for intervention orders, so members regard the 'committed' date as the date the intervention order was in place. Our analysis found errors in less than 1 per cent of the data we examined, which means these inaccuracies will not materially impact published statistics. Victoria Police is reviewing and amending this data. Date errors are discussed further under timeliness of recording.

Members can improve the accuracy of location information in crime data. Members with responsibility for analysis and intelligence gathering reported to us that location information is often poor—addresses can be entered with a street name but no street number, or one location can be reported in multiple ways. This means that police analysis may be incomplete or inaccurate.

To identify quality issues caused by members, RSD runs data queries across LEAP to identify missing and incomplete data and other quality issues. Where possible, RSD then develops fixes—an example of this is identifying and improving the completeness of LEAP's gender data. RSD does not oversee or provide data quality advice to members. Supervising sergeants are responsible for checking members' reports for errors before they are approved in LEAP.

Timeliness of recording

Victoria Police's timeliness standards are not specific, stating that 'in the majority of instances, the details of a crime should be submitted for approval by the end of the shift on which the reporting member was informed of the crime'.

Most members complete reporting within a reasonable time frame, but Victoria Police can enforce tighter time frames to reduce the amount of reports completed more than two days after members become aware of an incident. This will allow members to have confidence that they are fulfilling their duties based on the most complete and accurate information.

Of the 48 jobs we examined, 40 were entered into LEAP within 24 hours (83.3 per cent), and this increased to 93.8 per cent within a 48-hour period. All jobs were reported within 3.5 days.

The two date fields used to measure the timeliness of crime reporting are: the 'report date' field—the date when police first knew of the incident the 'create date' field—the date when the report was first created in LEAP.

However, the result from our timeliness test sample—that members recorded 93.8 per cent of jobs in LEAP within 48 hours during selected days in August 2017—is inconsistent with the average time between all 'report date' and 'create date' information entered in the system between January and September 2017, which is much higher (eight days). We found 557 744 sub‑incidents (16 per cent) with a gap greater than five days between the 'report date' and 'create date'.

Several issues affect the overall average:

• RSD data operators inadvertently save over the correct 'report date' when adding charges to existing sub-incidents.
• Members may record the date the alleged offence took place as the date reported to police, when it is more likely that there is a delay between the alleged offence occurring and its being reported to police.
• Offences such as the manufacturing of drugs, incest and manslaughter can involve information that needs to be restricted from LEAP users, and members may be unsure when it is appropriate to report the offences in LEAP.
• The average gap is also affected by some offences with disproportionately long gaps between the 'report date' and 'create date'.

Over the five years to 2017, the number of sub-incidents with a gap of over 100 days between 'report date' and 'create date' reduced by 15 per cent, from 9 402 in 2012 to 7 991 in September 2017.

So, while there is some improvement, Victoria Police members still make errors in recording dates, despite clearly defined standards. If this confusion is not corrected, the cleared crime could be reported in the wrong quarter. The number of errors will not have a material impact on CSA's quarterly published data, or on trends over time, but they should be rectified to ensure crime data is as accurate as possible. Victoria Police advises that it has started to address the incorrect LEAP reports.

The time taken to clear a crime is one of Victoria Police's BP3 measures, and its performance against this measure is published annually in the State Budget. Victoria Police uses CSA statistics to report on all crime performance measures except this one, which it generates using internal data. Victoria Police also now uses the 'report date' to calculate its performance results, which is likely to add additional time to the 30-day clearance rate reported in BP3 and provides the public with results that are 'worse' than if measured from the 'create date'. This demonstrates that Victoria Police has implemented the Victorian Ombudsman's recommendation to measure performance using the 'report date' field; but it now needs to improve the quality of the 'report date' data so that this result is more reliable. Working with CSA to use its statistics on crime clearance will help Victoria Police to avoid any perception of bias, as it is CSA's role to provide verified figures.

System and control effectiveness

Process noncompliance and control deficiencies can place critical business functions at risk. Victoria Police can improve its processes to ensure the reliability and integrity of its key systems and underlying data.

Quality control over data input

The quality of data input is paramount for accurate crime data. This means that reports need to be scrutinised before they are approved.

Supervising sergeants use the approval process to ensure members adhere to the Victorian Crime Recording Standards and include the correct information. We spoke with 18 members who felt this process needed to be more thorough to improve the quality of location information, identifying details and the description of the investigation.

Local area commanders for police service areas (PSA) also complete monthly workplace inspections to check compliance with a range of requirements, including investigation and reporting. A 2016 Victoria Police internal review found that workplace inspections—which include LEAP—are rudimentary, with assessments scored as either satisfactory or not satisfactory. This does not provide local area commanders with sufficient insight into reporting issues, and there is no evidence that this process has improved the quality of reporting. Victoria Police needs to make the methodology for the assessment more transparent so that reporting is meaningful and properly tracks performance. Victoria Police has delayed addressing the recommendations of the internal review until June 2019.

LEAP interoperability

LEAP has operated since 1993. LEAP remains the repository for crime data, but members now enter data using a modern interface known as LEAP Electronic Data Recorder Mark 2 (LEDR Mk2). This interface allows members to enter incidents themselves, rather than sending forms to RSD to complete. This promotes more accurate and timely reporting. RSD data operators still enter data on offenders into LEAP from faxed forms.

Other reporting and case management systems—such as the mobile data terminals (MDT) used in vans, and Interpose , Victoria Police's case management system—are not integrated with LEAP. Interpose contains restricted information. As a result, crime reporting is onerous and repetitive, with members retyping the same information in multiple systems, which increases the risk of error.

The MDTs in divisional vans can only capture limited information, and members do not always capture enough information to document events. Further, the format of ePDR reports makes it difficult for supervisors to track the sequence and outcome of events.

The lack of interoperability between crime recording systems continues to be a major source of frustration for members and adds to their time spent on administration. It limits the ability of members to easily add in offences containing sensitive information, such as drug crime, incest and manslaughter. This impacts the quality and hence usefulness of crime data.

Information technology general controls

We assessed the key information technology general controls (ITGC) that underpin the LEAP and LEDR Mk2 systems. They are operational but we identified a number of opportunities to increase their robustness. These include:

• reviewing and updating all LEAP and LEDR Mk2 policies and procedures on an annual basis to ensure processes and controls are implemented consistently
• expanding and performing regular system security reviews to ensure security risks are identified and responded to
• performing regular user access reviews to make sure access is appropriate and anomalies are addressed
• complying with Victoria Police's change management administration review process to provide assurance that documentation processes comply with the requirements of the change management process
• performing regular information risk assessments as per Victoria Police's Risk Management Framework requirement to ensure information assets are appropriately protected.

Published crime statistics

After extracting LEAP data, CSA completes a series of 'transformation' steps before publishing crime statistics. This involves reasonableness checks, such as seeing if increases or decreases in crime are in line with expectations. CSA also organises individual offence types into meaningful groupings, such as assigning 'theft of a motor vehicle' to the 'theft' category. CSA ensures the information is statistically significant and poses no risk of identifying people. It applies counting rules to derive specific populations. We found that the transformation processes are transparent and appropriate.

CSA applies another counting rule to Indigenous status data. It uses the 'most frequent' rule to improve reported data—for example, where a person has described his or her Indigenous status differently on separate occasions, CSA uses the most frequent response to assign the offender a status in its reports. CSA’s method may reduce the impact of any data entry error.

CSA categorises offences into groups that largely reflect the Australian and New Zealand Standard Offence Classification (ANZSOC), developed by ABS. CSA has modified ANZSOC where required to match Victorian criminal law. This is an appropriate adaptation to make the classifications relevant to Victoria. The location classifications are also useful, as they allow searches by police zone, suburb and local government area.

CSA does not audit, nor does it have any powers to direct improvement of, the quality of LEAP data. CSA has access to the descriptions of the investigation proceedings for LEAP incidents, known as narratives, which may help to identify missing or obviously incorrect data, but, as yet, Victoria Police and CSA have not developed guidelines for the use of this information.

CSA reports seven data tables on crime. We focused on two:

• The Recorded Offences table, which provides quarterly updates on the number of offences committed in Victoria and includes offences where no offender is identified or charged. The crime rate is often derived from this data.
• The Criminal Incidents table, introduced in December 2017, which is a count of all the criminal events rather than the individual offences and designed to more closely represent crime as it is experienced in the community.

The Criminal Incidents dataset only includes the offence type that is determined as the principal offence. If there are several offences, CSA selects the most serious as the principal offence for reporting. We assessed how CSA selected the principal offence by comparing its classification system to the National Offence Index (NOI). We found that CSA's methodology is sound and can be used to fairly reflect the principal offence in a crime.

We used LEAP data and CSA's methodology to replicate the CSA crime statistics for the Recorded Offences and Criminal Incidents tables and determine their reliability. We noted no material variance for the data from 2008 to 2017. This means that CSA's crime statistics are an accurate reflection of Victoria Police's crime data.

Using data for decision-making

Senior members of Victoria Police regularly meet to discuss crime data and participate in briefings on local crime clearance rates, shifts in offences, and comparative data. In addition, these members can use data analysis tools to perform a range of common data queries on the Victoria Police intranet at any time. This makes it easy for members to quickly access changes in offending that need to be discussed.

Recommendations

We recommend that Victoria Police:

1. conduct a training needs analysis to determine members' understanding of prima facie reporting and address any gaps in targeted training (see Section 2.2)

2. establish a centralised monitoring and reporting mechanism to ensure management reviews and updates policies and procedures in a timely manner, including:

• for crimes where a victim can be identified, mandating that members ask if the victim is comfortable disclosing their country of birth and, if so, that this data be recorded in the Law Enforcement Assistance Program (LEAP)
• requiring supervisors to routinely check Electronic Patrol Duty Returns and other crime and investigative reporting in line with Rule 2 of the Victorian Crime Recording Standards before they approve the reports
• setting a benchmark for timely input of data into LEAP and reporting against this annually
• reviewing and updating the policy and procedural documents for LEAP and the LEAP Electronic Data Recorder Mark 2 (LEDR Mk2) systems in accordance with the Information Management Policy (see Sections 2.3, 3.2, 3.6, 3.7 and 3.9)

3. improve the accuracy and timeliness of recording in LEAP, and monitor and reduce errors in LEAP that cause long gaps between the date the offence is reported and the date it is created on LEAP (see Section 3.9)

4. develop a data dictionary that defines the requirements for LEAP and LEDR Mk2 fields (see Section 3.3)

5. adopt a consistent approach to using performance measures computed by the Crime Statistics Agency in Victorian Budget Paper 3: Service Delivery reporting and document the methodology (see Section 3.9)

6. set a benchmark for what constitutes satisfactory LEAP and LEDR Mk2 reporting in workplace inspection reporting (see Section 3.3)

7. strengthen the information technology general controls for LEAP and LEDR Mk2 by:

• removing inactive user accounts
• ensuring compliance with Victoria Police's change management administration review process for documentation requirements (see Section 3.5)

8. perform regular information risk assessments as per Victoria Police's Risk Management Framework requirements and, based on the results, redesign and implement security controls to remediate the risks identified (see Section 3.6)

9. implement an appropriate compliance management framework and processes for monitoring, measuring, evaluating and reporting on the performance of crime data reporting and information technology general controls (see Sections 3.4, 3.5 and 3.6).

Responses to recommendations

We have consulted with DJR, Victoria Police, CSA and the Emergency Services Telecommunications Authority (ESTA), and we considered their views when reaching our audit conclusions. As required by section 16(3) of the Audit Act 1994, we gave a draft copy of this report to those agencies and asked for their submissions or comments. We also provided a copy of the report to the Department of Premier and Cabinet.

The following is a summary of those responses. The full responses are included in Appendix A.

Victoria Police acknowledged and accepted the report’s findings. It also accepted all of the recommendations and advised that it will develop an action plan within three months to detail how it will address them. We will follow this up with Victoria Police.

DJR and CSA thanked VAGO for the report’s valuable insights and remain committed to providing transparent and reliable crime statistics. ESTA did not respond.

Back to top

1.1 Crime statistics in Victoria

Accurate crime statistics assure the community that police are accountable and acting with integrity. Constables, sergeants and supervising sergeants spend a significant proportion of their time recording information about reported crimes. Police units rely on this data to understand patterns, determine resourcing and focus their efforts to prevent harm to the community.

Between 2012 and 2017, there were 3.5 million records relating to reported crimes created in Victoria Police's LEAP system. Quality control is paramount in providing reliable data, so centrally based operators in RSD check the quality of crime data entered in LEAP from the 54 PSAs. ITGCs can also reduce accidental or deliberate tampering with or release of crime data.

Crime recording standards

National Crime Recording Standard

ABS developed the NCRS to address the inconsistencies in crime reporting across Australia's various jurisdictions and make them more comparable. Victoria Police adopted the NCRS in 2009 and, by 2010, all states agreed to incorporate the NCRS into their internal policies.

Rule 2 of the NCRS requires that members record incidents on a prima facie basis. This means that if 'on the face of it' and on the balance of probabilities it is more probable than not that a crime has occurred and there is no credible evidence to the contrary, a member should record the incident at his or her earliest convenience.

This rule provides a common basis for recording an incident with one or more offences according to the judgment of the member (as distinct from evidentiary or prosecutorial reasons).

Victorian Crime Recording Standards

The Victorian Crime Recording Standards enshrine the NCRS in Victoria Police policy. They promote a consistent approach for recording crime and non-crime incidents. The objective of the standards is to ensure that records are accurate and timely, and to enable their effective use for operational, analytical and statistical purposes.

There are 10 recording rules and five recording requirements. Regarding the timely recording of data, Rule 3 states, 'Each incident and any associated status changes need to be recorded in the crime recording system as soon as they are known'. Regarding the complete recording of data, Rule 8 states, 'All distinct offences must be recorded including the date of occurrence'.

The Victorian Crime Recording Standards also define how Victoria Police should complete prima facie reporting. Prima facie reporting, if completed properly, should provide complete and reliable information about alleged perpetrators and victims. The standards state:

Before a crime is recorded corporately the reporting police member must:

• ensure an offence at law is being reported
• determine the facts of the crime.

The decision to record a crime is based on the information and evidence available at the time the crime is reported. In the majority of cases, the belief by the victim or reporting person that a crime occurred is sufficient justification to record the crime incident.

Members are required to record crimes and other incidents in LEAP. Crime statistics, which are published on the CSA website, are based on crime data recorded by Victoria Police in LEAP.

Victoria Police's organisational structure

Victoria Police is divided into four regions—Eastern, North West Metro, Southern Metro and Western. Regions are made up of divisions—there are 21 divisions overall, with between four and six per region. The 21 divisions are further divided into PSAs—there are 54 overall, with two to four per division.

Police stations report to a PSA within their division, and the assistant commissioner responsible for the region oversees the results. The assistant commissioners report to three deputy commissioners, who oversee three out of the five areas of Victoria Police executive command.

Relevant legislation and the Victorian Police Manual

Victoria Police operates under the Victoria Police Act 2013 (the Act). Part 13 of the Act deals with the confidentiality of police information, including access to and use and disclosure of information on a computerised database.

The Victoria Police Manual (VPM) guides members on how to conduct their duties. The VPM is updated regularly and includes policies and procedures for reporting and recording crime in LEAP. It explains:

• how and when to amend a LEAP record
• how to report on motor vehicle thefts
• when to seek assistance from RSD.

It does not include a prescriptive guide to completing reports in LEAP.

The introduction of the Family Violence Protection Act 2008 and its amendments in 2017 resulted in changes in the way that Victoria Police reports family violence incidents. The legislative changes have led to members recording more information about incidents that involve family violence and has increased focus on ensuring complete and accurate reporting. Members must capture more details about the parties involved and conduct structured risk assessments to determine the appropriate level of intervention.

The number of family violence incidents reported, including those that involve crimes, has increased as a result, and this is reflected in the CSA's published statistics. Family violence reports now form a distinct part of Victoria Police's reporting against BP3 measures.

1.2 Information management

Capturing crime data

Victoria Police captures crime data in the course of its work. The public may report potential crime in multiple ways, including calls to Triple Zero or attending a police station, or police may detect crime while on patrol. Members are required to report crimes in accordance with the VPM and the Victorian Crime Recording Standards.

Members record details such as offenders, witnesses, victims, locations where crimes occurred, and incident dates and times. RSD is responsible for processing offenders and adding in supplementary reports on thefts of motor vehicles, thefts from motor vehicles (particularly licence plates) and missing persons.

Having a centralised team within RSD to promptly report certain events makes information accessible to members in LEAP in a more timely manner. RSD data operators input the record into LEAP as soon as they receive instruction via telephone from a member to do so. Otherwise, the record would only be entered into LEAP once the reporting member had an opportunity to return to the station and complete the online forms.

Figure 1A outlines the main responsibilities for crime reporting.

Figure 1A
Crime recording and reporting responsibilities

Agency/division	Responsibility
Operational members—ranked mainly constable and senior constable	Gather data and input it into LEAP via LEDR Mk2 or hardcopy forms based on the requirements of the Victorian Crime Recording Standards Faxes crime reports to RSD
Operational members—ranked sergeant and above	Review LEDR Mk2 for breadth, quality, correctness Request rework and approve Review forms for transmission to RSD
RSD	Records some crimes and all offenders in LEAP based on the requirements of the Victorian Crime Recording Standards Performs quality assurance on recorded crime
Data Quality Unit (DQU), within RSD	Performs data quality and cleansing functions across LEAP based on risks and priorities
Corporate Statistics Unit (CSU)	Collects, collates, analyses and disseminates statistical data collected by Victoria Police
Capability Department—regional audit teams	Ensures LEAP audits are conducted on crime‑recording processes
CSA	Publishes crime statistics on a quarterly and annual basis

Source: VAGO based on Victorian Crime Recording Standards.

Members use a free-text field in LEAP, known as the 'narrative', to record details of incidents, including information about the scene of the crime, the people involved and the events leading up to police involvement. Members transcribe into LEAP information from the notes they take when an incident is reported. As investigations progress, members use the 'narrative' field to record updates as more information becomes known. The crime record also has an assigned status, which is updated as the investigation of the incident progresses.

After the initial recording of a crime or incident in LEAP, most members can only change the 'narrative' field and not fields that capture information such as dates and people involved in the incident. Supervisory sergeants can change the status of incidents in LEAP—from 'active' to 'pending' or 'completed' . Members have to fax requests to RSD to update other information such as adding in bail conditions or new sub-incidents and linking details of offenders to incident reports once they are identified.

Key systems—LEAP and LEDR

Victoria Police started using LEAP to record details of crime and incidents in 1993. Victoria Police first attempted to update the mainframe technology in 2006 when limitations in LEAP's capability became apparent—for example, the inability for members to directly enter data from their own desktop computers. However, the update project was cancelled five years later, costing $45 million, when it was determined it would cost $100 million more than originally expected. Some of the key challenges for upgrading LEAP included its many interfaces and integration with more than 25 existing information technology (IT) applications.

Since 2006, Victoria Police has been gradually introducing direct-entry capabilities for LEAP and upgrading the supporting infrastructure to keep the system operating while it determines its future plans for incident recording. In its 2015–16 annual report, Victoria Police indicated that these updates have given LEAP up to seven years' extra life, extending its use for 12 years, until 2027–28.

LEDR is one of the systems introduced to improve LEAP's capability. Members can use LEDR—a front-end web-based portal that interfaces with LEAP—to enter information and submit certain reports into LEAP. LEDR also introduced additional input controls on some data fields, more than were present in LEAP alone.

LEDR Mk2, the latest version rolled out in 2015, enables members to create initial reports for all incident types. However, some key limitations remain—we discuss these in Part 2.

Figure 1B outlines the current collection processes for crime data.

Figure 1B
Crime data collection processes

Source: VAGO based on information provided by Victoria Police.

Reporting on performance

BP3 performance measures

Victoria Police has performance measures relating to crime data in BP3. These measure rates of different types of crime, success in prosecution of crimes and the timeliness of crime recording.

The External Reporting team at Victoria Police manages the process for determining BP3 performance outcomes, which are published in Victoria Police annual reports. CSA assists in this process by providing Victoria Police with the counts of crime by category, but it is up to Victoria Police to determine the final performance outcomes for inclusion in BP3 and annual reports. Figure 1C shows Victoria Police performance measures related to crime data for each offence division. Appendix B provides more information on BP3 performance measures.

Figure 1C
BP3 performance measures related to crime data

Performance measure	Unit of measure	2017–18 target
Quantity
Crimes against property—excluding crime related to family violence (rate per 100 000 population)	number	<5 100
Crimes against property—crime related to family violence (rate per 100 000 population)	number	>150
Crimes against the person—excluding crime related to family violence (rate per 100 000 population)	number	<660
Crimes against the person—crime related to family violence (rate per 100 000 population)	number	>590
Timeliness
Proportion of crimes against the person resolved within 30 days	per cent	>42
Proportion of property crime resolved within 30 days	per cent	>22

Source: VAGO based on Victorian Government Budget Papers, 2017–18.

Victoria Police management reporting

CSU analyses large volumes of data, including data relating to crime, for internal management monitoring and reporting. CSU also manages the process for extracting crime data from LEAP to provide to CSA. CSA uses this data for its quarterly analysis and publication of crime statistics to the community.

Quality assurance

The quality assurance framework for crime data includes three levels of review:

• Sergeants are responsible for performing quality checks of LEDR Mk2 and LEAP reports to ensure that the stated information is accurate and complete before approving them.
• DQU within RSD performs data quality and cleansing work.
• RSD's data entry staff also conduct some data quality checks and error fixing, particularly in relation to logic errors and contradictions between selected data fields within the records.

Regional audit teams conduct audits at stations within their regions to ensure they are complying with VPM requirements.

1.3 Agency roles

Department of Justice and Regulation

DJR uses crime statistics to develop advice for government. It also uses them for research and developing strategies and policies for policing and the broader justice sector.

DJR is responsible for submitting Budget bids for funding on behalf of Victoria Police and for monitoring its performance against BP3 output measures.

Crime Statistics Agency

The Victorian Government established CSA under the Crime Statistics Act 2014. CSA sits within DJR and is responsible for processing, analysing and publishing Victorian crime statistics.

CSA's strategic objectives are to:

• improve the accessibility of crime statistics for all Victorians
• strengthen the integrity and quality of recorded crime data and instil public confidence in crime statistics
• build an evidence base to support decision-making and policy development
• provide tools that improve the statistical literacy of stakeholders and clients, including data users and commentators.

CSA receives monthly extracts of LEAP data from Victoria Police. At the end of each quarter, CSA performs basic reasonableness tests on the data to identify any obvious logic issues regarding the quantity and quality of the extract. CSA then applies various counting and categorisation rules to generate the different sets of crime statistics that it publishes on its website.

Victoria Police members record each offence that forms part of an incident in LEAP reports with crime codes from relevant legislation. Some offence types are more serious than others and attract harsher penalties.

Using the LEAP data extract, CSA reports on crime statistics in multiple tables, including:

• Recorded Offences table—this table captures all offences recorded by Victoria Police within a reporting period. As there can be multiple offences in one incident, the recorded offence population is higher than the incident-based population.
• Criminal Incident table—CSA applies classification rules to the recorded offence population to determine the principal offence, discussed further in Part 4. CSA first started reporting incidents in December 2017.

CSA was not granted audit powers in its enabling legislation, so it has limited ability to test the accuracy and reliability of LEAP data provided by Victoria Police.

Emergency Services Telecommunication Authority

ESTA manages all Triple Zero calls for emergency assistance and dispatches emergency services including Victoria Police. Victoria Police calls represent 55 per cent of all calls to Triple Zero.

ESTA uses computer-aided dispatch to transmit callout details to members in police vehicles through MDTs. If the callouts relate to crime or family violence matters, members are required to enter the appropriate reports into LEAP as soon as practicable.

Jobs dispatched from ESTA are captured in ePDR forms, which list details from the caller, including the date, time and address, if available. They can also record LEAP enquiries conducted by the member, such as licence plate checks. MDTs located in police cars allow members to enter notes about how they have responded to jobs.

1.4 Enhancing capability

The BlueConnect program

Victoria Police's $500 million BlueConnect program includes various projects that are intended to improve police IT capabilities across various areas. Figure 1D summarises projects in the BlueConnect program.

Figure 1D
Projects in Victoria Police's BlueConnect program

Project	Intended delivery and benefits	Time line
Intelligence management solution	A single-point search tool to allow members to access and analyse key information from multiple sources. This will improve intelligence capabilities and inform decision‑making.	Implementation—gradual rollout from 2017 to 2019
Mobile technology	Equipping members with devices in the field to give them timely access to relevant information. Eventual functionality to generate ePDRs and crime and event reports.	Phased deployment—at least 8 500 devices to be deployed by end of 2019
Police Assistance Line (PAL) and Online Reporting (OLR)	PAL and OLR will divert reporting for non-urgent matters away from frontline members, freeing them up for urgent matters. It will create two more avenues for the public to access police in addition to police stations and Triple Zero.	PAL and OLR channels will be fully operational by 2020
Body-worn cameras	Body-worn cameras will allow members to accurately capture evidence to support prosecution of offenders and assist victims.	Initial trial and rollout commenced in April 2018 and is ongoing

Source: VAGO based on information provided by Victoria Police.

Many of the projects that make up the BlueConnect program are intended to improve, or have implications for, the quality of LEAP data. Victoria Police is considering case management solutions to replace LEAP and provide a new system to capture crime records and other events.

1.5 Previous reviews on crime reporting

Previous reviews have identified risks in Victoria Police's incident recording practices and crime recording manuals, which have the potential to affect the integrity of crime statistics.

The Victorian Ombudsman has raised concerns about publicly reported crime data on multiple occasions. In 2009, the Victorian Ombudsman referred findings of under-reporting of assaults and lower-level offences to the former Office of Police Integrity (OPI). This under-reporting was based on discrepancies between the crimes reported to police and the crimes recorded in LEAP.

At the time, the Victorian Ombudsman linked these findings to issues with administrative systems and processes rather than deliberate manipulation, but also recommended the establishment of an independent agency with responsibility for publicly reporting crime statistics. The Victorian Ombudsman recommended that this body have the power to audit Victoria Police's collection of crime data and the practices behind it. New South Wales has an independent crime statistics agency with an audit function, and similar bodies exist in South Australia and Western Australia.

In 2011, the Victorian Ombudsman received and acted on a complaint about alleged manipulation of crime data for political gain. The Victorian Ombudsman found that the former police chief commissioner's claim that there was a 27.5 per cent reduction in assaults in Melbourne was 'misleading and inconsistent with other data' and reiterated the need for crime statistics to be managed and published independently from Victoria Police.

In May 2011, OPI found evidence of police clearing cases inappropriately. OPI reported that this resulted from 'system deficiencies, lack of clear policies, instructions and inadequate monitoring—some cases involved tacit approval by middle management'.

1.6 Why this audit is important

Crime statistics are a measure of safety and security that the public and decision-makers use to understand how safe their community is and to hold government to account. For these reasons, it is important that crime statistics accurately reflect criminal incidents and reported criminal activity actually occurring across the state. If crime data is inaccurate, there is a risk that resourcing and policy decisions concerning policing may not be appropriately informed. Crime statistics also need to be transparent and accessible to the community.

Previous reviews identified risks in Victoria Police incident recording practices, which have the potential to affect the integrity of crime statistics. In this audit, we sought to clarify if and how Victoria Police has mitigated these risks.

Victoria Police and DJR use crime data to inform resourcing decisions, research and criminal justice policies. This means that the data should be reliable and robust to ensure evidence-based decision-making.

1.7 What this audit examined and how

Our objective was to determine whether crime data in Victoria is reliable. In light of changes to requirements for reporting on family violence, we considered family violence incidents in our analysis and looked for evidence of how these changes influenced the quality of data collected.

Our approach included:

• matching reported crime from police divisional van ePDRs to crimes recorded in LEAP—we tested 380 records from Triple Zero calls sent to vans from 1 July to 30 September 2017 to determine whether they had been recorded in LEAP
• examining the ITGC environment that Victoria Police has to ensure the integrity of crime data
• assessing processes for the collection of crime data
• reviewing the methods used by CSA to generate publicly reported crime statistics and repeating these methods ourselves
• reviewing Victoria Police quality control processes to ensure complete and accurate recording of crime data
• considering Victoria Police crime reporting manuals and guidelines.

During the audit, we interviewed members of different ranks and operational staff in various roles throughout Victoria Police. We visited five police stations across the four police regions and spoke to approximately 47 operational members. The majority of these members were general duties members, responsible for attending the station and patrolling the division. At some stations, we were able to speak to station commanders and members of specialist units such as Sexual Offences and Child Abuse Investigation Teams (SOCIT), Family Violence Units, Investigation and Response, Tasking and Coordination, Divisional Intelligence Units and Crime Investigation Units. We also spoke to administrative staff within stations. These interviews gave us insight into how Victoria Police policies and procedures are used in practice.

At Victoria Police headquarters, we spoke to staff from CSU, Special Intelligence, RSD, and Tasking and Coordination. Many head office staff members were also sworn members who had diverse experience and expertise. We also spoke to regional audit teams.

We conducted our audit in accordance with section 15 of the Audit Act 1994 and ASAE 3500 Performance Engagements. We complied with the independence and other relevant ethical requirements related to assurance engagements. The cost of this audit was $960 000.

1.8 Report structure

The structure of the remainder of this report is as follows:

• Part 2 examines Victoria Police's collection and recording of crime data
• Part 3 focuses on data quality and controls
• Part 4 examines the process for analysing and releasing crime data.

Back to top

Reporting on crime and other events occupies a substantial amount of police time—depending on the circumstances, they may spend time observing, investigating and recording details of offences. Police members use their judgment to determine whether they need to record an incident reported to them and use a range of paper-based and electronic means to capture the details.

This part of the report assesses whether Victoria Police collects and records crime data according to the required standards.

2.1 Conclusion

Our test of jobs dispatched by ESTA to members in divisional vans showed that 96.1 per cent were recorded in LEAP. This means that members are appropriately recording offences that originate from Triple Zero reports and that LEAP contains a reasonably complete record of these jobs. However, the technology used to record jobs in divisional vans is not linked to LEAP and members do not always record enough information to determine the outcomes of their investigations. Therefore, LEAP may not contain all information that was available to members, and details that may be important for future investigations may be lost.

Victoria Police does not know if members are complying with prima facie reporting in accordance with its own policy. We spoke to four members who gave examples of not complying with this requirement. If members do not apply the prima facie rule, they may fail to record all potential offences in LEAP, which means it will not completely and accurately reflect all crime detected by members. There is little evidence that Victoria Police has emphasised the importance of this style of reporting for general crime data to the same degree as it has for reporting family violence incidents.

Technical limitations in crime recording systems continue to frustrate members who need to re-enter the same information repeatedly. This increases the risk of data being entered inconsistently, makes quality review challenging and affects how quickly members can access incident details.

2.2 Capturing crime information

Prima facie reporting

Prima facie reporting provides members with more complete and reliable information about alleged perpetrators and victims.

ABS measures reported and unreported victims of crime, using police data and household data from Australian jurisdictions. In the explanatory notes of its report Recorded Crime—Victims, Australia (2016), ABS states that recording of assaults by Victoria Police does not meet the standard for prima facie reporting:

For incidents of assault, Victoria differs from other jurisdictions in the interpretation and implementation of the NCRS. Some element of an investigation will be undertaken before deciding whether to record an incident on their crime recording system. A record of the incident may be taken on other systems however the incident will not be recorded on the police recording system until it is determined that a crime has been committed. As a result of the comparability issues arising from this difference in interpretation and implementation of the NCRS a decision has been made (by the ABS) not to make available assault data for Victoria.

Assault—the intentional application of force by a person to the body or clothing of another person which is without lawful excuse and results in the infliction of bodily injury, pain, discomfort, damage, insult or deprivation of liberty. —Adapted from Victoria's Crimes Act 1958

A review commissioned by Victoria Police in 2013 found that there was a disconnect between the policy requirement to record crime prima facie and members' interpretation. When members were asked if they would record a minor assault in LEAP:

• 61 per cent of respondents said they would not record the assault
• 75 per cent of respondents said they would not record the assault if the parties had no visible injuries.

Following the review, Victoria Police implemented modified, less comprehensive versions of the recommendations, including issuing a policy statement endorsing prima facie reporting. There has been no further review of prima facie reporting since 2013.

Our discussions with members during this audit make clear that more work is required to embed the policy:

• Four of the 14 constable-rank members we interviewed cited situations where they or their colleagues waited until they had completed further investigations before deciding whether to report an incident in LEAP.
• One member stated that they like to make sure it is really a crime before reporting it and that a supervising sergeant will hold off on signing an ePDR until further investigation is completed.
• A member of SOCIT spoke of receiving cases from members without any LEAP records, which conflicts with the intent of prima facie reporting. In addition to not complying with the Victorian Crime Recording Standards, this approach creates a risk that important details may never be recorded or that they may be forgotten by the time the incident is recorded, due to the time elapsed.

Victoria Police does not have evidence that the prima facie model of reporting is embedded in police practice, despite being official policy. It has limited understanding of how its members comply with the Victorian Crime Recording Standards. This means it does not know if the data includes the complete number of offences that should be reported.

Victoria Police needs to do further work to understand how members are reporting crime and to address any gaps. We have recommended that Victoria Police survey members on their reporting style to track their compliance with the Victorian Crime Recording Standards.

Reporting on family violence

Victoria Police released its Code of Practice for the Investigation of Family Violence in 2004. An update in 2014 significantly changed the way police report on family violence, and in 2017 further information was added. The 2016 Royal Commission into Family Violence also placed additional scrutiny on police responses to family violence.

Family violence incidents include non-criminal family incidents, such as verbal abuse, controlling and intimidating behaviour, and other behaviour or actions related to family violence that are typically considered civil matters—not handled by police—rather than criminal. These family violence interactions are recorded as incidents and referred to as 'event reports'. If a crime is committed as part of the event, it is recorded as a sub-incident within the event report. All members we interviewed during this audit stated that members comply with the requirements recording family violence incidents and record all family violence incidents regardless of whether an offence had occurred.

Incident—Victoria Police defines an incident as a related group of distinct course of conducts. One incident can include several sub-incidents. Sub-incident—Victoria Police defines a sub‑incident as a single distinct course of conduct or occurrence. It can be a specific crime or event that forms part of an incident. Usually a sub‑incident records the principal offence within an incident.

Members completed extensive training to transition to these new reporting requirements. To mandate reporting of additional information, Victoria Police also updated the data processes and electronic data entry interfaces that capture the new information. This included adding fields to allow for detailed risk assessment of the likelihood of repeated incidents and danger to affected family members.

Victoria Police's commitment to implementing the recommendations from the Royal Commission to reduce delays in serving intervention orders on perpetrators of family violence, warrants and charges means that timely recording and serving is a high priority. Reports on family violence—in particular, Family Violence Safety Notices, which are notices of intervention served on perpetrators—must be entered into LEAP by the end of the rostered shift. For non-family violence incidents, members sometimes do not create the record until a day or two after the incident, despite the same expectation for timeliness. Timeliness of reporting is discussed in more detail in Part 3.

Victorian Crime Recording Standards

Completeness

The following sections from the Victorian Crime Recording Standards stipulate when a crime needs to be recorded:

A crime MUST be recorded corporately as soon as the reporting member is satisfied that, on the balance of probability, a crime has occurred. For crimes where an offender is not readily identified, the initial recording of the crime MUST NOT be delayed to gather further evidence.

After making an assessment that a crime has occurred the reporting member MUST take the necessary steps to ensure the crime is recorded corporately in a timely manner even if the victim/s do not wish to take the matter/s further or refuses to co-operate with police.

Similarly, if the reporting member determines a crime has taken place but no victim can immediately be identified, the crime MUST be recorded corporately.

We assess completeness in Sections 2.3 and 3.7. Figure 2A shows the information that must be recorded in crime reports.

Figure 2A
Data that members need to capture in crime reports

Data field	Definition	Mandatory policy requirement
Report date	Time and date police became aware of crime (not the date the report is compiled)	Yes
Reporting member and station	Police member responsible for managing initial report of crime	Yes
Offence code	Most serious offence detected	Yes
Statute	The number of offenders associated with the crime	Yes
Person/property	The number of occurrences of the same offence, at the same location (same victim)	Yes
Time and date	Times and dates the victim believes the crime occurred	Yes
Location	Location of where offence was alleged to have occurred—the minimum requirement is a suburb or town	Yes
Record status	Current status of investigation—for example, unsolved, intent to summons, offender charged, no offence disclosed, complaint withdrawn	Yes
Victim	Victim's full name and date of birth, contact address and phone numbers and response to Standard Indigenous Question as provided to police	Yes
Other names	Full name, date of birth, contact address and phone numbers for other persons associated with the crime	No
Motor vehicle	Motor vehicle associated with the crime—for example, stolen vehicle or victim's vehicle	Yes, if theft of motor vehicle or theft from motor vehicle
Property (including bicycle or drugs)	Stolen property items or seized drugs associated with the crime	No
Modus operandi location	The type of location where the crime allegedly occurred	Yes
Modus operandi	Method offenders used to commit crime	No
Investigating member	Member responsible for the ongoing management of the investigation	Yes
Narrative	Describes the circumstances in which the crime was committed and the ongoing investigation	Yes

Source: VAGO based on information provided by Victoria Police.

Accuracy

The checking supervisor conducts a general review of the information as required in the Victorian Crime Recording Standards, including:

• the report time and date is when the police were informed of the crime
• the crime incident 'from' and 'to' dates are before or equal to the report time and date
• full location of the crime is recorded, for example, street number and street name
• correct offence and modus operandi is recorded.

However, no expectations or key performance indicators are defined for accuracy. We discuss accuracy further in Part 3.

Timeliness

The Victorian Crime Recording Standards describe when crimes should be recorded in LEAP. In the majority of instances, members should submit the details of a crime for approval by the end of the shift on which they were informed of the crime.

The timeliness indicator is vague and does not include a maximum expectation for reporting, such as within 48 or 72 hours. It does not differentiate between serious and less harmful crimes. This makes measuring the timeliness of reporting difficult. We discuss this further in Part 3.

2.3 Are all reported offences being recorded?

Reporting suspected crime

Victoria Police believes that the number of calls to Triple Zero is a measure of community perception of crime and safety rather than actual crime. Our analysis of calls to Triple Zero showed that in addition to investigating suspicious activity, police are highly involved in welfare checks, assisting people with mental health issues, and lower-priority issues such as noise complaints. Many non-serious jobs are categorised as 'no offence detected' once police have conducted initial investigations.

Electronic running sheets

MDTs in divisional vans generate reports called ePDRs which capture the jobs that are dispatched by Triple Zero operators at ESTA. These ePDRs list the jobs dispatched to a unit with details from the caller, including the date, time and address, if available. The ePDRs also record speed checks, breath tests and all licence plate checks completed by members. Some reports made direct to a police station by telephone or made in person are also dispatched to MDTs for members on patrol to investigate. MDTs allow members to set their status, such as 'on patrol', 'arrived at a job' or 'at the police station', and to enter notes about how they have responded to jobs.

MDTs and ePDRs provide members with the ability to look up some existing LEAP record details, such as person and location warnings, and supervisors at the stations can oversee activities conducted in divisional vans in real time.

No offence disclosed—when further information exists to indicate the crime did not occur or has insufficient evidence, the status of the crime incident is amended to 'no offence disclosed'. A sergeant or above must approve the change before its status can be updated to 'no offence disclosed'. —Adapted from the Victorian Crime Recording Standards

Members finalise ePDRs after their shifts, and supervising sergeants sign them off. When reviewing and approving ePDRs, the supervising sergeant needs to ensure that members have recorded appropriate actions or handovers to other units against the listed jobs, and that any required reporting of crimes, incidents and other details is completed.

Supervisors need to check that where 'no offence disclosed' is stated, members have exercised proper judgment. This is the primary way for Victoria Police to determine if jobs are assessed and recorded as they should be. This process is verbal unless a supervising sergeant asks a member to rework the report or investigate further—in this case, the ePDR is updated to clarify an outcome or capture additional information.

We found that the detail recorded in ePDRs is not always comprehensive and it is not always easy to determine the course of action taken by members nor the outcome of their investigations.

Testing LEAP data against ePDR jobs

We assessed whether members are recording incidents on a prima facie basis. If prima facie reporting is not embedded into practice, there is a risk that incidents reported to Triple Zero will not be recorded in LEAP when they should be. We also assessed whether members recorded ePDR jobs to LEAP in a timely manner.

For our tests, we could only assess information reported to divisional vans, as crimes reported to police stations in person or via telephone are not recorded in a consistent way or available electronically.

We reviewed ePDRs from randomly selected police stations. The methodology for our testing is contained in Appendix C.

Results

Our completeness test showed that, while most jobs are being recorded in LEAP, the level of detail recorded varies:

• Of the ePDR jobs we tested, we found that 96.1 per cent of jobs were either recorded in LEAP or had reason not to be recorded.
• We found that members did not always consistently record the outcome of jobs and that the details they added varied in length and quality. In some cases, it was not easy to understand how a job was completed because the outcome of the job was not detailed in the ePDR.
• Although the system cannot record a large amount of information, members can improve the information by clearly detailing their progress.

Current monitoring by Victoria Police does not ensure members include sufficient detail in ePDRs. Members do not have a thorough understanding of when to report and how to report so that ePDRs contain a clear course of action and outcome.

2.4 Issues impacting crime recording

LEAP is not fully integrated with police IT systems and other justice agencies' systems. In some instances, such as for highly sensitive or potentially prejudicial information, it is useful to segregate access and transfer of information. However, for day-to-day administration of crime recording, Victoria Police could achieve substantial benefits by better connecting data and systems to streamline processes and avoid duplicated effort.

A lack of system integration increases the risk of inconsistent or incorrect information—for example, offender or location information recorded in an ePDR does not transfer to LEAP, so manual input is required in two separate systems. Currently, if members wish to charge an offender, they need to retype the offender information into the brief prepared for court, as LEAP cannot automatically generate it. Administration staff at local police stations further retype the data to track the movement of the brief around the police station and to the courts.

When an offender is apprehended, instead of adding information into LEDR Mk2, the member must complete a form and fax it to RSD to enter into LEAP. This outdated process risks the forms not being received or updated in a timely manner. Upgrades to LEDR Mk2 to enable members to complete offender processing will require significant work.

Victoria Police did not complete upgrades to offender processing in 2017–18 as planned. The scoping exercise will carry over into 2018–19. The modifications, if implemented, will not improve ePDR functionality or integration with LEAP but are intended to integrate with the court system, allowing members to process offenders and prepare and lodge required briefs for court.

Victoria Police is aware of the limitations of its information systems. By its own admission, it has not kept pace with technological developments. To date, it has been unsuccessful in securing funding for a new system to replace LEAP. Upgrades to LEDR Mk2 in 2013 did speed up the recording and approval process, and modifications to improve functionality have been completed regularly since 2013. Victoria Police is planning to introduce the ability for members to directly process offenders, however, the timing of the release is unclear.

Back to top

Police members need to create incident records with the most complete and accurate information available so that crime is detected early and victims and perpetrators are treated fairly. Inaccurate or incomplete records in the LEAP database will affect the quality and reliability of published crime data.

This part of the report examines Victoria Police's control environment for ensuring it maintains the quality and security of crime data and assesses completeness, accuracy and timeliness of crime reports.

3.1 Conclusion

Victoria Police's central quality assurance mechanisms are targeted at the highest-risk errors. The central quality assurance process—including RSD and CSU—strengthens the reliability and accuracy of crime data but needs to incorporate regular monitoring of date entries to improve overall quality. RSD and CSU have limited control over the quality of data that members create, and these errors can impact the accuracy of publicly reported statistics.

We found no evidence of manipulation or intentional misrepresentation in the selection of data we reviewed. This shows that Victoria Police has improved its processes since the Victorian Ombudsman found evidence of falsely resolved crime in 2011.

However, problems with data quality persist. Some data fields—such as 'country of birth'—are not consistently completed and a lack of specific location information and investigation narrative means important details that could assist with intelligence operations or analysing crime trends are lost.

Victoria Police does not have clear expectations for timely reporting in LEAP, which reduces the incentive to report crime in a reasonable time frame. We could not determine whether Victoria Police records crime in LEAP in a timely way. On average, there is an eight-day gap between the date a crime is reported and the date its corresponding record is created in LEAP. However, in a limited test we found 93.8 per cent of potential crimes reported to Triple Zero were recorded in LEAP within a 48-hour period, far better than the average.

Without clear benchmarks for timely recording and better monitoring, timeliness data will remain unreliable and important incident information may not be available for members when they need it.

Victoria Police has ITGCs for system security, data protection and management of user access, and changes to its LEAP and LEDR Mk2 systems. However, it needs to further strengthen these controls to ensure the control environment is adequate for the level of risk associated with securing sensitive data and maintaining integrity.

3.2 Quality checking at police stations

Supervising sergeants monitor the tasks and activities allocated to patrol teams by reviewing ePDRs. This process allows senior sergeants to check whether junior members are complying with the requirement for prima facie reporting. However, there are no other checks or processes to verify whether members understand the standards and apply this style of reporting.

Once the supervisors approve ePDRs, they ensure members record any incident that is required in LEDR Mk2. According to the Victorian Crime Recording Standards, the checking supervisor ensures that:

• the report time and date is when the police were informed of the crime
• the 'crime incident from' and 'to' dates are before or equal to the 'report time and date'
• the victim's full personal details are recorded—for example, name and date of birth
• the full location of the crime is recorded—for example, street number, street name and suburb
• the offence is correctly coded—that is, that the member has not confused similar types of offences such as theft with burglary and that the correct offence code is recorded
• the modus operandi (the particular way a person commits a crime) is recorded
• for crimes against a person, the victim notification details are completed.

Once approved by the supervisor, the records are transmitted from LEDR Mk2 to LEAP and are available for internal review. These quality assurance processes are discussed in Section 3.4.

We interviewed 18 detectives and members who use crime data to analyse and report on crime trends and patterns. In these interviews:

• seven stated that supervising sergeants need to more thoroughly check the quality of information reported in LEAP
• five stated that it was not errors that were the problem but a lack of information—check boxes are skipped and the quality of the narratives in describing the steps taken during the investigation could be improved
• eight were frustrated with errors in recording location and name details.

Supervising sergeants indicated that time pressures often meant that they could not spend as much time doing quality reviews as they would like. Without detailed information about crimes, the accuracy and value of Victoria Police's crime data may be reduced.

Training and development

Victoria Police does not provide sufficient crime reporting training, nor does it set out clear expectations for staff regarding requirements for capturing accurate data.

Training manuals for LEAP and LEDR Mk2 provide guidance on how to report a range of scenarios and are available on the Victoria Police staff intranet. The manuals are clear and detailed. Victoria Police advises that members or staff are not permitted to access LEAP until they have completed training which lasts around half a day. They refer to the requirements in the VPM and the Victorian Crime Recording Standards for report quality and emphasise the importance of accurate data entry. These manuals are formatted as introductory modules for each system but are not intended to be a reference manual for recording investigations that comply with the VPM.

New members training at the Victoria Police Academy receive around one hour's training on data recording. This is inadequate for the significant amount of time members spend entering, updating and revising crime data. Training in the Victorian Crime Recording Standards needs to be included in data recording training to help members to understand their obligations. As it stands, the training will not provide members with sufficient understanding of how to complete reporting that meets the required standard.

Victoria Police runs other online training courses as necessary to introduce new requirements such as changes to crime recording. Members advised that it can be hard to fit online training into busy schedules and indicated that training was most effective when conducted in an 'on the job' environment—this is a reasonable approach given the range of scenarios that police encounter. However, the quality of on-the-job training varies from station to station, depending on the thoroughness of the member and the supervising sergeant.

To ensure all members understand the requirements for capturing accurate data, Victoria Police should provide additional training and define clear expectations. Further, senior members should reinforce the key messages with ongoing regular coaching, and follow up junior staff members' progress and understanding of the training material.

3.3 Regional oversight

Four regionally based teams have overall responsibility for undertaking appropriate audits of the crime-recording process. They may conduct audits when new senior members commence, to assist them in their new role, or when a risk has been identified. The teams look at a range of risk factors, including investigation workload and timeliness, and they conduct audits in accordance with the Victoria Police Workplace Inspection Manual .

The Workplace Inspection Manual guides the monthly Workplace Inspection Reports (WIR), which are completed at police stations to monitor compliance, workforce and workload. The regional audit teams collect data from LEAP and other Victoria Police systems and, if they identify risks, they may conduct training to rectify issues.

Monthly inspections

WIRs provide an indication of a PSA's workload, its progress against this workload, the quality of its reporting on investigations and the reasons for outstanding items—and they may prompt senior members to intervene. They also assess compliance with financial requirements, occupational health and safety, staff leave and investigations with an 'active' status for long periods.

WIRs include a review of all LEDR Mk2 entries with the status 'pending approval', 'draft' and 'reworked' to determine if they have been appropriately reviewed and actioned. Any entries with a 'pending approval', 'draft' or 'reworked' status for more than seven days are considered deficient. Local area commanders are expected to report on overdue, complex or deficient entries in the 'comments' section .

The scoring—'satisfactory', 'close to standard' or 'unsatisfactory'—for LEAP, family violence, LEDR Mk2 and other sections in WIRs is subjective. These descriptors are not based on a quantifiable measure but on judgment alone.

A 2013 internal Victoria Police report found that supervisors do not review LEDR Mk2 reports against computer-aided dispatch data from Triple Zero calls to identify reported incidents that should be in LEAP. The internal report noted the need to establish a coordinated approach for checking the completeness of crime records, also noting that station-level checks rely on members' diligence and the availability of resources. In response to this finding, Victoria Police updated the Workplace Inspection Manual to include this check in monthly reporting.

A further internal audit of the WIRs in 2016 found that Victoria Police assesses family violence and LEAP by the volume of the workload rather than the actions taken. This approach limits the WIRs' use for assessing compliance. For example, a PSA's LEAP reporting can be marked 'satisfactory' but there is no way to measure the checks that were undertaken or what the result means. The internal audit also found no evidence that WIRs are improving data quality or timeliness and that there is confusion as to who ultimately signs off WIRs. Many were left unsigned or were not submitted to the correct tier of management, despite being completed. The purpose and actual utilisation of WIR results for improving LEAP is unclear.

Victoria Police has not yet implemented the recommendations of the internal audit, including that it adopt a consistent methodology for assessing performance, but it intends to complete a compliance project to address these recommendations in June 2019.

Quality controls

Database dictionary

Database user manuals or dictionaries assist users to analyse data and are essential for maintaining data integrity during periods of technological and process change—particularly when data is input by many users locally and reported centrally. These manuals often describe the full set of fields that the system users see and the related technical field names. They also define the information that is captured within fields and include other technical information used to link and match data for analysis. A comprehensive database dictionary acts as a 'single source of truth'.

Victoria Police does not have a data dictionary for LEAP that defines all of its data fields and, as a consequence, there is significant reliance on the knowledge and experience of key staff for this important information. If these key staff leave the organisation, there is a risk that this knowledge will be lost and that LEAP data could be incorrectly captured or interpreted.

Preparing a comprehensive database dictionary would assist Victoria Police to maintain consistent and accurate data, particularly when field definitions change, or new fields are added. A database dictionary can be updated and communicated to all users.

A guide to navigating and searching LEAP is especially valuable for new users who may have difficulty navigating LEAP's old technology. While the LEDR Mk2 interface simplifies the process by allowing members to enter crime information with drop-down boxes and limited valid entries, newer members we interviewed struggled with using LEAP, not only because they were unfamiliar with the style of the old system but also because they use it infrequently. Without a database dictionary, there is a risk that information will be recorded, amended or approved inconsistently or inaccurately.

3.4 Central quality assurance

Data quality framework

Figure 3A shows how Victoria Police structures its data quality framework. Review of data quality is mostly completed at stations and PSAs by local supervising sergeants.

Figure 3A
LEAP data quality framework

Note: SAS is data storage and analytics software used by Victoria Police. CSS is Crime Scene Services which investigates property crime.
Source: Victoria Police.

RSD is responsible for adding offenders to incidents and performing quality checks. It also clears theft of motor vehicles, theft from motor vehicles (number plates) and missing-person reports based on a phone call from the member creating the report. RSD applies a risk-based approach to its quality checks. Data operators perform checks on each record for which they are responsible, and a supervisor checks the quality of the data. RSD supervisors select a random date from the previous month and perform quality checks of the work completed by the relevant data operators for that shift, equivalent to around 5 per cent of the data operators' completed work for the month. RSD supervisors monitor data operators closely and record and analyse errors to identify areas for further development and training.

The high-risk errors that RSD's data operators check for include:

• incorrect name record selected
• most recent address or telephone number details not added
• charges not linked to correct sub-incident
• member or station entered incorrectly on a charge record
• details of the accused's involvement added incorrectly
• duplicate charge added
• offender involvement not added.

In addition to the RSD supervisor checks, DQU runs daily queries across the dataset to review reports made by members via LEDR Mk2 and information entered by RSD staff from forms faxed by members.

In conjunction with the regional audits, CSU also checks compliance by comparing a division's reported crime against predetermined tolerance levels. It also completes a data analysis check to identify illogical, missing or duplicated entries—for example, a missing person's report with no person identified, missing victim information, ages more than 100 years or less than zero years recorded, and sub-incidents that appear duplicated.

The quality assurance conducted by RSD, DQU and CSU is reasonable, but the assurance that these mechanisms provide depends on the extent of quality control in the initial input of data. Ensuring that members understand how to complete reporting and that supervisors complete adequate checks is critical to improving data quality overall.

3.5 Information technology general controls

ITGCs are policies, procedures and controls to ensure the confidentiality, integrity and availability of systems and data—for example, an ITGC might require management to properly review and authorise access requests to the systems.

Ineffective ITGCs may affect the reliability and integrity of a system's underlying data and programs, and may affect management's ability to rely on underlying business and process controls.

Information technology general controls policies and procedures

Victoria Police's Enterprise Information Security Policy requires system-specific access control policies, standards and guidelines, and procedures that are documented, implemented and reviewed annually. Victoria Police has established enterprise-wide and LEAP and LEDR Mk2 specific policies and procedures, but it has not consistently reviewed and maintained them.

Figure 3B shows our testing results of 34 policy and procedure documents.

Figure 3B
Review of enterprise-wide and LEAP and LEDR Mk2 policy and procedure documents

Source: VAGO based on Victoria Police data.

Out-of-date policies and procedures may result in poor governance and inconsistent processes and controls implemented within the IT and business‑related applications. This may affect the confidentiality, integrity and availability of underlying systems and data.

Assessment of information technology general controls

User access management

User access management focuses on managing access to systems, including how access is approved, removed and periodically reviewed to ensure it aligns with staff roles and responsibilities. The main objective of managing user access is to maintain the confidentiality, integrity and availability of systems and data. Weaknesses in user access management controls can result in inappropriate and excessive privileges assigned to users and hence a risk of unauthorised access to systems and data.

Inactive user accounts

As employees take leave or resign, management should deactivate or remove user accounts in a timely manner to restrict access to systems and data. If not managed, these user accounts accumulate and can pose a security risk.

We identified a high number of enabled LEAP and LEDR Mk2 user accounts that had not been used for more than 90 days. LEAP and LEDR Mk2 systems have settings for user passwords to expire after 90 days to limit the security risk.

Victoria Police should improve its periodic user access management process to better identify, restrict and manage which users can access the LEAP and LEDR Mk2 systems.

Password configuration

The Australian Signals Directorate produces the Australian Government Information Security Manual—the standard that governs the security of government IT systems. The manual provides recommended password configuration guidelines for agencies.

Victoria Police has chosen not to fully comply with the recommendations of the Australian Government Information Security Manual for LEAP and LEDR Mk2 systems based on a risk assessment and system limitations. We noted that the systems' password configurations are robust. However, there are opportunities for improvement. Victoria Police has commenced an identity and access management project to review and improve the systems' password configurations.

System security review

Agencies periodically use vulnerability assessments and penetration tests to ensure continuous improvement of system security. Vulnerability assessment is a process that defines, identifies and classifies the security weaknesses in a computer system, and penetration tests simulate the actions of a hacker seeking to breach system security.

Victoria Police recently established vulnerability assessment and penetration test programs in 2018. The programs contain varying levels of system security reviews for LEAP and LEDR Mk2 systems. Victoria Police should improve its vulnerability assessment and penetration test programs to increase the security of LEAP and LEDR Mk2.

Program change management

Change management aims to ensure changes to an IT system or its environment are appropriate and preserve the integrity of the underlying system and data.

Figure 3C summarises a generic program change management life cycle.

Figure 3C
Generic program change management life cycle

Source: VAGO based on information provided by Victoria Police.

Weaknesses in change management can lead to an increased risk of unauthorised changes being made to systems and data, which could affect their availability and integrity.

Open change requests

We analysed LEAP and LEDR Mk2 change requests for the period 4 February 2016 to 6 February 2018 and identified that 65 per cent are still recorded as open in the IT service desk system. We tested a sample of 25 change requests and verified that management had approved them for implementation into the production systems. Victoria Police indicated that the change requests have been implemented. However, it has not performed the administrative review, and the reviewer has not closed the change request records in the IT service desk system.

Victoria Police should improve the administrative review process within its compliance activities for the change management process, to ensure it records and updates change request records as and when the administrative review is completed.

3.6 Information management

Victoria Police's Information Management Policy mandates the protection of information. The policy provides a framework for assessing the impact if the confidentiality, integrity or availability of information is compromised.

Figure 3D shows the four-step process in the information protection process.

Figure 3D
Victoria Police information protection process

Source: Victoria Police.

Information identification and assessment

Victoria Police performs information risk assessments to:

• identify information assets used as part of business and police operations
• assess the potential consequences if those information assets are compromised.

Victoria Police performed information risk assessments on the LEAP and LEDR Mk2 systems in February 2015 and April 2017 respectively. The risk assessment of the LEAP system used the version of the Information Management Policy relevant at that time, but a risk assessment has not been performed again since the policy was updated. This increases the risk that the assessment may not appropriately identify risks and that the systems may not apply relevant processes and controls to protect information.

Victoria Police indicated that it intends to refresh the LEAP and LEDR Mk2 information risk assessments in 2019 and, based on the results, redesign and implement security controls to remediate the risks identified.

Information marking and protection

Encryption is a form of information protection—it prevents data from being readable or usable in the event of unauthorised access to the system. Data encryption occurs at two stages—data in transit (electronic transfer of or access to data) or data at rest (electronic storage of data).

Affected family member—family members affected by events occurring during a family incident.

Victoria Police's existing processes only recommend the encryption of LEAP and LEDR Mk2 data when transmitted to a non-government agency. Victoria Police should improve its encryption controls over LEAP and LEDR Mk2 data. It has indicated that it intends to review its encryption controls for protecting sensitive personal data in its planned 2019 refresh of the LEAP and LEDR Mk2 information risk assessments.

3.7 Completeness

Master Name Index—numbers assigned to identify each individual who has interacted with police. Through these unique identifiers, Victoria Police can trace each individual's involvement with various incidents.

Consistent and missing data

Inconsistent identity data occurs when information about a person is recorded in LEAP differently for the same person across multiple interactions with police. As data can be re-entered without cross-referencing existing data, there is potential for conflicting data. Missing data occurs when a member skips a non‑mandatory field in LEAP. The more consistent and complete the data is, the more reliable it is.

To ensure crime data is accurate and can be fully utilised, LEAP should include reliable data on a person's country of birth, Indigenous status, gender and date of birth. This allows police, criminal justice researchers and policymakers to identify trends in crime such as the profile of victims of crime, offenders and affected family members.

When people interacting with police are recorded in LEAP in connection with an incident, they may already have an existing unique identifier known as a Master Name Index (MNI) number. LEAP assigns these unique numbers to identify each individual so that there is less risk of confusing individuals with a similar or the same name and identifying information. If an MNI does not already exist for an individual, LEAP generates a new one when they record the person's interaction with police.

'Unknown' and 'not stated' are options in some data fields for members to use if they cannot determine the answer. For non‑mandatory fields, the response can also be left blank.

To further mitigate the risk of incorrectly identifying a person, Victoria Police completes secondary checks of names against drivers' licence numbers, last known addresses and dates of birth. For other demographic information, however, it is not possible to reliably verify if it is correct. People interacting with police can provide contradictory advice, and police may also be hesitant to probe for personal information. We assessed the extent of inconsistent and missing data for critical data fields in LEAP. Figures 3E, 3F and 3G show the results of our analyses of key characteristics.

Figure 3E shows how well members record country-of-birth information.

Figure 3E
Consistency and completeness of country-of-birth data in LEAP, 2012 to 2017

Individual	Consistent	Inconsistent	Missing
Offender	78.9%	13.8%	7.3%
Victim	25.8%	2.1%	72.1%
Affected family member	59.1%	5.2%	35.7%
Other parties	72.1%	10.9%	17.0%

Note: 'Other parties' can include a witness or a family member or person present at the incident. Field responses selected as 'unknown' or 'not stated' were not considered in this analysis. Data for 2017 includes records for January to September only.
Source: VAGO based on Victoria Police data.

Victim of crime—a person who suffers harm as a direct result of an act committed by another person in the course of criminal conduct. Harm may include physical, psychological or psychiatric harm or may occur when property is deliberately taken, destroyed or damaged.

It is not mandatory for members to enter country-of-birth data, but for offenders they record it consistently 78.9 per cent of the time and it is completed 92.7 per cent of the time. Increasing the amount and consistency of country-of-birth data for victims of crime would help Victoria Police to better understand if people from different countries are targets of crime and could help members to better identify whether crime is racially motivated. The current lack of consistent data limits this kind of analysis.

Victoria Police is concerned that making more fields mandatory may decrease the quality of the data. As members already consider reporting requirements to be high, additional mandatory fields might lead to attempts to skip or rush the input of data.

Figure 3F shows how well members record gender.

Figure 3F
Consistency and completeness of gender data in LEAP, 2012 to 2017

Individual	Consistent	Inconsistent	Missing
Offender	95.9%	4.0%	0.1%
Victim	91.1%	1.0%	7.9%
Affected family member	97.4%	2.5%	0.1%
Other parties	96.1%	3.8%	0.1%

Note: 'Other parties' can include a witness or a family member or person present at the incident. Field responses selected as 'unknown' or 'not stated' were not considered in this analysis. Data for 2017 includes records for January to September only.
Source: VAGO based on Victoria Police data.

The rate of gender information is largely complete and has higher rates of consistency compared with other categories. However, victim gender data is missing 7.9 per cent of the time. Victoria Police completed a backlog of updates to gender data, which has improved the reliability of this information.

Figure 3G shows how well members record Indigenous status in LEAP. Once again, there is a larger amount of missing information about victims (41.2 per cent) compared with offenders (4.2 per cent). We discuss CSA's method for counting Indigenous data in Section 4.3.

Figure 3G
Consistency and completeness of Indigenous status data in LEAP, 2012 to 2017

Individual	Consistent	Inconsistent	Missing
Offender	87.1%	8.7%	4.2%
Victim	57.2%	1.6%	41.2%
Affected family member	82.4%	5.6%	12.0%
Other parties	85.2%	8.5%	6.3%

Note: 'Other parties' can include a witness or a family member or person present at the incident. Field responses selected as 'unknown' or 'not stated' were not considered in this analysis. Data for 2017 includes records for January to September only.
Source: VAGO based on Victoria Police data.

3.8 Accuracy

Downgrading offences

There is a risk that members may downgrade the seriousness of an incident if there is internal pressure to reduce serious crime, or members feel they need to 'improve' the rate of serious crime data by recording a less serious offence code.

Downgrading can also occur if a member decides to record a lesser offence when first detecting and reporting an incident. If this first decision to downplay the seriousness of an incident is not corrected by another member or supervising sergeant, it is difficult to detect unless victims or witnesses successfully challenge the facts of the incident.

Until a report is approved and committed to LEAP, members can save changes over the top of the original offence code. Such changes to the incident report may be necessary before approval—for example, if new information becomes known or errors need to be corrected—downgrading or updating offences occurs. LEAP allows offences to be downgraded for legitimate purposes. Police often communicate such changes in the 'narrative' field—for example, a sergeant might send a report back to a member with a note in the LEAP narrative or via LEDR Mk2 to change the offence code. In this case, the field will only record the last change made, not the original code. In this way, improper downgrading is also possible and difficult to detect on LEAP.

Victoria Police cannot easily search audit logs for changes made to offence codes because the logs contain so much information. LEAP audit logs show each screen a user has accessed, and each keystroke made, but not the actual changes made. The logs are used to try to back up claims of misuse once a mistake or manipulation has been alleged or discovered.

To see whether shifts exist in the prevalence of offences over time or movements indicate downgrading of offences, we looked at movements in the following crimes:

• robbery compared to theft
• burglary compared to theft
• burglary compared to trespass
• actual offences compared to attempted offences.

We used the following offences to see whether shifts were significant:

• trespassing—entering a private property without permission
• burglary (split by attempted and actual)—similar to trespass but with intent to commit a crime
• theft (split by attempted and actual)—taking someone's property with the intent to permanently deprive him or her of it
• robbery (split by attempted and actual)—similar to theft but with the use of physical force or fear.

Looking at the number and the percentage of movements from 2016 to 2017, it was difficult to draw any conclusions. The shifts in offences involve small numbers that are not material. One PSA had a 6 per cent reduction in theft and a 3 per cent increase in attempted theft. However, the count shows that 442 fewer thefts were recorded and only 11 more attempted thefts for the period. This is not a material shift, and it is not possible to conclude that the reduction in recorded theft offences correlates with the increase in recorded attempts.

Another PSA had a 1 per cent reduction in recorded theft and a 5 per cent increase in recorded attempted theft. Looking at the actual count, there were 17 fewer thefts and an increase of 16 attempted thefts. The numbers are too small to definitively say that there has been a downgrading of offences.

Crime clearance

Clearing or resolving crime—the investigation into the entire incident has been completed and all crime records have been marked with one of the following: 'offender processed' 'intent to summons' 'intent to summons not authorised' 'complaint withdrawn' 'no offence disclosed' 'other' (offender deceased, underage).

Cleared incidents do not always result in an offender being apprehended. A member can request to clear an offence if, after some investigation, the member determines that no offence occurred and evidence is insufficient to proceed to court. If approved, the incident will be counted as cleared.

Irregular outcomes

In May 2011, the former OPI found evidence of police clearing cases inappropriately, particularly for 'high volume crime such as theft, which has a low clearance rate'. We looked at incidents with outcomes that members could use to artificially clear unresolved crime. First, we looked at the 70 063 incidents from 2012 to 2017 with one of the following outcomes:

• 'no offence disclosed'
• 'complaint withdrawn'
• 'other'.

We reviewed the reasons for members recording these outcomes for 30 incidents recorded as 'other' and 30 recorded as 'no offence disclosed' and found that the outcomes were reasonable.

We also looked at incidents with 'complaint withdrawn' outcomes to assess whether a sample of the reports contained information to support that outcome. Inappropriate use of 'complaint withdrawn' could include a member failing to properly interview and obtain a statement from a victim, to verify the complaint is withdrawn or coercing the victim to sign such a statement. This will reduce the member's number of unsolved or open investigations and improve the clearance rate that Victoria Police is measured against.

We also checked a sample of cases open for long periods before being cleared with 'complaint withdrawn' to see whether there had been some incentive to reduce backlogs of old unsolved cases. There were 29 859 sub-incidents with 'complaint withdrawn' outcomes, and we reviewed a sample of the 12 per cent (3 583 incidents) that were resolved more than 120 days after being reported. These incidents represent less than 1 per cent of incidents for the period, so would not impact the clearance rates. We found that the LEAP narratives did not always sufficiently state how the outcome was reached. This is likely to be poor record-keeping rather than deliberate manipulation.

For crimes such as sexual offences, police manage the case through a dedicated case management system that has higher security protections, and a lot of information related to the case will not be recorded in LEAP. However, LEAP needs to remain a standalone reporting system. We found it was not always clear in the LEAP narrative how the member reached the stated outcome. It should be possible to record this without including sensitive information.

Victoria Police states that it is possible that members left Victoria Police without the incidents being resolved or that inexperienced staff were unsure how to record the information properly. Narratives should explicitly state how police reached outcomes so that information remains useful for other members and for compliance audits.

In our review of a selection of sub-incidents, we found no deliberate attempts to falsely clear reports. Victoria Police provided reasonable possible explanations for the outcomes, and the number of sub‑incidents that are affected is not material, so it does not significantly impact published data.

Illogical date sequencing

We tested whether LEAP data contained logical dates. The order in which we expected to see dates typically occur was:

• 'commit from' date—the date that an incident began
• 'commit to' date—the date that an incident ended
• 'report' date—the date that Victoria Police became aware of an incident
• 'create' date—the date that an incident was submitted to LEAP
• 'sub-incident result' date—the date that a result was recorded against a sub-incident.

Using this date sequence, we identified illogical date orders in 377 706 sub‑incidents, such as incidents created in LEAP before they were committed or where the 'sub-incident result' date is earlier than the 'commit to' date. This illustrates the potential for RSD staff and members to mix up key dates in LEAP and the magnitude of the problem.

We found 375 154 sub-incidents with intervention orders attached where the date the offence was committed was later than the date the sub-incident was resolved, or a result achieved. LEAP is not set up to record intervention orders. Victoria Police advised that it records the dates the court decides the intervention order is in place in the 'commit to' and 'commit from' fields'—which does not match the expected date sequence—but that this approach still allows members to understand how long the intervention order is active.

LEAP does not contain controls to prevent all possible illogical date sequences. LEDR Mk2 does use some date logic controls when members enter information, but once the data is transferred to LEAP, the dates can be edited. Incorrect dates create confusing or incorrect reports—this can obscure the number of incidents reported as being committed within a certain period, which means reporting on the time it takes to clear an offence will be wrong. However, the errors represent less than 1 per cent of the dataset. Therefore, the impact on public reporting is negligible. Victoria Police states that, since our audit, it has developed a date-checking tool to prevent this happening in future.

Location information

Street number is not a mandatory field in LEAP. When it is missing, it is difficult for members investigating crime to pinpoint exactly where an incident occurred. In some instances, members input a default value—for example, recording the street number as '1' when they are unsure of the actual address. For long streets, a missing street number may place the incident in the wrong suburb and distort reporting against a location, making the data inaccurate.

It is also possible to enter multiple address configurations for one location, such as a shopping centre, a school, a building taking up a large area or a site located on a corner, whose address might be recorded as either of the intersecting roads. Inconsistently recorded locations make it more difficult to gather reliable intelligence on sites of interest or high activity.

In our interviews, members who use LEAP data for intelligence purposes expressed frustration that location information is incomplete or inaccurate. They advised that it limits their ability to collect all available information on locations of interest. If the locations are not specific, there is a risk that locations may be under-reported or that incidents may be reported in the wrong location.

Detectives from one PSA spoke about their frustration with incorrect location information in LEAP:

A serious error is incorrect recording of location or addresses—it skews the data and affects intelligence products looking at trends across different locations. You need to do a manual search of narratives to link information together and you need knowledge of different names for places. We found three versions of the same school address with multiple offences and needed to manually match them up. Without the prior knowledge of the incident, we wouldn't know to look for it. The result was there were eight rather than three incidences at the address. We then mapped a sample of incidents using SAS [analysis software]—only 66 per cent had a correct address.

Another member spoke of missing data for theft of motor vehicles—the address where a vehicle is recovered may include a suburb or a street name with no street number. The member also described the challenge of identifying addresses for shopping centres, for which up to 50 registered address configurations can exist for a single location. Five members spoke about location information being incorrect or incomplete, limiting the usefulness of the data.

Information technology upgrades

The intelligence management system, one of the projects in the BlueConnect program, is intended to assist members to search narrative text in LEAP, including sites of interest and other important details. This should significantly reduce the time it takes to perform analysis. However, even with enhanced search functionality, user input still needs to be of a high standard so that information is as complete and relevant as possible.

Victoria Police plans to update LEDR Mk2 to introduce offender processing for members. Currently, RSD processes offenders, which may occur a considerable time after a sub-incident is recorded. This separate process may cause delays in LEAP being updated, but is likely to result in more accurate data because it is one of RSD's main responsibilities and RSD staff are trained to look for errors. Victoria Police needs strong controls over date inputs and quality assurance to make sure the data is correct.

3.9 Timeliness

As discussed in Part 2, Victoria Police's standard for timely input of crime data after members become aware of a crime is vague. This makes it difficult to hold members to account. We reviewed ePDRs from randomly selected police stations to understand how soon members were creating LEAP reports.

For the 48 ePDR jobs we tested, we found that:

• within 24 hours, 83.3 per cent (40 jobs) had been created in LEAP
• within 48 hours, 93.8 per cent (45 jobs) had been created in LEAP
• the remaining 6.2 per cent (three jobs) took up to 3.5 days to be created in LEAP.

Therefore, the majority of records in our selection were entered in LEAP in a timely fashion.

Measuring timeliness

To track how long it takes to clear a case, a member needs to input a start date. In LEAP, the 'report date' field records the date when police first knew of the incident. Another field—'create date'—generates automatically when the report is first created in LEAP. Victoria Police reports on timeliness of clearing cases by measuring the proportion of incidents resolved with 30 days. It uses the time elapsed between the report date and the date an incident is resolved.

In 2011, the Victorian Ombudsman recommended that Victoria Police should use the report date for case clearance reporting rather than the create date—which was previously used due to its higher reliability—and should work to improve the reliability of the report date field.

'The report time and date is when the incident is reported or attended by members and not the time and date details are entered on LEDR Mk2.' —LEDR Mk2 training manual

Prior to 2012, the report date was not used for reporting against BP3 measures because it was a non-mandatory field and the data was unreliable—our analysis confirmed that it was blank in a high proportion of entries. Since 2012, 'report date' has been a mandatory field.

Victoria Police now uses the 'report date' to calculate its BP3 timeliness measure results. However, there is no evidence that Victoria Police has attempted to improve members' understanding of the purpose of the field or improve the accuracy of data recorded in it.

Issues that impact timeliness data

We identified a number of issues that impacted our ability to accurately measure the timeliness data.

Long lag times between the 'report date' and 'create date' can indicate errors. RSD can inadvertently change dates in LEAP when adding information. Investigations can proceed over a long period of time and require many LEAP updates. When a witness is found after a sub-incident is approved or a suspect is re-interviewed, RSD may inadvertently enter these subsequent dates in the wrong field and accidentally override a reported date.

Members can also misunderstand the date concepts. Our analysis of irregular date patterns showed that members incorrectly enter the date an incident was committed as the date reported to police, particularly when people report historical offences.

Some offences, such as manufacturing drugs, involve information that needs to be restricted from LEAP, or require a longer period of time to establish that an offence occurred. Victoria Police needs to review whether these offences are being entered correctly and ensure processes involve close monitoring to avoid errors and improve the quality of the dataset.

Victoria Police advises that it has developed a program to scan LEAP for date irregularities, which should help to resolve this kind of error in future.

Time between the 'report date' and 'create date' fields

Timeliness of recording crime in LEAP

We analysed the timeliness of data entry by comparing 'report date' and 'create date' data for sub-incidents. Figure 3H shows the average number of days between the two dates by year since 2012. The average number of days has generally reduced during this period. However, it remains much higher than the VPM expectation that members complete the majority of reports within the same shift.

Figure 3H
Average number of days from 'report date' to 'create date', 2012–17

Note: Data for 2017 includes records for January to September only.
Source: VAGO based on Victoria Police data.

Figure 3I shows that entries for 45 per cent of sub-incidents were created on the same day they were reported to police. This proportion meets the VPM expectation. However, comparing the results in Figure 3I to the eight‑day average gap in 2017 shown in Figure 3H suggests that either a large portion of incidents are being created well after they are reported to police, or that 'report date' is still not a reliably completed field.

Figure 3I
Proportion of sub-incidents entered in LEAP by number of days from the date they were reported to police, 2012–17

Note: Data for 2017 includes records for January to September only. Excludes 4 per cent of sub‑incidents that had a 'create date' recorded as prior to 'report date'.
Source: VAGO based on Victoria Police data.

We analysed PSA data to see if some PSAs had particularly high averages that may be skewing the analysis. We found that all PSAs had an average of more than 6.8 days between the 'report date' and 'create date', ranging up to 14.5 days. Therefore, no PSA is performing well on timeliness, and it is likely that RSD and members across all PSAs are, at times, inaccurately recording reporting dates.

We also looked at the top five offences with the longest gap between the 'report date' and 'create date' to understand the impact offence type might have on the average. Figure 3J shows that offences such as manufacturing drugs, incest and manslaughter—which can all involve sensitive information that needs to be restricted from LEAP users—are created in LEAP after a substantial period of investigation.

Figure 3J
Top five offences with the longest gap between report and create dates, 2012 to 2017

Note: Data for 2017 includes records for January to September only.
Source: VAGO based on Victoria Police data.

In cases relating to drug manufacturing, manslaughter and incest that involve sensitive information that may need to be restricted, members may be unsure when to record the crime in LEAP. Victoria Police needs clear guidance on how to populate LEAP records correctly for these types of offences, so they do not distort timeliness data.

Older dates

Between 2012 and 2017, there were 3.5 million sub-incidents recorded in LEAP, and more than 500 000 of these (16 per cent) were created more than five days after the 'report date'. We performed further analysis of these sub-incidents and found significant differences between the 'report date' and 'create date', including:

• 788 records that had differences of five years or more
• 59 records that had differences of more than 20 years
• one record that had a difference of 47 years.

We asked Victoria Police to clarify how these large gaps in time points had occurred and whether they represented errors in the data. We provided a list of 59 210 sub-incidents for which the 'create date' was more than 100 days after the 'report date'.

Victoria Police advised that this likely occurred due to RSD incorrectly saving over the original report date when adding supplementary sub-incidents, as described above. Approximately 67.3 per cent of the records on our list might have had supplementary sub-incidents added to an incident at a later time. RSD's business rules for entering dates when members have failed to record them can also contribute to illogical date sequencing. There has been some improvement in the incidence of long gaps—from 9 402 sub-incidents in 2012 to 7 991 in September 2017.

Active—an investigation that is ongoing and further leads are to be followed up. Pending—an investigation that has been carried out as far as possible and all avenues of investigation have been exhausted with no final result. It is possible that crime incidents can remain in a pending state for many years.

During this audit, we saw no evidence that Victoria Police has attempted to improve the reliability of 'report date' recording since the Victorian Ombudsman reported the issue in 2011. Until the use of the 'report date' field is consistent, there is no way to understand whether members are meeting the standards or community expectations for crime recording. These issues are present in 1.6 per cent of the dataset so will have little impact on published crime statistics. However, Victoria Police can strengthen reporting training to reduce the risk of this occurring in future. RSD also needs to review how additional incidents are added to LEAP to make sure they do not contribute to the problem. Victoria Police advises that it intends to review these business rules.

Time taken to resolve cases

Police members clear crimes in a number of ways, such as by charging offenders, by investigating and finding that no offence has occurred, or by recording that a complaint has been withdrawn. In other instances, members may identify perpetrators but, for legal or other reasons, they cannot be charged—for example, if they are underage or deceased.

In 2011, the Victorian Ombudsman referred some allegations of police corruption relating to clearing of crime records to the former OPI for investigation. These allegations involved the mishandling of crime records, with the result that the data showed more crimes being resolved or 'cleared' than actual results.

In our interviews, members spoke of 'top-down' pressure that they or their superiors felt when there were increases in crime rates or breaches in time lines. Some saw this as indirectly encouraging members to artificially 'improve the numbers'. This pressure, if it does exist, will negatively impact on the completeness and accuracy of data. However, in this audit, we did not detect any manipulation of data or downgrading of offences to improve performance measures.

BP3 measures

Figure 3K shows the timeliness benchmarks that Victoria Police reports against in BP3.

Figure 3K
BP3 performance measures for Victoria Police—timeliness

Performance measures	Unit of measure	2017–18 target
Timeliness
Proportion of crimes against the person resolved within 30 days	per cent	>42
Proportion of property crime resolved within 30 days	per cent	>22

Source: VAGO based on Victorian Government Budget Papers, 2017–18.

Victoria Police publishes its performance against these measures in its annual reports. Over the past three years, it has requested data from CSA for the six crime-related BP3 measures—see Figure 1C in Part 1. However, during this period, Victoria Police used its own data to calculate the timeliness measures rather than using the figures provided by CSA.

Victoria Police used the 'report date', as recommended by the Victorian Ombudsman—rather than the 'create date'—and calculated timeliness based on all crime for the relevant period rather than just a proportion of solved cases. Victoria Police's results therefore include open cases that were taking longer than 30 days to close and provide a slightly less favourable but more transparent picture of timeliness.

We replicated the BP3 performance measure results based on the Victoria Police methodology and compared them to the published results for 2015–16 and 2016–17. Victoria Police uses statistics provided by CSA to report on the BP3 results, with the exception of the timeliness measures. We found that we could accurately replicate the results, which means that we have confidence that Victoria Police is reporting its performance results correctly.

To improve transparency, Victoria Police should work with CSA and ask CSA to compute timeliness measures for the Budget Papers. The Victoria Police methodology is sound, and our analysis showed that its results are accurate. However, it is CSA's role to provide verified figures, and Victoria Police can avoid any perception of bias if it uses CSA data.

Analysing case clearance times

Using LEAP data from the 'report date' and 'result date' fields, we analysed how long it takes to resolve or clear incidents. We found no evidence that police clear large volumes of reports close to their 30-day benchmark or at any other time point that would indicate manipulation of the data.

We also assessed individual PSAs for isolated pockets of case closure. We found that 43 per cent of cases were resolved within 10 days of the 'report date', and there were no spikes in case closures at the 30-day mark or any other points. This was true for all PSAs.

Figure 3L shows the number and proportion of cleared sub-incidents by days taken for the years 2012 to 2017. The chart shows that 84 per cent of sub‑incidents were cleared within 30 days, and 7 per cent were resolved over 120 days.

Figure 3L
Proportion of cleared cases within bands of days taken to clear, 2012 to 2017

Note: Data for 2017 includes records for January to September only.
Source: VAGO based on Victoria Police data.

Victoria Police advised that there are common reasons for older cases remaining open, such as inexperienced members failing to clear cases in LEAP when they were resolved. Another cause may be intervention orders left open with no date recorded against the 'resolved date' field—this occurs because LEAP is not set up to record intervention orders, particularly where the order is in place indefinitely. Victoria Police now enters a proxy end date for intervention orders to overcome this issue. Some cases take longer to resolve due to prolonged investigations.

Figure 3M shows the average days taken to clear cases for the years 2012 to 2017. The number of cases cleared on the same day they are reported has steadily risen over the five-year period.

Figure 3M
Average days to clear cases (banded) by year, 2012 to 2017

Note: Data for 2017 includes records for January to September only.
Source: VAGO based on Victoria Police data.

Back to top

CSA checks and sorts crime data—a process known as transformation—before it is considered reliable and meaningful enough to publish. Transformation can involve addressing errors, missing data and inconsistencies, and deciding whether to include or exclude very small datasets or outliers. It can also include grouping certain categories of data to make it easier to understand. Victoria Police provides CSA with raw recorded crime data, and CSA publishes it in accordance with a number of classifications, including its offence classification, based on ANZSOC.

4.1 Conclusion

CSA's crime statistics are complete and accurate, and the public can be confident that its published data tables contain reliable information on crime in Victoria since 2008. We recalculated the data in CSA's Recorded Offences reports for the 10 years from 2008 to 2017 and obtained an almost exact match between our results and those published by CSA. We also used LEAP to recalculate CSA's Criminal Incidents dataset from 2009 to 2017. Over the course of the years examined, we found differences of less than 2 per cent across each year in both calculations. From this, we conclude that CSA's reporting of crime rates over time closely reflects Victoria Police's actual performance.

CSA's methodology is transparent—it publishes information on its website about its processes and its counting and classification rules so that the public can understand how it uses Victoria Police data and how the public can interpret the statistics.

CSA developed an offence ranking system that closely aligns with the ABS NOI and adapted it to reflect Victorian legislation on crime. This ranking system is used to classify the offence type of incidents within the Criminal Incidents table. Again, the methodology is sound and allows the public to understand the level of crime that affects victims. However, the Criminal Incidents dataset should be used in conjunction with the Recorded Offences table rather than as a standalone measure of crime.

Victoria Police is using crime data to inform its decision-making. It uses data from LEAP to track and compare the level of crime in the community and to understand its progress against its publicly reported key performance indicators. Crime data is also helping PSAs to understand crime trends and where they can best focus their efforts.

4.2 Processes for transforming data into crime statistics

CSA publishes crime statistics in quarterly and annual reports. Its website contains information about how it transforms Victoria Police's LEAP data into published crime statistics.

CSA receives a monthly subset of LEAP data based on an extraction script written when CSA was first established. After each quarter, CSA conducts a series of high-level quality checks, compares the data to previous extracts to see if the number of records is within expectations, and tests for any missing data. When there are unexpected results, CSA consults with Victoria Police. CSA then classifies the data and applies a series of counting rules. CSA analyses notable movements and trends for the media release to help readers understand the information. Once the clearance processes and analysis are complete, it releases aggregated, de-identified data reports to the public on its website. CSA's documentation of its processes shows that it implements a clear and consistent approach for calculating crime statistics.

Individuals and researchers can easily access data on the CSA website, which includes an interactive ' crime by location ' tool. CSA also provides crime statistics in response to information requests—it receives about 1 000 requests each year from Victoria Police, government agencies (especially DJR), academics and the public. CSA has a process for managing and responding to data requests, which takes into account the nature of the request, its purpose and any potential for the data to identify individuals. Councils and ministers use data from CSA to explore potential crime risks for particular areas of a community. Government departments make data requests to inform policy and project work.

The monthly extract provided by Victoria Police does not contain the free-text 'narrative' field, which records specific details of incidents and actions taken to solve crime. This is an important part of the crime record, as members will usually only amend or add information to this field. However, it is possible the information recorded in the 'narrative' field may contradict data in the other fields and vice versa.

Victoria Police provided CSA with its first extract of 'narrative' data from LEAP in January 2018. This was a one-off provision of data. CSA considers 'narrative' information valuable as an additional source of information for validation. Victoria Police and CSA have yet to determine guidelines for the use of 'narrative' information in CSA's transformation process.

CSA publishes a number of data tables, as shown in Figure 4A. When the media reports on a rise or fall in the crime rate, they often use the Recorded Offences table to derive it while, in theory, other tables can also be used.

Figure 4A
Data tables published by CSA

Table	Description	Publishing frequency
Recorded Offences	The number of offences recorded by Victoria Police in LEAP during the reference period. This includes offences where no offender is identified or charged.	Quarterly
Alleged Offender Incidents	The number of offenders given an outcome by Victoria Police within the reference period. An individual may be an alleged offender multiple times within the period. If an incident includes multiple offences, it is assigned the offence category of only the most serious offence in the incident, known as the principal offence.	Quarterly
Victim Reports	The number of victim reports made to Victoria Police within the reference period. An individual may have reported multiple incidents of victimisation within the period.	Quarterly
Family Incidents	The number of family incident reports submitted by Victoria Police and recorded in LEAP within the reference period.	Quarterly
Unique Offenders	A snapshot of unique offenders within the reference period. An offender who has had multiple incidents is counted once.	Annually
Unique Victims	A snapshot of unique victims within the reference period. An individual who has reported multiple incidents of victimisation is counted once.	Annually
Criminal Incidents	A measure of criminal events. They may include multiple offences, offenders and victims recorded against one date and location.	Quarterly

Source: VAGO based on information provided by CSA.

Criminal Incidents table

CSA started reporting on incident-based crime in December 2017. Its intention in developing this additional table is to create a measure that is more aligned with how the community experiences criminal incidents—that is, as the criminal event that occurred rather than the unique offences that were involved. The introduction of the new measure is intended to complement the Recorded Offences table, and CSA publishes the incident crime rate alongside the Recorded Offences crime rate.

An incident is a distinct event that can be made up of one or many sub-incidents or offences occurring at the same time and location. It can also include multiple victims and alleged offenders. As a criminal incident can involve multiple offences and, therefore, multiple offence types or charges, CSA has developed a method for determining the 'principal offence', which becomes the offence type used for grouping incidents in the report. Figure 4B shows the method CSA uses.

Figure 4B
CSA's method for assigning principal offences or charges for incidents

Note: The fourth example shows the impact of charges when calculating the principal offence/charge for an incident—offences that have charges attached are prioritised over those that do not. In this example, a charge has been laid for assault but not for drug trafficking, thus assault becomes the principal charge even through drug trafficking has a higher ranking on CSA's Offence Index.
Source: VAGO based on CSA's explanatory notes.

The principal offence represents the most serious charge associated with the incident or the most serious offence committed (if no charges are yet laid) within an incident. The CSA Offence Index is largely based on the NOI, which ranks offence codes based on seriousness. CSA uses the police outcome associated with the most serious offence or charge to represent the overall investigation status of the incident.

We compared CSA's offence ranking against the NOI to test whether the rankings align and whether the assigned order of seriousness is reasonable. A direct alignment of the ranking systems is not possible because crimes are not named and defined in exactly the same way across the criminal legislation of different states. However, we observed that CSA has broadly applied the same order of priority for offences as the NOI, so we consider CSA's determination of principal offence for the Criminal Incidents table to be appropriate. The NOI measure of a crime's seriousness appears to reflect the harm imposed on victims, and we assessed this as a reasonable approach for assigning the principal offence of criminal incidents. We used CSA's offence ranking in our analysis of the Criminal Incidents table, which is discussed in Section 4.4.

Reporting based on incidents rather than just offences allows the public to understand the number of criminal events rather than the number of offences—CSA states that this is closer to how the community experiences a criminal event. However, the Criminal Incidents table may not highlight the increases and decreases in offences that do not amount to the principal offence. Although these offences are not the most serious part of an incident, they are still significant and relevant pieces of information about reported criminal activity. The Criminal Incidents table is most useful when considered alongside the Recorded Offences table.

4.3 Applying counting rules and classifications

CSA has a transparent process for the transformation of Victoria Police crime data. These transformations categorise the data in accordance with national recording standards such as ANZSOC. CSA has modified ANZSOC where required to match Victorian criminal law. This is an appropriate adaptation to make the classifications relevant to Victoria.

We reviewed the statistical transformations of the data and found that they comply with CSA's own policies and standards. The crime data that CSA receives from Victoria Police contains all sub-incidents that include an offence and selected other events. These events include family violence reports and traffic cautions issued to juveniles.

Adjustments

CSA applies two key counting rules to Victoria Police data:

• The first includes data on juvenile traffic cautions that Victoria Police does not view as offences but collects information on.
• The second excludes offences not managed by Victoria Police such as some animal cruelty offences, interstate motor vehicle theft and traffic infringements—which Victoria Police includes in its internal reporting.

These are reasonable adjustments.

Indigenous status data

Victoria Police records a person's response regarding his or her Indigenous status on each occasion that an incident is recorded against the person. Victoria Police suggests that members may avoid asking whether a person identifies as Indigenous because they perceive it to be sensitive and, instead, either guess the person's Indigenous status or record 'unknown'.

To manage the high number of Indigenous status fields marked 'unknown' in the data provided by Victoria Police, CSA applies a counting rule to the Indigenous status data to improve its quality and make it suitable for release. CSA uses the 'most frequent' rule to allocate an Indigenous status in its reports. According to the 'most frequent' rule:

• if a person has made only one meaningful response ('yes' or 'no'), CSA uses that response for all records
• if a person has made both 'yes' and 'no' responses, CSA allocates the most frequently appearing response.

CSA's method may reduce the impact of any data entry error on resulting crime statistics.

We accept there will be legitimate cases where 'unknown' is an appropriate option for members to record in response to the Standard Indigenous Question, but frequent recording of this response may suggest that reporting is burdensome or that members do not consider it important enough to do properly. This limits the usefulness of the data for analysis.

Victoria Police has an opportunity to strengthen this process by training members on the importance of collecting information accurately and having supervising sergeants monitor members' efforts to identify the correct response before they record 'unknown'. Further, Victoria Police should centrally monitor use of the 'unknown' response for Indigenous status.

Categorisation

Police need crime records to contain specific information, including geographical locations and location types, information about relationships between the people involved, and descriptions of people, places, injuries or weapons. LEAP records should capture very detailed information to allow police and intelligence teams to identify trends in criminal behaviour and develop advice for how to best respond to crime.

Published crime statistics, on the other hand, are easier to interpret at a less granular level. For example, the public may be more interested in knowing about a rise or fall in assaults as a category of offence rather than separating this out by the individual crime codes that make up the assault category.

Within CSA's public reporting, details of individual offences are grouped into categories to help the public more meaningfully measure the impact of crimes on the community. For example:

• the crime division 'property and deception offences' contains a crime subdivision for 'theft'
• the 'theft' subdivision contains seven groups that identify types of theft such as 'motor vehicle theft' and 'receiving or handling stolen goods'—these groups contain the individual criminal codes assigned to each offence.

The 'theft' subdivision contains more than 170 different offence codes that distinguish specific crimes, some of which are no longer used due to repealed legislation. This level of detail is unlikely to be of use to the general public seeking to understand crime trends.

CSA categorises geographic location in multiple ways—statewide level, the four police regions, the 54 PSAs, local government areas, and suburbs and postcodes that broadly fit within a PSA. Victoria Police also captures data on location types, such as car parks or residential streets, and CSA groups these into categories. Location types identify characteristics of certain locations so that where certain types of locations are repeatedly targeted for particular crimes, Victoria Police can identify the trend. This categorisation is reasonable, as it preserves the originating police region and allows the public to explore the statistics by local government areas or suburbs.

4.4 Reconciling Victoria Police data and published crime statistics

Offence count reconciliation

We compared the data extract that Victoria Police provided to us with published crime statistics in CSA's Recorded Offences table . To do this, we applied CSA's processes and counting rules to the data extract provided by Victoria Police. The result was an almost exact match between our analysis and the publicly reported figures across nine years (2008 to 2016). In 2017, our calculations differed by 1 per cent—this variance is most likely due to the different extract dates used for our data compared with that supplied to CSA, resulting in 29 additional offences being included in our data. We therefore concluded that the number of offences publicly reported by CSA reflects what is recorded in LEAP.

Incident count reconciliation

We also compared the extract that Victoria Police provided to us with the Criminal Incident table by applying CSA's process and counting rules for this data. Our analysis varied by 5 per cent overall across the nine-year period (2009 to 2017 ) and varied by 5 per cent or less in all but one of the offence divisions (the 'other' category).

As with the offence count reconciliation, the difference in the results may be due to the different dates on which our data and CSA's were extracted. Figure 4C shows the two sets of results.

Figure 4C
Comparison of VAGO and CSA calculation of crime incidents by year ending September, 2009 to 2017

Note: The difference in the 2017 results can be attributed to different data extraction dates in September. Years ending in September have been used for analysis as the data Victoria Police provided to VAGO was extracted in September.
Source: VAGO based on Victoria Police and CSA data.

Figure 4C indicates that despite the differences in raw figures, similar conclusions can be drawn from both calculations of the data—that criminal incidents reduced by roughly the same rate between September 2016 and September 2017—approximately 5 per cent.

4.5 Analysing and using crime data

We looked at whether Victoria Police uses crime data for decision-making. We reviewed the areas responsible for internal data analysis and the ways the data is used.

Within Victoria Police, CSU provides data for internal information requests, for thematic briefings to the chief commissioner, and for CSA's analysis and public reporting. Its responsibilities include tracking Victoria Police progress against BP3 measures and forecasting crime trends for the forthcoming quarter.

Victoria Police uses crime data at its regular meetings to discuss progress and inform strategic and resourcing decisions. Senior staff can access data analysis tools on the intranet to complete analysis at any time. This makes it easy for members to check data and crime trends for a range of common queries.

CompStat

Compare Statistics (CompStat)—a forum for discussing accountability, focusing on elements of police performance management including crime reduction, intelligence, strategy and resource management. The concept was originally developed by the New York City Police Department and has been widely adopted by other police departments worldwide.

Assistant commissioners participate in CompStat regularly, a forum for discussing, analysing and comparing the performance of regions. Through this forum, participants can use data to:

• evaluate performance against agreed corporate goals
• encourage effective use of resources to achieve agreed goals
• provide feedback to an area under review
• discuss meeting agreed performance measures.

Victoria Police's CSU provides a range of relevant data and analyses to support CompStat to fulfil its responsibilities. CompStat briefing packs include scatter plots to show how divisions compare on clearance rates and number of offences. For example, the briefing packs may compare clearance rates for the offence 'theft of motor vehicles' to the state average and PSAs that are outliers with very high or low clearance rates. Data in the CompStat briefing packs may also present performance as percentage increases in crime. However, if this involves small numbers, it is not always meaningful.

Completing and sharing this analysis at CompStat meetings may act to keep divisions accountable and keep senior management aware of outliers and trends that need a response.

CompStat provides a forum for police to debate the drivers of crime, preventative strategies, and the balance between achieving reductions in total crime and 'harm crimes'—crimes that cause greater damage to people and their perceptions of safety.

In our interviews, four senior members stated that reducing harm crimes such as aggravated burglaries does not seem to help with reducing total crime. Reducing total crime means devoting resources to solving high‑volume lower‑harm crimes, which may divert attention and resources from lower‑volume but higher-harm crimes. They expressed frustration that reducing total crime is typically the focus of crime statistics and media coverage, and then becomes an internal priority. They argue that the priority should be to reduce harm crimes rather than total crime, which includes a lot of petty crime.

Tasking and coordination

PSAs also conduct monthly tasking and coordination meetings. The purpose of these meetings is to look at crime trends using data and intelligence for individual PSAs, and to discuss local priorities and resource allocations for responding to emerging crime trends.

We reviewed tasking and coordination meeting documents from three PSAs and found that they include:

• regional intelligence updates, such as for drug crime
• the number of offender debriefs completed—informal reports of conversations with offenders
• the number of outstanding warrants
• family violence activities and trends
• trends in other crime categories
• scorecards—monthly crime comparisons with percentage changes
• information on known drivers of crime—for example, the link between alcohol consumption and assault
• updates on how data supports claims that crime reduction efforts are successful
• tasking opportunities—activities to help address issues identified
• discussion of the causes behind spikes in trends and links to other offending and proposed actions.

Victoria Police's Intelligence and Response Unit provides guidance on how to put tasks and priorities arising from tasking and coordination meetings into effect. Decisions made at divisional and regional meetings also influence PSA responses. These mechanisms can work together to influence PSAs and how they arrange their resources to tackle different categories of crime.

The ready availability of data on the intranet and through regular communications makes it easy for PSAs, regions and divisions to compare their performance to others.

Back to top

We have consulted with DJR, Victoria Police, CSA and the ESTA, and we considered their views when reaching our audit conclusions. As required by section 16(3) of the Audit Act 1994, we gave a draft copy of this report to those agencies and asked for their submissions or comments. We also provided a copy of this report to the Department of Premier and Cabinet.

Responsibility for the accuracy, fairness and balance of those comments rests solely with the agency head.

Responses were received as follows:

• DJR
• Victoria Police

RESPONSE provided by the Secretary, DJR

RESPONSE provided by the Chief Commissioner, Victoria Police

 

Back to top

Definitions for quantity performance measures

To calculate Victoria Police's quantity measures for the Victorian Budget Papers, the rule is the total offence counts for the division (such as crimes against the person) divided by the estimated resident population (ERP) in Victoria multiplied by 100 000. Figure 1C shows the BP3 measures. The BP3 measures use a count of all 'division A' offences for the three measures of crimes against the person, while the three measures of property crimes use a count of all 'division B' offences.

The rule is represented as:

Offence rate = (offence count / ERP count) x 100 000

The ERP is the official measure of Australia's population based on the number of people who usually reside in Australia regardless of citizenship status. It is based on the results of the census and adjusted for net undercount. ERP is compiled as at 30 June each census year and updated quarterly between censuses. CSA downloads ERP figures annually.

The quantity measures relating to property and deception—excluding crime related to family violence—are calculated as the total count of property and deception offences with a 'no' family violence flag, divided by ERP then multiplied by 100 000.

For the quantity measures relating to property and deception offences—including crime related to family violence—the calculation is the total count of property and deception offences with a 'yes' family violence flag, divided by ERP then multiplied by 100 000.

Definitions for timeliness performance measures

To calculate results against the timeliness measures—the proportion of crimes resolved within 30 days—a similar method is used as for the quantity measures. The numerator for the equation is a count of all offences with a timely resolved outcome within 30 days. The denominator is the total offence count. The rule is represented as:

Proportion of person/property crimes resolved within 30 days = (count of offences with a resolved outcome in less than or equal to 30 days from the report date to the result date / total offence count) x 100.

Back to top

There are two main ways that crime is reported—through members of the public reporting an incident in person at police stations or by calling Triple Zero, or through police members detecting an incident while on duty.

We tested completeness and timeliness by tracing a randomly selected sample of 380 records from Triple Zero calls sent to divisional vans in July, August and September 2017, and then from ePDRs into the LEAP system.

To search LEAP, we needed ePDRs to contain identifying names and places, but this was complicated due to:

• members receiving scant information from callers to Triple Zero
• members recording insufficient details on the ePDR.

One test focused on completeness, determining whether the incidents recorded in the ePDRs are also recorded in LEAP. We reviewed the ePDRs and identified 358 jobs that indicated that an offence had been detected or which, based on our understanding of the Victoria Police standards, we judged as needing to be recorded in LEAP. Using data analytics and assistance from Victoria Police, we then conducted a search of the LEAP data to locate corresponding dates, times and details for the records.

A second test used 48 jobs selected from shifts in August 2017 to measure the timeliness of LEAP recording based on the period elapsed between a job being generated in the ePDR and the corresponding LEAP record's 'create date'. Victoria Police does not have a specific target for how quickly members should complete reports after they become aware of a reportable incident, but it is expected that members complete required reports by the end of their shifts in most circumstances. We assessed that a record was completed in a timely fashion if it was created in LEAP within 24 hours of the ePDR job. This reflects the standards for timeliness.

Back to top