Early Intervention Services for Vulnerable Children and Families

Conclusions

Child FIRST and IFS are failing to provide effective services for vulnerable children and families. The increasing number of high-priority cases has made IFS less available to families who are 'at risk' and qualify for an early intervention response, and to professionals seeking to refer vulnerable children and families.

While the department is aware of the significant increase in the number and complexity of cases being referred to Child FIRST and IFS, it has not systematically analysed this demand or planned for early intervention services that can meet the needs of vulnerable children and families at different stages of their vulnerability. The current funding structure does not reflect the growth in the number and complexity of cases or the impact this has had on service providers' capacity to meet the needs of vulnerable families.

The partnership structure that brings together family service providers and department representatives in local alliances is a positive initiative of the department. However, there is great variability in the level of coordination and the maturity of alliances across the 24 catchments. In some catchments, weak partnerships and inadequate governance arrangements have impeded the delivery of integrated and coordinated Child FIRST and IFS. Although the department has worked to improve this, ineffective communication between and across department levels and family service providers remains a problem.

There are isolated examples of vulnerable children and families being better supported. However, the department does not know whether the services provided are effectively meeting the needs of vulnerable groups seven years after the establishment of Child FIRST and IFS. This is because there are significant limitations in the service performance data and a lack of outcomes monitoring at the system level.

Findings

Inadequate and reactive planning

The department's strategic planning for Child FIRST and IFS has been reactive and rudimentary. While the department has made a significant effort to build the capacity of alliances to undertake catchment planning, it has not forecast overall demand for services, assessed unmet or potential demand, or responded to emerging demand drivers—such as family violence—in a timely manner. It lacks the robust evidence base that would enable it to respond to emerging issues effectively at the catchment, divisional and state levels.

Child FIRST and IFS have implemented a range of demand management strategies to cope with the number and complexity of cases being referred. While alliances review their demand management strategies, the department has not identified and evaluated the overall implications of these demand management strategies on the effectiveness of service providers' interventions.

Child FIRST and IFS are operating above their funded capacity in the context of increasing demand and the growing complexity of family needs. The department's failure to adequately monitor or assess why services are delivering above their targets and the consequences of this limits its capacity to redirect scarce resources.

Serious limitations on the department's performance data, and a lack of sound analysis of the available data, limit its capacity to plan proactively and effectively, and reduce the ability of the alliances to plan effectively.

Inadequate partnerships and governance

Child FIRST and IFS rely on multiple providers to deliver services. Effective partnerships and governance arrangements are critical to the coordinated and efficient delivery of these services. The department needs to develop an effective statewide mechanism to engage better with current and potential service providers.

While there are some examples across the state of effective alliances and mature relationships within alliances, these are not always embedded in the organisational system and structure, but rather are dependent on the level of professional capability of individuals.

Effective governance requires clear roles, responsibilities and accountabilities. The department has set up alliances that require Child FIRST and IFS providers to undertake catchment planning together and apply demand management strategies to prioritise local needs. While these steps are positive, the department has not established effective governance mechanisms at the local catchment, divisional or statewide levels. This means there is great variability across catchments in terms of how coordinated planning is, and in the operational and strategic management of service delivery. The department could be doing more to support the development of these partnerships.

The department has identified the need for stronger governance and more clarity around roles and responsibilities between Child FIRST, IFS and Child Protection as key priorities for its next round of catchment plans.

Inadequate performance and outcomes monitoring

While the department has a monitoring framework to assess community-based child and family service compliance with service standards, there is little monitoring of the performance of alliances.

The department's monitoring of services focuses on outputs—such as the number of cases and service hours—rather than requiring service providers to show positive outcomes for families.

There are significant limitations to the department's Integrated Reports and Information System—such as inconsistencies in data collection and poor-quality data. Limited system-level analysis of service data makes it difficult for the department to know whether Child FIRST and IFS are improving outcomes for vulnerable families.

The department does not have a framework for measuring the effectiveness of services for vulnerable children and families. Its 2007 document, A Strategic Framework for Family Services, outlines a plan to develop measures to allow the monitoring of outcomes at the individual, program or catchment, and statewide levels. However, seven years after the framework was released, the department has still not implemented the outcomes component of the plan.

Recommendations

We made one overarching recommendation—that the Department of Health & Human Services takes the key shortfalls identified in this report as the starting point for a comprehensive and urgent review of its current approach to early intervention. This overarching recommendation is supported by nine specific recommendations.

That the Department of Health & Human Services takes the key shortfalls identified in this report as the starting point for a comprehensive and urgent review of its current approach to early intervention and, as part of this review:

1. improves planning by better demand forecasting and more systematic analysis of existing program performance data—including analysis of the level and nature of non‑substantive referrals—to understand gaps in service response
2. develops a regular statewide engagement mechanism to identify issues and risks in a timely manner and to design solutions with the input of the service sector
3. provides targeted training to service providers in catchment planning and data analysis
4. reviews its whole-of-system funding for early intervention to better reflect the impact of demand drivers on Child and Family Information, Referral and Support Teams and Integrated Family Services
5. provides targeted support to those Child and Family Services Alliance members whose partnerships are still underdeveloped, and supports them to become more collaborative in their interactions
6. investigates and implements ways of improving the effectiveness of its communications about operational and strategic issues between and across the department centrally, regionally and locally, and with community service organisations
7. provides explicit requirements for its local and divisional staff regarding the monitoring of operational risks, emerging issues, and the capacity and capability of the partnerships involved in the local Child and Family Services Alliances
8. develops a set of standard analytical data sets for the Child and Family Services Alliances to use to monitor and report on the timeliness and effectiveness of their engagement with clients at the program level, including rates of failure to engage, referral outcomes re-referrals and re-reports
9. undertakes statewide performance analysis using catchment data to facilitate sharing of practices among Child and Family Services Alliance members.

Submissions and comments received

We have professionally engaged with the Department of Health & Human Services throughout the course of the audit. In accordance with section 16(3) of the Audit Act 1994 we provided a copy of this report to the department and requested its submission or comments.

We have considered the department's views in reaching our audit conclusions and have represented them to the extent relevant and warranted. The department's full section 16(3) submissions and comments are included in Appendix B.

1.1 Introduction

The 1996 VAGO report,Protecting Victoria's Children: The role of the Department of Human Services, identified ongoing strain in the statutory child protection system. It recognised that effective prevention and early intervention strategies were key to preventing children from entering the child protection system.

In almost 20 years since the report, the landscape has changed significantly. Successive Victorian governments have commissioned reviews and inquiries into Victoria's child protection system and committed significant funding to supporting and improving this system. While substantial reform has taken place, the demand on the child protection system remains.

Figure 1A shows that there has been a steady growth in the number of children reported as being at risk of abuse and neglect in Victoria, with reports to the Victorian Child Protection Service (Child Protection) doubling between 2006–07 and 2013–14.

Figure 1A

Reports to Child Protection

Source: Victorian Auditor-General's Office, using Report on Government Services data.

VAGO's 2014 audit on Residential Care Services for Children found that the system was unable to respond to the growing demand and level of complexity of children's needs, and that diversion strategies in the Out of Home Care system had shown mixed results.

1.2 Vulnerable children and families

The Protecting Victoria's Vulnerable Children Inquiry (the Inquiry) was announced in January 2011 and the report was delivered in February 2012. The Inquiry considered Victoria's system for protecting its vulnerable children and young people as a whole. In response to the Inquiry, in 2013 the government released Victoria's Vulnerable Children—Our Shared Responsibility Strategy 2013–2022, which defined children and young people as vulnerable 'if the capacity of parents and family to effectively care, protect and provide for their long-term development and wellbeing is limited'.

1.2.1 Who is at risk?

A child or young person can be vulnerable for a range of reasons:

• a parent, family member or caregiver may havea history of family violence, alcohol or substance abuse, mental health issues, chronic physical illness or is experiencing financial stress or bereavement
• parents may be young, isolated,unsupported or havelimited parenting skills
• the child or young person may have health issues or a disability
• there may be societal, economic and environmental factors, such as poor social connections in the community, poverty and residential instability.

Some unborn children may be identified as vulnerable during a woman's pregnancy if risk factors for subsequent child abuse or neglect are present. Others only become known to the Department of Health & Human Services (the department) or Victoria Police when they are adolescents and their circumstances have made them vulnerable.

There is a risk that these factors, if not addressed, may escalate and lead to the child and their family becoming involved in the child protection system.

1.2.2 Early intervention

The Inquiry referred to early intervention services as interventions 'directed to individuals, families or communities displaying the early signs, symptoms or predispositions that may lead to child abuse or neglect'.

Figure 1B shows the public health model—used by the Council of Australian Governments in the National Framework for Protecting Australia's Children 2009‑2020—which describes the range of interventions that apply to protecting children.

Figure 1B

Public health pyramid

Source: Australian Institute of Family Studies (2009) adapted from Bromfield & Holzer model (2008).

As shown in Figure 1B, the model is represented as a pyramid of escalating interventions. Primary intervention (universal) services target the entire community to prevent the kinds of social problems that can lead to vulnerability. Secondary intervention services target families in need, where vulnerability has been identified and children are at risk of abuse or neglect. Tertiary intervention services target families where abuse or neglect has already occurred.

The delivery of early intervention services to vulnerable children and families is a secondary intervention.

1.3 Legislative framework

In 2005, the government released the White Paper Protecting Children: the next steps, which aimed to create a more integrated system of child, youth and family services—an accessible adaptable system with a focus on children's safety, health, learning, wellbeing and development.

Two key pieces of legislation were introduced—the Child Wellbeing and Safety Act 2005 (CWSA) and the Children, Youth and Families Act 2005 (CYFA).

1.3.1 Child Wellbeing and Safety Act 2005

The overarching principles for the wellbeing of children are established in Section 5 of the CWSA. These principles relate to the design and provision of services for children and families, and include protecting the rights of children and families and acknowledging the child's individual identity, including cultural identity.

The legislation requires that, when designing services, consideration be given to:

• minimising harm and strengthening the capacity of parents
• meeting the cultural needs of the local community
• giving the highest priority to those known to have the greatest need
• promoting continuous improvement in the quality of service provision.

Three fundamental principles underpin the provision of services:

• society as a whole shares responsibility for promoting the wellbeing and safety of children
• all children should be given the opportunity to reach their full potential and participate in society, irrespective of their family circumstances and background
• those who develop and provide services, as well as parents, should give the highest priority to the promotion and protection of a child's safety, health, development, education and wellbeing.

The CWSA also recognises that parents are the primary nurturers of a child and government intervention in family life should be limited to that necessary to secure the child's safety and wellbeing. However, government is ultimately responsible for meeting the needs of the child when the child's family is unable to provide adequate care and protection.

1.3.2 Children, Youth and Families Act 2005

One of the key objectives of the CYFA is to support a more integrated system of effective and accessible child and family services, with a focus on prevention and earlier intervention.

Under the CYFA, the key responsibilities of the secretary of the department include:

• promoting the prevention of child abuse and neglect
• assisting children who have suffered abuse and neglect and providing services to their families to prevent further abuse and neglect from occurring
• working with community services to promote the development and adoption of common policies on risk and need assessment for vulnerable children and families
• conducting research on child development, abuse and neglect and evaluating the effectiveness of community-based and protective interventions in protecting children from harm, protecting their rights and promoting their development
• leading the ongoing development of an integrated child and family service system.

The secretary through the department is also responsible for registering, funding, and setting performance standards for community-based child and family services.

Community-based child and family services

The CYFA defines a 'community-based child and family service', as a registered community service organisation established to meet the needs of children requiring care, support, protection or accommodation, or families requiring support.

Community-based child and family services are to:

• provide a point of entry into an integrated local service network that is readily accessible by families, that allows for early intervention in support of families and provides child and family services
• receive referrals about vulnerable children and families where there are significant concerns about their wellbeing
• undertake assessments of needs and risks to assist in the provision of services to children and families and to determine if a child is in need of protection
• make referrals to other relevant IFS providers if this is necessary to assist vulnerable children and families
• promote and facilitate integrated local service networks working collaboratively to coordinate services and supports to children and families
• provide ongoing services to support vulnerable children and families.

1.4 A strategic framework for family services

In 2007, the former Department of Human Services released A Strategic Framework for Family Services (the framework). The framework aims to improve outcomes for vulnerable children, young people and families, by focusing on the:

• 'safety, stability, health, development and learning of children and young people
• cultural connection for Aboriginal children, young people and families
• capacity of families to provide effective care, and of communities to support them
• effectiveness of the supports and services in meeting the changing needs of children, young people and families'.

The framework established an integrated system of family services that included a range of community service organisations and a new central community-based intake model—Child and Family Information, Referral and Support Teams (Child FIRST). The establishment of Child FIRST was based on the evidence of a successful trial that the department had piloted and evaluated from 2003 to 2006. Prior to the introduction of Child FIRST, there were multiple entry points into family services, which had led to inefficient and sometimes duplicated services.

Child FIRST is responsible for the intake and initial assessment phase, while family services are providers responsible for the case work response. This system is commonly referred to as Child FIRST and Integrated Family Services (IFS). Figure 1C shows the most common process once a referral is made to Child FIRST, although there can be local variations.

Figure 1C

The usual process for entering the integrated system of family services

Source: Victorian Auditor-General's Office.

There were 96 community-based child and family services, registered and funded by the department in 2013–14 to provide Child FIRST and IFS in Victoria. Appendix A provides a full list of these organisations. These include some of the largest faith‑based service providers whose services are underpinned by a strong ethical commitment to protecting the most vulnerable in our community.

In 2013–14, the budget for both the Child FIRST and IFS components was just over $88 million, including just over $6 million to facilitate Child and Family Services Alliances (alliance) coordination, program development and the employment of Early Childhood Development workers.

1.4.1 Child FIRST

The purpose of Child FIRST is to provide an identifiable and easily accessible entry point into the integrated system of family services within a designated catchment or geographical area. The intention is to effectively link vulnerable children and their families to relevant services based on assessed need and risk.

Child FIRST was rolled out from 2006 to 2009. There are 24 Child FIRST catchment areas across Victoria, as shown in Figure 1D.

Figure 1D

Child FIRST catchment areas

Source: Department of Health & Human Services.

The key functions of Child FIRST are to:

• provide information and advice
• identify initial needs and assess underlying risks to the child or young person in consultation with Child Protection, family services and other services or professionals
• identify the Aboriginal status of children and families and consult with an Aboriginal Liaison Worker or Aboriginal Community Controlled Organisation
• actively engage with the child and their family as appropriate to complete an initial risk assessment to determine the priority of a response and allocation of families to the integrated system of family services in consultation with family services and Child Protection, where required
• deliver timely responses through the provision or oversight of 'active-holding responses' that involves short-term work with children and families, before they are allocated to family services.

1.4.2 Integrated Family Services

The aim of IFS is to promote the safety, stability and development of vulnerable children, young people and their families, and to build capacity and resilience for them and their communities.

The primary client group for IFS is vulnerable children and young people aged 0 to 17 years—including unborn children—and their families who are:

• likely to experience greater challenges because the child or young person's development has been affected by the experience of risk factors and cumulative harm
• at risk of becoming involved with Child Protection if problems are not addressed.

1.4.3 Child and Family Services Alliances

To support an integrated and coordinated service response for vulnerable children and families, the department has established alliances as part of the operational and governance arrangements.

These alliances have been set up in each catchment and include Child FIRST, IFS, the department, Child Protection, and—where capacity exists—an Aboriginal Community Controlled Organisation. Other sector representatives and professional groups may be invited to participate, as agreed by the core alliance partners.

Alliances have three key functions:

• undertaking catchment planning
• providing operational management
• coordinating service delivery.

1.4.4 Role of Child Protection

The role of Child Protection workers is to:

• receive reports from people who believe a child needs protection from harm or abuse
• provide advice to people who report cases of abuse and neglect
• investigate when a child is believed to have been abused or is at risk of abuse or neglect
• refer children and families to services in the communityfor ongoing support and prevention
• take matters to the Children's Court if the child's safety within the family cannot be guaranteed
• supervise children on legal orders granted by the Children's Court.

Community-based Child Protection is the term used to describe a range of roles and functions that support partnerships between Child FIRST and IFS and Child Protection, as well as the delivery of family services.

As a central intake point, Child FIRST can receive referrals from Child Protection and can refer families to Child Protection.

1.4.5 Role of the Department of Health & Human Services

The department is responsible for registering, accrediting and funding the community service organisations that deliver services for vulnerable children and families. It is also responsible for leading the ongoing development of an integrated child and family service system.

The department is represented at divisional (regional) and local (catchment) levels.

1.4.6 The Shell Agreement

The operational requirements between the department and IFS are found in the 2013 Child Protection and Integrated Family Services State-wide Agreement (Shell Agreement). The purpose of the Shell Agreement is to bring together key partners from IFS and Child Protection to formalise a shared purpose and consistent approach for working together. The Shell Agreement articulates the importance of the relationship between IFS and Child Protection and the significant role of community-based Child Protection in supporting these relationships.

1.5 Past reviews and evaluations

In 2011, the former Department of Human Services commissioned an evaluation of Child FIRST and IFS. The evaluation identified a need to:

• improve coordination activities
• take a stronger statewide approach to demand management.
• improve how IFS and Child Protection work together
• strengthen partnership and governance.

The 2012 Report of the Protecting Victoria's Vulnerable Children Inquiry found the role of Child FIRST to be ill-defined and its governance structure inadequate. The Inquiry also identified that some Child FIRST networks were significantly under-resourced, while others were unable to meet their required service quotas. Recommendations to improve Child FIRST focused on governance—the creation of Area Reference Committees to oversee the monitoring, planning and coordination of services and management of operational issues within catchments—and a more consolidated intake model that would combine Child FIRST and statutory child protection intake processes.

In 2010, the Victorian Ombudsman's Own motion investigation into Child Protection – out of home care found that:

• Child FIRST was experiencing a level of demand that it could not satisfy
• there were variations in the thresholds that applied when Child FIRST was deciding whether to investigate a child or family
• performance measures needed to be more comprehensive and to measure effectiveness.

The Victorian Ombudsman recommended the development of a comprehensive strategy for improving understanding between Child Protection and Child FIRST workers regarding their respective roles and agreed processes.

1.6 Audit objective and scope

The objective of this audit was to determine the effectiveness of community-based child and family services for vulnerable children and families. Specifically we examined whether:

• community-based child and family services are improving outcomes for vulnerable children and families
• vulnerable children and families are able to access community-based child and family services as needed.

This audit focused on the department and its key family services program—Child FIRST and IFS.

The audit did not include other government departments with responsibilities for the provision of early intervention services for children at risk. It also did not examine the effectiveness and efficiency of services provided by community-based and contracted services providers to vulnerable children and families.

1.7 Audit method and cost

The audit methodology included:

• review of relevant documents and data
• interviews with relevant department staff
• site visits and interviews with a sample of community-based family services including executive and operational staff
• attendance at, and discussion with, a selection of Child FIRST and IFS alliance members
• focus groups with:
• Child FIRST and IFS workers
• chief executive officers of alliances
• chairs of alliances.

The audit was conducted in accordance with section 15 of the Audit Act 1994 and the Australian Auditing and Assurance Standards.

Pursuant to section 20(3) of the Audit Act 1994, unless otherwise indicated, any persons named in this report are not the subject of adverse comment or opinion.

The total cost of the audit was $445 000.

1.8 Structure of the report

The report is structured as follows:

• Part 2 examines planning forintegrated family services
• Part 3 examines partnerships and governance
• Part 4 examines performance monitoring and outcomes.

2.1 Introduction

Effective early intervention to support vulnerable children and families requires ongoing and evidence-based service planning at the strategic and local levels. In a devolved service delivery model, the Department of Health & Human Services (the department) is responsible for strategic planning and Child and Family Services Alliances (alliances) are responsible for developing and implementing catchment plans. Catchment planning is one of the three key functions of an alliance.

At both the strategic and alliance levels, effective planning needs to be informed by:

• changes in demand drivers that contribute to vulnerability—factors that are leading to more or different families becoming vulnerable and therefore in need of support services
• gaps in service supply—information about changing demographics and whether services are appropriate and sufficient to meet the populations changing needs
• an understanding of how well resources and processes are organised and prioritised to support the best possible delivery of services to achieve the intended outcomes.

This Part examines whether:

• the department's strategic planning has been based on a sound understanding of demand, and adequately addresses service gaps
• the department has provided adequate leadership and support for the service sector
• the current funding structure accurately reflects the services that are required.

2.2 Conclusion

The department's strategic planning for Child and Family Information, Referral and Support Teams (Child FIRST) and Integrated Family Services (IFS) has been reactive and rudimentary. While the department has made a significant effort to build the capacity of alliances to undertake catchment planning, it has not forecast overall demand for Child FIRST and IFS, assessed unmet or potential demand, or responded to emerging demand drivers in a timely manner. The department lacks the robust evidence base that would enable it to respond to emerging issues effectively at the alliance, divisional and state level. In the face of significant growth in demand for services, IFS are delivering beyond their funded capacity, casting doubt on the sustainability of the current model.

2.3 Understanding demand and supply

The department is aware of the significant increase in the number and complexity of cases referred to Child FIRST and IFS. However, it has not systematically analysed this demand at the alliance or state level to inform its strategic planning. It has not systemically planned for early intervention services that can meet the needs of vulnerable children and families at different stages of their vulnerability.

While acknowledging that the department is currently doing preliminary work on child protection demand drivers and their locations, a system-wide approach and understanding is needed if the department is to be assured that services and resources are appropriate and sufficient.

2.3.1 Understanding and forecasting demand

Demand for Child FIRST and IFS is driven by changes in:

• risk factors that contribute to vulnerability within a family and local community—such as unemployment, family violence, substance abuse, mental and physical disability, social exclusion
• referrals from the Victorian Child Protection Service (Child Protection) which have grown substantially since 2006–07.

Since Child FIRST and IFS were fully implemented in 2008–09, there has been a noticeable increase in the number and complexity of the cases being referred to them—as shown in Figure 2A. While the overall number of referrals to Child FIRST and IFS combined has only increased by 7 per cent, the number of referrals to Child FIRST—as the gateway—has increased significantly. However, we have not been able to gain sufficient assurance that the data in Figure 2A is reliable as the department does not routinely do this type of analysis, and the first data set it provided was inaccurate.

Figure 2A

Increases in demand and complexity of cases referred

(a) A non-substantive case is where less than two hours of service, typically through information and advice, is provided.

(b) Substantive referrals require the establishment of a case and allocation of a case worker.

Source: Victorian Auditor-General's Office based on data provided by the department.

Despite the significant increase in demand, the department has not sufficiently analysed the:

• nature and service outcomes of the non-substantive referrals
• capacity of Child FIRST and IFS to respond to the growing complexity of cases
• implications of not providing early intervention for those who—based on prioritisation of need—do not receive services from Child FIRST and IFS.

Non-substantive referrals

The department collects Child FIRST and IFS data at the service level using the Integrated Reports and Information System. It defines non-substantive referrals as those receiving less than two hours where one or more of the following applies:

• there is insufficient information about the family to officially register the family with a service provider and establish a file
• the family receives a one-off intervention
• there is no comprehensive assessment to identify the family's issues
• the family does not receive more than two staff-hours of services from all staff combined.

Alliances report that when demand is high, they record more referrals as non‑substantive because they have limited capacity to provide anything other than information and advice. Non-substantive referrals do not require the opening and registering of a case in the Integrated Reports and Information System. When demand is high this practice is also considered expedient but creates a risk that the real level and nature of family vulnerability is not being captured. Because no case is opened, the nature of the risk is not identified and potential precursors for escalating risk are not recorded or able to be tracked.

The department has not monitored or analysed data on non-substantive referrals. It does not know whether the families of non-substantive referrals represent unmet demand for early intervention or whether the lack of services being provided at this stage potentially leads to escalation of family issues that increases vulnerability.

Child FIRST is required to prioritise service provision to the most vulnerable families in keeping with the legislative intent of the Children, Youth and Families Act 2005. However, this creates tension for family services, which are also required to fulfil another important legislative mandate—to 'provide a point of entry into an integrated local service network that is readily accessible by families, that allows for early intervention in support of families'.

This gap between the need for early intervention and the alliances' capacity to respond to vulnerable families with lower levels of risk, was identified as early as 2009 when the department evaluated the roll out of Child FIRST and IFS across the state. However, it has not taken steps to understand and address the impact of this gap on the system. While the provision of appropriate child and family services is influenced by available government funding, the department's lack of robust, system-level demand forecasting has limited its ability to provide sound evidence to support government decision‑making.

The department's main focus has been on understanding the demand for Child Protection services. Over time it has gradually improved its demand forecast capability for these services. This has included identifying how different demographic groups respond to Child Protection services, the factors that contribute to growth in demand, and gaining a better understanding of pathways and flows within the Child Protection system.

These developments have been positive and have contributed to a more reliable prediction of demand for Child Protection services, which has been reflected in the department's recent budget submissions. However, it needs to apply a similar methodology to Child FIRST and IFS to improve its demand forecasting to enable an appropriate and sufficient service response.

In late 2014, the department began two pilot projects to examine:

• individuals' likely lifetime contact with Child Protection and associated services, such as family services
• complex client groups and the effective timing of interventions across the whole government service system—in a joint investigation with the Department of Premier and Cabinet as lead agency it is investigating those who use multiple government services.

Both initiatives are likely to inform demand planning, but it is too early for their effectiveness to be assessed.

2.3.2 Managing demand

Regular analysis of service response patterns at alliance and state levels should enable the timely identification of service gaps and capacity shortages. This is critical to effective service planning. While alliances review demand management strategies locally, the department has not identified and evaluated the overall implications of the demand management strategies being implemented by alliance members. Not knowing the impact of these strategies has limited the department's capacity to take a strategic, targeted and proactive approach to planning.

Faced with the challenges of increased demand and complexity of cases coming to Child FIRST and IFS, alliance members have adopted a range of demand management strategies, including:

• reducing the number of attempts to engage with families
• engaging with families for shorter periods of time than they might otherwise have done
• introducing a longer period for 'active holding' of families, during which information and advice is provided while waiting for a case manager to become available. Alliance members advise that prolonged delays in allocating families to services are a key reason for families becoming disengaged.

Figure 2B shows some of the implications for vulnerable families arising from the demand pressures at one local alliance.

Figure 2B

One alliance's approach to managing increased demand

Source: Victorian Auditor-General's Office based on information provided by one alliance member.

Engaging vulnerable families is critical to the success of family services work. Across the state, the increase in the demand and complexity of cases is compromising the quality of engagement and case support.

Between 2008–09 and 2013–14—as a result of demand increases—there has been:

• an increase of 37 per cent in the number of referrals that are diverted or not recorded as a 'case' at intake and therefore closed prior to being assessed or at assessment, before getting the full benefit of family service support
• an overall increase of 75 per cent in the number of cases closed at these early stages due to the family not engaging with the service.

As effective intervention is predicated on successful and meaningful engagement with vulnerable families, this trend should be of particular concern to the department.

The department has not identified whether services are adequately responding to this increase in demand and complexity. Figures 2C and 2D show great variation across the 24 Child FIRST catchments in the proportion of complex cases they responded to between 2008–09 and 2013–14. While most alliances have taken on more complex cases, catchments 2 and 22 have taken on smaller proportions of complex cases and increased the number of cases with no complex issues. The department has not identified or addressed the underlying reasons for these trends and variations or the implications for vulnerable families.

Figure 2C

Cases with no complex issues by alliance catchment (per cent),  2008–09 versus 2013–14

Source: Victorian Auditor-General's Office based on data provided by the department.

In 2013–14, 12 of the 24 alliances delivered more than 30 per cent of all their cases to clients with no complex issues, even though the intent of the program is to prioritise the most vulnerable clients.

Figure 2D

Cases with two or more complex issues by alliance catchment (per cent), 2008–09 versus 2013–14

Source: Victorian Auditor-General's Office based on data provided by the department.

Six alliances delivered less than 30 per cent of their cases to clients with more than two complex issues. This included alliances in areas identified by the department as containing some of the most vulnerable client groups—with high levels of reported domestic violence incidents, Child Protection reports and alcohol and other drug treatment clients. This raises the question of whether these alliances are appropriately targeting their services to the most vulnerable families.

2.4 Planning for diversity

An accurate understanding of family diversity and need is essential for effective local- and system-level planning

2.4.1 Aboriginal and Torres Strait Islander families

The department reports regularly at the alliance and system level on the number of cases involving Aboriginal and Torres Strait Islander families, in response to recommendations from the 2011 Protecting Victoria's Vulnerable Children Inquiry. Departmental data shows that the number of family service cases involving these families has increased by 51 per cent from 1 582 in 2009–10 to 2 388 in 2013–14.

However, the data shows the number of cases, not families. A case is measured as a single episode of support. As multiple cases can relate to one family, the department does not know:

• how many families were involved in these cases
• how many cases were provided to each of these families
• whether some families have presented in the system multiple times
• the level of unmet demand in the Aboriginal and Torres Strait Islander population.

The department should report on the number of Aboriginal and Torres Strait Islander families receiving family service support as well as the number of cases to improve its understanding of need and trends.

2.4.2 Culturally and linguistically diverse groups

Currently there is no monitoring or reporting requirement for culturally and linguistically diverse (CALD) family access to Child FIRST and IFS. Across the system, the proportion of families registered by Child FIRST in 2013–14 who were born overseas and who do not speak English well, or at all, is relatively small at 6.8 per cent. However, there has been noticeable population growth in CALD families across the state, especially in the growth corridors in the south-east and north-west metropolitan areas. For example, in the City of Greater Dandenong 60 per cent of the population was born overseas and 55 per cent are from nations where English is not the main language. The department needs to examine whether the very low representation in Child FIRST reflects a lack of awareness of services by these groups, a lack of access or a lack of appropriate cultural competency by the alliances.

Refining the measurement and reporting on Aboriginal and Torres Strait Islander and CALD family access will assist the department's planning in this regard.

2.5 Planning to support the service sector

The department has provided guidance material to support the service sector—such as promoting a common approach to needs and risk assessment, and providing professional development through the Office of Professional Practice. It has also undertaken initiatives—such as Cradle to Kinder and Early Childhood Development workers—to target specific groups of vulnerable children during their early years.

However, it has been slow to address the implications of changing demand trends on service providers. For example, as shown in Figure 2E, between 2008–09 and 2013–14, there was a dramatic rise in the number of family violence referrals to Child FIRST and IFS—an increase of around 52 per cent overall, or an average of around 9 per cent per year. Referrals associated with intellectual disability, substance abuse and mental health issues also grew in this period by 49 per cent, 36 per cent and 16 per cent respectively. Referrals of families previously involved with Child Protection grew by 22 per cent. All of these changes have affected the capacity of Child FIRST and IFS to effectively meet the needs of these vulnerable families.

Figure 2E

Number of new cases opened by recorded issue

Source: Victorian Auditor-General's Office based on data provided by the department.

Recognising the significance of family violence as a key driver of vulnerability in families, the department has led a statewide approach to responding to family violence and has developed appropriate risk assessment tools. Figure 2F provides two examples where the department has effectively supported the sector.

Figure 2F

Sector support materials for family violence

Source: Victorian Auditor-General's Office based on information provided by the department.

Despite these initiatives, the department's response has not been timely in terms of sector need. For example, in October 2010, Victoria Police reviewed its code of practice which led to an unprecedented increase in referrals for family violence being made to family violence specialists, Child Protection and Child FIRST. One alliance member reported receiving an average of over 15 reports a month from Victoria Police between 2011 and 2013, compared to two or three a month prior to 2010. This increase was accompanied by uncertainty, inconsistent practices and the duplication of effort by the multiple service providers that were receiving reports. This alliance reports that the average number of family violence referrals has increased to 22 a month in 2014 and 35 a month in 2015.

While Victoria Police did not intend to change the established referral pathways, the respective roles and responsibilities of Child Protection and Child FIRST became unclear. The department worked with Victoria Police to clarify and develop a common approach to referral pathways and later engaged with a small group of representatives from the service sector. It released a family violence referral protocol in May 2013 with the agreement of the sector. However, it took the department over 2.5 years to fully implement this change at a time when alliance members were struggling to cope with the high numbers of family violence-related referrals.

2.6 Funding to support service delivery

Currently, the demand for Child FIRST and IFS exceeds the funding provided to the sector. This has been the case since 2011–12. In the face of significant growth in demand for services, Child FIRST and IFS alliances are delivering beyond their funded capacity. The department needs to investigate whether this 'over-performance' is due to increased efficiency or alliance self-funding.

2.6.1 Service delivery funding

The department funds providers of Child FIRST and IFS according to agreed targets based on the number of service hours and cases delivered. Cases are classified as follows:

• entry level—up to 10 hours
• short response—between 10 to 40 hours
• long response—from 40 to 110 hours.

In 2013–14, just over $88 million was given to Child FIRST and IFS providers to deliver 33 527 cases of family support. This funding includes around $4 million for alliance facilitation and development. In 2013–14, the department's funding model allowed for 923 539 hours of services.

Adequacy of the funding model

The funding model no longer reflects what services are being delivered on the ground, or for how long. Most notably, in 2013–14, the department's data on actual cases delivered showed that Child FIRST—funded only as an intake gateway to IFS—delivered:

• short responses to 4 619 families
• long responses to 239 families
• services extending beyond 110 hours to eight families.

By contrast, IFS provided fewer entry level responses than funded, yet significantly more extended responses—which were not originally planned for when the program was first established.

The department has not analysed this data to gain a better understanding of why Child FIRST is going beyond its entry level role to take on so many short and long responses. The department advises that at least some of these cases may be the result of inaccurate recording but the extent of this is not known.

Between 2007–08 and 2013–14, the department increased funding to Child FIRST by 217 per cent, but decreased case targets to IFS by 2 per cent in the same period. The funding has not supported the progression of cases beyond Child FIRST to an IFS response. Insufficient IFS capacity means it is likely that some families who may be willing to engage with IFS are not getting beyond the intake gateway. This is counter to the intent of the Child FIRST role as originally conceived. The department has not analysed the data to determine whether there is a bottleneck at Child FIRST.

Actual cases and hours of service delivery

In 2013–14, while the department funded for 33 527 cases, alliances only delivered 32 346 cases. However, what was delivered used more hours, as evidenced by the number of cases exceeding 110 hours (2 071 cases). Service delivery above 110 hours was not budgeted for by the department. The department's funding approach does not reflect the changing reality of service demand. The department advises it is developing a new funding category of 200 hours in recognition of the extended time being taken to support vulnerable families.

In 2013–14, alliances delivered 982 441 hours of service while the department funded for a total of 923 539 hours. This means that providers self-funded more than 58 000 hours of service delivery.

Based on the 2013–14 unit price of $90.28 per hour, providers of Child FIRST and IFS have delivered an additional $5.3 million worth of services above the funding provided by the department. The department does not know if this is because of service provider inefficiency, inaccurate reporting or because of a strong ethical commitment to support families over and above funded capacity.

'Over performance' of IFS providers

The department does not assess the reasons for 'over-performance' or whether the reported performance is accurate or sustainable. This means the department has accepted the additional hours provided by alliance members without seeking a comprehensive picture of what is really happening. The department should be asking questions:

• Are these hours the product of considerable goodwill on the part of service providers that is not being recognised?
• Could service providers be using their time with families more efficiently and effectively?
• Are hours being reported accurately?
• Above all, are vulnerable families receiving the level and kind of support that they need?

The department needs to undertake a comprehensive and urgent review of its funding model, including the assumptions that underpin this model and the outcomes it is intended to deliver.

2.6.2 Alliance activity funding

In 2013–14, just over $6 million was provided to facilitate and develop Child FIRST alliances. This included:

• $2.7 million of program development funding—provided to all department‑funded alliance members at 3 per cent on top of service delivery funding per member, andintended to support member participation in alliance activities
• just under $1.4 million of alliance facilitation funding—provided to the designated lead member of each alliance, which may be used to supportcatchment planning, demand management, or to employ a project manager within the alliance
• around $2 million—provided to employ EarlyChildhood Development workers.

There is no requirement for alliance members to report how they use this funding. The department has advised that program and facilitation funding is recorded only by the department's local areas. The department's financial systems, which report based on the 17 departmental areas, do not provide information about funding at a catchment level. This makes it difficult for the department to gain a clear picture of whether funding at an alliance level is adequate for intended activities.

We analysed the self-reported funding activities of five lead organisations in different catchment areas and found that:

• It is common for the lead organisations to cross-subsidise their Child FIRST function using other state-funded program resources—in agreement with the department—including funding allocated to IFS. This includes IFS staff taking on referral intake and assessments, and the allocation of IFS staff to Child FIRST.
• Program management funding received from the department only partially covers the program management role for the lead members of alliances. Service provider-generated funding isused to subsidise this function, which providers considered to be unsustainable.
• These lead members have also contributed to program administration from their own funding.
• Two of the five service providers used local government funding to support their work with children from vulnerable families. Most of the five have used short-term responses to accommodate demand and limited resources.
• Current funding and service agreements no longer align with the services delivered. As cases become more complex, alliances report it is often not possible to close a case within 110 hours. The department advises that it is now working on a 200-hour funding category although it has been aware of the increase in complexity for the past six years.
• Significant variation in case complexity between alliances also indicates some areas may be over-serviced while others may be under-serviced.The department needs to undertake a system-level review of its funding structure. While it needs to enable local providers to make decisions about funding expenditure that reflect local priorities, it should nevertheless monitor how funds are being used and whether current performance in relation to set targets is sustainable.

2.6.3 Support for catchment planning

Catchment planning by alliances is aimed at achieving:

• a more integrated and coordinated service system
• earlier identification and intervention to support vulnerable families
• streamlined referral pathways among local community-based organisations involved in child and family services.

The department has made a significant effort to build the capacity of alliances to undertake catchment planning, including:

• holding workshops focusing on planning processes, the use of planning data and issues arising from catchment planning
• developing a catchment planning reference guide and template—released in July2009 and updated in August 2012.

However, effective catchment planning needs to be informed by reliable, accurate data on population trends and the associated demand for services. This includes Child Protection data as Child Protection notifications are a common demand driver for IFS. While issues to do with a lack of Child Protection data for catchment planning purposes were identified by alliances in discussions with the department—and the provision of such data centrally was made available to alliances in 2008–09—it was not until after 2013 that the department made data available to alliances statewide.

Recommendations

That the Department of Health & Human Services takes the key shortfalls identified in this report as the starting point for a comprehensive and urgent review of its current approach to early intervention and, as part of this review:

1. improves planning by better demand forecasting and more systematic analysis of existing program performance data—including analysis of the level and nature of non-substantive referrals—to understand gaps in service response
2. develops a regular statewide engagement mechanism to identify issues and risks in a timely manner and to design solutions with the input of the service sector
3. provides targeted training to service providers in catchment planning and data analysis
4. reviews its whole-of-system funding for early intervention to better reflect the impact of demand drivers on Child and Family Information, Referral and Support Teams and Integrated Family Services.

3.1 Introduction

Child and Family Information, Referral and Support Teams (Child FIRST) and Integrated Family Services (IFS) rely on multiple community organisations to deliver services. Effective partnerships and governance are critical to the coordinated and efficient delivery of these services.

The 2011 Protecting Victoria's Vulnerable Children Inquiry identified partnership and governance issues as inhibiting factors for integrated service delivery in the early stages of the Child FIRST and IFS rollout in 2009.

This Part examines whether the Department of Health & Human Services (the department) has established clear strategies and guidelines to enable the development of effective partnerships and governance.

3.2 Conclusion

Currently there is great variability in the level of coordination and the maturity of the Child and Family Services Alliances (alliance) across the 24 catchment areas. The department could be doing more to support the development of these partnerships.

Weak partnerships and inadequate governance arrangements have impeded the delivery of integrated and coordinated Child FIRST and IFS. Although the department has worked towards building effective alliances and strong governance over the years, there are still issues to do with a lack of clarity around roles and responsibilities, and inadequate communication between IFS providers and the department's staff locally in the catchment areas, divisionally and centrally.

3.3 Developing effective partnerships

3.3.1 Characteristics of mature alliances

We found examples of mature relationships between alliance members and effective alliances across the state—as shown in Figure 3A. However, many alliances function more as collections of individual members. In these alliances there is limited data and resource sharing, or transparency around funding capacity. Collaboration does not appear to be embedded in the organisational system and structure of member organisations.

Figure 3A

Characteristics of better practice in one alliance

Source: Victorian Auditor-General's Office based on information provided by one alliance.

3.3.2 The need to better support alliances

The department did not address the critical prerequisite of developing and facilitating the partnerships of alliance members when it rolled out Child FIRST and IFS. This was a significant strategic oversight.

During the department's catchment planning workshops in 2008, 2009 and 2010, alliance members consistently identified the need to strengthen partnership and governance arrangements within alliances.

In 2008, the department commissioned a three-year evaluation of the statewide implementation of Child FIRST and IFS. The consultant's 2009 interim and 2011 final reports both concluded that there was a critical need to strengthen the sustainability of alliance partnerships. The 2011 report proposed that the department should:

• provide the alliances with tools and resources to monitor the 'health' or soundness and sustainability of their partnerships
• enable effective management of leadership succession
• develop stronger accountability for partnership performance
• provide resources for an alliance project officer across all catchments.

In 2012, the department developed its Strengthening partnership strategy in response to the 2009 Ombudsman Victoria report Own motion investigation into the Department of Human Services Child Protection Program. It requested that alliances use this strategy to examine the health of their relationships and to identify problems and strengths. The department now requires this partnership 'health-check' process to be embedded as part of annual catchment planning activities. This kind of self-help is a positive step supporting the growth of stronger partnerships.

3.3.3 A continuum of partnership development

VicHealth's The partnerships analysis tool identifies the critical stages of a partnership. Figure 3B shows this continuum, with collaborating being the most advanced form of partnership.

Figure 3B

Stages of partnership

Source: Victorian Auditor-General's Office based on VicHealth's The partnerships analysis tool.

Based on our survey of alliances and review of relevant documents at a number of alliances—such as memorandums of understanding (MoU), demand management strategies and catchment planning documents—we found many alliances still at the 'coordinating' stage. Members participate in alliance activities as individual organisations within their funded capacity to take on cases referred to them, rather than working collaboratively for a common purpose. Most alliance members only share demand information and case allocations with each other. Sometimes assessments are shared. Few share their funding and case capacity.

There are a small number of alliances with advanced partnerships and strong performance in service delivery. Figure 3A shows the characteristics of an effective alliance performing at the 'collaborating' stage of partnership.

Figure 3C shows how client outcomes improved as a result of integrated service delivery. The family described has complex issues, with both parents having learning difficulties and the children having been diagnosed with learning difficulties, disability and developmental delay.

Figure 3C

An example of integrated service delivery

Source: Victorian Auditor-General's Office based on a de-identified case of one vulnerable family.

3.3.4 Governance arrangements

The department set up alliances by bringing together Child FIRST and IFS providers to undertake catchment planning together and apply demand management strategies to prioritise local needs. While these are positive steps, the department has not established effective governance mechanisms at the alliance, divisional and state levels. This means there is great variability across alliances in the degree of coordination that occurs in catchment planning and the day‑to-day operational and strategic management of service provision.

Despite efforts by the department over the years, attaining effective governance remains challenging as a result of weak partnerships at some alliances, constrained resourcing, and ineffective communications between and across levels of the department and the IFS providers.

At the alliance level, governance is managed through partnership agreements or an MoU between members. A fully developed local agreement includes the key elements listed in Figure 3D. The level of completeness and clarity of MoUs varies across the different alliances we examined. In practice, alliance members report that there are also significant differences in whether alliance members are held accountable to each other as set out in the MoUs.

Figure 3D

Key elements of a formal agreement between members of an alliance

Source: Victorian Auditor-General's Office based on a review of IPS' MoUs.

Child Protection and Integrated Family Services

Although the department has attempted to clarify the relationships and transactions between Child Protection and IFS, this is an ongoing issue.

Governance arrangements between Child Protection and IFS are articulated in the Child Protection and Integrated Family Services State-wide Agreement commonly referred to as the 'Shell Agreement'. This agreement consists of core statewide components with local variations. In particular, the Shell Agreement defines key roles and responsibilities and transactions between Child Protection, Child FIRST and IFS. This includes setting out the role and responsibilities of the department's Community Based Child Protection Practitioners (CBCPP).

Figure 3E outlines the key functions of CBCPPs. As the work of CBCPPs must be prioritised in the context of the overall demand for child protection, CBCPPs are not always available to support Child FIRST and IFS workers in responding to families whose vulnerability is assessed as less severe.

Figure 3E

Functions of Community Based Child Protection Practitioners

Source: Victorian Auditor-General's Office based on Department of Health & Human Services information.

As CBCPPs have a significant role in fostering positive working relationships and supporting the service delivery of IFS in catchments, it is important that community‑based child protection builds a strong profile and has a firm presence in the catchment, and that it is accessible to both Child Protection workers and IFS.

However, alliance members report that the role of the CBCPP has been diluted at the community level as a result of the high demand on Child Protection and the fact that the work of CBCPPs must be prioritised in the context of overall demand on the Child Protection program.

Service providers report significant issues in practice:

• CBCPPs are increasingly being taken away to take on Child Protection cases rather than being available to support Child FIRST and IFS with case referrals and risk assessment.
• CBCPPs are making referrals to IFS without providing adequate consultation. Although CBCPPs are not required to undertake a joint visit to family with the IFS providers, IFS providers consider it to be helpful in engaging the family referred by Child Protection.
• CBCPPs report that IFS providers are reluctant to 'hold the risk' and do not have the required capacity or confidence to deal with families that have complex needs.

The department has updated and reviewed the Shell Agreement in 2009 and 2010. Its latest review commenced in 2013 and has yet to be finalised. Although each review has sought to clarify the roles and responsibilities and pathways between Child Protection, Child FIRST and IFS, the documented arrangements are continually being tested because of changes in the external environment—such as the significant increase in demand and complexity of cases which have affected the whole integrated system of family services.

The department's role in alliance and family service providers

The department is represented in each of the alliances but there is considerable variation in how this role has been performed. For example, in one alliance, a department's representative attends meetings regularly, advocates on behalf of the alliance to the department, and proactively supports alliance members. In another alliance the department's representative rarely comes to alliance meetings and has not acted as an effective conduit between divisional and state level systems.

Of the 14 alliances that responded to our audit survey, only five indicated a positive working relationship with the department's divisional staff. The remaining alliances reported limited commitment and availability from the department's representatives. The effectiveness of this role appears to be dependent on individual capabilities rather than based on clearly articulated and closely monitored requirements.

Family service providers and universal and specialist providers

There is no consistent approach to managing the relationships between alliance members and community service organisations who are in a catchment but not an alliance, and who are not department funded. The department's 2007 A Strategic Framework for Family Services envisaged that local alliances would gradually build stronger links with these other service providers to provide a continuum of service responses. The degree to which this is being done in practice depends on the local relationships forged by alliance members with these external providers.

3.3.5 Effective outcomes-based planning and communications

Engagement by the department with the sector across the different levels—local, regional and state—is currently insufficient and inconsistent.

There is little understanding or timely reporting of how the system is functioning at the department's divisional and central levels, with the exception of reporting on the number of cases delivered against targets in the service agreements. While there have been department-led opportunities to exchange issues, information and better practice at the state level, alliance members report an increasing gap between the department's operations at the divisional and central level, particularly since the department's restructure in late 2012.

Currently the department's central office distributes policy or strategy-related information through the divisional staff who subsequently disseminate it to the IFS provider staff at catchment levels, resulting in delayed communication. In addition, the department disseminates emerging policy initiatives and discusses sector-wide issues with alliances' representatives through quarterly forums hosted by an independent research centre contracted to the department. But these discussions have not been outcomes-focused. Further, representations from the sector in these discussions have tended to be skewed towards larger providers, meaning fewer communication opportunities for smaller providers.

Currently, there is no single portal where the kind of information needed to support IFS providers can be easily located and accessed. Having a 'one-stop' shop for such a wide range of information—such as strategic planning, professional practice guidelines and service provider's good practices examples—would improve the effectiveness and efficiency of the department's information distribution.

3.4 Future directions

At the time of this audit, new initiatives had been developed, such as the piloting of the 'Services Connect' model which offers a more integrated approach to service delivery with one key worker rather than multiple service provider representatives. In May 2014, the department launched Children and Youth Area Partnerships (area partnerships). This initiative is one of the recommendations of the 2012 Victoria's Vulnerable Children's Report. The aim of the area partnerships is to bring together relevant government departments across health, education and justice, Victoria Police, local government and the community sector to improve outcomes for vulnerable children and young people. The area partnerships provide a sound platform from which to build more coordinated service delivery.

These initiatives aim to transform how services are delivered. However, this critically important goal of more integrated and coordinated service provision will only be realised when there are strong collaborative partnerships, appropriate and sustained levels of resourcing to enable partnerships to work together, and strong and clear governance supported by effective outcomes-based communications across all partners at all levels.

Recommendations

That the Department of Health &Human Services takes the key shortfallsidentified in this report as the starting point for a comprehensive and urgent review of its current approach to early intervention and, as part of this review:

1. provides targeted support to those Child and Family Services Alliance members whose partnerships are still underdeveloped, and supports them to become more collaborative in their interactions
2. investigates and implements ways of improving the effectiveness of its communications about operational and strategic issues between and across the department centrally, regionally and locally, and with community service organisations.

4.1 Introduction

Performance monitoring and reporting is a key element of effective governance and public sector accountability. Effective performance monitoring should identify emerging risks to successful delivery of core functions, including contractual obligations. Measuring outcomes is a key driver of continuous improvement. Such monitoring of performance and assessment of outcomes needs to occur at the Child and Family Services Alliances (alliance) or catchment level, the divisional level of the Department of Health & Human Services (the department) and the state level.

We examined whether the department has effectively monitored the performance of Child and Family Information, Referral and Support Teams (Child FIRST) and Integrated Family Services (IFS) and whether these services are improving outcomes for vulnerable children and families.

4.2 Conclusion

The department monitors the quality of Child FIRST and IFS and whether alliance members have met agreed targets, but does not monitor whether alliances are performing effectively. It has also not systematically assessed whether the outcomes for vulnerable children and their families have improved as a result of an integrated system of family services.

In monitoring performance, the department focuses on outputs rather than requiring service providers to show positive outcomes for families. Significant limitations in the current data system and a lack of meaningful, system-level analysis of service data mean that the department does not adequately understand whether funded IFS providers are delivering their services effectively.

Although there are isolated examples of IFS providers attempting to evaluate their service effectiveness, the department cannot show whether Child FIRST and IFS are effective in improving outcomes for vulnerable children and families as it does not capture meaningful outcomes data at the level of the individual family, IFS provider or state levels.

4.3 Performance monitoring

The department's performance monitoring is related mainly to whether Child FIRST and IFS alliance members are meeting their contractual obligations. Its current monitoring arrangements do not offer any insight into how local alliances—implementing a single intake and referral model with the support of a number of family services—are performing in terms of providing an integrated service network to support early interventions. This is because it does not measure this.

4.3.1 Monitoring contractual obligations

The department has developed a statewide monitoring framework that sets out an overarching set of policies, guidelines and tools to support its staff to monitor IFS provider compliance with standards and contract requirements. Under this framework, IFS providers are subject to a number of quality assurance processes, including:

• Initial demonstration of compliance with the department's standards prior to registration or renewal of registration every three years with the secretary. This performance is independently reviewed by auditors accredited by the department, and covers program management, financial sustainability, policies and procedures, case management and stakeholder feedback. Interim audits are conducted after 18 months to check if providers are still compliant with standards.
• Ongoing monitoring processes—conducted by departmental staff at the local catchment and divisional levels—to enable the effective management of service provider contracts. Such monitoring typically examines performance against core outputs contained in the service agreements. The department conducts desktop reviews of the funded organisations at least every 12 months around key risk areas, but does not generally examine any specific program area unless an issue has been identified.

The department's monitoring framework is based on risk management principles. The contractual performance of service providers involved in delivering Child FIRST and IFS—including the number of cases delivered and service hours—is regularly monitored and reported to the department's divisional management.

However, there is no ongoing performance monitoring of alliances to assess how they are performing against the intended objectives of Child FIRST and IFS or if they are identifying emerging risks.

4.4 Monitoring and reporting on outcomes

The department's key strategic policy—A strategic framework for family services 2007 (the Framework)—states that 'family services will be outcomes focused in their service delivery and practice approaches' as one of its nine family service principles. The Framework identifies the desired outcome for children to be 'safe, healthy, and continuing to develop, learn and achieve wellbeing' and outlines a plan to develop measures that will allow the monitoring of outcomes at the level of the individual family that has been referred, IFS provider who is managing the referral, or at the statewide population levels.

Although the Framework sets clear expectations for performance and outcome monitoring—accompanied by a well-articulated implementation plan—the department has still not implemented the outcomes component of the plan seven years after the Framework was released.

4.4.1 Limitations in data and analytical capabilities

The department needs to improve its oversight of the quality of its data collection, particularly for critical data items.

Child FIRST and IFS providers record operational information in the department's statewide Integrated Reports and Information System (IRIS). This information includes:

• number of referrals
• number of service hours in various activity categories
• results of needs assessments
• intake and referral decisions
• reasons for case closure.

Service providers export de-identified data to the department quarterly. IRIS allows the department's central office to prepare data tables for use by local staff at the catchment and divisional levels in their interactions with IFS staff. These data tables include data on:

• trends relating to new referrals
• levels of complexity of cases
• number of cases provided to Aboriginal or Torres Strait Islander families
• reasons for case closure.

However, these tables are of limited usefulness as they:

• are not presented in a format that allows easy identification of trends over time
• do not compare the performance of an alliance against statewide performance or against different alliances—the department advises that IFS providers are sensitive about sharing their specific performance issues, however, this comparison of performance and identification of trends could be achieved by using de-identified data.

Further, because there is a delay of three months between when data is entered and when IFS providers upload the local IRIS data for the department, IFS providers do not rely on these tables to discuss performance issues. Some IFS providers report that they have little confidence in IRIS and prefer using their local databases.

Inconsistently collected data is a waste of resources and a lost opportunity. Two such examples are provided in Figure 4A.

Figure 4A

IRIS data quality issues

Source: Victorian Auditor-General's Office based on the department's information.

4.4.2 Tracking client outcomes

Overall the department does not have a systematic and comprehensive approach to monitoring service outcomes. In 2012, it commissioned an evaluation of the outcomes on families who received support from registered family services as a result of a referral. However, the survey did not assess the particular impact of the Child FIRST and IFS system on families and does not provide a sufficient evidence base to inform department decision-making.

Case outcomes

The department does not track the timeliness of service provision or the effectiveness of services provided. The department acknowledges that IRIS is not designed as a case management system and that it has limited capacity to track clients and monitor client outcomes.

At the alliance level, IRIS can report on the completion of goals recorded by clients when cases are closed. However, alliance members report that local practice varies greatly in relation to how staff members interpret and record data on goal completion, which makes this measure of effectiveness even more limited.

There are some examples where local alliances have attempted to measure the effectiveness of their services, as shown in Figure 4B. This is a positive step towards gaining assurance that the services provided to vulnerable families are meeting need.

Figure 4B

Outcomes for families through before and after evaluation

Source: Victorian Auditor-General's Office based on information from an IFS provider.

Timely engagement with vulnerable families is critical in providing effective support. However, IFS can only offer services when the family agrees to participate, as these service providers do not have the same statutory powers that Victorian Child Protection Service (Child Protection) practitioners have under legislation. This means engagement with families facing issues can be challenging for providers.

Figure 4C illustrates the unique challenges associated with engaging and supporting families with multiple and complex needs and a history of previous interaction with Child Protection. The case study illustrates a typical dilemma confronting IFS providers. Although progress has been made with this particular vulnerable family, it has come at a high cost. When the case study was compiled, a total of 367 hours had been provided, which is more than three times the 'long' service time of 110 hours specified in the service agreement. The IFS provider anticipated that the case would remain open until February 2015, representing a two-year period of intervention.

The significant investment in this family comes at the expense of being able to provide services to other vulnerable families.

Figure 4C

Engagement challenge with a family of multiple and complex issues

Source: Victorian Auditor-General's Office based on a real life example provided by one IFS provider.

Re-reports and re-referrals

Re-reports occur when a child or family has been reported on more than one occasion to Child Protection. When the report to Child Protection is investigated and the risk is substantiated, the report becomes a substantiated report.

Re-referrals occur when a child or family engages with Child FIRST and IFS providers multiple times.

While the department monitors re-reports and re-substantiations at the local and statewide levels, it has not monitored the nature and causes of re-referrals at the divisional or state level.

As a proxy for effective system-level monitoring, it might be assumed that a low level of re-reports of clients who have received support from Child FIRST and IFS means that the system has been reasonably successful in diverting clients from further immersion in the Child Protection system. However, more detailed analysis of client-based outcomes is needed to understand whether families have merely resolved crises or have improved their parenting and support capability. The department does not monitor this.

A family who is referred to Child FIRST and IFS multiple times is not necessarily indicative of a poor intervention result. It also depends on the nature of the underlying risks, as shown in Figure 4D. IFS providers believe some families need episodic care over time and their re-referrals in Child FIRST and IFS shows that the system can serve as a buffer and prevent their vulnerability from becoming more entrenched.

Figure 4D

Providing episodic care

Source: Victorian Auditor-General's Office based on a de-identified real life example provided by one IFS provider.

Recent department budget submissions included data that showed the level of re‑reports of Child FIRST and IFS clients to Child Protection. The department indicates that tracking such clients requires considerable analytical resources and is not done regularly or at a system level. This inhibits its capacity to present a robust case for further funding or resources.

At our request, the department generated data on the number of re-reports to Child Protection within a year of these clients being referred to Child FIRST and IFS for the period between 2008–09 and 2013–14. The department also analysed the level of re‑referral of clients within Child FIRST and IFS between July 2006 and June 2014.

This analysis shows that:

• The proportion of families referred to Child FIRST from Child Protection who were subsequently re-reported to Child Protection within one year was around 18 per cent, 20 per cent, 20 per cent, 18 per cent, 16 per cent and 17 per cent of the total families referred from Child Protection, respectively, for each yearly period between 2008–09 and 2013–14. This equates to 325 families in 2008–09 and 626 families in 2013–14. The department advises that these are likely underestimates as data tracking cannot be completed for those records that have incomplete birth date data.
• Around 10 to 15 per cent of families referred to Child FIRST would be linked to a subsequent Child Protection report within 12 months. This rate was 15 per cent in 2008–09 (1 054 families), 12 per cent for 2009–10, 2010–11 2011–12 and down to 10 per cent in 2013–14 (1 092 families). This reduction in re-referral rates appears to suggest that family services have become better at supporting vulnerable families.
• Our analysis of data on re-referrals shows that about 25 per cent of clients were referred to Child FIRST and IFS more than once, out of a total of 74 750 clients with records across the eight-year period.
• The majority of clients with multiple contacts with Child FIRST and IFS—91.7 per cent—have up to three re‑referrals.
• More than half of the clients who had contact with Child FIRST and IFS up to three times had their first re-referral within six months of their case closure. This rate was 56 per cent, 59 per cent and 63 per cent for those who were re‑referred once, twice and three times, respectively.

The department has not undertaken further analysis to understand the reasons why families are being processed repeatedly through the Child FIRST and IFS system or to identify different service responses for these re-referred groups.

Recommendations

That the Department of Health & Human Services takes the key shortfalls identified in this report as the starting point for a comprehensive and urgent review of its current approach to early intervention and, as part of this review:

1. provides explicit requirements for its local and divisional staff regarding the monitoring of operational risks, emerging issues, and the capacity and capability of the partnerships involved in the local Child and Family Services Alliances
2. develops a set of standard analytical data sets for the Child and Family Services Alliances to use to monitor and report on the timeliness and effectiveness of their engagement with clients at the program level, including rates of failure to engage, referral outcomes, re-referrals and re-reports
3. undertakes statewide performance analysis using catchment data to facilitate sharing of practices among Child and Family Services Alliance members.