Contents

1. Numbers are people too Using data for decision making in a community services settingBen Smith, Robert Johnston, Owen Griffiths

2. Contents NSW Family and Community Services – What we do: purpose and interventions Evolution of data for decision making in Family and Community Services Client population forecasting Statistical analysis and hypothesis testing Evaluation, statistical modelling and economic analysis Demonstrating the success of our programs: economic analysis Development and analysis of Key Performance Indicators Demand modelling for NGO transition planning 2

3. NSW Family and Community Services purpose and interventions Our purpose is to protect children and young people most at risk of abuse and neglect and its effect Assessed Level of Intervention B. Safe or Safe with Plan and High or Very High Risk A. Safe or Safe with Plan and Low or Moderate Risk C. Unsafe at any Risk level Statutory intervention (e.g. court/case planning) Statutory intervention Universal Services. Child Youth and Family Support Services Brighter Futures Strengthening families FPIS/IFP/IFS/ IFBS* Out-of-Home Care (OOHC)/ Restoration Intervention pathways Early intervention and preservation Out-of-Home Care *FPIS = Family Preservation/Intensive Support Model, IFP = Intensive Family Preservation, IFS = Intensive Family Services, IFBS = Aboriginal Intensive Family Based Services

4. Evidence based decision making through timely information from our Business Intelligence System • The Corporate Information Warehouse (CIW) is a data warehouse • CIW went live in 2005 • Holds data from a number of sources and integrates information • Provides online access to information from desktops around NSW • Information is updated daily • Produces Management and Compliance reports • Produces reports for Annual, Quarterly and National reporting

5. Over time, the CIW expanded in scope and capability to deliver greater value. Insight-driven action • Capability provided beyond standard counts and measures (simulations, forecasting, etc) • Incorporation of external data sources to aid trend analysis and predictive modeling Today We are Here Organisation-wide capability • Potential for CIW to enable savings and improve efficiency across the organisation • Recognises issues exist across business functions Value Delivered • Leveraging the core information to provide value added insight • Discussions surrounding data integration and common needs begin to appear • Subject matter experts define priorities Growth Core Delivery • Focused on corporate reporting • Replaces manual or ineffective reporting • Proves the CIW concept Time

6. CIW is available to regional positions & to the Executives at the Head Office Regional Director 7 positions All Regions & CSCs trained Director Practice Standards 7 positions Director Child and Family 15 positions Regional Business Managers 7 positions Manager Client Services 76 positions Regional Executive Officers 7 positions Casework Specialist 76 positions Manager Casework 400 positions Max 6 per Mgr Client Service Service Support Manager 76 positions Caseworker >2100 positions Max 6 per Mgr Casework All Regions with this role has been trained ~500 Roles targeted within the region (and included in Pilot Training ~430 Currently not in scope for CIW – need additional licenses to cover No of Regions : 7 No of CSCs: 82 Now 12 months

7. COMS KiDS CIW OOHC MDS BF MDS Partner Portal CIW is a central component of Family and Community Services Software Solution Map FACS Internal Users FACS Internal Users FACS Internal Users FACS Internal Users (Management) (Policy and Planning) (COMS and MDS) (Client Service Delivery) Enterprise Applications Architecture SAP Finance CWU and other agencies (data matching) & FACS Internal Users (Finance and HR) SAP HR SAP Siebel BSDP Internal “Back Office” systems - Client facing systems (Federal)

8. Senior Executive dashboard – High level indicators, updated and analysed monthly, Executive and Minister briefed on arising issues • Full dashboard currently contains 8 KPIs. • All indicators are drillable to Summary and Trend levels

9. Indicator Dashboards - High level information is linked to “summary” reports by region and to “detail” reports at the child/record level

10. Data challenges • Data quality and collection compliance issues • Required information not collected • Lack of source system linkages (Master Data Management) • Insufficient data granularity • Lack of structured data • Complex counting rules used to overcome data quality issues

11. Our Continued Goal for the future …………To provide high quality information that meets the needs of all major stakeholders in their pursuit of delivering better services and improving performance.

12. Using the Data • Client population forecasting • Statistical analysis • Program and project evaluation • Cost benefit analysis • Performance indicators

13. Client population forecasting using simulation modelling • FACS has developed a ‘simulation’ model that projects 10 years demand for • child protection • early intervention and placement prevention (EIPP) • population for Out of Home Care (OOHC). • Evaluate / measure the longer term impacts of new policies and services aimed at improving client outcomes. • Assist policy development and matching future services to clients’ needs. 13

14. Client population forecasting using simulation modelling, cont’d The simulation model can be used to determine the demands / needs for the services: • Each client’s characteristics drive its pathways, including re-entry • Client characteristics also drive the type, intensity and duration of service it receives • Services can change client pathways • Each client’s characteristics also govern how long they stay in OOHC. The results of the simulation model impact on potential policy development and decisions: • Higher service intensity and duration means higher costs or fewer clients served • Affects outcomes and re-entry • Where budgets are fixed, increasing one service can be offset by reducing another • The model projects the trade-offs, allowing decision-makers to make informed policy choices. 14

15. Client population forecasting using simulation modelling, cont’d EIPP and CP service models Scenarios policy demand practice procedure budget Budget / unit costs = capacity Simulation Model (client movements) Future CP and EIPP (un/met) demand OOHC Services + Expenditure Demand, operational and cost implications of changes in one part of the system for the other parts of the system and for different client groups. 15

16. Demand modelling for NGO transition planning, cont’d As illustrated in the graph, as at 30 June 2011, the percentage of CYP in Statutory care provided by FACS is approximately 70%, while NGO provides the remaining 30%. The results of the demand modelling indicate that FACS’ provision of Statutory care will reduce to approximately 10% in four years, and a corresponding 90% of services will be provided by NGOs. In 10 years, it is expected that 100% of Statutory Care services will be provided by NGOs. 16

17. Conceptual Overview – NGO OOHC Service Levels Forecasting • Forecast the number of additional NGO placements needed to achieve the targets under the Transition Plan over the coming years. Estimated OOHC Service Levels Projected Capacity Increase Total Children & Young People Projected capacity increase per year +FY16 growth Transfers from FACS to NGO sector Entry and exit patterns +FY15 growth + + +FY14 growth Restoration rates Entries to NGO sector + +FY13 growth Adoption rates + +FY12 growth + Baseline NGO current contracted Baseline contracted including ICA conversions FY12 Placement prevention rates ICA conversions 17

18. Statistical analysis • 1. Operational improvements • Child and young person concern reports to FACS-CS increased by 324% from 1999/00 to 2008/09, and then plateaued following a change in criteria. • The OOHC population increased 41% from June 2007 to June 2011. These changes have major impacts on FACS and so it is important to understand their source. FACS undertakes monitoring of this information and much is published in the Annual Statistical Report on the web. 18

19. Statistical Analysis • Cohort analysis to look at re-reporting of children and young people by factors such as their age, Aboriginality, child protection history and OOHC history. • Analysis of sibling groups, focusing on sibling groups that are frequency encountered by FACS. Where more information is required, detailed statistical analyses are undertaken. Some of these are available on the FACS website, including:

20. Statistical analysis 2. Program refinement/hypothesis testing • As a variety of tools come into everyday use by FACS their performance needs to be assessed and impact monitored. • Example – the Child Assessment Tool • Refinement also applies to programs, with confirmation of better performance of some of the higher need families in Brighter Futures contributing to the development of Strengthening Families.

21. Statistical Analysis 3. Program and pilot evaluations • Establishment of formal trials allows comparison over time of performance, based on criteria such as substantiated reports and time in OOHC before and after an intervention. This work provides the substructure for the cost benefit analyses which help determine whether specific programs continue. • Example – Aboriginal Consultation Guide 4. Evidence base • Information accumulated from these activities needs to be available and accessible so that it can be applied in future program development.

22. An example: Evaluation - Applied Statistical & Economic Analysis • Evaluations assess the objective performance of programs and projects both qualitatively and quantitatively using: • Internal evaluation staff who provide the conceptual framework, highly targeted support, i.e. robust measurement and assessment, for key organisational initiatives with minimal resource impact on project areas. • External staff (consultants). Internal staff manage the procurement process, from development of the brief to contract preparation and management. 22

23. An example: Evaluation - Applied Statistical & Economic Analysis • Evaluations contribute to the decision making process, including: • Practice issues for FACS and its partners • Existing service models • Policy and program development • FACS, CS, State Plan and COAG indicators • Inter-agency programs • Future funding arrangements 23

24. An example: Evaluation - Applied Statistical & Economic Analysis Evaluation staff also: • Contribute to Multi-Agency evaluations, we providing advice, participating in governance and facilitating access to relevant data. • Contribute results to the evidence base underlying NSW government policy settings such as early intervention, and intensive intervention.

25. Conceptual Overview – Cost Benefit Analysis In performing a cost-benefit analysis, FACS typically adopts a five-step approach. Generally, the costs are determined using historical expenses of providing the programs. The benefits are derived from ‘cost savings’. Where the ‘cost savings’ exceed the expenses, there is a net benefit. 25

26. Development and analysis of Key Performance Indicators for strategic planning, performance improvement and budgeting, cont’d Implementing performance behaviour Developing a performance mgmt framework Developing the performance management framework requires input from all levels of the organisation 26

27. Development and analysis of Key Performance Indicators for strategic planning, performance improvement and budgeting, cont’d • The process for developing the performance management framework encompasses four stages. • Monitoring is one component of successful service delivery. • Monitoring should not be considered one off, rather an iterative process to continually refine practices. 27