Measurement for Improvement

Measurement for Improvement

Turn to your neighbor • What have been your biggest learningsor challenges regarding data gathering and measurement for your Early Years work? • Definitions • Collection • Reporting • Frequency • Analysis • Other?

Components of a Learning System • System level measures • Explicit theory or rationale for system changes • Segmentation of the population • Learn by testing changes sequentially • Use informative cases: “Act for the individual learn for the population” • Learning during scale-up and spread with a production plan to go to scale • Periodic review • People to manage and oversee the learning system From Tom Nolan PhD, IHI

The Model for Improvement Act Plan Study Do What are we trying to Accomplish? The three questions provide the strategy Our focus today How will we know that a change is an improvement? What change can we make that will result in improvement? The PDSA cycle provides the tactical approach to work Source: Langley, et al. The Improvement Guide, 1996.

The Quality Measurement Journey AIM(How good? By when?) Concept Measures Operational Definitions Data Collection Plan Data Collection Analysis PDSA Source: R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. Jones and Bartlett, 2004.

Case Study • Promoting Bedtime Reading • Lochrin and Grassmarket Nursery Schools • Edinburgh • Lead: Donna Murray

The Quality Measurement Journey AIM– Improvebedtime routines, attachment, literacy Concept – Increase bedtime reading Measures– Percentage of children receiving a bedtime story Operational Definitions– Number read story/Total Children Data Collection Plan – daily; no sampling Data Collection – teacher asks parents at drop off, spreadsheet Analysis– run chart PDSA

WORKSTREAM 3 (30 months to start of primary school) Theory of what actions will ensure developmental milestones are reached at the start of primary school Theory of what drives developmental milestones 2⁰ Aim 1⁰ Improved teamwork, communication and collaboration Poverty Quality Of Home Environment Improved uptake of benefits Domestic Abuse & Violence Improved child’s dental health Societal Issues Workforce Issues Improving child nutrition Transport, Community Capacity & Cultures Children have all the developmental skills and abilities expected at the start of primary school Improving brain development and physical play Access To Services Employment Improved family centred response Early Learning & Play Improved stability / permanence for LAC Child’s physical & mental health and emotional development Health Improved identification Attachment Improved joint working Detailed Aim: 90% of all children within each CPP have reached all of the expected developmental milestones at the time the child starts primary school, by end-2017 Additional Support Improved management, planning and quality of services Level of education Misuse of alcohol & drugs Improved sharing of information Carer’s physical & mental health and skills Nutrition Improved leadership, culture & planning` Disabilities & Mental health Identification & reasons for current resilience Parenting skills & knowledge Version: 06/03/2013

Project Aim Statements • What, by when, by how much. • “By the end of June, 90% of children at Grassmarket Nursery will receive a bedtime story at least 3 times a week.”

WORKSTREAM 3 (30 months to start of primary school) Theory of what actions will ensure developmental milestones are reached at the start of primary school Theory of what drives developmental milestones 2⁰ Aim 1⁰ Improved teamwork, communication and collaboration Poverty Quality Of Home Environment Improved uptake of benefits Domestic Abuse & Violence Improved child’s dental health Societal Issues Workforce Issues Improving child nutrition Transport, Community Capacity & Cultures Children have all the developmental skills and abilities expected at the start of primary school Improving brain development and physical play Access To Services Employment Improved family centred response Drivers Concept: Bedtime Reading Early Learning & Play Improved stability / permanence for LAC Child’s physical & mental health and emotional development Health Drivers Improved identification Attachment Improved joint working Detailed Aim: 90% of all children within each CPP have reached all of the expected developmental milestones at the time the child starts primary school, by end-2017 Additional Support Drivers Improved management, planning and quality of services Level of education Misuse of alcohol & drugs Improved sharing of information Carer’s physical & mental health and skills Nutrition Improved leadership, culture & planning` Disabilities & Mental health Identification & reasons for current resilience Parenting skills & knowledge Version: 06/03/2013

Driver Concept and Measure • Measures to provide feedback on the concept • % receiving a story • % of stories read at bedtime • % of parents reporting improved bedtime routine • % enjoying the bedtime story • % reporting increase in bedtime story reading

The Quality Measurement Journey AIM(Why are you measuring?) Concept Measures Operational Definitions Data Collection Plan Data Collection Analysis PDSA Source: R. Lloyd. Quality Health Care: A Guide to Developing and Using Indicators. Jones and Bartlett, 2004.

Operational Definitions • What does reading a story mean? • When is bedtime versus other time? • What is improved bedtime routine? • What is the definition of enjoyed?

Data Collection Planning & Doing All children are included, no sampling. 7 days a week. Donna will inquire with each parent at drop off Monday through Friday. Data will be captured on a paper form. Donna inputs daily data into spreadsheet following drop off.

Analysis – Run Charts weekly average displayed for parents Research information handed to parents. Books available at collection time. Grassmarket changes introduced.

Action – Testing Change Ideas

NHS Lanarkshire – Our 27-30 Month Review Data Journey Stacey Rooney

Background • NHS Lanarkshire Clinical Quality Service, supporting both North and South Lanarkshire CPPs from a health perspective • 27-30 month reviews commenced June 2013 in Lanarkshire – measurement data for stretch aim 2 • But we don’t have baseline data……. • ….so we need early access to our review data to develop a baseline……. • ISD Proposed quarterly reports…….November 2013 before we see our first set of results

Accessing our data – lots of questions.... • Where do the forms go? • Does anyone in Lanarkshire have access to the CHSP system? • Can we access the system? • Technical challenges accessing the system....patience.......patience........we’re in!! • We’re in but what do all these codes mean? • Expertise to interrogate the data

How did we access CHSP Business Objects Universe (AKA – our data)? • Contact the ATOS Helpdesk ITServiceDesk.NHSS-NS@Atos.net • Receive a form to complete via email, needs authorising by your Child Health Administrator • Receive complex instructions from ATOS on how to set-up your PC/laptop – stick with it, you’re getting there!! • Receive your Business Objects Universe login and password from your local Child Health Administrator

CHSP Business Objects

Benefits of accessing our data • Earlier access to our data – some delays come from the time to return the forms for data entry and data entry itself (approx 4 weeks) • Early identification of quality issues with form completion enabled us to feed this back and engage with staff • We have early baseline data • Stratify the data as we like (unit, locality, team) • Results are helping us identify early tests of change

Early data focus • Meaningful reviews (9 developmental milestones reviewed and recorded) • Stretch aim 2 (meeting all developmental milestones) • Future actions (request for assistance/referral) • Looked After Status • Breeches • Health Plan Indicator (HPI) – status change • Attendance / DNA – obtained locally

Coverage – A work in progress: A national definition of ‘eligible children’ in a reporting period is still in development. Eligible = denominator/starting point Invited: coverage measure 1 Attended: coverage measure 2 Meaningful: coverage measure 3

Coverage: An early example

Examples of our early data (p) Provisional data - incomplete

(p) Provisional data - incomplete

Limitations of our early data • We’re still developing a robust process to measure uptake - so results are only representative of those attending • Non attenders are potentially our more vulnerable children and this may be skewing our results – particularly stretch aim 2 • Non-meaningful reviews are being excluded, we need to reduce this (approx 25%)

Stage 2 data focus • What other data is available from the review? • Age of child at review • SIMD quintiles • Ethnicity • Place of review (home, GP practice, clinic) • Exposure to smoke • Registered with a dentist • Bi/Multi-lingual

Reporting

Monthly reports Monthly summary and locality level reports are in development

Next steps • Continue to work with colleagues to develop reporting that is meaningful and facilitates/guides improvement • Continue to feedback real time issues with data completeness and quality • Further work to link other reviews and develop data intelligence over time for each child e.g. HPI • Further work to consider how best to share this data with partners out with health to benefit children and families

Thank you. I hope that didn’t leave you feeling like this...................or this.......

I hope it did make you feel like this..... Thank you to Scott Purdie, Clinical Quality Co-ordinator, NHS Lanarkshire for his work in this area. And My niece Emily, age 9 months for brightening up the data talk! Stacey.rooney@lanarkshire.scot.nhs.uk

Measurement for Improvement