Monitoring clinical quality through indicators, metrics and dashboards National perspective

1. Monitoring clinical quality through indicators, metrics and dashboardsNational perspective Danny Keenan National Clinical Advisor Care Quality Commission

2. Quality metrics and indicators explained: what’s the difference between a metric and an indicator • Discussion re Clinical Quality • Augmenting national indicators with local measurements of quality • How the Care Quality Commission will monitor improvements in quality • Using clinical dashboards to monitor improvement in quality indicators and metrics NHS Performance Framework; Implementation guidance April 2011 Technical Guidance for the 2011/12Operating Framework; Jan 2011 The NHS Outcomes Framework 2011/12

3. Metric or indicator? • A metric is a measure • Patients in primary care have their blood pressure measured • An indicator is a way of using a metric for purposes of comparison: (Latin: indicare = to point out)

4. Metric versus indicator An indicator A metric Unit A has a caesarean Section rate of 15%

5. What is clinical quality?

6. Three domains of quality • Safety • Effectiveness • Patient experience

7. Outcomes and indicators for the first framework • Preventing people from dying prematurely • Enhancing the quality of life for people with long-term conditions • Helping people to recover from episodes of ill health or following injury • Ensuring people have a positive experience of care • Treating and caring for people in a safe environment and protecting them from avoidable harm EFFECTIVNESS Clinical quality PATIENT EXPERIENCE SAFETY The NHS Outcomes Framework 2011/12

8. Assessing quality • Measuring clinical quality: • Structure • Refers to the inherent characteristics which are associated with higher quality (e.g): • Procedure volumes • Advanced IT • Nurse staffing ratios • Stroke service • Processes • Do you comply with evidence-based care guidance? • Administration of aspirin and B blockers with acute coronary syndrome • Compliance with desirable practices such as frequent determination of HbA1c in diabetics • Outcomes • some general ones • some sector or condition specific

9. Example metrics for assessing quality Structure

10. Implementing the NHS Performance Framework www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/documents/digitalasset/dh_127007.pdf

11. Augmenting national indicators with local measurements of quality The NHS Outcomes Framework 2011/12 • Preventing people from dying prematurely • Enhancing the quality of life for people with long-term conditions • Helping people to recover from episodes of ill health or following injury • Ensuring people have a positive experience of care • Treating and caring for people in a safe environment and protecting them from avoidable harm EFFECTIVNESS Clinical quality PATIENT EXPERIENCE SAFETY There are a set of 50 indicators underneath these domains

12. Augmenting national indicators with local measurements of quality • The Commissioning Outcomes Framework • Derived from the NICE Quality Standards, among other things. • There will always be room for local measures: • North West : • Advancing Quality ---AQuA (Advancing Quality Alliance): • Heart Attack • Hip or knee replacement • Heart failure • Pneumonia • Coronary artery bypass surgery • Next iteration: • Mental health, including dementia • COPD • Diabetes

13. NHS Performance Guidance-Implementation Guidance. April 2011 • Measuring for Quality Improvement launched the development of a menu of Assured Quality Indicators to enable local clinical teams to identify indicators that support their improvement work and allow benchmarking with other clinical teams. • Quality indicators from this Assured Menu will have a number of uses, including in commissioner contracts (particularly the Commissioning for Quality and Innovation (CQUIN) payment framework), publication of Quality Accounts, and information for the public through NHS Choices.

14. Indicators for Quality Improvement

15. How CQC will monitor improvements in quality • The essential standards of quality and safety

16. CQC and registration

17. Quality and Risk Profile (QRP)

18. Quality Risk Profiles (QRP) are at the centre Rejects the use of a small number of numeric indicators to make judgements about service quality Bring together information about a care provider so as to estimate risk and prompt front line regulatory activity Act as a guide and aid CQC’s inspectors Help inform subsequent judgements and add to the knowledgebase about a care provider Essential tool to support monitoring of compliance Not a judgement (or a rating, or a league table) in themselves Capable of using both numeric (quantitative) and textual (qualitative) data Build over time

19. …but builds on 10 years development The philosophy draws on the work of Klein and Carter in the 1980s This approach is at the cutting edge of use of information, but we believe transferable between sectors The mathematical approaches have been being developed by CQC and its predecessors since the early 2000s and have substantial academic input The approach draws on and addresses Bevan and Hood’s reflections of the “measurement frenzy” in public service reform in the 2000s The qualitative analysis techniques allow us to treat textual information consistently draw on best practice in social research techniques 19

20. Updating and sharing of QRPs with providers QRPs are updated regularly (monthly or more often depending upon data feeds) This summer we will release an online service which will give access to data as live and which will have vastly increased functionality The QRPs will be shared with Providers as they are updated so that we are all working from the same data NHS trusts have had access to their QRP since September via a .pdf. This has been made available to commissioners, SHAs, Monitor and DH since October 20

22. How the Care Quality Commission will monitor improvements in quality

23. How CQC will monitor improvements in quality • Our main tool will be using Registration • and………. • Compliance with Registration • Previously we had the Annual Health Check…proved to be a blunt tool • Now that Registration has been in place, with Trusts, for a year compliance is becoming more important

24. Site visits • The aim of site visits is to gather evidence of compliance • They are short, focussed unannounced site visits, rather than set piece inspections that require the provider to spend a lot of time in preparation • Site visits are direct checks of compliance rather than assessing compliance through the assurance systems the organisation has in place. • Therefore site visits may include direct observation of care and spending time with people who use the service, their families and carers, unless not appropriate to do so. We may also talk to managers and staff. Experts by experience will join us on some site visits to help us engage with people who use services.

25. Regulatory judgement and response Judgement framework Regulatory response • Stage 2: Does the evidence show compliance? • Stage 3: What is the impact on people who use services and the likelihood of this happening? Is there: • No concern • Minor concern • Moderate concern • Major concern Stage 4: Validation • Maintain registration - no further action • Improvement actions: • eg improvement letter • Enforcement actions: • Statutory warning notice • Imposition or variation of conditions • Fines • Prosecution • Suspension of registration • Cancellation of registration Many Trusts have some compliance problems With enforcement to ensure compliance we believe this will drive improvement Regulatory judgement • Judgement of compliance or concerns • Translates minor, moderate or major concerns into regulatory judgement • Takes account of the provider’s capability to improve • Action will be proportionate

26. Using clinical dashboards to monitor improvement in quality indicators and metrics • This will be determined locally, bearing in mind that there • will be requirements laid down by the Department • (including QIPP local agenda); CQC and Monitor. • So a dashboard will have these through it: • Measures to reassure the Board/Managers that the • service they provide is effective, safe and patient responsive • Measures negotiated with local commissioners • Measures suggested by clinicians and for use in revalidation • Measures suggested by patients, including patient • outcomes and surveys

27. Using clinical dashboards to monitor improvement in quality indicators and metrics • Pitfalls: • Avoid that which is easy to measure but not useful • The adverse effects of any measure which is that the measured item improves at the expense of other important areas • The burden of measurement

28. Local dashboards • Some overarching themes: • Process: • Board Issues (Having a culture of quality and safety in an organisation and amongst staff) • Quality of Records • Information / HES quality/ NHS number • Commitment to safety • Issues relating to staff • Sickness/stress, staff satisfaction, absence, training, vacancies • Commitment to Audit and Service Improvement • Commitment to Research

29. Local dashboards • Some overarching themes: • Outcomes: • Mortality • Healthcare associated infections • Patients: • Satisfaction, complaints • Prescribing, administration and reconciliation of medicines • Patient Safety First outcomes • Never events • National audits; • Cardiac • Cancer • HES based outcomes

30. Thank you • Danny.Keenan@cqc.org.uk

31. Annex to the NHS Performance Framework; Implementation guidance April 2011 Technical Guidance for the 2011/12Operating Framework; Jan 2011 http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/documents/digitalasset/dh_123660.pdf

32. Mental Health www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/documents/digitalasset/dh_127007.pdf