Evidence based practice Questions of the individual clinician

1. Developing evidence based practice: what does it mean and can it be done?Miranda WolpertDirector CAMHS Evidence Based Practice UnitChair CAMHS Outcome Research Consortium

2. Evidence based practiceQuestions of the individual clinician • What is the best treatment for this particular child • What are the pros and cons of different treatments • What does the research show and what other factors do I need to take into account • Are there particular reasons for adopting a different approach in a particular case?

3. Evidence based practiceQuestions of the service developer • What does the evidence show should be provided by services and in what proportions • What skill mix is needed to provide child mental health services • What should be the ratio of investment in different options eg prevention/promotion programmes as opposed to direct interventions

4. Evidence based practicequestions of the academic • What does this research really show • Are there other interpretations • How can research be devised to answer the remaining questions

5. Evidence based practicequestions of the child and family • What does this research really show • Are there other interpretations • How can research be devised to answer the remaining questions

6. Answering these questions • Few straight or clear answers • Lots of complexity • Lots of gaps • Need a realistic way forward…

7. A realistic evidence based practice

8. When does information become evidence? Hierarchy of Evidence • Ia Evidence from meta-analysis of randomised controlled trials • Ib Evidence from at least one randomised controlled trial • IIa Evidence from at least one controlled study without randomisation • IIb Evidence from at least one other type of quasi-experimental study • III Evidence from descriptive studies such as comparative studies, correlation studies and case-control studies • IV Evidence from expert committee reports or opinions, or from clinical experience of a respected authority, or both. Alternative Hierarchies?

9. A non-evidence based approach? • Reliance on assumptions • More influenced by anecdote than statistics • Not testing theories • Unwillingness to change in light of new evidence • Most persuasive promoter wins out

10. Limitations of the evidence • Paucity of research • Skew in researched areas • Skew in researched populations • Generalisability to range of groups and settings questionable • Design flaws in studies • Lack of consensus on appropriate outcomes and perspectives • Lack of model for economic costings • Lack of focus on possible harm • Publication bias

11. Publication bias (from David Cottrell)

12. Publication bias (from David Cottrell)

13. Drawing on the Evidence Wolpert, Fuggle, Cottrell, Fonagy, Phillips, Target and Stein 2002 Based on systematic review: Peter Fonagy, David Cottrell, Mary Target, Zarrina Kurtz, Jeanette Phillips • DoH Mother & Child R&D Fund Revised edition 2006 -Updated in light NICE guidance and major randomised control trials

14. Possible summary of what we know works currently

15. Evidence based interventions • Cognitive behavioural therapies (CBT) • Behaviour therapy • Parent Training • Medications • Family Therapy • Interpersonal therapy (IPT) • Social skills training • Multi-systemic therapy (MST) • Treatment Foster Care • Individual psychodynamic therapies

16. BUT….complicating/mediating factors Demographic factors ? “attachment disturbance” “therapeutic alliance” Non-specific therapeutic factors Fidelity to model Ability to flexibly adapt model

17. Proposed checklist for evidence based interventions (adapted from Kazdin 2004) • What are the costs, risks and benefits of this intervention relative to no intervention? • What are the costs, risks and benefits of this intervention relative to other interventions? • What are the key components that appear to contribute to positive outcomes? • What parameters can be varied to improve outcomes (e.g. including addition of other interventions, non specific clinical skills etc)? • To what extent are effects of interventions generalizable across a) problem areas, b) settings, c) populations of children and d) other relevant domains Which of these can we answer now? How do we get answers?

18. Evidence base for service structures Lots of values much less clear evidence - Fort Bragg Studies - Pooled budgets impact -Suggestive work about impact of service user involvement Promising work on economic evaluation of early intervention in psychosis projects

19. Worcestershire EIS (2006 report Jo Smith)

20. Evidence base for skill mix (based on evidence based interventions) • 3 units of people able to provide behavioural, cognitive and interpersonal therapies • :1.5 units of people able to provide parent management training • : 1 unit of people able to provide systemic/multimodal therapy: • : 1 unit of a person/people able to provide physical treatments, prescription and monitoring. BUT doesn’t taken account of -under-researched interventions Non- specific therapeutic and assessment skills Possible needs of particular populations

21. Evidence base for children and families Choosing What’s Best for You What We Know (And What We Don’t) About the best ways of Helping Children and Teenagers With: Eg ADHD Information for: Children, Teenagers, Families

22. Choosing what's best for you booklet aims to help children young people and their families make informed choices about treatment options It gives information about what research up till now has shown to help. It is not designed to give you any general information. In this booklet we list the most evidence based treatments a the moment Each treatment option is rated using the following scale:. * * * = Very likely to help * * - = Quite likely to help * - - = Not that it will help

23. Choosing what's best for you • Points to remember • There are many treatments that we simply don’t know if they work or not yet because research has not been done or is inconclusive- they are not included here • Even when a treatment has been shown in research to work well for most people, as we are all different it may help some people more than others • You will have to weigh up the positives and negatives of any approach, including any possible side effects • Our knowledge is growing all the time so check if there have been further developments since this was published

24. Choosing what's best for you Points to remember • There are many treatments that we simply don’t know if they work or not yet because research has not been done or is inconclusive- they are not included here • Even when a treatment has been shown in research to work well for most people, as we are all different it may help some people more than others • You will have to weigh up the positives and negatives of any approach, including any possible side effects • Our knowledge is growing all the time so check if there have been further developments since this was published

25. Choosing what's best for you

26. Reflection and evaluation Routine outcome monitoring- “Mission Impossible” ?? (Einar Heiervang) • Case evaluation: To provide information about individual children and their families. • Clinician evaluation: To provide information about outcomes for the range of children and families seen by an individual clinician • Service evaluation: To provide information about the outcomes of particular projects or services • Strategy evaluation To provide information about the impact of a CAMHS strategy

27. Underpinning values • All services should routinely audit and evaluate their work • Data collected made available to clinicians, users and commissioners • Results used to inform service development • Collaboration essential

28. Evaluating outcomes Whose view? Child Parents Clinician Where get info from? Conversations Questionnaires Written communications Information held in a data set Population statistics For Whom?

29. What should be be evaluated • Change in difficulties • General adaptation • Feelings of burden and stress • Satisfaction • Population changes e.g. attendance rates; exclusions; youth crime; reported self harm; reported substance misuse, rates entering care, attainment rates

30. CAMHS Outcome Research Consortium (CORC) Members agreeing on a common approach • Creating reports for reflection on individual children/practitioners • Collating and centrally analysing data • Promoting use of data to inform service providers, commissioners, users and others • Facilitating sharing of ideas between members • Supporting dissemination and refinement of National CAMHS dataset

31. CORC aims • Develop and disseminate model of routine outcome evaluation that can be used across a range of services • Ensure data used to inform service providers, commissioners and users and other relevant stakeholders • Collate and centrally analyse data from all member sites • Collaborate in using outcome information to inform and develop good practice • www.corc.uk.net

32. CORC approach Child/parent perspective: Symptoms and burden: -Strengths and Difficulties Questionnaire (SDQ) for child (11-16) and parents of children aged 3-16 Experience of service: - Commission for Health Improvement (CHI) for child (9+) and parent Practitioner perspective Children’s Global Assessment Scale (CGAS) – measures overall functioning HoNOSCA where appropriate Consultation measure Being piloted

33. CORC protocol • Pre therapy measures for child and parent • First meeting measures for clinician • MDS • 6 month follow up (or case closure if before this) • Option for repeated follow ups for longer term contacts

37. Outcome measurement - SDQ

41. Interpreting the “evidence” • Credibility Self Evaluation: right measures, no.s of returns, quality of returns Research studies: right measures, right people, quality of controls • Context Self Evaluation: specific factors to be taken into account eg demographics, specialist focus Research Studies: generalisability • Comparison Self Evaluation: with baselines, with community with appropriate other services Research Studies: with other findings

42. Towards and evidence based practice approach Need to both acknowledge complexity and to promote clarity- is this possible? • More research • Explicit recognition of values base • Reflective practice