Evaluation of the Access to Allied Psychological Services (ATAPS) projects

1. Evaluation of the Access to Allied Psychological Services (ATAPS) projects General Practice Queensland Mental Health Network Meeting 24 March 2009 Bridget Bassilios, Justine Fletcher, Jane Pirkis, Kylie King, Fay Kohn Centre for Health Policy, Programs and Economics The University of Melbourne

2. Presentation Outline • Overview ATAPS Evaluation Findings • Minimum Data Set • T-CBT • Suicide Prevention • Perinatal Depression • Discussion / Questions

3. Participation in ATAPS Projects by GPs and Allied Health Professionals Over TimeData downloaded January 2009

4. Urban & rural referrals over timeN=135,994Data downloaded January 2009

5. Proportion QLD referrals Data Downloaded January 2009 * Total includes 2002 & 2009 data

6. CONSUMER PROFILEN= 100,854data downloaded January 08

7. Urban & rural sessions over time (N=534,618)Data downloaded January 2009

8. Proportion QLD sessions Data downloaded January 2009 * Total includes 2002 & 2009 data

9. SESSION PROFILE N= 420,555Data Downloaded January 2008

10. Consumer outcomes • Multiple outcome measures used across projects • Pre- and post- outcome scores available for 11,823 (15%) consumers from 54 (50%) projects • cf 5,288 consumers (7%) from 42 projects (38%) May 2007 • Overall large positive effect size • (80% projects show positive effects - 43% of projects showed large positive effects, 22% showed medium positive effects, 13% showed small positive effects) Data downloaded January 2008

11. Outcome Data: QLD DivisionsData Downloaded January 2008 • 11 of 18 QLD divisions entering useable data • 1,659/25,757 (6.4%) QLD consumers with pre- and post-test data • Represents 14% (1,659/ 11,876) of consumers nationally in outcome data

12. Minimum Data Set (MDS) • Web-based data entry system • Purpose is to obtain consistent and informative data from all ATAPS projects Data Collected • Patient demographics • Referral information • Session information • Pre & post outcome data

13. Use of MDS Data for Evaluation • Track ATAPS uptake overtime • Make comparisons between groups utilising ATAPS services, e.g. urban & rural consumers Evaluation Questions: • What is the level of uptake (consumers, GPs, AHPs)? • What are the sociodemographic characteristics of consumers? • What are the clinical characteristics of consumers? • What services are consumers receiving? • Are these services improving the mental heath of consumers? • What is the cost to the consumer?

14. Accessing the MDS • Username and password • Username and password can be used by more than one staff member (at your Division) to view the minimum dataset at the same time • only one person can make changes to the data at a time • GPs and AHPs can use the username and password provided to the Division and enter the data themselves. • The Minimum Dataset is accessed on the web at the following address: http://boimhc.org/bin/view. • Frequently Asked Questions link at the bottom of the home page • Questions regarding the minimum dataset support@boimhc.org

15. How Divisions can use the MDS: Data Entry • Methods of data entry • Direct entry to MDS • Upload data sets from Excel /Access etc • Enter data as soon as it is available • At the very least once per fortnight • Complete data, by entering zero instead of leaving fields blank

16. Changes to MDS

17. Changes to MDS

20. How Divisions can use the MDS: Reporting • Each division can view only their own data • Automatically generate a series of Individual Division Reports. • Compare with National Data • Compare with de-identified, aggregated ‘peer’ divisions • Enables Divisions to generate a series of tables for the MDS fields, including both consumer and session data. • These reports present data on consumers referred, and sessions provided, in a given period.

21. MDS Data for Individual Division Evaluations • Download data and import into an alternative program • Use excel or SPSS • Use for internal reporting and feedback to stakeholders • Three sets of data to download • Patient • Referral, including outcome measures • Session

23. Year of birth Gender Language at home English level Aboriginal Torres S.I. Low income Education Lives alone ICD-10 diagnostic categories Medication Strategies referred for Previous care MDS Generated TablesConsumer Reporting

25. Duration Individual / Group Strategies provided Co-payment MDS Generated TablesSession Reporting

27. Data entry tips • Each consumer should retain the same ‘patient key’ for all referrals • A new referral number is allocated after 12 sessions • When entering DASS scores enter either total or subscale scores. NOT BOTH • HADS, now use HADS dep or anx, the subscale scores will be automatically generated once item level data has been entered • If one lump sum copayment is taken for numerous sessions it should be divided by the number of sessions, and each session entered with the corresponding amount. • EG $120 in one payment for six sessions. The MDS should reflect that each sessions copayment was $20

28. Rationale for Evaluation • Accountability • Opportunities for improvement • Showcasing high levels of achievement • Making a case for ongoing support • Furthering knowledge about delivery of primary mental health care

30. Interim Evaluation Reports& Evaluation Support Reports available at: http://boimhc.org • Jane Pirkis • Bridget Bassilios • Justine Fletcher • Fay Kohn • Kylie King • Grant Blashki • Philip Burgess Evaluation support available via email: support@boimhc.org

31. List of Journal Publications • Bassilios B, Fletcher J, Pirkis J, Kohn F, Blashki G, Burgess P. (2008). Evaluation of the Better Outcomes in Mental Health Care program. APS Conference Proceedings, 16-20. • Fletcher J, Bassilios B, Kohn F, Naccarella L, Blashki G, Burgess P, Pirkis J (2008). Meeting demand for psychological services by people with depression and anxiety: Recent developments in primary mental health care. Medical Journal of Australia. 188 (12): S107-S109. • Hickie I, Pirkis J, Blashki G, Groom G, Davenport T. General practitioners' response to depression and anxiety in the Australian community: A preliminary analysis. Medical Journal of Australia 2004;181(7):S15-S20. • Kohn F, Pirkis J, Morley B, Naccarella L, Blashki G (2008). Utilization of findings from the evaluation of a major primary mental health care initiative in Australia. Evaluation Journal of Australia • Morley B, Pirkis J, Naccarella L, Kohn F, Blashki G, Burgess P. Improving access to and outcomes from mental health care for rural Australians. Australian Journal of Rural Health 2007;15(5):304-12. • Morley B, Pirkis J, Sanderson K, Burgess P, Kohn F, Naccarella L, et al. Better outcomes in mental health care: The impact of different models of psychological service provision on consumer outcomes. Australian and New Zealand Journal of Psychiatry 2007;41:142-149. • Naccarella L, Pirkis J, Kohn F, Morley B, Burgess P, Blashki G. Building evaluation capacity: Definitional and practical implications from an Australian case study. Evaluation and Program Planning , 30, 231-236. • Pirkis J, Kohn F, Morley B, Burgess P, Blashki G. Better Outcomes in Mental Healthcare? Primary Care Mental Health 2004;2:141-149. • Pirkis J, Morley B, Kohn F, Blashki G, Burgess P, Headey A. Improving access to evidence-based mental health care: General practitioners and allied health professionals collaborate. Primary Care Psychiatry 2004;9(4):125-130. • Pirkis J, Stokes D, Morley B, Kohn F, Mathews R, Naccarella L, et al. Impacts of Australia's Better Outcomes in Mental Health Care program for psychologists. Australian Psychologist 2006;41(3):152-159.