SIPP Quarterly Webinar March 30, 2012

1. SIPP Quarterly WebinarMarch 30, 2012 Presenters: Christy Hormann, LMSW, CPHQ Project Leader Don Grostic, MS Associate Director Perfect PIP Example

2. PIP Activities I-X • Select the study topic(s) • Define the study question(s) • Select the study indicator(s) • Use a representative and generalizable population • Use sound sampling techniques • Reliably collect data • Implement intervention and improvement strategies • Analyze data/Interpret study results • Assess for real improvement • Assess for sustained improvement

3. Activity ISelect the study topic(s) • Reflects high-volume or high-risk conditions. • Is selected following collection and analysis of data. • Addresses a broad spectrum of care and services. • Includes all eligible populations that meet the study criteria. • Does not exclude members with special healthcare needs. • Has the potential to affect member health, functional status, or satisfaction. (CE)

4. Activity ISelect the study topic(s) • Reflects high-volume or high-risk conditions. • Describe why the study topic is considered high-volume and/or high-risk. • Specify if the study topic was selected by the State. “There are three million individuals with co-occurring disorders having at least three disorders and one million people have four or more. (National Health Policy Forum, 1998).”

5. Activity ISelect the study topic(s) • Reflects high-volume or high-risk conditions cont. “There is considerable reason to be concerned about individuals with dual psychiatric and substance related disorders, often referred to as dually diagnosed. The past several decades has witnessed a growing population of dually diagnosed individuals related to de-institutionalization, increased availability of drugs, and changing social and economic conditions (Hegner, 1998). The National Co-morbidity Survey (NCS) indicated that individuals with mental illness are at least twice as likely to abuse drugs including alcohol as individuals with no mental illness (Kessler, 1994). Research has demonstrated that dually diagnosed individuals have a greater tendency for psychiatric relapse and re-hospitalization (Drake & Mueser, 1996). Dually diagnosed individuals also are more likely to experience divorce, social isolation, unemployment, poverty, homelessness, crime and incarceration, and violence (Drake & Mueser, 1996; Kessler, 1994).”

6. Activity ISelect the study topic(s) • Is selected following collection and analysis of data. • Provide plan-specific historical data. • If plan-specific data are not available, provide rationale. “On April 2, 2008, the plan completed a study…which included an examination of claims data and a random sample of plan discharge summaries of members with Co-Occurring Diagnosis (COD)…” “The findings from the April 2, 2008 study report supported implementation of an ongoing plan quality improvement activity aimed at improving the quality of care for members with COD…study serves as historical data for this PIP.”

7. Activity ISelect the study topic(s) • Selected following collection and analysis of data cont. “Private inpatient facilities: 34% were referred for any substance related disorders…32% were referred for a combination of substance related disorders and psychiatric services.” “RMHIs: 15% of members were referred for substance related disorders follow; 10% were referred for a combination of substance related disorders and psychiatric services.”

8. Activity ISelect the study topic(s) • Addresses a broad spectrum of care and services. “The overall objective of the Co-Occurring Diagnosis PIP is to improve the quality of discharge planning for members with co-occurring diagnosis.”

9. Activity ISelect the study topic(s) • Addresses a broad spectrum of care and services cont. “In the late 1970s, practitioners began to recognize that the presence of substance related disorders in combination with mental disorders had profound and troubling implications for treatment outcomes. This growing awareness has culminated in today’s emphasis on the need to recognize and address the interrelationship of these disorders through new approaches and appropriate adaptations of traditional treatment. In the decades from the 1970s to the present, substance related disorders treatment programs typically reported that 50 to 75 percent of their clients had COD, while corresponding mental-health settings cited proportions of 20 to 50 percent. During the same period of time, a body of knowledge has evolved that clarifies the treatment challenges presented by the combination of substance abuse and mental disorders and illuminates the likelihood of poorer outcomes for such clients in the absence of targeted treatment efforts.”

10. Activity ISelect the study topic(s) • Includes all eligible populations that meet the study criteria. • Describe the eligible population. • Explain if all eligible populations were included in the study. • Include reference if the population was selected by the State. “ ‘Member’ is defined as being enrolled in the program at the time of discharge or being eligible for plan benefits at the time of discharge. The target population for this PIP is members with co-occurring diagnoses discharged after an inpatient stay – both acute psychiatric hospitalizations and residential treatment facility admissions.” “This PIP will include all eligible populations. No eligible members will be excluded.”

11. Activity ISelect the study topic(s) • Does not exclude members with special healthcare needs. • Include a statement about the inclusion of members with special health care needs. “Members with special health care needs were not excluded from the study.”

12. Activity ISelect the study topic(s) • Has the potential to affect member health, functional status, or satisfaction.(CE) • Explain how study topic will affect health/functional status/satisfaction. • Explain link between study topic and outcomes of care. “The overall objective of the Co-Occurring Diagnosis PIP is to improve the quality of discharge planning for members with co-occurring diagnosis.” “A carefully developed discharge plan, produced in collaboration with the client, will identify and relate client needs to community resources, ensuring the supports needed to sustain the progress achieved in treatment.”

13. Activity IIDefine the study question(s) • States the problem to be studied in simple terms. (CE) • Is answerable (CE).

14. Activity IIDefine the study question(s) • States the problem to be studied in simple terms. • Study question in CMS protocol “Does doing X result in Y?” format. • Define any unclear terms used in study question. • Is answerable. • Answerable through specified data collection method and study indicators. “Do targeted interventions increase the percentage of members who had a follow-up visit with a substance abuse practitioner within 7 calendar days of an inpatient discharge?”

15. Activity IIISelect the study indicator(s) • The indicators are well-defined, objective, and measurable. (CE) • Are based on current, evidence-based guidelines, pertinent peer-reviewed literature, or consensus expert panels. • The indicator(s) allow for the study question to be answered. (CE) • The indicator(s) measure changes (outcomes) in health or functional status, member satisfaction, or valid process alternatives.

16. Activity IIISelect the study indicator(s) • The indicator(s) have available data that can be collected on each indicator. (CE) • The study indicator(s) are nationally recognized measures such as HEDIS specifications, when appropriate. • Include the basis on which indicator(s) was adopted, if internally developed.

17. Activity IIISelect the study indicator(s) • The indicator(s) are well-defined, objective, and measurable. (CE) • Objective and measurable study indicators with description of numerator and denominator. • Define terms used in indicator; include relevant codes. Study Indicator:Percentage of members who had a follow-up visit with a substance abuse practitioner within 7 calendar days of an inpatient discharge. Numerator:Total number of members who had a follow-up visit with a substance abuse practitioner within 7 calendar days of an inpatient discharge. Denominator:Total number of members discharged with COD.

18. Activity IIISelect the study indicator(s) • Are based on current, evidence-based practice guidelines, pertinent peer-reviewed literature, or consensus expert panels. • Based on current guidelines/research, with sources cited. • If the State provided the study indicators, include this in the documentation. • This evaluation element would be scored Not Applicable for this PIP.

19. Activity IIISelect the study indicator(s) • The indicator(s) allow for the study question to be answered. (CE) • The study indicator(s) and study question(s) should align. Study Question: “Do targeted interventions increase the percentage of members who had a follow-up visit with a substance abuse practitioner within 7 calendar days of an inpatient discharge?” Study Indicator: “Percentage of members who had a follow-up visit with a substance abuse practitioner within 7 calendar days of an inpatient discharge.”

20. Activity IIISelect the study indicator(s) • The indicator(s) measure change (outcomes) in health or functional status, member satisfaction, or valid process alternatives. • Objectively measure member outcomes such as health/functional status/member satisfaction or valid alternatives. • The study indicator(s) should be completely and correctly defined to receive a Met score for this evaluation element.

21. Activity IIISelect the study indicator(s) • The indicator(s) have available data that can be collected on each indicator. (CE) • Data from administrative sources, medical records, surveys, or other readily available sources. • The data that will be collected for each study indicator should be clearly documented.

22. Activity IIISelect the study indicator(s) • The study indicator(s) are nationally recognized measures such as HEDIS specifications, when appropriate. • Nationally recognized measures should be used when available. • If using nationally recognized measures, provide explanation and year. • This evaluation element is scored Met or Not Applicable. For this PIP, it would be scored Not Applicable.

23. Activity IIISelect the study indicator(s) • Include the basis on which indicator(s) was adopted, if internally developed. • Include rationale for why each study indicator was chosen. “The plan has received qualitative feedback internally from staff, as well as from external providers and advocates suggesting that members experience difficulty accessing treatment, receive psychiatric diagnoses inappropriately in order to access services, and do not always receive treatment and appropriate discharge plans related to substance abuse when hospitalized in psychiatric facilities.”

24. Activity IVUse a representative and generalizable population • The study population is accurately and completely defined. (CE) • The study population includes requirements for the length of a member’s enrollment in the plan. • The study population captures all members to whom the study question applies. (CE)

25. Activity IVUse a representative and generalizable population • The study population is accurately and completely defined. (CE) • Clearly define inclusion/exclusion and diagnosis criteria. • Include list of diagnosis/system codes used as identifiers. • Include any anchor dates used to identify age criteria. “The target population for this PIP is members with co-occurring diagnoses (COD) discharged after an inpatient stay – both acute psychiatric hospitalizations and residential treatment facility admissions. “Member” is defined as being enrolled in the program at the time of discharge or being eligible for benefits at the time of discharge. “Co-occurring diagnosis” (COD) is defined as having both psychiatric and substance related disorders /dependence related diagnoses. An acute inpatient stay is defined as having a length of stay of between two and thirty (inclusive) days. Members must be continuously enrolled through 7 days after discharge. The denominator is based on discharges, not members.”

26. Activity IVUse a representative and generalizable population • The study population is accurately and completely defined cont. (CE) • “Exclusions: • Member refusal for services prior to scheduled appointment • Member readmits to an inpatient level of care within seven (7) days. • Discharges that indicated length of stays were less than two days and more than 30 days (calculated as the number of days elapsed between admit and discharge dates). • Applicable codes: 304.XX (drug dependence codes), 305.XX (Nondependent abuse drug codes), mental disorders-broad range of codes: 209-319.”

27. Activity IVUse a representative and generalizable population • The study population includes requirements for the length of a member’s enrollment in the Plan. • Define continuous enrollment/new enrollment/allowable gaps in enrollment. • Include any dates used to define continuous enrollment. “Members must be continuously enrolled through 7 days after discharge.”

28. Activity IVUse a representative and generalizable population • The study population captures all members to whom the study question applies. (CE) • Should include all members to whom the study question applies. • The study population must be completely and correctly defined. • The study population and study question should align.

29. Activity VUse sound sampling techniques • Consider and specify the true or estimated frequency of occurrence. • Identify the sample size. • Specify the confidence level to be used. • Specify the acceptable margin of error. • Ensure a representative sample of the eligible population. (CE) • Ensure that the sampling techniques are in accordance with generally accepted principles of research design and statistical analysis.

30. Activity VUse sound sampling techniques • Consider and specify the true or estimated frequency of occurrence. • The true/estimated frequency of occurrence should be provided. • Identify the sample size. • Specify the confidence level to be used. “The entire eligible population was used. No sampling techniques were utilized.”

31. Activity VUse sound sampling techniques • Specify the acceptable margin of error. • Ensure a representative sample of the eligible population. (CE) • Representative sampling should ensure generalizable information. • If NCQA certified software used, include certified software seal. • Ensure that the sampling techniques are in accordance with generally accepted principles of research design and statistical analysis. • Use valid/replicable sampling techniques for all study indicators. “The entire eligible population was used. No sampling techniques were utilized.”

32. Activity VIReliably collect data • Identification of data elements to be collected. • Identification of specified source data. • A defined and systematic process for collecting baseline and remeasurement data. • A timeline for the collection of baseline and remeasurement data.

33. Activity VIReliably collect data Evaluation Elements 5-9 for manual data collection • Qualified staff/personnel to abstract manual data. • A manual data collection tool that ensures consistent and accurate collection of data according to indicator specifications. (CE) • A manual data collection tool that supports interrater reliability (IRR). • Clear and concise written instructions for completing the manual data collection tool. • An overview of the study in written instructions.

34. Activity VIReliably collect data Evaluation Elements 10 and 11 for administrative data collection • Administrative data collection algorithms/flowcharts that show activities in the production of indicators. • An estimated degree of administrative data completeness.

35. Activity VIReliably collect data • Identification of data elements to be collected. • Clearly define the data elements to be collected. • If using HEDIS, submit Final HEDIS Audit Report. “Applicable codes: 304.XX (drug dependence codes), 305.XX (Nondependent abuse drug codes), mental disorders-broad range of codes: 209-319.” *taken from Activity IV page C-7

36. Activity VIReliably collect data • Identification of specified sources data. • Sources of data should be clearly specified. [ X ] Administrative Data Data Source [ X ] Programmed pull from claims/encounters “Transcription of data from the plan Discharge Summaries into free-standing database…”

37. Activity VIReliably collect data • A defined and systematic process for collecting baseline and remeasurement data. • A systematic method for data collection should be defined. • If NCQA vendor used, include vendor’s name. “Transcription of data from the plan Discharge Summaries into free-standing database…” “Inpatient providers (acute hospital and residential treatment facilities)…” “Data entry staff receives / prints discharge summary and reviews for completeness/accuracy…” “Data entry staff forwards discharge summary to the plan Medical Records for filing in member’s medical record.”

38. Activity VIReliably collect data • A defined and systematic process for collecting baseline and remeasurement datacont. “E-Solutions programmer/analyst writes “Co-Occurring Diagnosis Script.doc” (appended), which commands database to filter / select / extract data sets for this PIP.” “Data are populated into spreadsheet format for study leader to analyze.”

39. Activity VIReliably collect data • A timeline for the collection of baseline and remeasurement data. • Include starting/ending dates for all measurement periods. “Baseline: January 1, 2009 – December 31, 2009 Remeasurement 1: January 1, 2010 – December 31, 2010 Remeasurement 2: January 1, 2011 – December 31, 2011”

40. Activity VIReliably collect data • Qualified staff/personnel to abstract manual data. • Relevant education/experience/training of all manual data collection staff members should be included. • For this PIP, Evaluation Elements 5-9 of Activity VI would be scored Not Applicable because manual data collection was not used. This example contains only collection and analysis of administrative data.

41. Activity VIReliably collect data • A manual data collection tool that ensures consistent and accurate collection of data according to indicator specifications. (CE) • Include a copy or screenshot of the manual data collection tool. • For mailed surveys, include a copy of cover letter and survey. • For phone surveys, include a copy of script and training process for staff.

42. Activity VIReliably collect data • A manual data collection tool that supports interrater reliability (IRR). • Discuss in detail the IRR process. • Clear and concise written instructions for completing the manual data collection tool. • Include a copy or screenshot of the written instructions. • An overview of the study in written instructions. • Include a brief statement about the purpose of the study in the written instructions for the manual data collection tool.

43. Activity VIReliably collect data • Administrative data collection algorithms or flowcharts that show activities in the production of indicators. • Narrative/flowchart/algorithm that defines an ordered sequence of steps where each step depends on the outcome of the previous step. • “Data collection process: • Transcription of data from the plan Discharge Summaries... • Inpatient providers (acute hospital/residential treatment facilities)… • Data entry staff receives / prints discharge summary... • Data entry staff forwards discharge summary... • Data are populated into spreadsheet format…to analyze.”

44. Activity VIReliably collect data • An estimated degree of administrative data completeness. • Include estimated degree of completeness and a description of how the estimated degree of completeness was determined. “The data completeness of 97.5 % was based on the plan’s claims/lag report that is generated by our finance department on a quarterly basis. Ninety-five percent of medical claims are processed within 180 days of the date of service and 97% are processed within 30 days of receipt. The data for this study was extracted at least 120 days after the last day in the study period and thus, at least four months of claims run out was used to compute the rates.”

45. Activity VIIImplement intervention and improvement strategies VIIa:List interventions chronologically.

46. Activity VIIImplement intervention and improvement strategies VIIb:Describe interventions. • Whether they are related to causes/barriers identified through data analysis and quality improvement (QI) processes. (CE) • Whether they are system changes that are likely to induce permanent change. • Whether they are revised if original interventions are not successful. • Whether they are standardized and monitored if interventions are successful.

47. Activity VIIImplement intervention and improvement strategies • Whether they are related to causes/barriers identified through data analysis and quality improvement (QI) processes. (CE) • Describe causal/barrier analysis and how interventions are related. • “Prior to initiation of this PIP…completed a barrier analysis to identify possible root causes of the lack of post-inpatient (ambulatory follow-up) mental health and substance abuse services for members with COD. The primary identified barriers were: • Lack of knowledge…regarding the need for ensuring that members with COD receive mental health and substance abuse services post discharge; • Lack of knowledge…regarding the need to complete and submit accurate and complete discharge summaries;”

48. Activity VIIImplement intervention and improvement strategies • Whether they are system changes that are likely to induce permanent change. • Include interventions that will likely have a permanent effect. “Case management database was expanded to enable all fields from the inpatient discharge summaries to be captured for analysis.”

49. Activity VIIImplement intervention and improvement strategies • Whether they are revised if original interventions are not successful. • If repeat measures do not yield improvement, explain how problem solving and data analysis were performed to identify possible causes. • Identify revised interventions and explain how they were planned, developed and implemented. 01/10: Revised inpatient provider Discharge Summary Form to include substance related disorders services after discharge. 03/10: Case management database was expanded to enable all fields from the inpatient discharge summaries to be captured for analysis.

50. Activity VIIImplement intervention and improvement strategies • Whether they are standardized and monitored if interventions are successful. • If there was demonstrated improvement, document any interventions that were standardized and monitored. • “Clinical Department: • Monitored discharge summaries for substance related disorders …according to a standardized procedure. • Sent scorecards to providers on a monthly basis, along with a request for CAPs to be submitted to the Clinical Department for deficiencies. • The plan QI clinical reviewers measured compliance with COD service standards during regularly-scheduled audits of inpatient facilities and provided feedback on audit findings both informally during audits and formally via audit reports...”