1 / 80

June 13, 2011 2:00 - 4:30 p.m.

Data Collection and Reporting Methods for Effective Implementation, Evaluation, and Sustainability 11th Annual New England School of Prevention Studies. June 13, 2011 2:00 - 4:30 p.m. WELCOME!. Who Are We?. Craig Love, PhD, Chief of Epidemiology, CAPT

lance
Télécharger la présentation

June 13, 2011 2:00 - 4:30 p.m.

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data Collection and Reporting Methods for Effective Implementation,Evaluation, and Sustainability11th Annual New England School of Prevention Studies June 13, 2011 2:00 - 4:30 p.m.

  2. WELCOME!

  3. Who Are We? • Craig Love, PhD, Chief of Epidemiology, CAPT • Kristen Clements-Nolle, PhD, MPH, Epidemiologist, West Resource Team, CAPT • May Yamate, MS, Epidemiologist, Northeast Resource Team, CAPT

  4. What is SAMHSA’s CAPT? The Center for the Application of Prevention Technologies (CAPT) is a national technical assistance contract funded by the Substance Abuse and Mental Health Services Administration (SAMHSA).

  5. Activity 1: Getting to Know Each Other • Introduce yourselves to each other. • What is your name? • Where do you work? What do you do? • Two truths and a lie • Question: What do you want get from this course?

  6. Course Objectives - Day 1 • Learn why it is important to collect data • Learn how to use data to improve prevention activities and mobilize program support • Learn how to mine existing data sources to better understand a community’s drug and alcohol problems

  7. Course Objectives - Day 2 • Learn how to collect new data (e.g., define a target group, write and/or locate useful questions, implement a data collection effort that yields quality information) • Learn how to choose appropriate data collection methods (e.g., surveys, focus groups, key informant interviews) • Learn how to use data to mobilize groups to address issues raised by the findings

  8. What’s data got to do with it?2:00 – 4:30 p.m.

  9. Learning Objectives • Understand the importance of collecting data • Identify types of data—other than consumption and consequence data—that can be used to understand substance abuse • Describe the three data collection methods that can be used to obtain data related to substance abuse

  10. Why Collect Data? Reason #1 • Data can help you understand your community’s substance abuse problem • Magnitude • Trends • Comparisons • Seriousness/Severity

  11. Why Collect Data? (cont’d.) Reason #2 • Data can guide substance abuse prevention activities for your community. It can help you define: • target population • design • implementation

  12. Why Collect Data? (cont’d.) Reason #3 • Program evaluation requires data. • To determine effectiveness, including: • impact • accountability—to your agency, funders, and community • sustainability: helps garner the support and funding • To identify areas that may need to be improved • To identify ways to refine activities, as needed

  13. Why Collect Data? (cont’d.) Reason #4 • Data can help mobilize support from the community • They may not realize that a problem exists • They may overreact to a dramatic incident that is not representative of community problems • They may see the effective use of money spent on prevention Result • Sustainability: support from community, stakeholders, and funders

  14. Why Collect Data? (cont’d.) Reason #5 • Data support funding requests. It can show that: • a community has a real need • a community is making a difference • Funding applications supported by data are more likely to be funded than those that offer only anecdotal evidence of success

  15. Why Collect Data? (cont’d) Reason #6 • Funders (State, Federal, private) may require data. • to judge program effectiveness • to advocate for continued legislative support of programs

  16. Why Collect Data? (cont’d) Reason #7 • Data contribute to the field of substance abuse prevention

  17. Types of Data • Consumption indicators • Consequence indicators • Sociodemographics • Risk and protective factor indicators • Mental and behavioral health indicators • Community knowledge and attitudes

  18. Consumption and Consequence Indicators

  19. Sociodemographics • Gender • Age • Ethnicity • Race • Income • Poverty Level • Educational Attainment

  20. Risk and Protective Factor Indicators • Age of onset • Youth perception that parents approve of alcohol or drug use • Peers engaging in problem behavior • Early and persistent problem behaviors (e.g., risk-taking, high sensation-seeking) • Parental monitoring (or perception of monitoring)

  21. Risk and Protective Factor Indicators (cont’d.) • Parent or older sibling drug use (or perception of use) • Low perception of harm • Strong parent and adolescent relationship and family cohesion • Youth access and availability • Poor school achievement and school bonding

  22. Mental and Behavioral Health Indicators • Mental health diagnoses • Suicide attempts and completions • Major depressive episodes • Self-harm • Domestic abuse/child abuse

  23. Community knowledge and attitudes • How much of a concern is substance abuse in your community? • Are there any efforts to address substance abuse in your community? • Are substance abuse efforts supported by the community? • What is the community’s attitude about substance abuse? • Is local data about substance abuse available in your community?

  24. Finding Data • Existing data (national, local) • Survey data • Focus groups • Key informant interviews

  25. Mining Existing Data Sources • Handout 1 • Web resources • Download reports • Download datasets • Interactive database

  26. Review of Day 1

  27. Sneak Preview of Tomorrow • Surveys • Focus groups • Key informant interviews • Data makeover • Data dissemination

  28. Data Collection and Reporting Methods for Effective Implementation,Evaluation, and Sustainability11th Annual New England School of Prevention Studies June 14, 2011 8:30 - 4:30 p.m.

  29. Welcome Back! • Review of Day 1 • Overview of Day 2: • Designing and Conducting Surveys, Focus Groups, and Key Informant Interviews • Survey Design • Introduction to Focus Groups • Key Informant Interviews • Data Makeover • Data Dissemination

  30. Survey Design 9:00 – 10:45 a.m.

  31. Learning Objectives • Describe strengths and weaknesses of different modes of survey administration • Describe key considerations for deciding whether or not to use a survey • Identify problems with survey questions and strategies for addressing these problems • Develop an action plan for adapting, developing, and implementing surveys

  32. Activity 2: Puzzle Activity Instructions: On your table, you will find puzzle pieces that represent different types of data. As a group, please complete the puzzle. You have 10 minutes to complete this activity.

  33. Why Do a Survey? • When information for planning does not exist • When existing data are not culturally appropriate • When need to collect data across a large geographic area • When it’s important that items are consistent

  34. Modes of Administration • Face-to-face interviews • Telephone interviews • Self-administered questionnaires • In-person, mail, web • Combination of methods

  35. Face-to-Face Interviews • Advantages • High cooperation and low refusal rates • Allows for longer, more complex interviews • High response quality • Takes advantage of interviewer presence

  36. Face-to-Face Interviews (cont’d.) • Disadvantages • Costly • Takes longer than other methods • Interviewer bias • May not be good for sensitive information

  37. Telephone Interviews • Advantages • Less expensive than personal interviews • Faster than personal interviews • Covers a large geographic area

  38. Telephone Interview (cont’d.) • Disadvantages • Miss homes without phones and unlisted numbers • Low response rate • Difficult for discussing sensitive topics • Difficult to build rapport

  39. Self-Administered (Mail or Web) • Advantages • Generally lowest cost • No field interviewers needed • Respondents can take their time • No need for data entry (if web-based) • Good for collecting sensitive information

  40. Self-Administered (Mail or Web) • Disadvantages • Low response rate (often need incentive) • Must be very short; not good for complex issues • Difficult to clarify questions (no interviewer)

  41. Important Issues to Consider • Is a survey the best method? • Do you have appropriate resources? • What are your objectives? • Who is your target population? • How will you sample? • What mode of administration will you use? • Who needs to approve the survey? • Who owns the data? • How will the data be shared and used?

  42. Activity 3: Big Picture Exercise The Issue: • Your organization wants to conduct a survey to determine the frequency of nonmedical prescription drug use among young adults (18 to 24 years) in your community. • Work with other members at your table to answer Questions 1-7 in Activity #3. Record your results.

  43. Developing Good Survey Questions A good question: • Measures the underlying concept it is intended to measure. • Doesn’t measure other concepts. • Means the same thing to all respondents.

  44. Developing Good Survey Questions Avoid: • technical terms and jargon • vague questions • complex questions • double-barreled questions • biased or leading questions • double-negative questions

  45. Developing Good Survey Questions Include: • Reference frames (i.e., all responders’ answers should refer to same time and place) • All possible response options • Mutually exclusive response categories • Balanced response scales • “Don’t know” and “Prefer not to answer” options, when appropriate

  46. Activity 4: Developing Good Survey Questions • Review the following set of questions. • Identify the problem with each question. • Revise each question based on what you’ve learned about “good” survey questions.

  47. Introduction to Focus Groups 11:00 am – 12:00 pm

  48. Learning Objectives • Understand the strengths and limitations of using focus groups to collect data • Obtain basic tools to design focus groups • Understand how to develop good focus group questions

  49. What is a Focus Group? • A carefully planned group interview guided by a moderator • An interactive discussion on a predetermined topic • A qualitative research method to gather new data or validate existing data

  50. Use of Focus Groups for Prevention • At the start of a program, intervention or activity: • To design other data collection efforts • To guide the needs assessment process • To guide resource assessment activities • During a program, intervention or activity: • To improve or refine a program or activity • To inform process evaluation • After a program, intervention, or activity: • To inform outcome or impact evaluation

More Related