We Will Be Starting Shortly

1. Welcome to This CAPT Webinar Evaluating Retail Access Campaigns June 3, 2010 We Will Be Starting Shortly

2. Introduction to Connect Pro Welcome to Connect Pro. We will be using the following windows: • PowerPoint Window • Attendee List • Chat • Note We will also use the “Raise your Hand” feature

3. Introduction to Connect Pro Two ways to ask questions By Writing: • Write question in chat window • Press enter • Only the moderator will see the question

4. Introduction to Connect Pro By “Raising your Hand:” Do this by clicking • You will see your status change in attendees list • Un-mute your phone (press * #) when called upon • Remember to mute your phone again once you have finished speaking • Lower your hand by again clicking

5. Today’s Presenters Helaine Hornby, MA Sarah Krichels Goan, MPP Hornby Zeller Associates, Inc. Lourdes Vázquez Training and Technical Assistant Specialist CSAP’s Northeast Regional Expert Team

6. Introductions Erie County Office of Drug and Alcohol Abuse Lackawanna County Commissioner Crawford County Drug and Commissioners Schuylkill County Commissioners Mercer /County Behavioral Health Commission, Inc. Berks County Council on Chemical Dependency Washington County Drug and Alcohol Commission Bucks County Council on Alcoholism and Chemical Dependency Greene County Human Services Program Delaware County Office of Behavioral Health Montgomery County MH/MR D & A Behavioral Health Armstrong/ Indiana D&A Commission Lancaster County Drug & Alcohol Commission Juniata Valley Tri-County Drug and Alcohol Abuse Commission Westmoreland Drug and Alcohol Commission, Inc. Chester County Department of Drug & Alcohol Services Bedford Personal Solutions, Inc.

7. Today’s Learning Objectives Retail Access Strategies and Evaluation • Make key decisions in designing the evaluation • Select appropriate methods and resources • Analyze the data and draw conclusions

8. Access Strategies in PA • Responsible Alcohol Management Program (RAMP) • Alcohol Purchase Survey • Sticker Shock and Shoulder Taps (social access strategies in a retail setting)

9. Part I Making key decisions about evaluation

10. Things to Consider for Evaluation • Strategy Purpose • e.g., change behavior, change policy • Strategy Target • e.g., retailers/merchants, furnishers • Strategy Focus • e.g., what community factors are being addressed?

11. Responsible Alcohol Management • Purpose: help licensees and their employees to serve alcohol responsibly. • Target: licensees operating restaurants, hotels, clubs, distributors. • Focus: alcohol sales to minors and visibly intoxicated persons.

12. Alcohol Purchase Survey • Purpose: obtain information about illegal sales to minors and increase merchant perception of being monitored. • Target: off-premise establishments selling alcohol. • Focus: alcohol sales to minors.

13. Process Evaluation Questions • How many establishments are there in the community? How many were approached? Engaged? How many staff participated? • To what degree have merchants implemented RAMP guidelines? • To what degree have merchants sustained RAMP guidelines? • How many establishments were surveyed?

14. Short-Term Questions • How many attempted purchases are made/ rejected? • How many have policies/procedures that follow the best practice guidelines? • To what extent are staff and management aware of PA law, policy and liability? • How many establishments have participated in RAMP as a result of APS activities? Measures: Number of sales /number of rejections; number of policies implemented/changed; knowledge of staff

15. Medium-Term Questions • To what extent have compliance rates improved? • To what extent are citations related to underage drinking decreasing? • To what extent are youth obtaining alcohol from local establishments? Measures: Rates of compliance; number of citations for intoxication/DUI; source of alcohol for citations.

16. In-Store Social Access Strategies • Purpose: raise awareness about PA furnishing laws and deter adults from providing alcohol to minors. • Target: adults who are legally purchasing alcohol. • Focus: provision of alcohol to minors (social access).

17. Process Evaluation Questions • How many retail alcohol outlets are there in the defined community? • How many establishments were approached? Of those, how many participated? • How many materials were distributed? How many activities were conducted? • How many adults were approached? • How many were willing/unwilling to purchase?

18. Short-Term and Medium-Term Questions • Short-Term: Are more adults aware of the laws and penalties associated with furnishing? • Medium-Term: • To what extent are citations for furnishing alcohol to a minor decreasing? • Are fewer adults likely to furnish alcohol to minors? • To what extent do youth report obtaining alcohol from adults? Measures: Number of citations furnishing; reports of likelihood of providing; source of alcohol for citations.

19. Long-Term Questions • Is there a change in youth perception that it is easy to obtain alcohol? • Have rates of past-month alcohol consumption declined? • Is there a decrease in the rate of alcohol-related car crashes/fatalities? DUIs? Measures: Perceived ease of access; rate of past month use; alcohol-related car crashes/fatalities in target age group.

20. PART II METHODS FOR DATA COLLECTION

21. Sources of Evaluation Data • Pre- and Post-Knowledge Tests • Retailer Logs/Records • Retailer Self-Assessment • Environmental Observations • Purchase Survey Data • Brief Exit Interview • Police Department Logs/Records • Existing Survey Data

22. Pre- and Post-Test (Knowledge) • Brief survey reflecting 5 to 10 key components of a training • Conducted before and after a training with participants to track immediate learning acquisition • Can be used as follow-up measure of sustainability

23. Retailer Alcohol-Related Occurrence Log • Simple log on which staff keep track of alcohol-related occurrences (suspicious, questionable or problematic) • Coalition can examine incidents by date/time, staff, attempt type, actions taken • Data from multiple establishments allows for community-level analysis and trends over time

24. Pennsylvania Liquor Board Incident Documentation Form

25. Retailer Self Assessment • Self-administered/reported assessment survey or rating checklist • Captures management and hiring procedures, intent to change or actual changes, new employee training, knowledge of laws • Can be asked of managers and staff periodically

26. Card ME On Premise Licensee Checklist

27. Environmental Observations • Brief scan for environmental factors that promote/prevent access to alcohol • Observe practices and environment at local on-premise establishments (and retail establishments) • Track when scans occur and which establishments participated in RAMP training (and when)

28. Business Alcohol Scan

29. Purchase Survey Data • Observations and key details outlining purchase attempts • Recorded into standard form by the person attempting to purchases • Results can be aggregated, analyzed and tracked over time

30. Communities Mobilizing for Change in Alcohol (CMCA) Alcohol Outcome (Buyer Form)

31. Brief Exit Interviews • Brief interview addressing exposure, attitudes, intent to change behaviors. • Can be asked of patrons exiting a store • Youth: Saw signage, signage as deterrent, perceived ease of access, knowledge of laws • Adults: Saw stickers, stickers as deterrent, willingness to furnish, knowledge of laws

32. Sticker Shock: Team Feedback Form

33. Police Department Records • Local law enforcement records relating to alcohol-related activities or citations conducted by police • Include compliance checks, violations and citations data • Small departments may lack electronic records • Consider creating a simple worksheet

34. Existing Data • Pennsylvania Youth Survey (PAYS) • National Survey on Drug Use and Health (NSDUH) • NOMs Community Survey • Fatality Analysis Reporting System (FARS)

35. PART IIIPUTTING IT ALL TOGETHER: ANALYSIS AND CONCLUSIONS

36. Comprehensive Approach • Use a combination of measures and methods to gauge success • Use appropriate data sources • Look at medium- and long-term measures in relation to short-term ones • Present data over time • Where measures are missing, use state or national data sources

37. Use Appropriate Data Sources

38. Comprehensive Approach: Example 1 Courtesy of the Regional Center for Healthy Communities (Metrowest) www.healthier-communities.org

39. Comprehensive Approach: Example 2 • By Year 5, 21 out of 28 (75%) local establishments had participated in RAMP, training 112 employees. • Across all years, 82% of staff were sufficiently knowledgeable 6 months after RAMP training, (e.g., identify fake identification, PA laws, store policies). • Between Years 3 and 5 of RAMP implementation, youth reports that alcohol was easy to obtain decreased by 13% (from 79%).

40. Comprehensive Approach: Example 3 • By Year 5, 21 out of 28 (75%) local establishments had participated in RAMP, training 112 employees. • In Year 5, 19 out of 28 establishments (68%) reported training new employees, and logs show that 73 attempts were rejected due to improper identification (52) or visible intoxication (21).

41. Discussion and Sharing

42. Next Steps • Webinar materials available: • EZ Learning Management System: • http://learn.aero.und.edu • Webinar evaluation: An email will be sent to you to complete the evaluation. Please respond!

43. Contact Information Helaine Hornby, MA Sarah Krichels Goan, MPP Hornby Zeller Associates, Inc. 373 Broadway South Portland, ME 04101 (207) 773-9529 hhornby@hornbyzeller.com sgoan@hornbyzeller.com Lourdes Vázquez, MSW Training and Technical Assistant Specialist CSAP’s Northeast Regional Expert Team lvazquez@edc.org