1 / 24

National Evaluation of the Demonstration to Improve the Direct Service Workforce

National Evaluation of the Demonstration to Improve the Direct Service Workforce. Sarah Hunter RAND corporation Laura Steighner American Institutes for Research November 16, 2009. Briefing Outline. Overview of Demonstrations and Evaluation Implementation Analysis Methods Findings

jburchfield
Télécharger la présentation

National Evaluation of the Demonstration to Improve the Direct Service Workforce

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. National Evaluation of the Demonstration to Improve the Direct Service Workforce Sarah HunterRAND corporation Laura SteighnerAmerican Institutes for Research November 16, 2009

  2. Briefing Outline • Overview of Demonstrations and Evaluation • Implementation Analysis • Methods • Findings • General Lessons Learned

  3. Principal Investigator John Engberg, Ph.D. RAND Project Advisor Donna Farley, Ph.D. RAND Team Leaders Outcomes Evaluation John Engberg, Ph.D. RAND Nick Castle, Ph.D. Univ of Pittsburgh Implementation Evaluation Sarah Hunter, Ph.D. RAND Laura Steighner, Ph.D. AIR Project Team Structure

  4. Overview of Study Demonstrations Evaluation Plan Multiple Case study Approach

  5. Demonstrations • Demonstrations were intended to improve recruitment and retention of direct service workers • 5 grants awarded in 2003, 5 grants awarded in 2004 • Grantees implemented various methods for accomplishing goals (details on next slide) • Web-based reporting overseen by Lewin • Technical assistance provided by Lewin and PHI • Each site had a local evaluator

  6. National Evaluation • Purpose • To provide a summative evaluation that would allow a comparison across the sites • Data Collection • Site visits • Surveys • Evaluation • Process (implementation) • Outputs (participation, initiative cost) • Outcomes (job satisfaction, consumer satisfaction, turnover, retention, vacancies, turnover cost)

  7. Multiple Case Study Approach • Understand implementation context across different grantees • Describe initiatives, implementation, and their perceived outcomes • Identify patterns across grantees • Explore costs and sustainability • Collected from multiple sources to increase validity • Reviewed extant documents (grant proposals, Lewin & PHI reports, quarterly reports) • Developed logic models in collaboration with grantees • Conducted telephone interviews and site visits and attended NFI conference

  8. Implementation Analysis Crosswalk of Initiatives Initiative Purpose Implementation Perceived Outcomes

  9. Crosswalk of Initiatives

  10. Health Care • Purpose • To provide coverage or improve accessibility to health care • Implementation • Demonstration is a difficult vehicle to study coverage • Variation across Grantees • Affordability • Sustainability • Skepticism over Free Good • Full-time status is a barrier to getting coverage • Health care coverage is complicated • Perceived Outcomes • Need for Coverage • Past Unmet Need • Improved Climate

  11. DSW Training • Purpose • To improve competence and job satisfaction among DSWs • Implementation • Training Content • Professional Development • Training Mode • Scheduling • Incentives for Participation and Completion • Participation by Supervisors/Managers • Voluntary versus Mandatory Participation • Perceived Outcomes • Training Effectiveness • Community Building among DSWs • Improved DSW Self-Care

  12. Supervisor / Consumer Supervisor Training • Purpose • To improve DSW job satisfaction by improving communications with supervisors and/or consumers • Implementation • Target Audience • Needs Assessment • Training Technique • Perceived Outcomes • Training Effectiveness • Retention and Turnover • Community Building among Participants

  13. Realistic Job Previews • Purpose • To set realistic candidate expectations for the job and improve retention • Implementation • Match between RJP Content and Job Context • Third-Party Implementation of RJP • Consumer Participation • Perceived Outcomes • Person-Job Fit • Turnover • Quality of Care

  14. Peer Mentoring Initiative • Purpose • To provide support to newly hired workforce • Implementation • Targeted new staff and mentors selected from veteran staff • Matching Process, Contact, Intensity, and Variability in Documentation across Grantees • Perceived Outcomes • Resistance among Supervisory Staff • Mentors enjoyed program • Unlikely to Improve Retention

  15. Recognition • Purpose • To improve DSW retention by providing either merit-based and longevity incentives • Implementation • Initiated longevity award with early retention goals (6th months, 1 year) and later added longer retention awards to acknowledge all staff • Logging Tenure, Applying for Award • Type of Recognition • Public Recognition • Perceived Outcomes • Staff Appreciation

  16. Worker Registry • Purpose • To improve both recruitment and retention by establishing better match between job and worker • Implementation • Development Time • Worker Eligibility • Consumer Eligibility • Perceived Outcomes • Income Stabilization • Balancing Supply and Demand • Consumer Choice • Employment and Costs

  17. Marketing • Purpose • To increase public awareness and recruit new DSWs to the field • Implementation • Understanding the Community • Mode of Marketing • Perceived Outcomes • Effectiveness of Mass Marketing Efforts • Effectiveness o f Targeted Marketing Efforts

  18. Targeted Recruitment Strategies • Purpose • To identify non-traditional populations to recruit to the field • Implementation • Sources for Recruitment • Limited Resources or Staff to Recruit Candidates • Overcoming the Barriers to Working as a DSW • Case Management Approach • Perceived Outcomes • Participation and Employment

  19. General Lessons Learned Marketing and Participation Implementation Evaluation Outcomes

  20. Marketing and Participation of Initiatives • Research the population you are trying to reach • Develop and showcase program champions • Support behavioral change through case management and modeling

  21. Implementation of Initiatives • Consider who is in charge • Initiatives take time • Don’t overwhelm workforce with options • Carefully plan incentives

  22. Local Evaluation of Demonstration • Consider the heterogeneity among worker sample • Engage local evaluators early and often • Don’t set up to assess a “moving target”

  23. Demonstration Outcomes • Consider ways to improve job satisfaction • Initiatives may have positive, unanticipated benefits • Improved Agency Support • Worker Community Building • Consider a “package” of initiatives

  24. Additional Information Contact Information Link to Report Sarah Hunter RAND(310) 393-0411 ext 7244shunter@rand.org Laura SteighnerAIR(202) 403-5064lsteighner@air.org http://www.rand.org/pubs/technical_reports/TR699

More Related