1 / 10

California Afterschool Outcome Measures Project

Research project by UC Davis and UC Irvine to develop and pilot outcome measures for afterschool programs in California, focusing on behavior change and skill development. The study involves surveys of students, staff, and teachers. Initial phase involved literature review and measure selection; current phase includes pre-piloting and piloting of protocols. Next phase will focus on field study.

schulman
Télécharger la présentation

California Afterschool Outcome Measures Project

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. California Afterschool Outcome Measures Project University of California, Davis and University of California, Irvine In collaboration with the California Department of Education and with support from the David and Lucile Packard Foundation

  2. Project Overview: Three Phases • Phase 1—INSTRUMENT DEVELOPMENT (2008-2009) • Literature review • Survey of the field (expected outcomes and current practices) • Selection of measures with input from the CDE and Accountability and Outcomes Subcommittee • Phase 2—PILOT STUDY (currently underway) • Pre-piloting to test procedures and instructions to students, staff, and teachers; and to determine feasibility of online administration of surveys • Piloting to test NEAR-FINAL protocols under varying conditions • Phase 3—FIELD STUDY (2010-2011)

  3. Outcome Measures • Surveys based on existing measures with demonstrated reliability and validity: • BEHAVIOR CHANGE (social efficacy, social skills with peers, child behavior, misconduct, and substance use) • SKILL DEVELOPMENT (work habits and task persistence) • Three sets of instruments: • Surveys of students who attend ASES afterschool programs • Surveys of ASES program staff • Surveys of classroom teachers

  4. Profile of Pre-Pilot Sites • Nine pre-pilot sites—5 Southern CA, 4 Northern CA • 5 elementary school sites • 3 intermediate/middle school sites • 1 center-based site serving K-8 • Seven school districts • 3 urban • 3 suburban • 1 rural • Range of school/program sizes • School enrollments 478 - 1,519 • ASES enrollments 81 - 475

  5. Pre-Pilot Activities • Staff directed the administration of the online surveys to groups of students (2 to 6 groups per site) • Staff were provided a script and guidelines about the survey administration • Students completed online surveys about their skill development and behavior and about their experience taking the survey • Researchers observed surveying process and noted student feedback • ASES staff and classroom teachers completed the online surveys about individual students’ behavior and were interviewed about their experiences completing the surveys • Instructions and surveys were tweaked (mid pre-pilot) based on issues identified in the observations and interviews

  6. Pre-Pilot PhaseNovember 2009—February 2010 • Completed the test of initial versions of measures • Completed the test of initial protocols and procedures • Tested feasibility of online administration • Minor modifications of procedures and protocols during the pre-pilot

  7. Pre-Pilot Data Collected

  8. Survey Data Summary Tables Elementary School Students Middle School Students

  9. Survey Data Summary Tables Teachers Program Staff

  10. Next Phase: PilotMarch—June 2010 • Pilot measures at a broad spectrum of ASES programs • 12 Northern CA and 12 Southern CA • Size of program (small, medium, large) • Grade level (8 elementary, 4 middle/intermediate) • Program status (challenged to well functioning) • Location (urban, suburban, rural) • Diverse student characteristics (ethnicity, % ELL and FRL) • Purpose of pilot: • Identify issues at a range of programs with & without our physical presence • 12 sites with UC researchers present • 12 sites to administer independently [sent packet; support by email and phone only] • Test administration of teacher survey at 8 elementary sites • 4 with teacher compensation • 4 without teacher compensation • Test hard copy administration of student survey at sites without computer and internet access • Determine reliability of the measures under these different conditions

More Related