1 / 32

Measuring Our Success: Providing Evidence for CTE Outcomes

Measuring Our Success: Providing Evidence for CTE Outcomes. KC Greaney , Santa Rosa Junior College Mallory Newell, De Anza College Terrence Willett, RP Group. RP Conference, Kellogg West, CA April 2013.

gaetano-lavery
Télécharger la présentation

Measuring Our Success: Providing Evidence for CTE Outcomes

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Measuring Our Success: Providing Evidence for CTE Outcomes KC Greaney, Santa Rosa Junior College Mallory Newell, De Anza College Terrence Willett, RP Group RP Conference, Kellogg West, CA April 2013

  2. -CTE Survey Outcomes - CTE Launchboard Project- CTE Users Feedback Survey- CTE OS Data Visualization Tool A Brief Overview Measuring Our Success, RP Conference, April 2013

  3. 2012 Statewide Effort:CTE Employment Outcomes Survey Pioneered at Cabrillo College over past 12+ years • Replicated at other individual colleges including Chaffey, Los Rios, Riverside, Mt. SAC, Santa Monica Statewide effort instigated by Bay Area Vocational Deans Measuring Our Success, RP Conference, April 2013

  4. CTE Employment Outcomes Survey Collaborative effort among community college consortia, the Chancellor’s Office, and the RP Group Dual purposes • Provide insightful information for local program improvement • Collect statewide data for documentation of CTE outcomes Practitioner-Driven Effort • Statewide meetings, conference calls, collaboration • All methodology, implementation and survey questions vetted • Report format developed in response to practitioner needs Measuring Our Success, RP Conference, April 2013

  5. CTE Employment Outcomes Survey Methodology 2 Surveys: Completers, Leavers 3 Modalities: e-mail, then US mail, then phone calls Survey conducted approximately 1.5 years after completing/leaving Sample 15 Colleges All students included who met the criteria • By 2009/10, had either earned a vocational award of 6+ units OR met VTEA “concentrator” definition AND did not enroll in 2010/11 Total = 15,105 (7,681 Completers and 7,424 Leavers) Measuring Our Success, RP Conference, April 2013

  6. CTE Employment Outcomes Survey Data Chancellor’s Office MIS data wherever possible Local data supplement (student contact information) Survey data Complete data set provided to colleges for further analysis Measuring Our Success, RP Conference, April 2013

  7. CTE Employment Outcomes Survey Response Rates Overall, 30% (Completers: 34%, Leavers: 26%) By College, overall response rate ranged from 19% to 37% Reports Statewide reports for Completers and Leavers College-level reports for Completers and Leavers Measuring Our Success, RP Conference, April 2013

  8. Statewide Findings: Completers The big questions… Are our CTE students employable? Do they find jobs in the field in which we trained them? Do they see an increase in income? …in other words, was the program worth it? Measuring Our Success, RP Conference, April 2013

  9. Statewide Findings: Completers 34% Response rate 94% Satisfaction rate with education and training 30% transferred (25% pursuing related field) 77% working in their field of study (or “close”) 73% are interested in taking more classes or training in their field in the near future 45% successfully engaged in a job search, 14% did not (the remainder were not looking or were already employed) Measuring Our Success, RP Conference, April 2013

  10. Findings: Completers 79% of respondents are employed for pay Measuring Our Success, RP Conference, April 2013

  11. Findings: Completers More students worked full-time after completing studies and training Measuring Our Success, RP Conference, April 2013

  12. Findings: Completers The hourly wage of degree/certificate earners increased 39% from their hourly wage before their studies ($17.78) to their hourly wage after completing a degree or certificate ($24.79) Measuring Our Success, RP Conference, April 2013

  13. Statewide Findings: Leavers The big questions… Are our CTE leavers “losers”? Or have we failed them? Are leavers employable? Do they find jobs in the field in which we trained them? Do they see an increase in income? …in other words, were the courses and training worth it, even if the student did not complete a program of study? Measuring Our Success, RP Conference, April 2013

  14. Statewide Findings: Leavers Similar to Completers, except: Lower Response Rate (26% versus 34% for Completers) Fewer with the goal of earning a certificate or degree (60% versus 80% for completers) Twice as many with the goal of upgrading job skills and/or renewing a license/permit (19% versus 10% for Completers) Slightly higher transfer rate (34% versus 30% for Completers) Slightly fewer report working in their field of study (or “close” – 70% versus 77% for Completers) Earnings increase is lower (26% increase versus 39% for Completers) Measuring Our Success, RP Conference, April 2013

  15. Basic Demographics Measuring Our Success, RP Conference, April 2013

  16. “Swirl” Measuring Our Success, RP Conference, April 2013

  17. CTE Survey: Changes in 2013 Implement all the lessons we learned from the pilot year 35 colleges participating Move away from “Completers” versus “Leavers” • Move towards “CTE Skill Builders” Broaden cohort to students who completed 9+ units of SAM-coded vocational courses (to include those who completed a certificate or vocational degree) Tighter timeline (reports out by the end of June) Add questions re: industry certification, impact of coursework, more responses taking transfer into account Measuring Our Success, RP Conference, April 2013

  18. CTE Survey: Changes in 2014 Santa Rosa Junior College will continue to administer survey, working directly with interested colleges Chancellor’s Office endorses survey and will continue to define cohort via MIS and provide basic data RP Group is supportive, but no longer directly involved CCCAOE continues to support survey • At Spring 2013 annual meeting, the CCCAOE “Excellence in Collaboration” award given to RP Group (Kathy Booth), Bay Area Vocational Deans Group (Rock Pfoetenhauer and Kit O’Dougherty) and Santa Rosa Junior College (KC Greaney) CalPASS+ will host CTEOS survey data in future Measuring Our Success, RP Conference, April 2013

  19. CTE User Feedback Survey Measuring Our Success, RP Conference, April 2013

  20. CTE User Feedback Survey A link to the survey was emailed to all college contacts who participated in the survey in the 2011-12 academic year Received 7 valid responses Respondents suggested that the results had largely been used to inform decision making in the areas of: • Enrollment management • Accreditation • Perkins or other reporting • Program improvements • Changes to program degree offerings Measuring Our Success, RP Conference, April 2013

  21. CTE User Feedback SurveyAdditional Highlights Respondents found the following to be most valuable about the survey: • Ease of administration by the RP Group • The multiple ways students were contacted to increase response rates • The overall findings from the survey from my college • The model report that was provided Respondents stated the following technical support would have been beneficial: • Assistance with the initial data pull of students • Assistance with analyzing the results • Examples of how other colleges shared the results and with whom Measuring Our Success, RP Conference, April 2013

  22. Launchboard Project Measuring Our Success, RP Conference, April 2013

  23. CTE Launchboard 4 Community Colleges participating in the pilot Includes data elements from “original” Launchboard Metrics (OLB) and metrics from the “Common Metrics” (CMLB) associated with the Doing What Matters Framework: http://doingwhatmatters.cccco.edu/. Currently includes 67 data elements All metrics are presented by program TOPS code Some metrics presented by region as defined in the Doing What Matters Framework Includes data from: COMIS, CalPASS Plus, EDD, Centers for Excellence, and data files submitted by colleges Measuring Our Success, RP Conference, April 2013

  24. CTE Launchboard Includes measures in the following areas: Alignment – middle school to high school and high school to CCC Enrollments – course completion in high school and CCC in CTE pathway Capacity – FTEF, students served in CTE programs by CCC, students served in similar programs in non CCC colleges Support – completed work readiness program, internship, took basic skills course, received financial aid, DSPS or EOPS Return on Investment – number of years of income to equal cost per students Measuring Our Success, RP Conference, April 2013

  25. CTE Launchboard Milestones Completed major requirements in a CTE area (Primary Goal) (CMLB) Earned more than 8 units in the same TOP code within 3 years of first enrollment (OLB) Term-to-term retention (OLB) Completed college-level English or math within 3 years of first enrollment (OLB) Course completion and success (OLB) Measuring Our Success, RP Conference, April 2013

  26. CTE Launchboard Credentials Industry-recognized certificates (OLB) Non-credit certificates (OLB) Chancellor’s Office approved credit certificates in and not in CTE pathway (OLB) Associate’s degree in and not in CTE pathway (OLB) Transfer in and not in CTE pathway (CMLB) Measuring Our Success, RP Conference, April 2013

  27. CTE Launchboard Employment: Students are employed in same or similar field(OLB) Wage gain in field (OLB) Wage gain for students earning 9+ units (OLB) Average wage of students who are employed in same or similar field (OLB) Job retention (OLB) Measuring Our Success, RP Conference, April 2013

  28. CTE Launchboard Demand: Recent demand in Doing What Matters Economic Region (OLB) Projected demand in Doing What Matters Economic Region (OLB) Present demand in Doing What Matters Economic Region (OLB) Median (50th percentile) Hourly Earnings (OLB) Measuring Our Success, RP Conference, April 2013

  29. CTE Launchboard Live look at the Launchboard: http://www.calpass.org/ Measuring Our Success, RP Conference, April 2013

  30. CTE OS Data Visualization Tool Proposed Help users access CTE Outcomes Survey data with web based reporting tool Measuring Our Success, RP Conference, April 2013

  31. Questions? Measuring Our Success, RP Conference, April 2013

  32. Find Out More KC Greaney, Santa Rosa Junior College kgreaney@santarosa.edu Mallory Newell, De Anza College newellmallory@fhda.edu Terrence Willett, RP Group twillett@rpgroup.org (831) 461-4586 http://www.rpgroup.org/projects/CTE-Employment-Outcomes-Survey

More Related