National Data and the Picture it Presents

1. National Data and the Picture it Presents NACTEI THURSDAY MAY 9, 2013 EMBARGOED UNTIL 9/30/2013 JOHN HAIGH CHIEF PERFORMANCE AND ACCOUNTABILITY BRANCH OVAE/DATE US DEPARTMENT OF EDUCATION WASHINGTON, DC

2. Accountability Match Game _____CEDS _____ARRA _____Waiver/EDFLEX _____Blueprint _____BLS _____NAICS _____Apprenticeships _____RPOS _____SPAC _____Clusters _____PCRN _____EDEN/EDFacts _____NACTE _____OSDS _____DQC A State accountability folks interested in indicator consistency B Six studies of CTE participation and outcomes C Administrations outline for reauthorization of Perkins D Refined systems of school accountability that includes growth E Education categories of industries / employment F Post training field experience G National evaluation of CTE H Website for CTE resources and information I Agency responsible for measuring labor market activity J Federal funds used for school reform K Voluntary standards for a key set of educational data elements L Allows for flexibility from key provisions of NCLB M National standards for classifying business established for collecting, analyzing, and publishing statistical data N System to assist with analysis of supply and demand of the labor market O Provides national forum to facilitate cross-state sharing of data

3. CTE Education Level Participation Trend Table

4. CTE Education Level Participation Trend Graph

5. CTE Gender Participation Trend

7. Gender Participation Trend Graph

8. Secondary Ethnicity Participation Trend

9. Secondary Ethnicity

10. Postsecondary Ethnicity Participation Trend

11. Postsecondary Ethnicity

13. Secondary & Postsecondary 2011-2012 Career Clusters

14. BLS Selected Occupational Projections Data Matched With Similar Career Clusters

15. Programs of study are those activities and programs that meet or exceed the State adjusted levels of performance as stated in the requirements of the State plan Section 122: • ‘‘(c) PLAN CONTENTS.—The State plan shall include information that— • ‘‘(1) describes the career and technical education activities to be assisted that are designed to meet or exceed the State adjusted levels of performance, including a description of— • ‘‘(A) the career and technical programs of study, which may be adopted by local educational agencies and postsecondary institutions to be offered as an option to students (and their parents as appropriate) when planning for and completing future coursework, for career and technical content areas that— • ‘‘(i) incorporate secondary education and postsecondary education elements; • ‘‘(ii) include coherent and rigorous content aligned with challenging academic standards and relevant career and technical content in a coordinated, non-duplicative progression of courses that align secondary education with postsecondary education to adequately prepare students to succeed in postsecondary education; • ‘‘(iii) may include the opportunity for secondary education students to participate in dual or concurrent enrollment programs or other ways to acquire postsecondary education credits; and • ‘‘(iv) lead to an industry-recognized credential or certificate at the postsecondary level, or an associate or baccalaureate degree;

16. 3 Year Trend of Secondary Performance and States Missing by 90%

18. 3 Year Trend of Secondary Performance and States Missing by 90%

19. 3 Year Trend of Secondary Performance and States Missing by 90%

20. 3 Year Trend of Secondary Performance and States Missing by 90%

21. 3 Year Trend of Postsecondary Performance and States Missing by 90%

23. 3 Year Trend of Postsecondary Performance and States Missing by 90%

24. 3 Year Trend of Postsecondary Performance and States Missing by 90%

25. AN EVALUATION OF TEXAS CAREER AND TECHNOLOGY EDUCATIONPROGRAMS FUNDED UNDER THECARL D. PERKINS VOCATIONALEDUCATION ACT Region VI Education Service Center3332 Montgomery Rd. Huntsville, TX 77340936-435-8400 www.esc6.netFinal Report June 2007Prepared for the Texas Education Agency • Purpose • The purpose of the CTE evaluation study was to gather and analyze both quantitative and qualitative data and information that would provide measurable evidence of the effectiveness of CTE programs in Texas in reaching the goals for students set by the Texas Legislature: • “Each public school student shall master the basic skills and knowledge necessary for: • 1. managing the dual roles of family member and wage earner; and • 2. gaining entry-level employment in a high-skill, high wage job or continuing the student’s education at the post-secondary level” (Texas Education Code, § 29.181). • In addition to the objective student performance data gathered by TEA, this evaluation collected information regarding the perceptions of stakeholders involved in the CTE program at the secondary level. Worthen and Sanders (1987) note that while there may be a statistical significance in measures like test scores, the educational and practical significance or value of a program must also be judged against what is delivered relative to what is expected by stakeholders. For this reason, it was important that the evaluation provide an opportunity for various audiences to review and react to the program and its impact.

26. AN EVALUATION OF TEXAS CAREER AND TECHNOLOGY EDUCATIONPROGRAMS FUNDED UNDER THECARL D. PERKINS VOCATIONALEDUCATION ACT Region VI Education Service Center3332 Montgomery Rd. Huntsville, TX 77340936-435-8400 www.esc6.netFinal Report June 2007Prepared for the Texas Education Agency • Study Objectives • The overall objectives of the CTE evaluation study are: • 1) to determine the scope and effectiveness of statewide professional development training for CTE teachers during each of the 2000-2005 school years; • 2) to determine the relationship between the size, scope and quality of CTE programs and the attainment of academic skills and increase in graduation rates; • 3) to determine the effect that the Tech-Prep program of study, Advanced Technical Credit (ATC) opportunities, and dual credit opportunities have on improving graduation rates and post-secondary enrollment, retention and completion rates; • 4) to examine the effectiveness of curricular Career and Technology Student Organizations (CTSOs) in developing leadership skills, keeping students engaged in school, and demonstrating academic and technical skill attainment; • 5) to evaluate the effectiveness of the following special projects in achieving their stated goals: • a. Project Lead The Way (PLTW) – increasing academic and technical skills • b. Special Populations Resource Center (SPACE) – providing resources and technical assistance to teachers to meet the needs of special populations • c. Advanced Technical Credit – the documentation of the application, training, and eligibility approval of ATC program teachers • 6) to evaluate the effectiveness of the six Educational Excellence grants, including the quality of curriculum resources provided to teachers.

28. AN EVALUATION OF TEXAS CAREER AND TECHNOLOGY EDUCATION PROGRAMS FUNDED UNDER THECARL D. PERKINS VOCATIONALEDUCATION ACT Region VI Education Service Center3332 Montgomery Rd. Huntsville, TX 77340936-435-8400 www.esc6.netFinal Report June 2007Prepared for the Texas Education Agency The following are the key findings from the evaluation: • CTE enrollment data show the percentage of each Public Education Information Management System (PEIMS) subgroup population is represented quite similarly to the non-CTE student population. However, the Hispanic subgroup consistently has at least 4.7% more students enrolled in the Tech-Prep program than in the non-CTE program. The white subgroup, however, consistently has over 3.4% fewer students enrolled in the Tech-Prep program than in the non-CTE program.

29. AN EVALUATION OF TEXAS CAREER AND TECHNOLOGY EDUCATION PROGRAMS FUNDED UNDER THECARL D. PERKINS VOCATIONALEDUCATION ACT Region VI Education Service Center3332 Montgomery Rd. Huntsville, TX 77340936-435-8400 www.esc6.netFinal Report June 2007Prepared for the Texas Education Agency Focus group participants indicated that Career and Technology Student Organizations (CTSOs) are a high priority due to their focus on real-world skills, and many administrators and teachers felt that these organizations are effective in developing students’ leadership skills and other positive outcomes. However, the student participation rates do not reflect this priority. Future Farmers of America (FFA) was considered the most effective by school staff and had the highest proportion of participating students. • Administrator and teacher survey responses indicated that many respondents were “neutral” with regard to the Project Lead the Way (PLTW) and Special Projects Resource Center (SPACE) programs. This may be due to a lack of knowledge about the programs. • There is a performance gap between CTE and non-CTE students on both the Texas Assessment of Knowledge and Skills (TAKS) and Higher Education Readiness Component (HERC),

30. AN EVALUATION OF TEXAS CAREER AND TECHNOLOGY EDUCATION PROGRAMS FUNDED UNDER THECARL D. PERKINS VOCATIONALEDUCATION ACT Region VI Education Service Center3332 Montgomery Rd. Huntsville, TX 77340936-435-8400 www.esc6.netFinal Report June 2007Prepared for the Texas Education Agency The analyses of TAKS and HERC results also show that the more structured the CTE program, the better the students performed on the reading/ELA TAKS and HERC, although still not as well as the non-CTE students. This dynamic is especially true for students from ethnic minority backgrounds on the reading HERC. Students of Hispanic origin participating in Tech-Prep programs outperformed non-CTE students of Hispanic origin on the reading HERC. • Findings on CTE’s impact on graduation rates are encouraging. CTE students are more likely to remain in school and graduate than non-CTE students. This difference grows as the structure of the CTE program increases as well. • Analyses of the HERC and graduation plans show that CTE students who plan to go to college lag behind non-CTE students who plan to go college in meeting the HERC components in reading and math.

31. CAREER AND TECHNICAL EDUCATION EVALUATION, Austin Independent School District Department of Program Evaluation February 2010 District questions: • What were the demographic characteristics of students in the CTE program, and how did they compare with characteristics of the general student population? • How did the academic achievement of CTE students (as measured by the Texas Assessment of Knowledge and Skills, or TAKS) compare with that of non-CTE students? • Did taking a series of CTE courses play a role in determining whether students pursued postsecondary education upon graduation? • Did taking a series of CTE courses play a role in determining whether students were working right after graduation?

33. CAREER AND TECHNICAL EDUCATION EVALUATION, Austin Independent School District Department of Program Evaluation February 2010 The demographic analysis found that just over half of high school students in AISD (55%) took a CTE course in 2008–2009, although the percentage varied widely by school. About 15% of students took a sequence of courses that taught them skills for a specific career; these students were considered CTE course sequence takers. On average, larger schools and those with a higher percentage of economically disadvantaged students had a higher percentage of CTE students and CTE course sequence takers than other schools.

34. CAREER AND TECHNICAL EDUCATION EVALUATION, Austin Independent School District Department of Program Evaluation February 2010 Similar to CTE students nationally, CTE students in AISD were different from the district’s general student population. A significantly higher percentage of CTE students (students who took at least one CTE course) and CTE course sequence takers than general district students were male, Hispanic, and economically disadvantaged. Female, Asian, white, and special education students were underrepresented among those who took CTE courses. Female, white, African American, special education, and English language learning (ELL) students were underrepresented among CTE course sequence takers, compared with non-sequence takers and the district as a whole.

35. CAREER AND TECHNICAL EDUCATION EVALUATION, Austin Independent School District Department of Program Evaluation February 2010 • The number and type of CTE courses available at each campus played a role in determining the demographic characteristics of CTE students. • Increasing the number and diversity of courses available at each campus, and ensuring that programs attractive to both male and female students are offered, would likely diminish the demographic differences between CTE and non-CTE students. • Demographic differences also could be the result of students seeing CTE as a program for the non-college-bound because the characteristics of CTE students mirrored those of students who did not attend college.

36. CAREER AND TECHNICAL EDUCATION EVALUATION, Austin Independent School District Department of Program Evaluation February 2010 CTE course sequence takers had higher TAKS scores and passing rates on all TAKS tests, on average, than other students. When students who took random CTE courses were included in the analysis, the TAKS passing rate for most tests remained significantly higher for CTE students; however, their average TAKS scale scores were lower than the TAKS scores of non-CTE course takers. These results demonstrated that CTE was successful, especially with course sequence takers, in attracting high achievers and was not, as historically has been the case nationally, a “dumping ground” for low achievers.

38. CAREER AND TECHNICAL EDUCATION EVALUATION, Austin Independent School District Department of Program Evaluation February 2010 CTE participation did not appear to play a role in determining whether students pursued postsecondary education or work after graduation. Although CTE might have been successful at preparing students for postsecondary education, other factors, such as economic disadvantage, may have been stronger determinants of postsecondary enrollment. It is not clear why CTE course sequence taker status did not play a role in whether students were working after graduation. More research is needed to determine whether the skills taught in CTE classes helped students obtain employment.

39. Mini Meta Chart

41. Pilot Project on Improving Data Exchange Between Industry Certification Organizations and State Education and Workforce Longitudinal Data Systems The project is based on a multi-year roadmap for the development of a national data exchange clearinghouse that will allow states and educational institutions to gain access to data on industry-recognized certifications earned by students. Current industry partners include CompTIA and The Manufacturing Institute (an affiliate of the National Association of Manufacturers, or NAM). The Department of Education is also interested in the development and outcome of this project. Industry certifications and licensure are a growing part of the expected outcomes of career and technical education (CTE) programs, and the Department is interested in the identification of solutions to the issues surrounding state collection of valid and reliable data. ACTE, in cooperation with a number of national and state partners (listed below), is supporting a project to expand and improve data exchange between industry certification organizations and state longitudinal data systems.

42. Pilot Project on Improving Data Exchange Between Industry Certification Organizations and State Education and Workforce Longitudinal Data Systems • States Selected for the First Round: • California Kansas Maryland North Carolina Ohio Oklahoma • Certification Project Advisory Committee: Keith Brown, SLDS State Support Team •  Marie Buker, Department of Education •  Ross Cohen, U.S. Chamber of Commerce •  Christopher Coro, Department of Education •  Dean Folkers, National Association of State Directors of CTE Consortium •  Pam Frugoli, Department of Labor •  Amilcar Guzman, Data Quality Campaign  •  John Haigh, Department of Education •  Lisa Hudson, National Center for Education Statistics •  Gretcher Koch, CompTIA •  Stanley Koutstaal, Department of Health and Human Services •  Kate Louton, WRIS •  Mary Alice McCarthy, Department of Education  •  Jennifer McNelly, National Association of Manufacturers •  Scott Parke, Florida College System •  Tom Richardson, ASE •  Doug Shapiro, National Student Clearinghouse  •  Robert Sheets, University of Illinois at Urbana-Champaign  •  David Stevens, FEDES •  Roy Swift, ANSI   •  Robin Utz, Department of Education •  Andy Van Kleunen, National Skills Coalition •  Andrala Walker, Department of Labor     •  David Wilcox, GSX  •  Nathan Wilson, Illinois Community College Board 

44. NACTE Interim Report Perkins IV required the U.S. Department of Education to undertake a National Assessment of Career and Technical Education (NACTE) to assess the current status of state and local CTE programs. The NACTE is designed to address the following questions: • How have secondary and postsecondary student enrollments in CTE programs changed? • Do students who participate in CTE realize any educational or workforce benefits? • How are Perkins IV funds distributed and used? • To what extent are Perkins IV accountability data valid, reliable, and comparable, and to what extent are decision-makers using these data? • Are grantees meeting the requirement to develop and implement programs of study that integrate academic and technical content?

45. NACTE Interim Report Highlights This interim report presents preliminary findings regarding CTE enrollment patterns, student outcomes, and international comparisons of CTE participation. The final report will provide a more comprehensive summary of findings from the six NACTE studies in order to address the full set of research questions. Key findings from the interim report include: • The percentage of students completing a sequence of CTE courses at the high school level declined between 1982 and 2004. •  High school graduates in the U.S. less frequently completed a CTE concentration than their counterparts in Europe and Asia. •  An analysis of national longitudinal data of high school students found that taking a concentrated sequence of CTE courses neither increased nor decreased math achievement or high school completion. •  Data from a natural experiment in one city found that students who chose to apply and were admitted to a CTE school by lottery were more likely to graduate on time and complete a sequence of college preparatory math courses than those who applied but were not admitted; however, the study did not find an effect on math or reading improvement.

46. NACTE Interim Report Measuring CTE Participation • Measuring CTE Participation • This analysis focuses on student enrollment in CTE occupational courses offering specific labor market preparation. CTE courses in general fall into three groups: • Family and consumer sciences education (FCSE) • General labor market preparation (GLMP) • Specific labor market preparation (SLMP) or occupational education • FCSE courses prepare students for family and consumer roles outside the paid labor market. GLMP courses teach general employment skills that are not specific to one occupational area, such as basic computer literacy, keyboarding or typing and general work experience courses. In contrast, occupational courses (SLMP education) are designed to prepare students for work in a specific occupational field or for related postsecondary education or training. • The CTE occupational courses being studied are divided into the following 11 mutually exclusive occupational areas (Bradby and Hudson, 2008; Hudson and Laird, 2009): • Agriculture and Natural Resources; • Architecture, Construction, and Science Technology; • Business; • Communications and Design; • Computer and Information Science; • Consumer and Culinary Services; • Engineering Technologies; • Health Sciences; • Manufacturing, Repair, and Transportation; • Marketing; and • Public Services.

47. NACTE Interim Report Measuring CTE Participation • Measuring participation in CTE in terms of occupational coursetaking can be done in two ways: the number of CTE occupational credits earned or some threshold number of courses representing a significant investment in CTE (e.g., Hoachlander, Kaufman, and Levesque, 1992). Continuous measures of the number of credits may be less useful because their representation as numerical averages can mask substantial variation within the population and overstate the influence of particularly high or low values. In considering whether CTE has particular outcomes associated with it, some minimal level of exposure seems necessary. In particular, some level of concentration in CTE occupational courses may be particularly important for postsecondary work opportunities. • This descriptive report expresses a concentration level for CTE participation in terms of CTE investors (Silverberg et al., 2004). This category is defined on the basis of the number of occupational courses a student takes. Those taking three or more are classified as CTE investors, while those taking fewer than three are non-investors.

48. Questions • john.haigh@ed.gov