National Reporting System for the 21 st Century Congress of Adult Education State Directors

1. National Reporting System for the 21st Century Congress of Adult Education State Directors Pre-Meeting Webinar July 28-29, 2010

2. Webinar Agenda • Introductions • Purpose of meeting • Review of issues to be discussed • Meeting structure • Questions 2

3. Purpose of Meeting • To get your input on 6 changes planned or under consideration for the NRS • Changes that can be implemented by July 2011 and do not require reauthorization • Other changes being considered that will require more time, authority through reauthorization, or more research 3

4. Goal Setting Changes— Issues #1, 2, and 3 • Goal setting for employment, GED, and postsecondary entry— problematic • Difficulty identifying appropriate goals for students • Number of goals set appears too low • High fluctuation of performance rates within states and across years 4

5. Issue #1: Employment Outcomes • Discontinue goal setting; move to “automatic cohort definition” • Automatically select students for outcome based on student characteristics • Options considered: • Include all students seeking work or only unemployed seeking work • Set a federal standard for number of students to include 5

6. Issue #1: Recommendations • Automatically designate all students in the labor force who are unemployed as the cohort for which “entered employment” must be tracked. • Automatically designate all students who enter the program employed as the cohort for which “retained employment” must be tracked. 6

7. Issue #1: Discussion Questions • How will this policy affect the population that local programs serve? • Should we continue to use program “exit” as a criterion for cohort identification and reporting? • What changes, if any, will states need to make to state-level MIS to implement the new policy, and at what cost? 7

8. Issue #1: Discussion Questions (Cont.) • What are implications for the follow-up methods (survey v. data match)? • How can states overcome barriers to collecting valid Social Security Numbers? • What are the implications for state training to local providers? • How will this change affect local program funding, especially in states with performance-based funding? • Is this policy change feasible to implement in PY 2011-2012? 8

9. Issue #2: Postsecondary Follow-up • Eliminate goal setting; options considered: • Include all students, track for multiple years • Set a standard for number of students to include for follow-up • Include only students who have a GED, high school diploma or enrolled in transition class. 9

10. Issue #2: Recommendation • Automatically designate all students who - have earned a GED, - have a secondary credential, or - are enrolled in a class specifically designed for transitioning to community college (e.g., bridge program, college readiness) as the cohort for which “entry into postsecondary education” must be tracked. 10

11. Issue #2: Discussion Questions • What effect will this policy have on local programs—both desirable and unintended effects? Are there ways to ameliorate unintended effects on service delivery? • How will states identify and report on those enrolled in aclass specifically designed for transitioning to community college? • What changes will states need to make to state-level MIS, and at what cost? 11

12. Issue #2: Discussion Questions (Cont.) • What are implications for the follow-up methods? • How can states that do not have adequate postsecondary data systems implement this change? • How can states that cannot or do not collect Social Security Numbers overcome barriers? • What are the implications for state training to local providers? • What effect, if any, will this change have on states using performance-based funding? 12

13. Issue #3: Secondary Credential Follow-up • Options considered: • Eliminate goal setting; include all students who take the GED Tests • Eliminate goal setting; include all students at ASE levels who do not have a credential • Maintain goal setting—no change. 13

14. Issue #3: Recommendation • Match GED test records for all students who take tests during the year to calculate a pass rate. • For states with adult high school, report the number of students in high adult secondary education (ASE) who obtain a high school diploma. • For states with EDP, report the number of students enrolled in the assessment phase who obtain a high school diploma. 14

15. Issue #3: Discussion Questions • What effects will this policy have on the populations served by local programs? • What changes will states need to make to state-level MIS, and at what cost? • What are the implications for the follow-up method (survey v. data match)? • How can states that cannot or do not collect valid SSNs overcome barriers? • What are the implications for state training to local providers? 15

16. Issue #4: Other Educational Gain Measures—Reporting Test Scores • Should OVAE require states to report test scores in addition to educational levels gains? • Long-term issue, requiring further study • Psychometric expert and OVAE will discuss methods and implications 16

17. Issue #6: Measuring Progress in Postsecondary Education • Collect outcome data on students after transition to postsecondary education • Options: • Require programs to follow all students after transition • Require students in integrated education and training programs to be tracked 17

18. Issue #6: Recommendation • Require programs with integrated education and training (IET) models to track progress towards and completion of a credential in the program of study in which the student is enrolled. 18

19. Issue #6: Discussion Questions • How should postsecondary retention and completion be defined at the federal level? • How can states collect postsecondary retention and completion data at the postsecondary level? • What are the implications for State MIS? 19

20. Issue #6: Discussion Questions (Cont.) • Do states have postsecondary-level databases that provide this information? Can this information be collected within local postsecondary institutions? • Are there unintended consequences of this policy that may impede the use of the IET model? How can the effect of such consequences be ameliorated? 20

21. Issue #5: Refining Outcomes Measures for GED Students • GED students exit before posttesting–negative effect on educational gain • Educational gain not always appropriate measure for GED prep students • Options considered: • Count GED as educational advancement • Allow posttesting at fewer hours • Create separate track for GED students, exempt from educational gain 21

22. Issue #5: Under Consideration • Create a separate level for GED prep students to include all students who score at the secondary level on any NRS-approved test battery. • Educational gain not reported for these students; they are not counted in calculation of ed gain for any NRS level. • The only reportable NRS outcome for these students is attainment of a secondary credential. 22

23. Issue #5: Discussion Questions • Would adding this track be a positive change in your state? Are there negative implications? Would the net result be positive or negative? • How would dropping the requirement for educational gain posttesting affect you? • Is this approach better aligned with GED delivery models in your state? • How much time would it take to implement? • Would your state support this recommendation? 23

24. Meeting Structure Dupont Hotel August 4-5, 2010 24

25. Six Issues for Discussion • Issues 1, 2, 3, and 6—Day 1 • Issues 4 (panel) and 5—Day 2 • Think Sheets for states to prepare responses to questions • Feedback Forms for states to offer comments—to ensure that OVAE hears from and considers input from every state 25

26. Four Break-Out Rooms • Process • 10 Minutes for States to Reflect on Issue and Questions (Using Think Sheets) • Round-Robin Responses to Questions to Allow Input from All States • General Discussion of Issue • States Complete and Submit Feedback Forms 26

27. Ground Rules for Round-Robin Process • One state speaks at a time; One State—One Voice • No one individual dominates discussion • When identifying challenges, also identify potential solutions or ways to overcome challenges (Don’t say only “We can’t/ my state can’t” without offering some solutions.) • If nothing to add when it’s your state’s turn, say “pass.” • No interrupting round-robin approach; opportunity for clarification after every state has responded to the questions • Use Parking Lot for long-term issues 27

28. Questions?Comments? 28

29. Thank You! See you August 4-5 in Washington, DC! Safe Travels!