Managing Yout h Customer Performance Data

1. Managing Youth Customer Performance Data Part 2: Literacy/Numeracy Gains Presented by Jennifer Medhaug & Ada Conner-Coash August 21, 2013

2. Objectives • Review I-Trac DOL Common Measures Report • Understand data elements that factor into performance reporting • Common Data Validation Errors • Impact of Exit • Impact of Co-enrollment in WIA Adult/DW • Tips for Success – Minimize Common Data Issues • Support & References

3. DOL Common Measures Report • Aggregate Report • Shows the performance as a percentage (Numerator/Denominator) • Indicates how cohort is defined • Lit/Num Measure: reporting Year/Quarters determined by anniversary of Participation Date and Performance Year/Quarter

4. DOL Common Measures Report • List Report • Shows all customers in denominator • Has a numerator column • Successfully Meets Measure = Y • Does not Meet Measure = N

5. Literacy/Numeracy Gains • PY13 Performance Goal: 53% • Of those out-of-school youth who are basic skills deficient, the number of participants who increase one or more educational functioning levels.

6. Literacy/Numeracy Gains Denominator Elements • I-Trac Calculation • Based on the Literacy/Numeracy Assessments control data (refer to Youth I-Trac Data Entry Manual for more information) • Performance Year/Quarter • Year 1 Performance Denominator for that performance quarter = Yes • OR Year 2 Performance Denominator for that performance quarter = Yes • OR Year 3 Performance Denominator for that performance quarter = Yes

7. Literacy/Numeracy Gains Denominator Elements • Basic Skills at Registration Status • Required to determine Out-of-School Youth (OSY) status • Required for ALL customers – regardless of OSY or ISY status (WIASRD #130, CCWD MIS Manual C_CLBSKILL) • Determining Basic Skills Status | Approved Assessment • TEGL 28-11 Data Validation Source Documentation Requirements: • Standardized assessment tests (including CASAS Appraisal) • School record • Case notes – formal note of how the information was obtained by an appropriate source

8. Literacy/Numeracy Gains Denominator Elements • Out-of-School Definitions • TEGL 17-05 • Out-of-School: An eligible youth who is a school dropout, or who has received a secondary school diploma or its equivalent but, is basic skills deficient, unemployed, or underemployed (WIA section 101(33)). For reporting purposes, this term includes all youth except: (i) those who are attending any school and have not received a secondary school diploma or its recognized equivalent, or (ii) those who are attending post-secondary school and are not basic skills deficient.

9. Literacy/Numeracy Gains Denominator Elements • Out-of-School Definitions • I-Trac Definition • In/Out-of-School: will show the customer as In-School (ISY) or Out-of-School (OSY). The following is how OSY is calculated. If the customer does not meet one of the following conditions for calculating OSY, then they are ISY: • Education Status = Not Attending School – High School Dropout • Education Status = Not Attending School – Diploma/GED/Equivalent • Education Status = Student in Post High School AND Basic Skills Characteristic = Basic Skills Deficient

10. Literacy/Numeracy Gains Denominator Elements • Pre-Tests • DOL Approved Assessment: TEGL 17-05 and National Reporting System (NRS) tests for educational functioning levels • WorkKeys • CASAS ABE (Not CASAS Appraisal) • TABE 7-8, 9-10 • Best Plus (ESL) • CASAS ESL (Not CASAS Appraisal) • TABE CLAS-E • Best Literacy (ESL) • Previous year’s Post-Test will become current year’s Pre-Test (as appropriate)

11. Literacy/Numeracy Gains Denominator Elements • Program Year Anniversary • Calculated on enrollment date (I-Trac Participation Date) • Year 1: All customers in the denominator must be pre-tested and if BSD in one subject, must have a post-test in that subject prior to the earliest of Exit Date or Anniversary Date • Year 2: Only customers that are not exited before Year 2 Anniversary and are still BSD in either Literacy or Numeracy subject should be post-tested for a gain during year 2. • Year 3: Only customers that are not exited before Year 3 Anniversary and are still BSD in either Literacy or Numeracy subject should be post-tested for a gain during year 3.

12. Literacy/Numeracy Gains Numerator Elements • I-Trac Calculation • Customer is in the denominator • AND based on the Literacy/Numeracy Assessments (refer to I-Trac Manual for more information about calculations in this control) • Year 1 Performance Numerator for that performance quarter = Yes • OR Year 2 Performance Numerator for that performance quarter = Yes • OR Year 3 Performance Numerator for that performance quarter = Yes

13. Literacy/Numeracy Gains Numerator Elements • Post-Tests • Must compare to a Pre-Test of the same test name/test type combination • If the customer is determined to be in the denominator of the following year, the Post-Test will become the following performance year’s Pre-Test

14. Literacy/Numeracy Gains Numerator Elements • Gains • A gain means the customer improves in DOL EFL by at least one digit for the performance year (based on customer anniversary date) • A gain can only count in the subject where the Pre-Test has a DOL EFL < 7 • Where a gain is achieved in two subjects during a performance year, only one gain is counted • Gains from a previous year cannot count for the following year

15. Literacy/Numeracy Gains Numerator Elements • No longer Basic Skills Deficient (BSD) • Customer has tests in BOTH Literacy and Numeracy subjects where DOL EFL ≥ 7

16. Common Data Validation Errors • Missing Test Scores • Customer does not have pre-test scores in BOTH Literacy and Numeracy subjects

17. Impact of Exit OSY/BSD Customers • If the exit date is…. • Before Year 1 Anniversary: customer must gain before exit • Between Year 1 and Year 2 Anniversary: customer is not required a year 2 gain (Denominator = No) • Year 2 Anniversary: customer is required to have a gain recorded anytime during year 2 (Denominator = Yes) • Between Year 2 and Year 3: customer is not required a year 3 gain (Denominator = No) • Year 3 Anniversary: customer is required to have a gain recorded anytime during year 3 (Denominator = Yes)

18. Impact of Co-enrolling into WIA Adult/DWOSY/BSD Customers • I-Trac does not co-exit WIA Youth and WIA Adult/DW • CCWD does co-exit WIA Youth and WIA Adult/DW • Customers that are engaged with WIA Adult/DW prior to the 90th day following the customer’s Exit Date in the WIA Youth enrollment, may continue to be in this measure as long as they are OSY and BSD (max Year 3 Anniversary of WIA Youth enrollment date)

19. Tips for Success Minimize Common Data Issues • Auto Exit | Exit Management Dashboard Report • Selective Service Auto Exit | Selective Service After Registration Dashboard Report • Post-Test EARLY & OFTEN • I-Trac will report the best score • Use EFL decimal field to target gains • After Year 1: Don’t wait for Year 2 Anniversary to Post-Test for gains • After Year 2: Don’t wait for Year 3 Anniversary to Post-Test for gains • Consider waiting (where possible) to co-enroll WIA Youth into WIA Adult/DW at least 90 days following Exit Date.

20. Support & References • DOL TEGL 17-05 Common Measures • DOL TEGL 28-11 Data Validation and Source Documentation • National Reporting Systems approved tests for Educational Functioning Levels • I-Trac Youth Data Entry Manual • I-Trac Reports Manual • Email I-Trac Support:support@i-trac.org

21. Objectives • Review I-Trac DOL Common Measures Report • Understand data elements that factor into performance reporting • Common Data Validation Errors • Impact of Exit • Impact of Co-enrollment in WIA Adult/DW • Tips for Success – Minimize Common Data Issues • Support & References

22. Thank you Questions?