WIA Net Impacts: Preliminary Estimates ADARE Project Briefing #3

1. Welcome to the Marshall Center PowerPoint Template. To add text, click on the visible text, delete the template titles, and enter your data. To add additional slides, use Ctrl + M. You can modify the slide layout (i.e. select a pre-set layout for tables instead of a layout for text) by right-clicking on the slide background and selecting Slide Layout. Commonly used sample layouts are included. To delete slides you don’t want to use, choose Delete Slide from the Edit menu. You can also Duplicate Slides through the Insert menu. Pre-set slide transitions are included. They can be changed through the Slide Show menu. Finally, to hide comment fields (like this box), select Comments from the View menu. Please email Sarah Looney at looneys@mail.utexas.edu if you have any questions. WIA Net Impacts: Preliminary EstimatesADARE Project Briefing #3 Kevin Hollenbeck, W. E. Upjohn Institute Christopher T. King & Daniel Schroeder, Ray Marshall Center, University of Texas-Austin May 7, 2003 W.E.Upjohn Institute for Employment Research

2. Overview Use this page to create numbered lists. • Background & WIA Context • Approach • Preliminary Impact Estimates • Discussion • Next Steps

3. Background & Context This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. • Key to assessing program impacts: answer two questions: • What happened to those who participated in the program (the ‘treatment group’)? • What would have happened to them if they had not participated? • First question is straightforward, but the second is more complicated…

4. Background & Context This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. • The second question is answered by creating a group of people who are as similar as possible to the treatment group, except that they do not receive the treatment. • Outcomes for the treatment group and this ‘comparison group’ are tracked over time. • Program impact is the difference between the outcomes for the two groups.

5. Background & Context This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. • Ideal Approach: Experimental design, which uses random assignment of participants to groups to ensure the similarity of the treatment and control groups. • This is the “gold standard,” but it can be expensive to implement, and is not always feasible.

6. Background & Context This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. • An experiment is not possible for WIA impact estimation because eligible clients cannot be denied service. • Next-Best Approach? Quasi-experimental design, with a comparison group selected to ensure similarity to the treatment group. • But, must be carefully done to get at true program impacts, and not just differences that already existed between the groups.

7. Background ... This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. History of quasi-experiments: Multiple, complicated answers to the impact question are not useful to policymakers. Examples: CETA & CLMS, 1st Job Corps Evaluation —> 1985 JTLS Advisory Committee —> National JTPA Study, 2nd Job Corps Evaluation, & state evaluations (e.g., TX, UT, WA)

8. Background ... This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. • WIA Context— • Service Sequencing: Core, Intensive & Training Services • Young, highly devolved program

9. Approach This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. Data: WIASRD files for FL, GA, IL, MD, MO, TX, & WA; ES registrant data for two states only; and linked UI wage records for all states. Impacts estimated for WIA Adults & Dislocated Workers, not Youth.

10. Approach ... This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. Treatment group = those who received intensive or training services and exited WIA in PY2000. Comparison group = those who received core services only, and exited in PY 2000. Thus, this design estimates the incremental impact of intensive/training, above and beyond the receipt of core services.

11. Approach ... This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. • Impact estimates will be presented for these Labor Market Outcomes— • Employment in 4th post-exit quarter • Earnings in 4th post-exit quarter • Note: States de-identified until state clearance is granted. Criterion for employment is a minimum of $100 quarterly earnings.

12. Preliminary Impact Estimates This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. • Outcomes for 6 states: employment & (conditional) earnings in 4th post-exit quarter. • Employment, 4th Post-Exit Quarter— • Adults: participation —> positive impacts in 3 states, negative impacts in 3 states. • Dislocated Workers: participation —> positive impacts in 4 states, negative in 2 states.

13. Impact Estimates ... This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style.

14. Impact Estimates ... This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. • Earnings, 4th Post-Exit Quarter— • Adults: participation —> positive impacts in 4 states, zero impact in 1 state, negative impact in 1 state. • Dislocated Workers: participation —> positive impacts in 2 states, negative in 4 states.

15. Impact Estimates ... This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style.

16. Discussion This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. [Still editing from here on…. -DGS] Considerable impact variation across subgroups (i.e., adults, dislocated workers) and across states Small sample sizes, especially for WIA core services Alternative matching techniques, e.g., caliper matching and others Local stratification

17. Next Steps This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. Use ES registrant data to construct comparison groups for all 7 current ADARE states (and add new states) Further refine and test alternative statistical matching techniques Estimate impacts using PY2001 WIA participant data & longer-term UI wage records Vet methods and results more widely

18. Use this page as the final slide in your presentation. For More Information Kevin Hollenbeck 269/385-0431 hollenbeck@upjohninstitute.org Chris King 512/471-2186 ctking@uts.cc.utexas.edu Daniel Schroeder 512/471-2196 schroed@uts.cc.utexas.edu