Teaching Improves Graduate Student Research Skills

1. Teaching Improves Graduate Student Research Skills David F. Feldon, Melissa Hurst, Michelle A. Maher Deepalakshmi Muniraj Completed Findings from NSF-0723686 Research Team: Jie Chao, Joanna Gilmore, James Peugh, Cindy Stiegelmeyer, Denise Strickland, Briana E. Timmerman Discover. Create. Change.

2. Motivation • Effective preparation of graduate students in STEM is critical • Advancement of human knowledge • Stimulation of the economy • Quality of instruction linked to quality and quantity of student outcomes • Teaching often given “short shrift” Discover. Create. Change.

3. Current Research • Minimal research on graduate education • Overreliance on narrow range of methods • Interview, survey, reflection • Incapable of addressing causality or efficacy of specific practices • Field in need of performance-based data Discover. Create. Change.

4. Teaching & Research Traditional faculty roles, BUT… FACULTY STUDENTS Best students get top research-only slots Stipends higher for RAs than TAs Advisors actively discourage teaching Students’ “dirty secret” • Reward structure tied only to research (Tier I) • Teaching often viewed as distraction/ detraction • Grants used to buy out of teaching

5. Integrated Professionals? • Programs like PFF created to assume a range of professorial responsibilities • Graduate school as a socialization process by which “One enters a discipline …by become a successful disciple” (Damrosch, 2000) • Yet, “disciples” hear “mixed messages” about the value of research and of teaching (Austin, 2002)

6. Economic Realities • Mission of universities (esp. public) entails teaching • Budget constraints reduce adjunct/clinical teaching faculty • Funding agencies capping faculty support levels • Graduate students can be a low-cost source of instructors

7. Background • Meta-analyses of research to date indicate no correlation between teaching quality and research publications for faculty in science disciplines (Feldman, 1987; Hattie & Marsh, 1996) BUT: • Faculty are likely to have highly stable teaching and research skills (Simonton, 2004) • Self-explanation effect suggests metacognitive practice can enhance skills (Chi et al., 1994) • Qualitative self-reports of teaching experiences benefitting research (Becker & Kennedy, 2006; Lyons et al., 2005)

8. Inquiry-Based Teaching • Not all teaching experiences are equal • Inquiry teaching asks students to: • Frame their own questions for investigation • Use some discretion in determining appropriate evidence and means for collecting it • Evaluate evidence and draw conclusions • Teacher supports and scaffolds these efforts while monitoring content knowledge

9. Research Question • To what extent do graduate student participation in • inquiry-based teaching • advisor-directed research experiences • both teaching and research experiences impact the trajectory and magnitude of research skill development?

10. Study Design • Mixed methodologies • Synthesis and triangulation of: • Performance-based measures • Standardized assessments • Interview data from students and advisors • Two-wave data collection • Early fall and late spring • Three annual cohorts

11. Study Design • Participants • Masters and Ph.D. students (n=124) • 30 different STEM disciplines • STEM Education (math or science) • 3 institutions • Large, public research-intensive • Medium, public research-extensive • Small, private research-extensive

12. Universal Lab Rubric • Extensively validated in biology • Slightly revised for broader STEM writing • Modification for mathematics

13. Two-wave HLM Model Context (1,2) Primary Literature (1,2) Time Testable Hypotheses (1,2) Experimental Design (1,2) Validity/Reliability (1,2) Data Analysis (1,2) Data Presentation (1,2) Group Data Selection (1,2) Conclusions (1,2) 1=Teaching & Research 0=Research Only Limitations (1,2) MLR Estimator

14. Unmitigated Effects

15. Model Adjustments • Covariates to increase comparability of groups: • Prior research experience • Lawson’s Test of Scientific Reasoning • ACT Test of Science Reasoning • Change of assumptions for missing data: • 3 Mathematics participants for concerns regarding non-ignorable missingness • MCAR  MAR

16. With Covariates

17. Final HLM Model Context (1,2) Primary Literature (1,2) Time Testable Hypotheses (1,2) Experimental Design (1,2) MLR Estimator CFI ≥ 0.954 X2 ≥ 206.6, p=.000 Validity/Reliability (1,2) Data Analysis (1,2) Data Presentation (1,2) Group Data Selection (1,2) Conclusions (1,2) 1=Teaching & Research 0=Research Only Limitations (1,2)

18. Alternative Analysis • MANCOVA including covariates: • Prior research experience • Lawson’s Test of Scientific Reasoning • ACT Test of Science Reasoning • Rubric pre-scores

19. MANCOVA Findings • Significant differences under ML: • Testable hypotheses (d=0.58) • Experimental design (d=0.63)

20. Next Steps • Pending NSF proposal for a 5-year longitudinal study using a national sample • IGERT and traditional programs from: • Brown University • Carnegie Mellon University • Columbia University • George Washington University • University of Illinois • Indiana University • Kent State University • University of Michigan • Norfolk State University • University of Rochester • Stanford University • Vanderbilt University • Washington State University Discover. Create. Change.