Abigail J. Lynch and William W. Taylor

1. Learners & Learning environments:Comparison of cognition level between lecture-based classroom and experiential study abroad assessment Abigail J. Lynch and William W. Taylor Michigan State University, Department of Fisheries and Wildlife, Center for Systems Integration and Sustainability 30 April 2013 FAST Fellowship Symposium

2. What is our objective as instructors? Fostering learning environments which result in learning. What is our objective as scholars? Evaluating effectiveness of learning environments to ensure learning. Research need Research approach Preliminary results Implications

3. Study abroad: not available on campus • “My study abroad gave me tools and experiences that studying on campus could never provide. In fact, without studying abroad, I would never have known how valuable the experiences were.” Research need

4. Study abroad: experiential learning • “Most of the time they not only told us about their work and knowledge, but took us out into the field to show us too. This alone made this program a unique educational experience, because rather than just talking about what we were learning, we actually went to see where policy making or science and environmental restoration work takes place.” Research need

5. Study abroad: experience with impact • “This learning did not happen in a classroom and there was barely any lecture time…These people had devoted their lives to fish and water preservation, so at no point were they teaching us. They were actually physically showing us the things they were most passionate about. This is a type of learning that makes an impact on students.” Research need

6. Learning Environments Traditional Classroom Study Abroad • Lecture-based • Experiential How can you compare? zaidlearn.blogspot.com Research need

7. Comparing learning environments • Foreign language competency • Segalowitz et al. (2004): U.S. classroom vs. study abroad in Spain • Freed et al. (2004): U.S. classroom vs. summer immersion vs. study abroad in France • Immersion students: significant gains in oral proficiency • Study abroad students: significant gains in speech fluidity • Classroom students: no statistically significant gains • Gains related more to hrs/week speaking and writing French than anything else • Foreign language competency • Segalowitz et al. (2004): U.S. classroom vs. study abroad in Spain • Freed et al. (2004): U.S. classroom vs. summer immersion vs. study abroad • Other subject areas? • Limited opportunities for paired design • Foreign language competency • Foreign language competency • Segalowitz et al. (2004): U.S. classroom vs. study abroad in Spain • Study abroad students: significantly greater gains in oral proficiency, oral fluency but not grammar or pronunciation. • Cognitive thresholds for second language learning readiness (regardless of location) Research need

8. Paired comparison: FW 481Global issues in Fisheries and Wildlife • Classroom: Spring 2010, Spring 2012 • Study abroad: Summer 2011, Summer 2013 • Classroom: Spring 2010, Spring 2012 • Study abroad: Summer 2011, Summer 2013 • Same learning objectives • Same essay assignments • Similar number of students • Similar demographics Research approach

9. Teaching-as-Research Objective To compare cognition level between lecture-based students and experiential study-abroad students Research approach

10. Cognition level • Bloom’s Taxonomy (1956): hierarchical classification of thinking skills • Most assessment is at lower levels of the taxonomy • Students remember more when they handle a topic at the higher levels of the taxonomy Garavalia, Hummel, Wiley, & Huitt (1999) edweb.sdsu.edu Research approach

11. “Blooming” • Scored essay assignments using Bloom’s Taxonomy (1956) • Coded for highest competence at the sentence, paragraph, and paper level • 1: knowledge • 2: comprehension • 3: application • 4: analysis • 5: synthesis and evaluation • Standardized by number of sentences, paragraphs • Weighted by each level • 0.5: sentence • 0.3: paragraph • 0.2: paper Research approach

12. “Blooming” example • Scored essay assignments using Bloom’s Taxonomy (1956) • Coded for highest competence at the sentence, paragraph, and paper level • Standardized by number of sentences, paragraphs • Weighted by each level full score Research approach

13. Research Questions Is there a difference between cognition level between: Classroom and study abroad students? Students the beginning and end of the course? Students with stated preference for active over lecture-based pedagogy? Research approach

14. Learning environment effects • Could learning environment affect demonstrated cognition level? • T-test for learning environment effects: Is there a difference between demonstrated cognition level in the final essay between study abroad and classroom students? t = 4.0933 df = 13.947 p-value = 0.001105 study abroad essay mean significantly higher cognition level than classroom essay mean study abroad classroom Preliminary results

15. Student effects • Could the course influence demonstrated cognition level? Self-selection? • T-test for prior effects: Is there a difference in the cognition level between the beginning and end of the study abroad course? t = -3.1145 df = 7 p-value = 0.01697 final essay mean significantly higher cognition level than initial essay mean initial final Preliminary results

16. Student effects • Could the course influence demonstrated cognition level? Self-selection? • Could pedagogical preference affect demonstrated cognition level? • T-test for pedagogical effects: Is there a difference in demonstrated cognition level between students who prefer active vs. lecture-based pedagogy? Preliminary results

17. Lecture vs. active learning preferences • Conducted a unit of the 2012 classroom course (three class periods) using different pedagogical styles • Lecture • Active learning • jig-saw and problem-based learning • Distributed survey on pedagogical preferences • Likert scale • Open ended questions Preliminary results

18. Lecture vs. active learning preferences Lecture Active learning • “Greater trust in [lecture] information because it comes from the TA/professor instead of classmates.” • “I like small group discussions because you hear multiple viewpoints and interpretations to better comprehend the material” Preliminary results

19. Student effects • Could the course influence demonstrated cognition level? Self-selection? • Could pedagogical preference affect demonstrated cognition level? • Test for pedagogical effects: Is there a difference in demonstrated cognition level between students who prefer active vs. lecture-based pedagogy? t = 1.7805 df = 7.999 p-value = 0.1129 preference for active learning mean NOT significantly higher cognition level than preference for lecture mean active lecture Preliminary results

20. Implications • Evidence that learner preference does not impact demonstrated cognition level • Small sample size • Evidence that experiential study abroad learning environments can promote higher order thinking • Final assignment mean cognition score: • Significantly higher than initial assignment • Significantly higher than classroom final assignment • Need to compare initial assignments from both learning environments to isolate effects Implications

21. Next steps • Add 2010 and 2013 students, 2012 initial essay • Add second reader to ensure reproducibility • Consider regression analyses? • Consider publication? • Any recommendations? • Journal of Natural Resources and Life Sciences Education? Implications

22. AcknowledgementsFW 481 students FAST Fellowship Steering Committee FAST Fellows Thank You!lynchabi@msu.edu Abigail J. Lynch and William W. Taylor Michigan State University, Department of Fisheries and Wildlife, Center for Systems Integration and Sustainability 30 April 2013 FAST Fellowship Symposium