Intensity of Engagement in Foreign Language Lessons SASLI: June 2007

1. Intensity of Engagement in Foreign Language LessonsSASLI: June 2007 Benjamin Rifkin Temple University b r i f k i n @ t e m p l e . e d u

2. Time and Participant Roles • Time as a factor in foreign language learning • Participant roles as a factor in FL lesson planning

3. Statistical Consultants David B. Dahl Steven LeMire University of Wisconsin-Madison

4. What can we expect students to achieve? • Four years of college instruction • Summer immersion programs • Study abroad opportunities

5. Learning Outcomes Studies • Carroll (1967) • Magnan (1986) • Brecht, Davidson and Ginsberg (1993) • Tschirner (1996) • Thompson (1996) • Davidson (1998, 2002, 2003, ongoing) • Rifkin (2003, 2004, 2005)

6. Level of Interest: Advanced (2) Minimum working competency is advanced according to federal standards and the experience of private sector (business) practices

7. Carroll (1967) After four years of instruction in Russian: advanced-high level proficiencies* *Proficiency Guidelines not codified yet in 1967: no standardization of testing, poor correlation to today’s standards *No control for instruction before college enrollment, summer immersion, study abroad: but there was virtually no study abroad in Russia anyway. *He found these results disturbinglypoor!

8. Magnan (1986) After four years of instruction in French: advanced level oral proficiency* *No control for instruction before college enrollment, summer immersion, study abroad. *French is a Category 1 language.

9. Brecht, Davidson, Ginsberg (1993) 77% students going on study abroad had pre-program IM or IH oral proficiency; 83% had IM listening proficiency; 44.5% had IH reading proficiency.* *No control for years of prior instruction, but most had 2-3 years. *consistent with ongoing ACTR research

10. Tschirner (1996) After two years of instruction in German: intermediate-mid oral proficiency* (consistent with Magnan’s findings for 2nd year French) *No control for instruction before college enrollment, summer immersion, study abroad. *German is a Category 2 language.

11. Thompson (1996) U. of Iowa Summer Program (1st-2nd year): NM and NH in all modalities Middlebury Summer Program (3rd-4th year): IH and A in all modalities (in 1996 no AL/AM distinctions in any modalities) *No control for years of prior instruction except for placement, differences in student populations (Iowa / Middlebury)

12. Davidson (2003) Students going on ACTR study abroad for a semester, pre-program testing: 95% intermediate listening proficiency 67% intermediate reading proficiency 81% intermediate oral proficiency (N= over 1200 students)

13. Rifkin (2003, 2004, 2005) • Tests piloted in 2000 • Tests administered in 2001-2003: listening, speaking, reading, writing, grammar • All tests but grammar correlated with proficiency guidelines • Speaking test: small random sample actual OPI, balance — oral exams based on OPI

14. Sample Listening Test Items • Novice: What’s your name? • Intermediate: Provide specified details from a weather forecast • Advanced: Summarize radio report about a mail-order bride scam • Superior: Analyze a report about a cultural controversy for rhetoric and logic of argument Students answer in English in writing; tests scored by hand.

15. Sample Reading Test Items • Novice: Identify signs in airport or store • Intermediate: Read a movie theater schedule and provide specified details • Advanced: Summarize newspaper article about new organization of emergency services in Moscow • Superior: Analyze a news report about a cultural controversy for rhetoric and logic of argument Students answer in English in writing; tests scored by hand.

16. Sample Writing Test Items Students responded in writing in Russian to same probes with written responses at different levels (consistent with ACTFL Writing Proficiency Test currently under development) • Describe your hometown • Tell a story about something funny that happened to you • Pick an important political or social problem and identify its possible causes and its impact on people’s lives, Outline a solution to the problem and the consequences were it to be implemented. • How would your life be different if you hadn’t …? Tests scored by hand of course.

17. Sample Speaking Test Items • Why do you like (dislike) your hometown? • Tell me about your best friend. • What did you do on your last vacation? • Tell me the story of a favorite film. • Describe how you celebrate a holiday. • Explain opposing interpretations of a political or social controversy Tests tape-recorded; testers filled out assessment grids and oral proficiency rating was assigned to each student’s test performance.

18. Sample Grammar Test Items 100 questions embedded in a cloze-test adapted from Russian prose text written for native speakers: Лиляужаснорасстроилась, (1when) ___ (2 father) ___ сказал (3 her) ___ ,чтосейчасонинемогуткупитькотёнка, которогодевочкаужесчитала (4 her own) ___ . Смотреть (5 after) ___ этогонаЛилиныглаза, полныеслез, былоневозможно. Test items include verbs, verbs of motion, pronouns, prepositions, participles, time expressions, котoрый-clauses, чтобы-clauses, and so forth.

19. The Student Sample • Over 300 learners • 3 years of testing • From over 100 institutions across North America, employees from federal agencies and members of the US armed services: a national sample

20. Amherst Beloit Berkeley Columbia Georgetown Harvard Indiana Michigan Oberlin Occidental Penn Princeton Reed St. Olaf Stanford Swarthmore U. Texas-Austin UNC-Chapel Hill Utah Virginia Washington U. (St. Louis) Yale Over 100 Home Institutions

21. US Government Employees • CIA • FBI • US Navy • US Air Force • Future US Government employees in these and other agencies and branches of the armed forces

22. Not Typical Learners:They Chose • Summer intensive study: lost wages • Middlebury immersion • Isolation from loved ones • Language pledge Atypical commitment? Degree of motivation?

23. Middlebury Language Pledge I promise to use only Russian until the end of the summer program.

24. What doesimmersionmean at the Middlebury Russian School?

25. Residential Life

26. Sports Clubs

27. Social Life: Karaoke

28. And of course CLASSES!

29. Number of Students Tested 2001 = 106 (19 with OPI) 2002 = 125 (34 with OPI) 2003 = 118 (27 with OPI) Total N = 349

30. Hours of Prior Instruction 5 hours per day, 5 days per week for 10 weeks = about 250 hours 5 hours per week for one academic year = about 150 hours 4 hours per week for one academic year = about 120 hours 3 hours per week for one academic year = about 90 hours Example: 4 hours per week for 4 years = 480 hours

31. Small Surveys (SEELANGS/FLASC) N = 34 (Russian) / 32 (Other Langs) : adjusted for 30 weeks/yr First-year average hours = 4.48 / 4.188 Second-year average hours = 4.06 / 3.68 Third-year average hours = 3.09 / 3.19 Fourth-year average hours = 2.64 / 3.00 Total average classroom hours in 4-year college curriculum = 428.21 / 421.74

32. Test Correlations Data presented in the following slides were collected in Middlebury RS tests demonstrated earlier. Test correlations: .75 to .86 Correlations confirm Brecht, Davidson, Ginsberg’s (1993) finding of importance of grammatical competence for language gain.

33. Acquisition Rate Scale Novice: 1-3 = NL, NM, NH Intermediate: 4-6 = IL, IM, IH Advanced: 7-9 = AL, AM*, AH Superior: 10 = S *AL/AM distinction for speaking/writing only

34. Oral Acquisition

35. Listening Acquisition

36. Reading Acquisition

37. Writing Acquisition

38. Acquisition of Four Skills

39. Grammar “Acquisition”

40. Robin’s Proficiency Paradox In 1967, the typical Russian major had no proficiency-based instruction, no opportunities for length stays in country, no possibility of homestays, little in the way of strategy instruction …[and] little recourse to authentic … Russian. Yet even with the advent of goal oriented instruction and materials designed to help us meet those goals, … apparent lack of meaningful progress. As we in pedagogy look at outcomes, we can’t help but ask if the last three decades [since the publication of Carroll's study] have been a waste of time (Robin, 2000: 29).

41. Proficiency Paradox or Ceiling Effect? • Why Carroll’s study is not a reliable indicator • Progress since 1967 in pedagogy, input access, opportunities for communicative practice But what if there is a ceiling limiting students’ gains in classroom instruction?

42. The Ceiling Effect Data show that learners do not attain advanced-level proficiency in a Category 3 language in listening, speaking, reading or writing without over 700 hours of classroom instruction.

43. Oral Acquisition

44. Listening Acquisition

45. Reading Acquisition

46. Writing Acquisition

47. Grammar “Acquisition”

48. Middlebury / SASLI? • Student motivation • Giving up social support • Giving up potential summer earnings or vacation • Engaging in the study of a foreign or heritage language in unknown (for many) circumstances

49. Role of Immersion Immersion is necessary for learners to break through the “ceiling” to higher levels of attainment.

50. (1) Classroom Instruction Build the intermediate-level foundation: students should be able to ask and answer questions, participate in conversations in predictable contexts, understand simple texts (listening and reading) Build basic lexical and grammatical foundation