CRELLA University of Bedfordshire May 2012 Parvaneh Tavakoli

1. Effects of Task Design on Native and Non-native Task Performance CRELLA University of Bedfordshire May 2012 Parvaneh Tavakoli

2. Background • Tasks as units of Teaching and Testing • Task Design in Language Teaching • Sequencing and syllabus design • Cognitive processes and SLA • L2 developmental processes • Task Design in Language Testing • Are language testing tasks of equal difficulty (Pollitt, 1991) • Defining task difficulty remains a challenge (Iwashita, et al, 2001) • A hierarchy of task difficulty yet to be established (Bachman, 2002) • What can task difficulty help with? • Selecting & categorizing tasks for test purposes • Providing a more reliable assessment • Improving the validity of the interpretations and uses made on test results

3. Investigating Task Design Study 1: Effects of degree of task structure on L2 spoken performance Study 2: Effects of storyline complexity and structure on L2 & L1 spoken performance Study 3: Effects of storyline complexity on L2 written performance (work in progress)

5. Study 1: Effects of Task Structure • Task structure & Planning Conditions • 6 oral narrative tasks (picture stories) were used with 80 Iranian EFL learners under testing conditions • Degree of task structure was operationalized through tightness of the structure presented in the picture stories • Structure was defined based on knowledge structures (Mohan, 1991; Hoey, 1983) • Tight structure (fixed sequence of events) • Loose structure (arbitrary sequence of events) • Findings: • Task structure proved to have an impact on L2 performance: the more structured tasks elicited more accurate and more fluent language • There was no systematic impact on syntactic complexity or lexical diversity • Planning improved performance in all different measures across the two LP levels

6. Study 2: Research Focus • Storyline Complexity foreground only (simple) foreground and background (complex) • Structure loose structure tight structure • Context of language learning • learning English in Tehran • learning English in London • Native speakers & L2 learners

7. Study 2: Research Design

8. (Main) Research Questions • Does task design have an impact on L2 learners’ performance? • Accuracy, fluency, syntactic complexity, lexical diversity • Does task design have an impact on native speakers’ performance? • Fluency, syntactic complexity & lexical diversity

9. Study 2: Different Phases of the Research

10. Study 2: Methods • Independednt variables structure: within-participant storyline complexity: between-participant • Narratives Oral narratives typically used in EFL settings They are popular in language testing, e.g. previously used by ETS; currently used by different local, national and international institutions • Setting in one-to-one sessions with the researcher

11. Participants Tehran 60 female Iranian L2 learners of English in a language school in Tehran aged between 18-34; Intermediate level (B1-B2) London 40 male/female L2 learners of English in a college in London aged between 19 and 47, different L1s, Intermediate level (B1-B2) London 40 male/female native speakers of English first year undergraduate students of English program/psychology in a university in London aged between 19 and 60

12. Procedures & Measures • Data transcribed and coded • Task accomplishment • Fluency • repair, speed and breakdown fluency • Mid-clause vs end-clause pauses • Syntactic complexity subordination and MLU • Accuracy percentage of error-free clauses • Lexical Diversity • corrected measure of D calculated by “vocd” • qualitative analysis of lexical selection • A range of statistical analyses • Factor analysis, MANOVAs, ANOVAs, T-tests

13. Findings: Impact of Task Design on L2 & L1 Performance A complex storyline encourages L2 performance of more syntactic complexity L2 learners both in Tehran and in London It affects native speakers’ performance Presence of structure encourages L2 performance of higher accuracy and fluency L2 learners both in Tehran and in London It doesn’t affect native speakers’ performance

14. Patterns of syntactic complexity across tasks and groups • Means of subordination in native speakers’ performance • Picnic vs. Football 1.71 vs. 1.50 F=2.56, p<.01*, η 2= .150 • Walkman vs. Journey 1.86 vs. 1.54 F=2.62, p<.01*, η 2=.150

15. Patterns of MLU across the tasks ad groups • Means of length of utterance in native speakers’ performance • Picnic vs. Football 10.81 vs. 9.09 F=2.64, p<.01*, η 2= .160 • Walkman vs. Journey 10.73 vs. 9.33 F=1.87, p<.07ns, η 2= .08

16. Syntactic complexity across the three groups (ratio of clauses to AS Units)

17. Fluency across the three groups(Total silence mid-clause per minute)

18. Patterns of mid-clause pausing in the three groups

19. Patterns of end-clause pausing in the three groups

20. Accuracy of L2 Performance

21. Patterns of lexical diversity across tasks and groups

22. “D” across the Three Groups Lexical Diversity

23. Discussion Task design as a source of variance in L1 & l2 performance (construct-irrelevant variance) task variability may well introduce error into the assessment of the oral ability Studying task design and its effects on performance can provide insight into some systematic and predictable variation These findings are in contrast with Elder et al., 2002 & Iwashita et al., 2001 Reliability: a statement of the accuracy, consistency and fairness of a measuring instrument (Banerjee, 2000)

24. Does task design affect speaking and writing in the same way? • Researching task design: speaking From Ellis 1987 up to now • Researching task design: writing • Kuiken & Vedder (2007) • Ong & Zhang (2010) • Kormos (2011, 2011) • How do they relate to one another? • Does task difficulty affect speaking and writing production processes in the same way? • Can a single model/index of task difficulty account for both modes of production?

25. Study 3: Storyline complexity in writing (compared with speaking) • Written narratives: the same picture stories • Participants: 40 intermediate learners in London • 20 minutes to complete their writing (no strategic planning, no drafts, no dictionaries) • Research questions: • What are the effects of storyline complexity on syntactic complexity of L2 learners’ narrative writing? • Are the effects of storyline complexity on L2 written performance comparable to those of oral performance?

26. Storyline complexity & L2 written and oral performance Syntactic complexity in written and oral L2 performance

27. REFERENCES Bachman, L. (2002). Some reflections on task-based language performance assessment. Language Testing, 19(4), 453-476. Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press. Banerjee, J. (2000). Reliability. Routeledge Encyclopedia of language teaching and learning. London: Routeledge, 513-5. Elder, C., Iwashita, N., & McNamara, T. (2002). Estimating the difficulty of oral proficiency tasks: What does the test-taker have to offer. Language Testing, 19(4), 343-368. Fulcher, G. (2003). Testing second language speaking. London: Longman. Fulcher, G., & Marquez Reiter, R. (2003). Task difficulty in speaking tests. Language Testing, 20(3), 321-344. Hoey, M. (1983). On the surface of discourse. London: George Allan and Unwin. Iwashita, N., McNamara, T., & Elder, C. (2001). Can we predict task difficulty in an oral proficiency test? Exploring the potential of an information-processing approach to task design. Language Learning, 51(3), 401-436. Mohan, B. A. (1991). LEP students and the integration of language and content: Knowledge structures and tasks. In C. Simich-Dudgeon (Ed.), Proceedings of the first symposium on limited English proficient students' issues . Washington, DC: Office of Bilingual Education and Minority Language Affairs. Pollitt, A. (1991). Giving students a sporting chance: Assessment by counting and judging. In J. C. Alderson & B. North (Eds.). Language testing in the 1990s (pp. 46-59). London: Modern English Publications in Association with the British Council.