Criterial Features and the CASP Model of SLA John A. Hawkins

1. Criterial Features and the CASP Model of SLA John A. Hawkins Department of Linguistics, UC Davis

2. From 2005-2011 I co-directed a research program on second language learning at Cambridge University (the “English Profile Programme”) with three components: (a) Cambridge Learner Corpus [CLC]: 40+ million words of written English from learners around the world; (b) Computational techniques: CLC was first searchable lexically, with 76 error codes; subsequently tagged for parts of speech and parsed using an automatic parser, RASP, Briscoe et al 2006; (c) New research methodology: designed to yield practical benefits for learning/teaching/assessment, and to make a theoretical contribution to SLA.

3. The key theoretical and methodological innovations include: (i) the concept of criterial features, as a means of distinguishing levels of proficiency in the learning of a second language; (ii) the development of a general learning model, CASP (“complex adaptive system principles of SLA”) informed by the criterial features of the CLC and by other empirical studies in the literature. These two innovations, especially CASP, were developed on the basis of joint work with my principal collaborator, Luna Filipović (then Cambridge, now University of East Anglia). Further assistance from members of the Cambridge team is recognized in the Acknowledgements at the end of this talk.

4. Criterial features are properties of learners’ English that are distinctive and characteristic of L2 proficiency at the different levels. The CLC scripts have been graded by Cambridge examiners and assigned grades A-F at each of six levels of proficiency, following the Common European Framework of Reference (CEFR), see the Council of Europe 2001 : Levels: C2 Mastery [CPE] C1 Effective Operational Proficiency [CAE] B2 Vantage [FCE] B1 Threshold [PET] A2 Waystage [KET] A1 Breakthrough When searching for criterial differences between levels we focussed on the scripts of students who had achieved passing grades of A-C at each level.

5. The questions that motivated this research are: 1) how much of the grammar and lexicon of English do learners actually know and/or produce at each of these CEFR levels? 2) what empirical patterns and principles are there in these developing second language stages of English? and 3) what are the practical benefits, for learning, teaching and assessment of gathering this information?

6. Electronic corpora of learner English make it possible for us to answer these questions. The CLC is the biggest learner corpus of English or of any language. It gives us empirical evidence for developmental stages in the learning of new constructions, words and word meanings. It gives us quantitative data on learner errors in syntax, morpho-syntax and lexical choice. It was originally searchable only lexically, in conjunction with the error codes. Subsequently it has been tagged and parsed using the automatic parser, RASP.

7. Sample Error Codes in the CLC RN Replace nounHave a good travel (journey) RV Replace verbI existed last weekend in London (spent) MD Missing determinerI spoke to President (the) I have car (a) AGV Verb agreement errorThe three birds is singing (are) IV Incorrect Verb InflectionI spended last week in London (spent) FJ Wrong Adjective FormThe situation got worst (worse) UQ Unnecessary QuantifierA little bit quite common (quite common) DY Derivation of Adverb It happened fastly (fast)

8. Briscoe’s RASP (Robust Accurate Statistical Parser) • identifies parts of speech (PoS) probabilistically tagging • generates a parse forest representation containing all possible subanalyses with associated probabilities • weighted Grammatical Relations yielded by the n-best parses of the input.

10. Criterial features can be found in all areas of English: syntax, morphology, phonology, the lexicon, semantics, and discourse. They distinguish higher proficiency levels from lower levels in an efficient way. In this talk some of these features are illustrated, as is the theoretical learning model derived from them, “CASP”.

11. An analogy: Languages change over time, and when historians of English examine Old English, Middle English and Early Modern English, they focus on important differences between these stages, not on what stayed the same. Similarly in a learning context, we are interested in changes from one level to another.

12. There are different types of criterial features. Here we focus on just two: Positive Criterial Features These refer to positive, i.e. correct, linguistic properties of English that have been acquired at a certain L2 level and that generally persist at all higher levels. A property P (e.g. a new construction type) acquired at B2 may differentiate that level and higher levels from [A1, A2, B1] and will be criterial for the former. Or P may be acquired at C2 and differentiate this level from all lower levels.

13. Negative Criterial Features These are incorrect properties of English, or errors, that occur at a certain level or levels and with a characteristic frequency. Both the presence versus absence of the errors, and especially their frequency (the "error bandwidth"), can be criterial for the level(s).

14. Examples of Positive Criterial Features (from Hawkins & Filipović 2012 and using data from the CLC) The A levels (A1 and A2) Simple intransitive clauses (NP-V) and the slightly more complex transitive (NP-V-NP) sentence types are present from the beginning: He went. (NP-V) A1 He loved her. (NP-V-NP) A1

15. Modal auxiliary verbs like may, might, can and must appear first at A1 or A2, but only in some of their senses. Can is first attested in the PERMISSION sense at A1 and in the POSSIBILITY sense at A2: And if you want, you can bring pencils or pens. (PERMISSION) A1 In this magazine you can see all the new C.D.[s] and all the dates of the concerts. (POSSIBILITY) A2

16. Noun Phrase sequences of Pronoun plus Infinitive are found at A2: something to eat A2 nothing to do A2 as are postnominal modifiers with participial –ed: beautiful paintings [painted by famous Iranian painters]A2

17. Lexical verbs appearing at the A levels are typically among the most basic and frequent verbs of English; they appear first in their most basic and frequent senses. Verbs attested at A1 include: catch, eat, give, put, take and walk

18. New lexical verbs appearing at A2 include: break, cut, hit, push, stand, and fall again typically in their most basic and literal senses. For break this includes its primary physical sense: I broke a beautiful glass. A2 for cut it includes the following example in its primary sense: First I cut the cake with my mother. A2

19. The B Levels (B1 and B2) The new features at B1 involve more complex syntax, e.g. an “Object Control” structure such as: I ordered him [to gather my men to the hall] B1 him is both the object of ordered and the logical subject of gather here. This is a criterial construction for B1 and higher levels which distinguishes them from the A levels.

20. Structures like the following with finite or non-finite subordinate clauses and movement of the WH-word (how, where, etc) to the front of its clause are also first attested at B1: I don’t know [how I could have done it] B1 I did not know [where to look for it] B1

21. And postnominal modifiers in participial –ing become productive at B1: I received your mail [asking for the sales report] B1

22. Structures with a finite subordinate clause positioned to the right of predicates like is true and seems with a subject it are also criterial for B1 and higher levels: It’s true [that I don’t need a ring to make me remember you] B1  i.e. so-called “Extraposition” structures

23. A large number of new lexical verbs appear for the first time at B1 including: divide, fit, grab, spill, stick and tear And the meanings of the verbs that appeared first at A1 and A2 begin to expand from their basic senses.

24. break appears for the first time in the extended sense of INTERRUPT at B1: At last I managed to break the routine of the city … B1

25. Constructions that are criterial for B2 and higher levels include “secondary predications” go and paint the houses yellow and blue B2 with yellow and blue predicated of the direct object houses

26. “Extraposition” structures with a non-finite subordinate clause positioned to the right of its predicate are B2 It would be helpful [to work in your group as well] B2

27. And so-called “Pseudocleft” structures with an initial what functioning as subject of its verb: What fascinated me was [that I was able to lie on the sea surface] B2

28. “Subject-to-Subject Raising” constructions appear first at B2 with most of the higher verbs and adjectives that trigger this rule, for example prove: The car has proved [to be one of the most important inventions of our century] B2 Similar examples are found at B2 with other raising verbs and adjectives (The car happened to be …, The car appeared to be …, The car turned out to be …, The car is likely to be …, etc)

29. New lexical verbs at B2 include acquire, capture, drag, rush, spread,swallow and new meanings and uses are attested for the verbs that appeared earlier.

30. For break, first attested at A2, these include new collocations such as break a promise or break the law: For cut, also an A2 verb, they include new meanings at B2 such as REDUCE in cut the cost

31. The C Levels (C1 and C2) “Subject-to-Object Raising” constructions with the verb believe appear first at C1 and are criterial for the C levels: I believe her [to be this country’s best representative] C1

32. Passivized Subject-to-Object Raising constructions such as the following with assumed are also criterial for C1: the low cost of membership and entry was assumed to be an advantage. C1

33. Sequences of two prenominal –s genitives are found at C1: in the bride’s family’s house C1 Structurally: in [[[the bride’s] family’s] house]

34. New lexical verbs appearing first at C1 include accumulate, boast, quote, reassure, shape and stain along with new meaning possibilities for the verbs already introduced. E.g. break appears first in the idiomatic sense of break the bank at C1.

35. New features appearing at C2 include less common Subject-to-Object Raising constructions with higher predicates such as presume, declare and remember: He presumed work [to be the way to live] C2

36. New lexical verbs at C2 include stagger, sway, limp, saunter, raid,squander New meanings for break at C2 include original figurative senses such as the attested break the wall that surrounds him.

37. Negative Criterial Features One major distinguishing feature of the C levels can be seen in the low frequencies for “negative features” or error types such as those illustrated above. There are significant improvements in ALL of the syntactic and morpho-syntactic error types at the C levels.

38. By contrast, at the B levels improvements are relatively modest, and for many error types the scores actually get worse, especially at B2, before they get better again at C1.

39. The error codes involve morpho-syntactic errors of inflection, derivation and grammatical form, syntactic errors of omission, positioning and co-occurrence, and errors of appropriate lexical choice. It is clear that learners at the C levels are increasingly mastering these rules of English, whereas B-level learners are not (see Hawkins & Filipović 2012 for details).

40. We must now ask: WHY do we see these patterns in the data and why do we see the criterial features changing the way they do at the different levels? In particular, WHAT is it about the features of the higher proficiency levels that makes them late acquired rather than early?

41. It cannot simply be that learners are imitating the words and constructions they are explicitly taught in their textbooks. First, because there are many different textbooks and teaching methods around the world. But secondly because learners learn more than they are explicitly taught, from their reading materials, papers, magazines, movies, TV, conversations, etc. I.e. second language learning shares many similarities with first language learning, but not all obviously.

42. For example, more frequently occurring words and constructions are learned before less frequent ones, and simpler words, constructions and meanings are learned before more complex ones, in both first and second language acquisition.

43. E.g. learning English nouns and verbs with high frequencies of use is easier than learning those with lower frequencies, because they are encountered more frequently (greater exposure); frequent lexical items are overrepresented at first in L2 English, moving gradually to L1 English norms (seeHawkins & Buttery 2009, Hawkins & Filipović 2012)

44. The constructions of English that are learned earliest are those that occur most frequently in the input, as reflected in e.g. the British National Corpus. This could be established by comparing the CLC with the British National Corpus (BNC), see Williams (2007).

45. The new constructions that are criterial for A2, B1 and B2, in Williams’ (2007) data, appear to be learned in direct proportion to their frequency in the input, as reflected in the BNC. The more exposure, the earlier the acquisition and the easier the learning. This is shown in Tables 1, 2 and 3. Table 1 lists the new construction types found, for example, at B1. Tables 2 and 3 give the frequency correlations between the CLC and the BNC for the different levels.

46. Table 1 New B1 Verb Co-occurrence Frames NP-V-NP-NP She asked him [his name] NP-V-VPinfin (Wh-move) He explained [how to do it] NP-V-NP-V(+ing) (Obj Control) I caught him stealing NP-V-NP-PP (P=to) (Subtype: Dative Movement) He gave [a big kiss][to his mother] NP-V-NP-(to be)-NP (Subj to Obj Raising) I found him (to be) a good doctor NP-V-NP-Vpastpart (V=passive) (Obj Control) He wanted [the children] found NP-V-P-Ving-NP (V=+ing) (Subj Control) They failed in attempting the climb NP-V-Part-NP-PP I separated out [the three boys] [from the crowd] NP-V-NP-Part-PP I separated [the three boys] out [from the crowd] NP-V-S (Wh-move) He asked [how she did it] NP-V-PP-S They admitted [to the authorities] [that they had entered illegally] NP-V-Part She gave up NP-V-S (whether = Wh-move) He asked [whether he should come] NP-V-P-S (whether = Wh-move) He thought about [whether he wanted to go]

47. Table 2 Frequencies for Verb Co-occurrence Frames in English Corpora (including BNC) Average Token Frequencies in the BNC for the new Verb Co-occurrence Frames appearing at the learner levels A2 B1 B2/C1/C2 1,041,634 38,174 27,615

48. Table 3 Frequency Ranking Average Frequency Ranking in the BNC for the new Verb Co-occurrence Frames appearing at the learner levels A2 B1 B2/C1/C2 8.2 38.6 55.6

49. These kinds of data enable us to set up the following principle of second language learning (for which there are also well-attested parallels in first language learning, see e.g. Tomasello 2003, Diessel 2004, MacWhinney 2005):

50. (1) Maximize Frequently Occurring Properties (MaF) Properties of the L2 are learned in proportion to their frequency of occurrence (as measured, for example, in the BNC): more frequent exposure of a property to the learner facilitates its learning and reduces learning effort. I.e. more frequent properties will result in earlier L2 acquisition, more of the relevant properties learned, and fewer errors, in general. Infrequency makes learning more effortful, with precise predictions depending on other factors.