1 / 29

A Dynamic Learning Model For Categorizing Words Using Frames

A Dynamic Learning Model For Categorizing Words Using Frames. Hao Wang, Toben Mintz Department of Psychology University of Southern California. The Problem of Learning Syntactical Categories. Grammar includes manipulations of lexical items based on their syntactical categories.

jerry
Télécharger la présentation

A Dynamic Learning Model For Categorizing Words Using Frames

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Dynamic Learning Model For Categorizing Words Using Frames Hao Wang, Toben Mintz Department of Psychology University of Southern California

  2. The Problem of Learning Syntactical Categories • Grammar includes manipulations of lexical items based on their syntactical categories. • Learning syntactical categories are fundamental to the acquisition of language.

  3. The Problem of Learning Syntactical Categories • Nativist approach • Children are innately endowed with the possible syntactical categories. • How to map a lexical item to its syntactical category or categories? • Empirical approach • Children have to figure out the syntactical categories in their target language, and assign categories to lexical items. • There is no or little help from syntactical constraints.

  4. Approaches Based on Semantic Categories • Grammatical Categories correspond to Semantic/Conceptual Categories(Macnamara, 1972; Bowerman, 1973; Bates & MacWhinney, 1979; Pinker, 1984)object  noun action  verb • But what about • action, noise, love • to think, to know(Maratsos & Chalkley, 1980)

  5. Grammatical Categories from Distributional Analyses • Structural LinguisticsGrammatical categories defined by similarities of word patterning (Bloomfield , 1933; Harris, 1951) • Maratsos & Chalkley (1980): Distributional learning theory • lexical co-occurrence patterns • (and morphology and semantics) • the cat is on the mat • cat, mat

  6. Grammatical Categories from Distributional Analyses • Patterns across whole utterances(Cartwright & Brent, 1997) • My cat meowed. • Your dog slept. • Det N X/Y. • Bigram co-occurrence patterns(Mintz, Newport, & Bever, 1995, 2002; Redington, Chater & Finch, 1998) • the cat is on the mat

  7. Frequent Frames (Mintz, 2003) • Frames are defined as “two jointly occurring words with one word intervening”. • “would you put the cans back ?” • “you get the nuts .” • “you take the chair back . • “you read the story to Mommy .” • Frame: you_X_the

  8. Sensitivity to Frame-like Units • Frames lead to categorization in adults (Mintz, 2002) • Fifteen-month-olds are sensitive to frame-like sequences (Gómez & Maye, 2005)

  9. Distributional Analyses Using Frequent Frames (Mintz, 2003) • Six corpora from CHILDES (MacWhinney, 2000). • Analyzed utterances to children under 2;6. • Accuracy results averaged overall corpora.

  10. Limitation of the Frequent Frame Analyses • Requires two passes through the corpus • Step 1, identify the frequent frames by tallying the frame frequency. • Step 2, categorizing words using those frames. • Tracks the frequency of all frames • E.g., approximately 15000 frame types in one of the corpora in Mintz (2003).

  11. Goal of current study • Provides a psychological plausible model of word categorization • Children possesses limited memory and cognitive capacity. • Human memory is imperfect. • Children may not be able to track all the frames he/she has encountered.

  12. Features of current model • It processes input and updates the categorization frames dynamically. • Frame is associated with and ranked by a activation value. • It has a limited memory buffer for frames. • Only stores the most activated 150 frames. • It implements a forgetting function on the memory. • After processed a new frame, the activation of all frames in the memory decreased by 0.0075.

  13. Child Input Corpora • Six corpora from CHILDES (MacWhinney, 2000). • Analyzed utterances to children under 2;6. • Peter (Bloom, Hood, Lightbown, 1974; Bloom, Lightbown, Hood, 1975)Eve(Brown, 1973)Nina (Suppes, 1974)Naomi(Sachs, 1983)Anne(Theakston, Lieven, Pine, Rowland, 2001)Aran(Theakston et al., 2001) • Mean Utterance/Child: ~17,200 • MIN: 6,950 ; MAX: 20,857

  14. Procedure • The child-directed utterances from each corpus was processed individually • Utterances were presented to the model in the order of appearance in the corpus • Each utterance was segmented into frames • “you read the story to Mommy” • you read the • read the story • the story to • story to Mommy

  15. Procedure continued… • you read the • read the story • the story to • story to Mommy

  16. Procedure continued… • The memory buffer only stores most activated 150 frames. • It becomes full very quickly after processing several utterances.

  17. Procedure continued… • “you put the” • Frame: you_X_the • Look up you_X_the frame in the memory • Increase the activation of you_X_the frame by 1 • Re-rank the memory by activation

  18. Procedure continued… • “you have a” • Frame: you_X_a • Look up you_X_a frame in the memory • story_X_Mommy < 1 • Removestory_X_Mommy • Add you_X_a to memory, set the activation to 1 • Re-rank the memory by activation

  19. Procedure continued… • A new frame not in memory • The activation of all frames in memory are greater than 1 • There is no change to the memory.

  20. Evaluating Model Performance • Hit: two words from the same linguistic category grouped together • False Alarm: two words from different linguistic categories grouped together • Upper bound of 1

  21. V V V ADV V V Hits: 10 False Alarms: 5 Accuracy: Accuracy Example

  22. Noun, pronoun Verb, Aux., Copula Adjective Preposition Adverb Determiner Wh-word Negation -- “not” Conjunction Interjection Ten Categories for Accuracy

  23. Averaged accuracy across 6 corpora

  24. The Development of Accuracy • Accuracy are very high and stable in the entire process

  25. Compare to Frequent Frames • After processing about half of the corpus, 70% of frequent frames are in the most activated 45 frames in memory.

  26. Memory of Final Step of Eve Corpus

  27. Stability of Frames in Memory • Big changes of frames in memory in early stage, but become stable after processing 10% of the corpus

  28. Summary • After processed the entire corpus, the learning algorithm has identified almost all of the frequent frames by highest activation. • Consequently, high accuracy of word categorization is achieved. • After processing fewer than half of the utterances, the 45 most activated frames included approximately 70% of frequent frames.

  29. Summary • Frames are a robust cue for categorizing words. • With limited and imperfect memory, the learning algorithm can identify most frequent frames after processing a relatively small number of utterances. Thus yield a high accuracy of word categorization.

More Related