1 / 15

Identifying Expressions of Opinion in Context

Identifying Expressions of Opinion in Context. Eric Breck and Yejin Choi and Claire Cardie IJCAI 2007. Introduction. Traditional information extraction: answer questions about facts Extract answers to subjective questions: how does X feel about Y?

pabla
Télécharger la présentation

Identifying Expressions of Opinion in Context

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Identifying Expressions of Opinion in Context Eric Breck and Yejin Choi and Claire Cardie IJCAI 2007

  2. Introduction • Traditional information extraction: answer questions about facts • Extract answers to subjective questions: how does X feel about Y? • Subjective information extraction and question answering will require techniques to analyze text below the sentence level

  3. Introduction: System Requirement • Is its polarity positive, negative, or neutral? • With what strength or intensity is the opinion expressed: mild, medium, strong or extreme? • Who or what is the source, or holder, of the opinion? • What is its target, i.e. what is the opinion about?

  4. Introduction: Examples • Minister Vedrine criticizedthe White House reaction. • the agent role = “Minister Vedrine” • the object/theme role = “White House reaction” • 17 persons were killed by sharpshooters faithful tothe president. • Tsvangirai saidthe election result was “illegitimate” and a clear case of “highway robbery”. • Criminals have been preying on Korean travelers in China.

  5. Introduction • Direct subjective expressions (DSEs) • criticized, faithful to • Said (speech event, if subjective) • Expressive subjective elements (ESEs) • illegitimate, highway robbery • preying on (instead of mugging) • None has directly tackled the problem of opinion expression identification.

  6. Subjective Expressions • The expressions can vary in length from one word to over twenty words. • They may be verb phrases, noun phrases, or strings of words that do not correspond to any linguistic constituent. • Subjectivity is a realm of expression where writers get quite creative, so no short fixed list can capture all expressions of interest. • Also, an expression which is subjective in one context is not always subjective in another context.

  7. Approach • This task is treated as a tagging problem. • Conditional random field • Class variable • IOB vs IO • Features • A linear-chain conditional random field is chosen, using MALLET toolkit.

  8. Features (1) • Lexical features • The word at position i relative to the current token. • Lex-4 ~ Lex4, , 18,000 binary features per position (vocabulary size) • Syntactic features • POS (45 binary features) • prev, cur, next (CASS partial parser, constituent type), 100 binary features each. • Dictionary-based features

  9. Features (2) • Dictionary-based features: 4 sources • WordNet: WordNet hypernyms (29,989 binary features) • Levin: Levin’s categorization of English words • Framenet: word in the categorization of nouns and verbs in Framenet • Wilson clues (subjective): strong or weak (two binary features)

  10. Statistics of Data MPQA corpus, 535 documents. 135 for training, 400 for testing. 10-fold cross validation

  11. Evaluation • Metric: Precision/Recall/F-measure • Exact • Overlap • Baselines: dictionary-based • two dictionaries of subjectivity clues: Wiebe vs. Wilson • Wilson is incorporated in this experiment

  12. Results (DSE/ESE)

  13. Results (DSE and ESE)

  14. Results (Dictionary-based) • WordNet is the most useful • The other dictionaries only help a little

  15. Discussion • Rules of boundary agreement is not defined for the annotations: order 1 outperform order 0 • DSEs includes speech events like “said” or “a statement”, which may be objective. • Expressions of subjectivity tend to cluster, therefore density-based features might help. • Inter-annotator agreement of DSE: 0.75; ESE:0.72

More Related