80 likes | 299 Vues
Question AnswerinG. Deliverable 3 – Question Processing John Gilmer Michael Foster Adam Ledyard. Overview. Question Classification Question Reformulation Query Expansion Results Related Reading. Question Classification. Used Roth and Li’s Question Classification Taxonomy
E N D
Question AnswerinG Deliverable 3 – Question Processing John Gilmer Michael Foster Adam Ledyard
Overview • Question Classification • Question Reformulation • Query Expansion • Results • Related Reading
Question Classification • Used Roth and Li’s Question Classification Taxonomy • Trained a MaxEnt Classifier on the 5500 training set • Features used: • Unigrams • POS tag unigrams • NP chunks found by nltk trained chunker using BIO tagged CoNLL 2000 corpus (WSJ text) • Semantic feature based on Roth and Li’s semantic lists • Results: 84% accuracy based on TREC 10 labeled test data.
Question Reformulation • Topic insertion: Topic text is always added to a query: • Replace the first pronoun in the sentence with topic. • Leads to duplicate query terms if partial topic text and a pronoun are in a question. • Potential replacement of incorrect pronoun (none observed). • “How long was it used as a defense?” → “How long was Great Wall of China used as a defense?” • If no pronouns are present, identify capitalized terms in the question that are also in the topic. • Look forward and backward to make sure that only missing terms are inserted. • “What company acquired Wynn’s Mirage Resorts in 2000?” → “What company acquired Steven Wynn’sMirage Resorts in 2000?” • Otherwise, if it isn’t present, insert topic at beginning of query. • “How old was the dam?” → “Johnstown floodhow old was the dam”?
Question Reformulation • Exact topic text queries: • When topic text is inserted, create a second query with the topic text in “double quotes” to force an exact phrase search. • Add quotes regardless of position in the query. • ngrams that came from queries containing double quoted text are given a multiplier to their score during answering processing.
Query Expansion • - Use Question Topic for Wikipedia Query • - If Wikipedia returns a page, it is unambiguous • - Extract first sentence of page and parse and chunk with NLTK • - Extract relevant Noun Phrase Chunks and use them to expand query terms • ISSUES: • - Sometimes topics will not return any Wikipedia Page (i.e. “The tourist massacre in Luxor in 1999”) • -Frequently term expansions are not that helpful as is typical of query expansion efforts
Related reading • Fang, Hui. "A re-examination of query expansion using lexical resources." Proceedings of ACL-08: HLT (2008): 139-147. • Li, Xin, and Dan Roth. "Learning question classifiers." Proceedings of the 19th international conference on Computational linguistics-Volume 1. Association for Computational Linguistics, 2002. • Stoyanchev, Svetlana, Young Chol Song, and William Lahti. "Exact phrases in information retrieval for question answering." Coling 2008: Proceedings of the 2nd workshop on Information Retrieval for Question Answering. Association for Computational Linguistics, 2008.