1 / 50

Yun - Nung ( Vivian ) Chen, Yu Huang, Sheng -Yi Kong, Lin-Shan Lee

Automatic Key Term Extraction from Spoken Course Lectures Using Branching Entropy and Prosodic/Semantic Features. Yun - Nung ( Vivian ) Chen, Yu Huang, Sheng -Yi Kong, Lin-Shan Lee. National Taiwan University, Taiwan. Introduction. Definition. Key Term Higher term frequency Core content

zaina
Télécharger la présentation

Yun - Nung ( Vivian ) Chen, Yu Huang, Sheng -Yi Kong, Lin-Shan Lee

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Automatic Key Term Extractionfrom Spoken Course LecturesUsing Branching Entropy and Prosodic/Semantic Features Yun-Nung (Vivian) Chen, Yu Huang, Sheng-Yi Kong, Lin-Shan Lee National Taiwan University, Taiwan

  2. Key Term Extraction, National Taiwan University Introduction

  3. Key Term Extraction, National Taiwan University Definition • Key Term • Higher term frequency • Core content • Two types • Keyword • Key phrase • Advantage • Indexing and retrieval • The relations between key terms and segments of documents

  4. Key Term Extraction, National Taiwan University Introduction

  5. Key Term Extraction, National Taiwan University Introduction language model n gram hmm acoustic model hidden Markov model phone

  6. Key Term Extraction, National Taiwan University Introduction bigram language model n gram hmm acoustic model hidden Markov model phone Target: extract key terms from course lectures

  7. Key Term Extraction, National Taiwan University Proposed Approach

  8. Key Term Extraction, National Taiwan University Automatic Key Term Extraction ▼ Original spoken documents Learning Methods K-means Exemplar AdaBoost Neural Network Archive of spoken documents ASR trans ASR Branching Entropy Feature Extraction speech signal

  9. Key Term Extraction, National Taiwan University Automatic Key Term Extraction Learning Methods K-means Exemplar AdaBoost Neural Network Archive of spoken documents ASR trans ASR Branching Entropy Feature Extraction speech signal

  10. Key Term Extraction, National Taiwan University Automatic Key Term Extraction Learning Methods K-means Exemplar AdaBoost Neural Network Archive of spoken documents ASR trans ASR Branching Entropy Feature Extraction speech signal

  11. Key Term Extraction, National Taiwan University Automatic Key Term Extraction Phrase Identification Learning Methods K-means Exemplar AdaBoost Neural Network Archive of spoken documents ASR trans ASR Branching Entropy Feature Extraction speech signal First using branching entropy to identify phrases

  12. Key Term Extraction, National Taiwan University Automatic Key Term Extraction Phrase Identification Key Term Extraction Learning Methods K-means Exemplar AdaBoost Neural Network Archive of spoken documents ASR trans ASR Branching Entropy Feature Extraction speech signal Key terms entropy acoustic model : Then using learning methods to extract key terms by some features

  13. Key Term Extraction, National Taiwan University Automatic Key Term Extraction Phrase Identification Key Term Extraction Learning Methods K-means Exemplar AdaBoost Neural Network Archive of spoken documents ASR trans ASR Branching Entropy Feature Extraction speech signal Key terms entropy acoustic model :

  14. Key Term Extraction, National Taiwan University How to decide the boundary of a phrase? Branching Entropy represent is of is hidden Markov model • “hidden” is almost always followed by the same word in can : : : :

  15. Key Term Extraction, National Taiwan University How to decide the boundary of a phrase? Branching Entropy represent is of is hidden Markov model • “hidden” is almost always followed by the same word • “hidden Markov” is almost always followed by the same word in can : : : :

  16. Key Term Extraction, National Taiwan University How to decide the boundary of a phrase? Branching Entropy represent is of is hidden Markov model in can : : : : boundary • “hidden” is almost always followed by the same word • “hidden Markov” is almost always followed by the same word • “hidden Markov model” is followed by many different words Define branching entropy to decide possible boundary

  17. Key Term Extraction, National Taiwan University How to decide the boundary of a phrase? Branching Entropy represent is of is hidden Markov model in can : : X xi : : • Definition of Right Branching Entropy • Probability of children xi for X • Right branching entropy for X

  18. Key Term Extraction, National Taiwan University How to decide the boundary of a phrase? Branching Entropy represent is of is hidden Markov model in can : : X : : boundary • Decision of Right Boundary • Find the right boundary located between X and xi where

  19. Key Term Extraction, National Taiwan University How to decide the boundary of a phrase? Branching Entropy represent is of is hidden Markov model in can : : : :

  20. Key Term Extraction, National Taiwan University How to decide the boundary of a phrase? Branching Entropy represent is of is hidden Markov model in can : : : :

  21. Key Term Extraction, National Taiwan University How to decide the boundary of a phrase? Branching Entropy represent is of is hidden Markov model in can : : : :

  22. Key Term Extraction, National Taiwan University How to decide the boundary of a phrase? Branching Entropy represent is of is hidden Markov model in can : : X : : boundary • Decision of Left Boundary • Find the left boundary located between X and xi where X: model Markov hidden Using PAT Tree to implement

  23. 4 6 5 1 2 3 Key Term Extraction, National Taiwan University How to decide the boundary of a phrase? Branching Entropy • Implementation in the PAT tree • Probability of children xi for X • Right branching entropy for X hidden X: hidden Markov x1: hidden Markov model x2: hidden Markov chain Markov X state variable chain model distribution x2 x1

  24. Key Term Extraction, National Taiwan University Automatic Key Term Extraction Phrase Identification Key Term Extraction Learning Methods K-means Exemplar AdaBoost Neural Network Archive of spoken documents ASR trans ASR Branching Entropy Feature Extraction speech signal Key terms entropy acoustic model : Extract prosodic, lexical, and semantic features for each candidate term

  25. Key Term Extraction, National Taiwan University Feature Extraction Speaker tends to use longer duration to emphasize key terms • Prosodic features • For each candidate term appearing at the first time duration of phone “a” normalized by avg duration of phone “a” using 4 values for duration of the term

  26. Key Term Extraction, National Taiwan University Feature Extraction Higher pitch may represent significant information • Prosodic features • For each candidate term appearing at the first time

  27. Key Term Extraction, National Taiwan University Feature Extraction Higher pitch may represent significant information • Prosodic features • For each candidate term appearing at the first time

  28. Key Term Extraction, National Taiwan University Feature Extraction Higher energy emphasizes important information • Prosodic features • For each candidate term appearing at the first time

  29. Key Term Extraction, National Taiwan University Feature Extraction Higher energy emphasizes important information • Prosodic features • For each candidate term appearing at the first time

  30. Key Term Extraction, National Taiwan University Feature Extraction • Lexical features Using some well-known lexical features for each candidate term

  31. Key Term Extraction, National Taiwan University Feature Extraction Key terms tend to focus on limited topics • Semantic features • Probabilistic Latent Semantic Analysis (PLSA) • Latent Topic Probability tj: terms Di:documents Tk: latent topics

  32. Key Term Extraction, National Taiwan University Feature Extraction Key terms tend to focus on limited topics • Semantic features • Probabilistic Latent Semantic Analysis (PLSA) • Latent Topic Probability non-key term key term How to use it? describe a probability distribution

  33. Key Term Extraction, National Taiwan University Feature Extraction Key terms tend to focus on limited topics • Semantic features • Probabilistic Latent Semantic Analysis (PLSA) • Latent Topic Significance Within-topic to out-of-topic ratio non-key term key term within-topic freq. out-of-topic freq.

  34. Key Term Extraction, National Taiwan University Feature Extraction Key terms tend to focus on limited topics • Semantic features • Probabilistic Latent Semantic Analysis (PLSA) • Latent Topic Significance Within-topic to out-of-topic ratio non-key term key term within-topic freq. out-of-topic freq.

  35. Key Term Extraction, National Taiwan University Feature Extraction Key terms tend to focus on limited topics • Semantic features • Probabilistic Latent Semantic Analysis (PLSA) • Latent Topic Entropy non-key term key term

  36. Key Term Extraction, National Taiwan University Feature Extraction Key terms tend to focus on limited topics • Semantic features • Probabilistic Latent Semantic Analysis (PLSA) • Latent Topic Entropy non-key term Higher LTE key term Lower LTE

  37. Key Term Extraction, National Taiwan University Automatic Key Term Extraction Phrase Identification Key Term Extraction Learning Methods K-means Exemplar AdaBoost Neural Network Archive of spoken documents ASR Branching Entropy Feature Extraction ASR trans speech signal Key terms entropy acoustic model : Using unsupervised and supervised approaches to extract key terms

  38. Key Term Extraction, National Taiwan University Learning Methods • Unsupervised learning • K-means Exemplar • Transform a term into a vector in LTS (Latent Topic Significance) space • Run K-means • Find the centroid of each cluster to be the key term The terms in the same cluster focus on a single topic The candidate term in the same group are related to the key term The key term can represent this topic

  39. Key Term Extraction, National Taiwan University Learning Methods • Supervised learning • Adaptive Boosting • Neural Network Automatically adjust the weights of features to produce a classifier

  40. Key Term Extraction, National Taiwan University Experiments & Evaluation

  41. Key Term Extraction, National Taiwan University Experiments • Corpus • NTU lecture corpus • Mandarin Chinese embedded by English words • Single speaker • 45.2 hours 我們的solution是viterbi algorithm (Our solution is viterbi algorithm)

  42. Key Term Extraction, National Taiwan University Experiments some data from target speaker • ASR Accuracy SI Model Bilingual AM and model adaptation AM CH EN trigram interpolation LM Adaptive Background In-domain corpus Out-of-domain corpora

  43. Key Term Extraction, National Taiwan University Experiments • Reference Key Terms • Annotations from 61 students who have taken the course • If the k-th annotator labeled Nk key terms, he gave each of them a score of , but 0 to others • Rank the terms by the sum of all scores given by all annotators for each term • Choose the top N terms form the list (N is average Nk) • N = 154 key terms • 59 key phrases and 95 keywords

  44. Key Term Extraction, National Taiwan University Experiments • Evaluation • Unsupervised learning • Set the number of key terms to be N • Supervised learning • 3-fold cross validation

  45. Key Term Extraction, National Taiwan University Experiments • Feature Effectiveness • Neural network for keywords from ASR transcriptions F-measure 56.55 48.15 42.86 35.63 20.78 Pr: Prosodic Lx: Lexical Sm: Semantic Prosodic features and lexical features are additive Three sets of features are all useful Each set of these features alone gives F1 from 20% to 42%

  46. Key Term Extraction, National Taiwan University Experiments AB: AdaBoost NN: Neural Network F-measure • Conventional TFIDF scores w/o branching entropy • stop word removal • PoS filtering • Overall Performance 67.31 62.39 55.84 51.95 23.38 Branching entropy performs well Supervised approaches are better than unsupervised approaches K-means Exempler outperforms TFIDF

  47. Key Term Extraction, National Taiwan University Experiments AB: AdaBoost NN: Neural Network F-measure • Overall Performance 67.31 62.70 62.39 57.68 55.84 52.60 51.95 43.51 23.38 20.78 Supervised learning using neural network gives the best results The performance of ASR is slightly worse than manual but reasonable

  48. Key Term Extraction, National Taiwan University Conclusion

  49. Key Term Extraction, National Taiwan University Conclusion • We propose the new approach to extract key terms • The performance can be improved by • Identifying phrases by branching entropy • Prosodic, lexical, and semantic features together • The results are encouraging

  50. Key Term Extraction, National Taiwan University Thanks for your attention! Q & A Thank reviewers for valuable comments NTU Virtual Instructor: http://speech.ee.ntu.edu.tw/~RA/lecture

More Related