1 / 15

The SRI 2006 Spoken Term Detection System

Dimitra Vergyri, Andreas Stolcke, Ramana Rao Gadde, Wen Wang Speech Technology & Research Laboratory SRI International, Menlo Park, CA. The SRI 2006 Spoken Term Detection System. Outline. STD system overview STT systems BNews system description CTS system description

faunia
Télécharger la présentation

The SRI 2006 Spoken Term Detection System

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Dimitra Vergyri, Andreas Stolcke, Ramana Rao Gadde, Wen Wang Speech Technology & Research Laboratory SRI International, Menlo Park, CA The SRI 2006 Spoken Term Detection System

  2. STD-06 Workshop Outline • STD system overview • STT systems • BNews system description • CTS system description • ConfMtg system description • Indexing • N-gram index from word lattices • NNet based posterior estimation • Retrieval • Time and memory requirements • ATWV Results • Future work

  3. STD-06 Workshop SRI STD System Search Terms Audio N-gram Index with posteriors Word Lattices INDEXER RETRIEVER STT Terms with Times and Probabilities Indexing step

  4. STD-06 Workshop English BN STT System • Single front-end : PLP (52  39 dim) • HLDA, feature-space SAT • Gender-independent acoustic modeling • Decision-tree clustered within-word and cross-word triphones • MLE followed by alternating MPE-MMIE acoustic training • Acoustic training: Hub4, TDT2+TDT4+TDT4a, BNr1234 subset • MLE training: 3300 hours, MPE training: 1700 hours • 2500 x 200 Gaussian for nonCW triphones • 3000 x 160 Gaussians for CW triphones • Word bigram and 5-gram LMtrained on Hub4, TDT, BNr1234 transcripts, Hub4 LM training data, and NABN (cutoff date Nov. 30, 2003) • 62k words, 29M bigrams, 27M trigrams, 15M 4-grams, 2.4M 5-grams • Duration rescoring (word-specific phone durations) • Two-pass decoding • First decoding stage unadapted nonCW model with bigram LM • Adapted CW models to nonCW output after 5-gram LM and duration model lattice rescoring • Lattice constrained decoding with MLLR adapted, SAT, cw model

  5. STD-06 Workshop English BN STT System PLP MPE CW Adapted 5-gram Lattices PLP MPE nonCW 2-gram Lattices 4-gram Lattices Legend Decoding/rescoring step Hyps for MLLR or output Lattice generation/use Lattice or 1-best output • Runtimes: • 2.5xRT for unadapted lattices • 5.4xRT for adapted lattices • ~10% relative WER improvement after adaptation • Both decoding stages use shortlists.

  6. STD-06 Workshop English CTS STT System • Two front-ends: • MFCC + voicing + MLP-features (52 + 10 + 25  39 + 25 dim) • PLP (52  39 dim) • HLDA, feature-space SAT • Gender-dependent acoustic modeling • Decision-tree clustered within-word and cross-word triphones • MLE followed by alternating MPE-MMIE acoustic training • Acoustic training: all Hub5 + Fisher training • 2500 x 128 x 2 Gaussians for nonCW triphones • 3000 x 128 x 2 Gaussians for CW triphones • Prosodic rescoring (word-specific phone durations, pause trigram) • Word bigram and 4-gram LM • Interpolated + pruned LM trained on CTS, BN, and Web data • 48k words, 16M bigrams, 16M trigrams, 12M 4grams • First lattice generation uses phone-loop MLLR nonCW MFCC and 2-gram LM • Second constrained lattice generation uses cross-adapted CW SAT PLP models.

  7. STD-06 Workshop English Meeting STT System[Stolcke et al., MLMI’05; Janin et al., MLMI’06] • Based on CTS system architecture (2-pass system) • Combination of CTS (narrow-band) and BN (wide-band) base models • Acoustic models adapted to distant-mic meeting recordings using MMI-MAP • MLP features adapted for meeting recordings by incremental training • Mixture language model trained on meetings, CTS, and Web data • System used in RT-06S meeting evaluation, co-developed with ICSI

  8. STD-06 Workshop English CTS & Confmtg STT Systems Adapted 4-gram lattices MFCC-MLP MPE nonCW PLP MPE CW 2-gram Lattices 3-gram Lattices Legend Decoding/rescoring step Hyps for MLLR or output Lattice generation/use Lattice or 1-best output • CTS runtime: • 1.8xRT for unadapted lattices • 2.5xRT for adapted lattices • Confmtg runtime: • 5.4xRT for unadapted lattices • 6.8xRT for adapted lattices • CTS system uses Gaussian shortlists in first pass only • Confmtg system does not use shortlists.

  9. eval02 eval03 STD-dev06 BN 10.7% 10.5% 23.2% eval02 eval03 dev04 STD-dev06 CTS 23.7% 24.0% 17.0% 17.4% dev04 eval04s STD-dev06 Confmtg 36.9% 37.2% 44.2% English STT Result Summary (WER) • STD-dev06 WER measured using references constructed from RTTM files • Systematic differences compared to standard STT references • For example, BN scoring does not exclude commercial segments • Note: STT systems were not especially tuned for STD; used configurations inherited from STT evaluations. STD-06 Workshop

  10. STD-06 Workshop Indexing of Word Lattices • SRILM lattice-tool dumps all word 1-grams to 5-grams in lattices, along with side information • Posterior probabilities based on normalized recognizer scores • Start/end times, channel, waveform name • 0.5s time tolerance to merge same N-grams with different times • Pronunciations (to detect OOV words, not used yet) • N-grams with posterior < 0.001 are omitted to keep index size reasonable • Index = term occurrence table sorted by N-gram • Indexing function incorporated in SRILM release 1.5.1 • Lattice-tool –write-ngram-index option • Downloadable from www.speech.sri.com/projects/srilm/

  11. STD-06 Workshop Score Calibration • Neural net maps posteriors to unbiased STD scores • Input features used: audio source (bnews/cts/confmtg), LM joint probability, LM N-gram length, #words, duration, lattice posterior • Used LnkNet software for training MLP to predict correctness of hypothesized term (1 hidden layer with 10 nodes) • Cross entropy objective function • Neural net trained using the dev06 term list • Training on raw data improved Occurrence Weighted Value, not A-Term Weighted Value • Also required re-tuning the posterior threshold. • Resample training data to approximate ATWV • Downsample/upsample within occurrences of each term to have equal number of training samples for each term. • Posterior threshold 0.5 ended up being optimal for ATWV (at least on the training data).

  12. STD-06 Workshop Searching & Retrieval • Convert the search terms into a sorted list • Run the Unix “join” command between the index list obtained in indexing and the term list • YES/NO decision based on the posterior threshold 0.5 • Run time almost independent of the size of the search list (depends on the index size)

  13. STD-06 Workshop BN (3h) CTS (3h) Confmtg (2h) Indexing STT run time 58560 s 26760 s 40440 s Index from lattice 493 s NNet run time 2711 s Index size (# terms/MB) 944K 74Mb 602K 37Mb 530K 37Mb Search time needed for all terms 13 s Time and Memory Requirements • The system was run on 3GB, 3.4 GHz Intel hyperthreading CPU • Both index size and search time can be significantly reduced if we keep only candidates with high posterior • STT runtimes were incorrectly measured in submitted sysdesc.

  14. STD-06 Workshop Occ.WV/ATWV Thres. dev06 dryrun06 Extra dev eval06 BN No NNet With NNet 0.3 0.5 0.914/0.850 0.914/0.865 0.906/0.802 0.905/0.818 0.887/0.801 0.889/0.817 --- / 0.824 CTS No NNet With NNet 0.3 0.5 0.881/0.692 0.881/0.714 0.860/0.615 0.860/0.660 0.792/0.681 0.800/0.712 --- / 0.665 Confmtg No NNet With NNet 0.3 0.5 0.585/0.275 0.515/0.427 0.566/0.205 0.491/0.358 0.631/0.462 0.536/0.461 --- / 0.255 All No NNet With NNet 0.3 0.5 0.821/0.787 0.804/0.817 0.802/0.700 0.782/0.739 0.790/0.687 0.784/0.718 STD Results • Extra dev consists of RT02, RT03 (BN+CTS), dev04 (CTS+ConfMtg), RT04s (ConfMtg) • Difficult to debug eval06 (no references were given), but the result on meetings seems much lower than on dev sets. • Possibly overtrained neural net on meetings condition.

  15. STD-06 Workshop Future Work • Current system does not cover detection of terms with OOVs. Possible approaches: • Map the unknown search terms to the known vocabulary (OGI work, gave about 2-3% improvement on BNews). • Use of phone recognition and phone-based indexing for OOVs • Hybrid word+graphone recognizer outputs both words and “graphone” units that can match OOVs (Bisani & Ney 2005) • Improve the score mapper • Bigger devset needed to avoid overtraining • Other models (decision tree, logistic regression) • Found some mismatch between ASR vocabulary and term lists. Apply normalization rules to fix common problems (found about 0.3% relative improvement with few simple rules) • Tune STT systems for indexing speed

More Related