ICASSP 2007 Robustness Techniques Survey

1. ICASSP 2007 Robustness Techniques Survey Presenter: Shih-Hsiang Lin

2. Topic • Word Graph based Feature Enhancement for Noisy Speech Recognition • Stereo-based Stochastic Mapping for Robust Speech Recognition • Combination of Recognizers and Fusion of Features Approach to Missing Data ASR Under Non-Stationary Noise Conditions

3. WORD GRAPH BASED FEATURE ENHANCEMENT FOR NOISY SPEECH RECOGNITION Zhi-Jie Yan1Frank K. Soong2Ren-Hua Wang1 1iFlytek Speech Lab, University of Science and Technology of China, Hefei, P. R. China, 230027 2Microsoft Research Asia, Beijing, P. R. China, 100080 SPE-L3: Robust Features and Acoustic Modeling

4. Introduction • This paper presents a word graph based feature enhancement method for robust speech recognition in noise • The word graph based approach would have more chance that the correct hypotheses exist in the graph with relatively lower posterior probabilities (or likelihoods) than the incorrect first best hypothesis • The proposed method is based upon Wiener fitering of the Mel-filter bank energy, given • the input noisy speech • a signal processing based estimate of noise • a clean trained Hidden Markov Model (HMM) • Therefore, the enhanced speech feature after Wiener filtering can match the clean speech model better in the acoustic space, and thus leads to an improved recognition performance

5. Algorithm Overview normalized speech model based estimate of the clean speech rough estimate of noise spectrum corresponding clean speech mean re-estimate noise final estimate of clean speech

6. More Details… • Kernel posterior probabilities for each Gaussian component of the model can be calculated • These posterior probabilities will serve as the weighting coefficients for synthesizing the model based clean speech for Wiener filtering • Using the word graph, the posterior probability of kernel k at time t, given the entire observation sequence can be formulated as: • The model based clean speech estimate for Wiener filtering is constructed in two steps • First step, for each time frame t, the expected values of the mean and covariance of the clean speech feature are calculated using the kernel posterior probabilities along with the kernel parameters Word Posterior Probability (WPP) State Occupancy Probability Kernel Occupancy Probability

7. More Details… (cont.) • Second step, clean speech S3 can be synthesized in ML sense • the ML solution of S3 can be obtained by solving the weighted normal equation synthesized clean speech

8. More Details… (cont.) • Wiener filtering of the Mel-filter bank energy is performed in the linear spectral domain • In the last step, is converted to the cepstral domain, and then rescore the word graph • Re-decode S4 within the constrained search space defined by the word graph final estimate of clean speech

9. Experimental Results • signal processing based feature enhancement consistently improves the recognition performance, and the overall relative error rate reduction is 35.44% • the GER of the decoded word graph is significantly lower than the WER of the first best hypothesis (only about 1/4 ∼ 1/5) • The results show that an overall relative error rate reduction of 57.89% is obtained • Using word graph constrained second pass decoding, this result is obtained with a minor increase of the computational cost • The experimental results suggest that the difference between the two decoding scenarios is minimal

10. STEREO-BASED STOCHASTIC MAPPING FOR ROBUST SPEECH RECOGNITION Mohamed Afif, Xiaodong Cui, and Yuqing Gao IBM T.J. Watson Research Center 1101 Old Kitchawan Road, Yorktown Heights, NY, 10598 SPE-L3: Robust Features and Acoustic Modeling

11. Introduction • The idea is based on building a GMM for the joint distribution of the clean and noisy channels during training and using an iterative compensation algorithm during testing. • Also interpreted as a mixture of linear transforms that are estimated in a special way using stereo data • Stack both the clean and noisy channels to form a large augmented space and to build a statistical model in this new space • The observed noisy speech and the augmented statistical model are used to predict the clean speech

12. Algorithm Formulation • Assume we have a set of stereo data {(xi, yi)} • Define z ≡ (x, y) as the concatenation of the two channels • The first step in constructing the mapping is training the joint probability model for p(z) • Once this model is constructed it can be used during testing to estimate the clean speech given the noisy observations • The problem of estimating x in Equation looks like a mixture estimation problem where

13. Algorithm Formulation (cont.) • Hence, we will iteratively optimize an EM objective function given by • is the value of x from previous iteration

14. Algorithm Formulation (cont.) • By differentiating Equation with respect to x, setting the resulting derivative to zero • An interesting special case arises when x is a scalar

15. Experimental Results Digit recognition in the car • The first three lines refer to train/test conditions where the • clean refers to the CT and noisy to the HF • It can be observed that the proposed mapping outperforms SPLICE for all GMM sizes with the difference decreasing with increasing the GMM size • Both methods are considerably better than the VTS result in the last row of Table 1 • Using a time window gives an improvement over the baseline SSM with a slight cost during runtime • These results are not given for SPLICE because using biases requires that both the input and output spaces have the same dimensions

16. Experimental Results (cont.) English large vocabulary speech recognition MFCC Feature • SSM brings considerable improvement over MST even in • the clean speech condition LDA+MLLT Feature • Building maps for the final feature space (after LDA and • MLLT) looks to be slightly better than the original cepstral • space

17. Combination of Recognizers and Fusion of Features Approach to Missing Data ASR Under Non-Stationary Noise Conditions Neil Joshi and Ling Guan Department of Electrical and Computer Engineering Ryerson University Toronto ON M5B 2K3, Canada SPE-P14: Robustness II

18. Introduction • This paper proposes a method a enhance speech recognition performance using missing data techniques for non-stationary noise conditions • By incorporating more resilient feature sets into the decoding process • Two separate HMM based models • One using spectral features • The other MFCC features • The statistical dependencies found in the models are based upon a coupled HMM methodology, the Fused HMM model • One using standard ASR techniques (traditional MFCC based HMM models) • The other missing data based (missing data theory spectral HMM models)

19. Coupled Fused HMM • The fused HMM model models the relationship between HMMs using a probabilistic fusion model • The statistical dependencies between two HMM process is thus,

20. Experimental Results • the fused decoder is found to significantly • increase recognition performance over • conventional missing data decode process