1. Wavelet-Based Denoising Using Hidden Markov Models M. Jaber Borran and Robert D. Nowak Rice University

2. Some properties of DWT • Primary • Locality  Match more signals • Multiresolution • Compression Sparse DWT’s • Secondary • Clustering Dependency within scale • Persistence  Dependency across scale

3. pS(1) fW|S(w|1) pS(2) fW|S(w|2) fW (w) S W Probabilistic Model for an Individual Wavelet Coefficient • Compression many small coefficients few large coefficients

4. t f Probabilistic Model for a Wavelet Transform Ignoring the dependencies  Independent Mixture (IM) Model Clustering  Hidden Markov Chain Model Persistence  Hidden Markov Tree Model

5. Parameters of HMT Model • pmf of the root node • transition probability • (parameters of the) conditional pdfs e.g. if Gaussian Mixture is used q : Model Parameter Vector

6. Signal w1 w1 Wavelet t t T T t 0 t T w2 w2 T/2 T/2 t 0 t T/2 w2 w2 T T/2 T t t T/2 T/2 T Dependency between Signs of Wavelet Coefficients

7. pS(2) fW|S(w|2) pS(4) fW|S(w|4) pS(1) fW|S(w|1) pS(3) fW|S(w|3) fW (w) S W New Probabilistic Model for Individual Wavelet Coefficients • Use one-sided functions as conditional probability densities

8. Proposed Mixture PDF • Use exponential distributions as components of the mixture distribution m even m odd

9. PDF of the Noisy Wavelet Coefficients Wavelet transform is orthonormal, therefore if the additive noise is white and zero-mean Gaussian process with variance s2, then we have Noisy wavelet coefficient, m even m odd

10. Training the HMT Model • y: Observed noisy wavelet coefficients • s: Vector of hidden states • q: Model parameter vector Maximum likelihood parameter estimation: Intractable, because s is unobserved (hidden).

11. Model Training Using Expectation Maximization Algorithm • and then, • Define the set of complete data, x = (y,s)

12. EM Algorithm (continued) • State a posteriori probabilities are calculated using Upward-Downward algorithm • Root state a priori pmf and the state transition probabilities are calculated using Lagrange multipliers for maximizing U. • Parameters of the conditional pdf may be calculated analytically or numerically, to maximize the function U.

13. Denoising • MAP estimate:

14. Denoising (continued) • Conditional Mean estimate:

18. Conclusion • We observed a high correlation between the signs of the wavelet coefficients in adjacent scales. • We used one-sided distributions as mixture components for individual wavelet coefficients. • We used hidden Markov tree model to capture the dependencies. • The proposed method achieves better MSE in denoising and the denoised signals are much smoother.