1 / 53

Maximum a Posterior

This presentation explores the concept of Maximum a Posterior (MAP) estimation in Hidden Markov Models (HMMs), focusing on both discrete and semi-continuous cases. We delve into using Dirichlet and normal-Wishart priors to derive MAP estimates, emphasizing the significance of prior information in parameter estimation. The session covers the derivation of initial estimates through modes and means of prior densities, alongside detailing segmental MAP estimates. An appendix introduces matrix calculus concepts relevant to the discussions, providing a comprehensive understanding for practitioners in the field.

yon
Télécharger la présentation

Maximum a Posterior

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Maximum a Posterior Presented by 陳燦輝

  2. Maximum a Posterior • Introduction • MAP for Discrete HMM • Prior Dirichlet • MAP for Semi-Continuous HMM • Prior Dirichlet + normal-Wishart • Segmental MAP Estimates • Conclusion • Appendix-Matrix Calculus

  3. Introduction • HMM parameter estimators have been derived purely from the training observation sequences without any prior information included. • There may be many cases in which the prior information about the parameters is available, ex : previous experience

  4. Introduction (cont)

  5. Discrete HMM Definition :

  6. Discrete HMM Q-function :

  7. Discrete HMM Q-function :

  8. Discrete HMM Q-function : 同理

  9. Discrete HMM R-function :

  10. Discrete HMM Q

  11. Discrete HMM Initial probability

  12. Discrete HMM Transition probability

  13. Discrete HMM observation probability

  14. Discrete HMM

  15. Discrete HMM • How to choose the initial estimate for ? • One reasonable choice of the initial estimate is the mode of the prior density.

  16. Discrete HMM • What’s the mode ? • So applying Lagrange Multiplier we can easily derive above modes. • Example :

  17. Discrete HMM • Another reasonable choice of the initial estimate is the mean of the prior density. • Both are some kind of summarization of the available information about the parameters before any data are observed.

  18. SCHMM

  19. SCHMM independent

  20. Model 1 Model 2 Model M SCHMM

  21. SCHMM Q-function :

  22. SCHMM Q-function :

  23. SCHMM

  24. SCHMM Initial probability • Differentiating w.r.t and equate it to zero.

  25. SCHMM Transition probability • Differentiating w.r.t and equate it to zero.

  26. SCHMM Mixture weight • Differentiating w.r.t and equate it to zero.

  27. SCHMM • Differentiating w.r.t and equate it to zero. • Differentiating w.r.t and equate it to zero.

  28. SCHMM • Full Covariance matrix case :

  29. SCHMM • Full Covariance matrix case :

  30. SCHMM • Full Covariance matrix case :

  31. SCHMM • Full Covariance matrix case : (1) (2) (3)

  32. SCHMM • Full Covariance matrix case :

  33. SCHMM Full Covariance • The initial estimate can be chosen as the mode of the prior PDF • And also can be chosen as the mean of the prior PDF

  34. SCHMM • Diagonal Covariance matrix case : • Then and

  35. SCHMM • Diagonal Covariance matrix case :

  36. SCHMM Diagonal Covariance • Diagonal Covariance matrix case :

  37. SCHMM Diagonal Covariance • Diagonal Covariance matrix case :

  38. SCHMM • Diagonal Covariance matrix case :

  39. SCHMM • Diagonal Covariance matrix case : (1) (2) (3)

  40. SCHMM Diagonal Covariance • Diagonal Covariance matrix case :

  41. SCHMM Diagonal Covariance • The initial estimate can be chosen as the mode of the prior PDF • And also can be chosen as the mean of the prior PDF

  42. Segmental MAP Estimates

  43. Segmental MAP Estimates DHMM

  44. Segmental MAP Estimates SCHMM

  45. Conclusion • The important issue of prior density is discussed. • Some application : • Model adaptation, HMM training, IR(?)

  46. Appendix-Matrix Calculus(1) • Notation:

  47. Appendix-Matrix Calculus(2) • Properties 1: • proof • Properties 1— Extension: • proof

  48. Appendix-Matrix Calculus(3) • Properties 2: • proof

  49. Appendix-Matrix Calculus(4) • Properties 3: • proof

  50. Appendix-Matrix Calculus(5) • Properties 4: • proof

More Related