Hidden Markov models

1. Hidden Markov models Sushmita Roy BMI/CS 576 www.biostat.wisc.edu/bmi576/ sroy@biostat.wisc.edu Oct 24th, 2013

2. Key concepts • What are Hidden Markov models (HMMs)? • States, transitions, emissions • Key algorithms to solve these problems in HMMs • Probability of a sequence: Forward algorithm • Most likely set of states: Viterbi • Parameter estimation: Baum-Welch • Formulating HMMs for simple problems

3. Revisiting the CpG question • Given a sequence we can use a Markov chain model of a CpG island that tells whether it is a CpG island or not. • What we do if we were asked to “find” the CpG islands in the genome?

4. A+ A- G+ G- C+ C- T+ T- An HMM for identifying CpG islands Background CpG Island Note, there is no more one-to-one correspondence between states and observed symbols

5. What does an HMM do? • Enables us to describe observed sequences that are generated by an underlying hidden system • We assume the system moves between states and at each state it generates a symbol • Note there are two sequences • Sequence of states: path • Sequence of observations

6. Describing a Hidden Markov model • States • State Transition probabilities • Define the probability of transitioning from state at time t to state at time t+1 • Emission probabilities • Defines the probability of emitting symbols from a state • Not all states need to have emission probabilities • E.g. the begin and end state

7. Notation • Observed sequence • Hidden state sequence • Transition probabilities: Probability of transitioning between states • We will use a0k to denote transitions from the begin state • Emission probabilities: Probability of emitting a symbol b from a state k

8. 0.4 0.2 A 0.4 C 0.1 G 0.2 T 0.3 A 0.2 C 0.3 G 0.3 T 0.2 0.8 begin end 0.6 0.5 1 3 0 5 A 0.4 C 0.1 G 0.1 T 0.4 A 0.1 C 0.4 G 0.4 T 0.1 0.5 0.9 0.2 2 4 0.1 An example HMM probability of a transition from state 1 to state 3 probability of emitting character A in state 2 0.8

9. An HMM for an occasionally dishonest casino 0.95 0.9 1/6 1/6 1/6 1/6 1/6 1/6 1/10 1/10 1/10 1/10 1/10 1/2 0.05 0.1 1 2 Fair Loaded What is hidden? What is observed?

10. Three important questions in HMMs • Given a sequence of observations how likely is an HMM to have generated it? • Forward algorithm • What is the most likely sequence of states that has generated a sequence of observations • Viterbi • How can we learn an HMM from a set of sequences? • Forward-backward or Baum-Welch (an EM algorithm)

11. Probability of a sequence of states and observations State transition between consecutive time points Emitting symbol xt Initial transition

12. begin end Probability of a sequence of states and observations 0.4 0.2 A 0.4 C 0.1 G 0.2 T 0.3 A 0.2 C 0.3 G 0.3 T 0.2 0.8 0.6 0.5 1 3 0 5 A 0.4 C 0.1 G 0.1 T 0.4 A 0.1 C 0.4 G 0.4 T 0.1 0.5 0.9 0.2 2 4 0.1 0.8

13. What is the Probability of a sequence of observations? • It is however not possible to observe the state sequence • What we need to do is sum over all possible state sequences of length T • T is the length of the observation • For a sequence of length T, how many possible paths are there? • Forward algorithm enables us to compute this efficiently

14. Forward algorithm • Based on dynamic programming • Efficiently computes the probability of a sequence by summing over all state sequences • Probability of observing x1..xtand state at t is k • This can be written recursively as follows

15. Steps of the Forward algorithm • Initialization: denote 0 for the “begin” state • Recursion: for t=1 to T • Termination End state

16. 0.4 0.2 A 0.4 C 0.1 G 0.2 T 0.3 A 0.2 C 0.3 G 0.3 T 0.2 0.8 0.6 0.5 1 3 begin end 0 5 A 0.4 C 0.1 G 0.1 T 0.4 A 0.1 C 0.4 G 0.4 T 0.1 0.5 0.9 0.2 2 4 0.1 0.8 Forward algorithm example What is the probability of sequence TAGA ?

17. Table for TAGA t=1 t=2 t=3 t=4 T A G A 0 1 2 States 3 4 5 This entry is also P(x). Does not require f1(4) and f2(4)

18. Three important questions in HMMs • Given a sequence of observations how likely is it for an HMM to have generated it? • Forward algorithm • What is the most likely sequence of states that has generated a sequence of observations • Viterbi • How can we learn an HMM from a set of sequences? • Forward-backward or Baum-Welch (an EM algorithm)

19. Viterbi algorithm • Viterbi algorithm gives an efficient way to find the most likely/probable state • Also based on Dynamic programming • Consider the dishonest casino example • Given a sequence of die rolls can you infer when the casino was using the loaded versus fair die? • Viterbi algorithm gives this answer • We have seen a similar idea in another problem, do you remember?

20. Notation for Viterbi • Very similar to forward algorithm notation • The probability of the most likely state sequence path for x1..xtand being in state k at time t • What we want is vN(T), N is the end state

21. Steps of the Viterbi algorithm • Initialization: • Recursion: • Termination:

22. Traceback in Viterbi • Traceback for t=T to 1

23. 0.4 0.2 A 0.4 C 0.1 G 0.2 T 0.3 A 0.2 C 0.3 G 0.3 T 0.2 0.8 0.6 0.5 1 3 begin end 0 5 A 0.4 C 0.1 G 0.1 T 0.4 A 0.1 C 0.4 G 0.4 T 0.1 0.5 0.9 0.2 2 4 0.1 0.8 Viterbi algorithm example What is the most likely path for TAG ?