CS 154, Lecture 4: Limitations on DFAs (I), Pumping Lemma, Minimizing DFAs

1. CS 154, Lecture 4:Limitations on DFAs (I), Pumping Lemma, Minimizing DFAs

2. Non-Regular Languages Regular or Not? D = { w | w has equal number of occurrences of 01 and 10 } REGULAR! C = { w | w has equal number of 1s and 0s} NOT REGULAR! • How can we prove that there is no DFA for a particular language? • Surprising Algorithms (even in restricted models) are routinely being discovered

3. The Pumping Lemma: Structure in Regular Languages Let L be a regular language Then there is a positive integer P s.t. for all strings w  L with |w| ≥ P there is a way to write w = xyz, where: • 1. |y| > 0 (that is, y ε) • |xy| ≤ P • For alli ≥ 0, xyiz L Why is it called the pumping lemma? The word w gets pumped into longer and longer strings…

4. q|w| Proof: Let M be a DFA that recognizes L Let P be the number of states in M Let w be a string where w  L and |w| ≥ P |y| > 0 |xy| ≤ P xyiz L for alli ≥ 0 We show: w = xyz y x w z … q0 qj qk Claim: There must existj and k such that 0 ≤ j < k ≤ P, and qj = qk

5. Generalized Pumping Lemma: Let L be a regular language Then there is a positive integer P s.t. for all strings awb L with |w| ≥ P there is a way to write w = xyz, where: • 1. |y| > 0 (that is, y ε) • |xy| ≤ P • For alli ≥ 0, axyizb L

6. Applying the Pumping Lemma Let’s prove that EQ={ w | #1s = #0s} is not regular. By contradiction. Assume EQ is regular. Let P be as in pumping lemma. Let w = 0P1P EQ.  Can write w = xyz, with |y| > 0, |xy| ≤ P, such that for all i≥ 0, xyiz is also in EQ Claim: The string y must be all zeroes. Why? Because |xy| ≤ P and w = xyz = 0P1P But then xyyz has more 0s than 1s Contradiction!

7. Applying the Pumping Lemma Prove: SQ = {0n2 | n ≥ 0} is not regular Assume SQ is regular. Let w = 0P2  Can write w = xyz, with |y| > 0, |xy| ≤ P, such that for all i≥ 0, xyiz is also in EQ So xyyz∊ SQ. Note that xyyz = 0P2+|y| Note that 0 < |y| ≤ P So |xyyz| = P2 + |y| ≤ P2 + P < P2 + 2P + 1 = (P+1)2 and P2 < |xyyz| < (P+1)2 Therefore |xyyz| is not a perfect square! Hence 0P2+|y| = xyyz∉ SQ, so our assumption must be false. SQ is not regular!

8. Does this DFA have a minimal number of states?

10. Is this minimal? How can we tell in general?

11. Theorem: For every regular language L, there is a unique (up to re-labeling of the states) minimal-state DFA M* such that L = L(M*). Furthermore, there is an efficient algorithmwhich, given any DFA M, will output this unique M*. If these were true for more general models of computation, that would be an engineering breakthrough

12. Note: There isn’t a uniquely minimal NFA

13. Extending transition function  to strings Given DFA M = (Q, Σ, , q0, F), we extend  to a function  : Q Σ* → Q as follows: (q, ε) = q (q, ) (q, ) = (q, 1 …k+1) = ( (q, 1 …k), k+1 ) (q, w) = the state of M reached after reading in w, starting from state q Note: (q0, w)  F  M accepts w Def. w  Σ* distinguishes states q1 and q2 if exactly one of (q1, w), (q2, w) is a final state Def. w  Σ* distinguishes states q1 and q2 if (q1, w)  F  (q2, w)  F

14. Distinguishing two states Def. w  Σ* distinguishes states q1 and q2 if exactly one of (q1, w), (q2, w) is a final state I’m in q1 or q2, but which? How can I tell? Ok, I’m accepting! Must have been q1 Here… read this W

15. State p is distinguishable from state q if there is w  Σ* that distinguishes p and q ( there is w  Σ* so that exactly one of (p, w), (q, w) is a final state) Fix M = (Q, Σ, , q0, F) and let p, q  Q Definitions: State p is indistinguishable from state q if p is not distinguishable from q ( for all w  Σ*, (p, w) F (q, w)  F) Pairs of indistinguishable states are redundant…

16. Which pairs of states are distinguishable here? ε distinguishes all final statesfrom non-final states

17. Which pairs of states are distinguishable here? The string 10 distinguishes q0 and q3

18. Which pairs of states are distinguishable here? The string 0 distinguishes q1 and q2

19. Fix M = (Q, Σ, , q0, F) and let p, q, r  Q Define a binary relation ∼ on the states of M: p ∼ q iff p is indistinguishable from q p≁q iff p is distinguishable from q Proposition: ∼ is an equivalence relation p∼p (reflexive) p ∼ q  q ∼ p (symmetric) p ∼ q and q ∼ r  p ∼ r (transitive) Proof?

20. Fix M = (Q, Σ, , q0, F) and let p, q, r  Q Therefore, the relation ∼ partitions Q into disjoint equivalence classes Proposition: ∼ is an equivalence relation [q] := { p | p ∼ q }

22. Algorithm: MINIMIZE-DFA Input: DFA M Output: DFA MMIN such that: L(M) = L(MMIN) MMIN has no inaccessible states MMIN is irreducible || For all states p  q of MMIN, p and q are distinguishable Theorem: MMIN is the unique minimal DFA that is equivalent to M

23. Intuition: The states of MMIN will be the equivalence classes of states of M We’ll uncover these equivalent states with a dynamic programming algorithm

24. The Table-Filling Algorithm Input: DFA M = (Q, Σ, , q0, F) (1) DM = { (p, q) | p, q  Q and p ≁ q } Output: (2) EQUIVM = { [q] | q  Q } • We know how to find those pairs of states that the string ε distinguishes… • Use this and iteration to find those pairs distinguishable with longer strings • The pairs of states left over will be indistinguishable High-Level Idea:

25. The Table-Filling Algorithm Input: DFA M = (Q, Σ, , q0, F) (1) DM = { (p, q) | p, q  Q and p ≁ q } Output: (2) EQUIVM = { [q] | q  Q } Base Case: For all (p, q) such that p accepts and q rejects  p ≁ q

26. The Table-Filling Algorithm Input: DFA M = (Q, Σ, , q0, F) (1) DM = { (p, q) | p, q  Q and p ≁ q } Output: (2) EQUIVM = { [q] | q  Q } Base Case: For all (p, q) such that p accepts and q rejects  p ≁ q Iterate: If there are states p, q and symbol σΣsatisfying: D D  (p, ) = p mark p ≁ q  ≁ q  (q, ) = D Repeat until no more D’s can be added

27. D D D D D D

28. D D D D

29. Claim: If (p, q) is marked D by the Table-Filling algorithm, then p ≁ q Proof: By induction on the number of steps in the algorithm before (p,q) is marked D If (p, q) is marked D at the start, then one state’s in F and the other isn’t, so εdistinguishes p and q Suppose (p, q) is marked D at a later point. Then there are states p, qsuch that: 1. (p, q) are marked D  p≁ q(by induction) So there’s a string w s.t.(p, w)  F (q, w)  F 2. p = (p,) and q = (q,), where   Σ The string w distinguishes p and q!

30. Claim: If (p, q) is not marked D by the Table-Filling algorithm, then p ∼ q Proof (by contradiction): Suppose the pair (p, q) is not marked D by the algorithm, yet p ≁ q (call this a “bad pair”) Then there is a string w such that |w| > 0 and: (p, w)  F and (q, w)  F (Why is |w| > 0?) We have w = w, for some string w’ and some   Σ Of all such bad pairs, let p, q be a pair with the shortest distinguishing string w Let p = (p,) and q = (q,) Then (p, q) is also a bad pair, but with a SHORTER distinguishing string, w !

31. Algorithm MINIMIZE Input: DFA M Output: Equivalent minimal-state DFA MMIN 1. Remove all inaccessible states from M 2. Run Table-Filling algorithm on M to get: EQUIVM = { [q] | q is an accessible state of M } 3. Define: MMIN = (QMIN, Σ, MIN, q0 MIN, FMIN) QMIN = EQUIVM, q0 MIN = [q0], FMIN = { [q] | q  F } MIN( [q],  ) = [ ( q,  ) ] Claim: L(MMIN) = L(M)

32. D D D D D D D D D

33. D D D D D D D D D

34. Thm: MMIN is the unique minimal DFA equivalent to M Claim: If L(M)=L(MMIN) and M has no inaccessible states and M’ is irreducible  there is an isomorphism between M and MMIN • Suppose for now the Claim is true. • If M’is a minimal DFA, then M’ has no inaccessible states and is irreducible (why?) So the Claim implies: • Let M’ be a minimal DFA for M. •  there is an isomorphism between M’ and the DFA MMIN that is output by MINIMIZE(M). •  The Thm holds!

35.   Thm: MMIN is the unique minimal DFA equivalent to M Claim: If L(M)=L(MMIN) and M has no inaccessible states and M’ is irreducible  there is an isomorphism between M and MMIN Proof: We recursively construct a map from the states of MMINto the states of M Base Case: q0 MIN q0 Recursive Step: If p  p Then q  q q q

36.   Base Case:q0 MIN q0 Recursive Step: Ifp  p Thenq  q q q

37.   We need to prove: The map is defined everywhere Base Case:q0 MIN q0 The map is well defined Recursive Step: Ifp  p The map is a bijection Thenq  q The map preserves all transitions:If p  p then MIN(p, )  ’(p’, ) (this follows from the definition of the map!) q q

38.   The map is defined everywhere Base Case:q0 MIN q0 That is, for all states q ofMMIN there is a state q of M such that q  q Recursive Step: Ifp  p If q  MMIN, there is a string w such that MIN(q0 MIN,w) = q Thenq  q q q Let q = (q0,w). Then q  q

39.   The map is well defined Base Case:q0 MIN q0 Proof by contradiction. Suppose there are states q and q such that q  q and q  q Recursive Step: Ifp  p Thenq  q We show that q and q are indistinguishable, so it must be that q = q q q

40. Suppose there are states q and q such that q  q and q  q Suppose q and q are distinguishable MMIN M u u w w Accept Accept q q q0 q0 MIN Contradiction! v w v w Reject Reject q q q0 q0 MIN

41.   The map is onto Want to show: For all states qof M there is a state q of MMIN such that q  q Base Case:q0 MIN q0 For every q there is a string w such thatM’ reaches state q’ after reading in w Recursive Step: Ifp  p Thenq  q Let q be the state of MMIN after reading in wClaim: q  q q q

42. The map is one-to-one Proof by contradiction. Suppose there are states p q such that p  q and q  q If p  q, then p and q are distinguishable MMIN M u u w w Accept Accept q p q0 q0 MIN Contradiction! v w v w Reject q Reject q q0 q0 MIN

43. How can we prove that two regular expressions are equivalent?

44. Parting thoughts: Pumping for contradictions DFAs can’t count DFAs can be optimized Later: Can DFAs be learned? Questions?