CS 154, Lecture 2: Finite Automata, Closure Properties Nondeterminism,

1. CS 154, Lecture 2:Finite Automata, Closure PropertiesNondeterminism,

2. Why so Many Models?

3. Streaming Algorithms 0 0 1 42

4. Deterministic Finite Automata

5. 1 0 0,1 accept states (F)‏ states 1 0 0 1 states Anatomy of Deterministic Finite Automata transition: for every state and alphabet symbol q1 q0 q2 start state (q0)‏ q3

6. 11 1 0 0,1 1 0111 111 1 0 0 The DFA reads its string from left to right 1 The automaton accepts the string if this process ends in a double-circle (accept state). Otherwise, the automaton rejects the string.

7. What and Why DFA Simple: Constant-size memory & read-once input Both as important? Foundational or archaic object? So “close” to a Turing Machine and yet so far: nondeterminism (verified guessing) is “weak”, optimizing (and some settings of learning) is easy Allows us to talk about – useful vocabulary, nondeterminism, (learning), limitations, streaming, communication complexity

8. IBM JOURNAL APRIL 1959 Turing Award winningpaper

10. Some Vocabulary An alphabetΣ is a finite set (e.g., Σ = {0,1})‏ A stringover Σ is a finite length sequence of elements of Σ Σ* = the set of all strings over Σ For string x, |x| is the length of x The unique string of length 0 is denoted by ε and is called the empty string

11. Languages A language over Σ is a set of strings over Σ In other words: a language is a subset of Σ* Languages = Boolean functions over strings Every language L overΣcorresponds to a unique function f : Σ* {0,1}: Definefsuch that f(x) = 1 if x  L = 0 otherwise

12. Definition. A DFA is a 5-tuple M = (Q, Σ, , q0, F) Q is the set of states (finite) Σ is the alphabet (finite)‏  : Q Σ → Q is the transition function q0 Q is the start state F  Q is the set of accept/final states‏

13. Definition. A DFA is a 5-tuple M = (Q, Σ, , q0, F) • Let w1, ... , wnΣ and w = w1... wnΣ* • M acceptsw if there are r0, r1, ..., rnQ, s.t. • r0 = q0 • (ri, wi+1) = ri+1for all i = 0, ..., n-1, and • rnF

14. M = (Q, Σ, , q0, F) where Q = {q0, q1, q2, q3} Σ = {0,1}  : Q Σ→ Qtransition function* q0 Q is start state F = {q1, q2}  Q accept states *

15. A DFA is a 5-tuple M = (Q, Σ, , q0, F) L(M) = set of all strings that M accepts = “the language recognized by M” = the function computed by M

16. 0,1 M 1 010 0 10 1010 0 0,1 { w | w begins with 1} L(M) =

17. 0,1 q0 {0,1}* L(M) = 

18. 0 0 1 q0 q1 1 { w | w has an even number of 1s} L(M) =

19. 0 0 Theorem: L(M) = {w | w has odd number of 1s } 1 q p 1 Proof: By induction on n, the length of a string. Base Case n=0: ε  L and ε  L(M) Induction Step Suppose for all w Σ*, |w| = n, M accepts w  w has odd number of 1s A string of length n+1 has the form w0 or w1, |w|=n, continue with a case analysis …

20. 0,1 1 0 0 0 1 q q0 q00 q001 1 Build a DFA that accepts exactly the strings containing 001

21. Definition: A language L’ is regular if L’ is recognized by a DFA; that is, there is a DFA M where L’ = L(M). L’ = { w | w contains 001} is regular L’ = { w | w has an even number of 1s} is regular

22. Closure Properties for Regular Languages Union: A  B = { w | w  A or w  B } Intersection: A  B = { w | w  A and w  B } Complement:A = { w  Σ* | w  A } Reverse: AR = { w1 …wk | wk …w1 A, wi Σ} Concatenation: A  B = { vw | v  A and w  B } Star: A* = { s1 … sk | k ≥ 0 and each si A } Theorem: if A and B are regular then so are: A  B, A  B, A, AR, A  B, and A*

23. Union Theorem for Regular Languages Given two languages L1 and L2 recall that the union of L1 and L2 is L1 L2 = { w | w  L1 or w  L2 } Theorem: The union of two regular languages is also a regular language

24. Theorem: The union of two regular languages is also a regular language Proof: Let M1 = (Q1, Σ, 1, q0, F1)be a finite automaton for L1 and M2 = (Q2, Σ, 2, q0, F2)be a finite automaton for L2 1 2 We want to construct a finite automaton M = (Q, Σ, , q0, F)that recognizes L = L1 L2

25. = pairs of states, one from M1 and one from M2 Q = { (q1, q2) | q1 Q1 and q2 Q2 } = Q1 Q2 1 2 q0 = (q0, q0)‏ Proof Idea: Run both M1 and M2 “in parallel”! 1 M1 = (Q1, Σ, 1, q0, F1)recognizes L1 and M2 = (Q2, Σ, 2, q0, F2) recognizes L2 2 F = { (q1, q2) | q1 F1OR q2 F2 } ( (q1,q2), ) = (1(q1, ), 2(q2, ))

26. 0 0 1 q0 q1 1 1 1 0 p0 p1 0 Theorem: The union of two regular languages is also a regular language

27. 1 q0,p0 q1,p0 1 0 0 0 0 1 q0,p1 q1,p1 1 What about the INTERSECTIONof two languages?

28. = pairs of states, one from M1 and one from M2 Q = { (q1, q2) | q1 Q1 and q2 Q2 } = Q1 Q2 1 2 q0 = (q0, q0)‏ Proof Idea: Run both M1 and M2 “in parallel”! 1 M1 = (Q1, Σ, 1, q0, F1)recognizes L1 and M2 = (Q2, Σ, 2, q0, F2) recognizes L2 2 F = { (q1, q2) | q1 F1AND q2 F2 } ( (q1,q2), ) = (1(q1, ), 2(q2, ))

29. Union Theorem for Regular Languages: The union of two regular languages is also a regular language “Regular Languages Are Closed Under Union” Intersection Theorem for Regular Languages: The intersection of two regular languages is also a regular language “Regular Languages Are Closed Under Intersextion”

30. Complement Theorem for Regular Languages The complement of a regular language is also a regular language: If L is regular than so is L= { w  Σ* | w  L } Proof ?

31. Closure Properties for Regular Languages Union: A  B = { w | w  A or w  B }  Intersection: A  B = { w | w  A and w  B }  Complement:A = { w  Σ* | w  A }  Reverse: AR = { w1 …wk | wk …w1 A, wi Σ} Concatenation: A  B = { vw | v  A and w  B } Star: A* = { s1 … sk | k ≥ 0 and each si A } Theorem: if A and B are regular then so are: A  B, A  B, A, AR, A  B, and A*

32. The Reverse of a Language Reverse of L:LR = { w1 …wk | wk …w1 L, wi Σ} Theorem: If L is regular, then is LR also regular If L is recognized by the usual kind of DFA, Then LR is recognized by a DFA that reads its strings from right to left How can every “Right-to-Left” DFA be replaced by a normal “Left-to-Right” DFA?

33. Reversing DFAs Assume L is a regular language. Let M be a DFA that recognizes L We want to build a machine MR that accepts LR If M accepts w, then w describes a directed path in M from the start state to an accept state First Attempt: Try to define MR as M with the arrows reversed, turn start state into a final state, turn final states into starts

34. Problem: MR is not always a DFA! It could have many start states Some states may have more than one outgoing edge, or none at all!

35. 0,1 1 0 0 0 1 1

36. Non-deterministic Finite Automata (NFA) 0,1 1 0 0 0 1 1 What happens with 100? We will say this new machine accepts a string if there is some path that reaches some accept state from some start state

37. Another Example of an NFA At each state, we can have any number of out arrows for some letter  Σ, including ε Set of strings accepted by this NFA = {w | w contains a 0}

38. Multiple Start States We allow multiple start states for NFAs, and Sipser allows only one Can easily convert NFA with many start states into one with a single start state: ε ε ε

39. Yet Another Example of an NFA L(M) = {1,00}

40. A non-deterministic finite automaton (NFA) is a 5-tupleN = (Q, Σ, , Q0, F) where Q is the set of states Σ is the alphabet  : Q Σε! 2Qis the transition function Q0 Q is the set of start states F  Q is the set of accept states 2Q is the set of all possible subsets of Q Σε = Σ {ε}

41. Def. Let w Σ*. Let N be an NFA. N accepts w if there’s a sequence of states r0, r1, ..., rk Q and wcan be written as w1... wk with wi Σ {ε}s.t. • r0  Q0 • ri+1 (ri, wi+1 ) for all i = 0, ..., k-1, and • rn F L(N) = the language recognized by N = set of all strings machine N accepts Language L’ is recognized by an NFA N if L’ = L(N).

42. N = (Q, Σ, , Q0, F) Q = {q1, q2, q3, q4} Σ = {0,1} Q0 = {q1, q2} F = {q4}  Q {q4} (q2,1) = (q3,1) =  00 L(N)? 01 L(N)? {q3} (q1,0) =

43. NFAs are generally simpler than DFAs A DFA recognizing the language {1} An NFA recognizing the language {1}

44. Deterministic Computation Non-Deterministic Computation reject accept accept or reject Are these equally powerful???

45. Parting thoughts: Regular Languages can be manipulated When life hands you ambiguity define Nondeterministic Finite Automata Is verifying easier than computing (I)? Questions?