CSCI 4325 / 6339 Theory of Computation

1. CSCI 4325 / 6339Theory of Computation Zhixiang Chen

2. Chapter OneFinite Automata

3. Outline • Finite Automata • Deterministic vs Nondeterministic finite automata, and their equivalence • Finite Automata vs. regular expressions, and their equivalence • Pumping lemma and non-regular languages

4. What Is a Finite Automaton? Input tape a b a b a a b b a … • Input Tape • Reading tape • States, initial state, final states • Moves • Computation • Accepting an input string • Rejecting an input string Reading head q0 q5 q1 q4 q2 q3 Finite Control

5. Formal Definition • A deterministic finite automaton is a quintuple M = (K, Σ, δ, s, F) where • K is a finite set of states; • Σ is a finite alphabet, • s  K is the initial state, • F  K is the set of final states, • δ is the transition function from K x Σ  K • Explain K, Σ, δ, s, F. • The hardest one is δ. • What does δ(p, a) = q mean?

6. Configurations • How to describe moves involved in a computation process? • Configurations • Formal definition of configuration is very simple • A configuration is any element of K x Σ*. • Given a configuration (q, w), qK, wΣ*, • What is its physical meaning? • For example, (q, aabbab)?

7. Yield Relation • The yield relation • We must understand that the yield relation describes exactly one step of a computation processing.

8. Because the yield relation is a relation, we can find a unique reflexive and transitive closure of . Denote the closure as • Now, Given any two configurations (p, w) and (p’, w’), • If (p,w) (p’,w’) • then we say that (p,w) yields (p’,w’) • stop! Think about the physical meaning of (p, w) yielding (p’, w’)!

9. The Language Accepted by a DFA • The language accepted by an DFA M is • L(M) = { w: w  Σ*, (s, w) (q, e) for some q  F } • Explain the definition.

10. Ex’s of DFA’s • M = (K, Σ, δ, s, F) • δ is given in the following table • Show computation form • What is L(M)?

11. a b a b qo q1 State Diagram • State Diagram of a DFA M= (K, Σ, δ, s, F): • A directed graph: • for each state in K, draw a node • for each final state in F, draw a double circle. • Mark the initial node of the initial state >. • If δ(p, a) = q, then draw an edge from p to q and label it with a. • Ex. • The state diagram representation of a DFA is simple and intuitive.

12. Design DFAs to Accept • L1 = {w: w  {a,b}*, w has no three consecutive b’s} • L2 = {w: w  {a,b}*, w has abab as a substring} • L3 = {w: w  {a,b}*, w has no aa nor bb}

13. L1

14. a b a b b a,b q0 q2 q3 q4 q1 a a b L2 a a

15. a b a q2 q3 q1 b b q4 a q5 L3

16. Nondeterministic Finite Automata (NFA) • Motivation of nondeterminism • Play chess, factor a number • The key point is at each step, there are a fixed number of possible choices for the next. Some choices are good, some are bad. • When we try to study the power of nondeterminism, we assume that a machine knows how to make a good choice.

17. a q0 a b a a q0 q1 q2 q3 q1 b e a b a q2 b q0 q1 a b NFA Examples • Ex. Three NFAs accepting the same language defined by the regular expression • (ab v aba)*

18. Definition of NFA • An NFA M = (K, ∑, ∆ , s, F) where, • K is a finite set of states • ∑ is a finite alphabet • s is the initial state • F  K is the set of finite states • ∆ is the transition relation from • K x (∑ ⋃ {e})  K • Understand ∆ ! • ∆ (p, a) may have more than one value • M is nondeterministic, because ∆ may have multiple values for a given pair of (p, a)

19. Configurations of NFAs • Configuration • Pairs in K X ∑* • Meaning of a configuration? • Yield Relation • (q, w) (q’, w’) • if w = aw’ for a Є ∑ ⋃{e} • and ∆ (q, a) = q’

20. The Reflexive and Transitive Closure • Because the yield relation is a relation, we can find a unique reflexive and transitive closure of . Denote the closure as • Now, Given any two configurations (p, w) and (p’, w’), • If (p,w) (p’,w’) • then we say that (p,w) yields (p’,w’) • stop! Think about the physical meaning of (p, w) yielding (p’, w’)!

21. The Language Accepted by an NFA • The language accepted by an NFA M is • L(M) = { w: w*, (s, w) (q, e) for some qF }

22. a, b b b a e b a, b NFA Example

23. Design an NFA to accept • L = {w: not all symbols in  occured in w }, • where  = {a, b, c, d } a,b,c e a,b,d e e b,c,d e a,c,d

24. Theorem. For each NFA, there is an equivalent DFA. • Remark. Because each DFA has an equivalent NFA, the above result tells us that the power of NFA is the same as DFA.

25. Proof of the above Theorem • Proof by Construction: Construct a DFA to simulate the given NDF • Let the NFD M = ( k, , , s, F ) • Wanted a DFA M’ = ( K’, , , s’, F’ ) • Such that L( M ) = L( M’ ) • Construction of M’ •  =  • K’ = 2K • s’ = { qK: (s, e) |---* (q, e ) } • F’ = { A K’: A  F  } • For A  K, a • ( A, a ) = { qK: ( p, a ) |---* ( q, e ) for some p A }

26. Claim 1. M’ is deterministic by definition of  • For any qK, define E(q) = { pK: ( q, e ) |---* ( p, e )} • Claim 2. For any string w*, p, qK ( q, w ) |---* ( p, e )  ( E(q), w ) |---* ( P, e ) for some PK’ such that pP. • Proof By induction on |w| • Basic step . For |w| = 0, w = e, we need to show: (q, e) |---* (p, e)  ( E(q), e )|---* (P, e) for some PK’ such that pP

27. Induction Hypothesis • Suppose the claim is true for all string w such that |w|  k • Induction Step • We prove the claim is true for all string w such that |w| = k+ 1. Let w = ua for some a , u  * First prove: (q, w) |---* (p, e)  ( E(q), w ) |---* (P, e) for some P with pP • (q, w) |---* (p, e)  r1, r2 such that (q, w) |---* (r1, a) |---(r2, e) |---(p, e) • apply induction hypothesis to (q, w) |---* (r1, a) Second we prove (E(q), w) |---*( P, e)  (q, w) |---*(p, e) for some P with pP Again, let w = ua for some a, u* • (E(q), w) |---* (P, e)  (E(q), w) |---* (R1, a) |--- ( R2, e) |---* (P, e) • Apply induction hypothesis to (E(q), w) |---* (P, e)  (E(q), w) |---* (R1, a)

28. Claim3: L(M) = L(M’) • By claim 2, for any w*, (s, w) |---* (p, e), for pF  (E(s), w) |---*, (P, e), for some P with pP • Note E(s) = s’, PF’

29. e e a a e b b Equivalent DFA of NFA a e • NFA • DFA a a b b a a,b

30. FA and Regular Expressions • Theorem 1. The class of languages accepted by FA is closed under • (a) Union • (b) Concatenation • (c) Kleene star • (d) Complementation • (e) Intersection

31. Proof of Theorem 1 • Proof. • (a) Union: • Given • Need to prove that there is a FA such that • Idea for proving (a): by construction • Ideas for proving (b), (c) and (d): also by construction • Idea for proving (e):

32. Theorem 2. A language is regular if and only if it is accepted by a FA. • Proof. • (The if-part) If a language is regular then it is accepted by a FA • Recall that the class of regular languages is the smallest class of languages containing the empty set Ø and the singleton a, where a is a symbol, and closed under union, concatenation, and Kleene star. It is evident the empty set and all singletons are indeed accepted by FA; and by closure property of FA, every regular language is accepted by some FA.

33. (The only-if-part) If a language is accepted by a FA, then it is regular. • Idea: Starting from the directed graph representation, find all paths from the initial state to the final states. For each path write a regular expression. Use Kleene star to deal with loops. • Let be a FA. We shall construct a regular expression R such that . We shall represent L(M) as the union of many (but a finite number of) simple languages. • Let . For i, j =1,…,n and k=0,…,n, we define R(i,j,k) as the set of all strings in that may drive M from state to state without passing through any intermediate state numbered k+1 or greater.

34. Note. • By Math Induction, R(i,j,k) is regular for k=0,1,…,n. • Hence, L(M) is regular.

35. The Pumping Theorem • Theorem. Let be an infinite regular language. There is an integer such that any string with can be rewritten as such that and for each . • Proof. Follows from the graph representation.

36. Irregular Languages w has equal number of a’s and b’s} • The first two can be proved with the Pumping Theorem • The third can be proved by contradiction and closure property of regular languages:  If L3 is regular, so is L1. But L1 is not

37. Algorithms for FA • Theorem. • (a) There is an exponential time algorithm which, given an NFA, constructs an equivalent DFA. • (b) There is a polynomial time algorithm which, given a regular expression, construct an equivalent NFA • (c) There is an exponential time algorithm which, given an NFA, constructs an equivalent regular expression

38. (d) There is a polynomial time algorithm which, given two DFAs, decides whether they are equivalent • (e) There is an exponential time algorithm which, given two NFAs, decided whether they are equivalent; similarly for equivalence of two regular expressions. • For (d) and (e), note the following

39. More Algorithms • Theorem. If L is a regular language, then there is an algorithm which, given tests whether it is in L in time. • Proof. Construct a DFA to accept L.

40. More Algorithms • Theorem. Let be an NFA. then there is an algorithm which, given tests whether it is in L(M) in time. • Proof. Simulate the accepting process of M: • Step 0: Find E(s), which is s plus all states that are reachable with e-moves • Step 1: Start at E(s), find all states reachable passing the first letter of w plus their e-reachable states. • … • Step |w|: find the current state set as in Step 1 and decide whether a final state is in the current state set. • Estimate the time of the above process.

41. Conclusions • NFA have the same computing power as DFA • NFA have the same computing power as regular expressions • Pumping Theorem vs. irregular languages • Algorithms for • Constructing an equivalent NFA for a regular exp. • Constructing an equivalent regular exp. for an NFA • Constructing an equivalent DFA for an NFA • Deciding whether two FA are equivalent or not • Deciding whether a string is accepted by a DFA (or NFA) or not