CSC 3130: Automata theory and formal languages

1. Fall 2008 The Chinese University of Hong Kong CSC 3130: Automata theory and formal languages Undecidable Languages Andrej Bogdanov http://www.cse.cuhk.edu.hk/~andrejb/csc3130

2. Are Turing Machines computers? computer input data output data instructions (program) A Turing Machine does not take the programas part of its input. ... but it can!

3. The Universal Turing Machine U input x M on input x program 〈M〉 The universal TMU takes as inputs a program Mand a string x and simulates M on x The program M itself is specified as a TM!

4. Turing Machines as strings • This Turing Machine can be described by the string M ☐/☐R qacc 0/0R q0 1/1R qrej 〈M〉 = ((q0, qacc, qrej), ((q0, q0, ☐/☐R), (q0, qacc, 0/0R), (q0, qrej, 1/1R)))

5. The universal Turing Machine U input x 001 program 〈M〉 ((q0, qacc, qrej), ((q0, q0, ☐/☐R), state q0 U on input (〈M〉, x) will simulateM on input x

6. Languages about automata • A TM can take another TM (or DFA, CFG) as input decidable ADFA = {(〈D〉, w): Dis a DFA that accepts input w} decidable APDA = {(〈P〉, w): Pis a PDA that accepts w} ATM = {(〈M〉, w): Mis a TM that accepts w} Which of these is decidable?

7. Languages about automata • Idea: Simulate M on input w • Simulation is correct: • If M accepts w, we accept • If M rejects w, we reject • If M loops on w, we loop • This TM recognizes but does not decideATM ATM = {(〈M〉, w): Mis a TM that accepts w}

8. Recognizing versus deciding • This kind of TM is called a decider • The textbook uses different terminology: The language recognized by a TM is the set ofall inputs that make it reach qacc A TM decides language Lif it recognizes L anddoes not loop on any input recognizable = recursively enumerable (r. e.) decidable = recursive

9. Undecidability • Turing’s Theorem: • Before we prove this, let’s observe one thing: The language ATMis undecidable. A Turing Machine M can be given its own description〈M〉 as an input!

10. Proof of undecidability • Proof by contradiction: Suppose ATM is decidable. Then there is a TM H that decides ATM: accept ifM accepts w 〈M〉, w H reject ifM rejects wor M loops on w accept ifM accepts 〈M〉 〈M〉, 〈M〉 What happens whenw = 〈M〉? reject ifM rejects 〈M〉or M loops on 〈M〉

11. Proof of undecidability accept ifM accepts 〈M〉 H 〈M〉, 〈M〉 reject ifM rejects 〈M〉or M loops on 〈M〉 Let H’ be a TM that does the opposite of H H’ acc acc H rej rej

12. Proof of undecidability accept ifM accepts 〈M〉 H 〈M〉, 〈M〉 reject ifM rejects 〈M〉or M loops on 〈M〉 ifM rejects 〈M〉 or M loops on 〈M〉 accept H’ 〈M〉, 〈M〉 reject ifM accepts 〈M〉

13. Proof of undecidability ifM rejects 〈M〉 or M loops on 〈M〉 accept H’ 〈M〉, 〈M〉 reject ifM accepts 〈M〉 Let D be the following TM: 〈M〉 H’ 〈M〉, 〈M〉 copy

14. Proof of undecidability ifM rejects 〈M〉 or M loops on 〈M〉 accept D 〈M〉 reject ifM accepts 〈M〉 What happens when M = D? If D accepts 〈D〉 then D rejects 〈D〉 ifD rejects 〈D〉 or D loops on 〈D〉 so D does not exist! 〈D〉 If D rejects 〈D〉 then D accepts 〈D〉 reject ifD accepts 〈D〉

15. Proof of undecidability: conclusion • Proof by contradiction • We assumed ATM was decidable • Then we had Turing Machines H, H’, D • But D does not exist! • Conclusion The language ATMis undecidable.

16. What happened? input w … 00 e 0 1 M1 acc rej rej acc M2 … rej acc loop rej Turing Machine M3 rej loop rej rej … … We can write an infinite table for every pair (M, w)

17. What happened? … 〈M4〉 〈M1〉 〈M2〉 〈M3〉 M1 acc loop rej acc M2 … rej acc rej acc M3 loop rej loop rej … … Now let’s look only at those wthat describe a TM M

18. What happened? … 〈M4〉 〈M1〉 〈M2〉 〈M3〉 M1 acc loop rej acc M2 … rej acc rej acc … … D rej rej acc rej … … If ATM is decidable, then TM D is in the table

19. What happened? … 〈M4〉 〈M1〉 〈M2〉 〈M3〉 M1 acc loop rej acc M2 … rej acc rej acc … … D rej rej acc rej D does the opposite of the diagonal entries 〈M〉 accept ifM rejects or loops on 〈M〉 D reject ifM accepts 〈M〉

20. What happened? … 〈M4〉 〈M1〉 〈M2〉 〈M3〉 〈D〉 M1 acc loop rej acc loop M2 … rej acc rej acc acc … … ? … D rej rej acc rej We run into trouble when we look at (D, 〈D〉)!

21. Unrecognizable languages • How about languages that are not recognizable? The language ATMis recognizable but not decidable. ATM = {(〈M〉, w): Mis a TM that does not accept w} = {(〈M〉, w): Mrejects or loops on input w} The language ATMis not recognizable.

22. Unrecognizable languages • General Theorem • We know ATM is recognizable, so if ATM were also,then ATM would be decidable • Impossible by Turing’s Theorem If L and L are both recognizable, then L isdecidable.

23. Unrecognizable languages • Proof idea If L and L are both recognizable, then L isdecidable. accept ifw ∈ L M w accept rej/loop ifw ∉ L w accept ifw ∉ L M’ w reject rej/loop ifw ∈ L

24. Unrecognizable languages • Proof attempt If L and L are both recognizable, then L isdecidable. Problem: If M loops on w, we will never get to step 2! Let M = TM for L, M’ = TM for L On input w, Simulate M on input w. If it accepts, accept. ① Simulate M’ on input w. If it accepts, reject. ②

25. Bounded simulation • Proof If L and L are both recognizable, then L isdecidable. Let M = TM for L, M’ = TM for L On input w, For k = 0 to infinity Simulate M on input w for ksteps. If it accepts, accept. Simulate M’ on input w for ksteps. If it accepts, reject.

26. Another undecidable language • To prove this, we will show that … so by Turing’s Theorem it is undecidable. HALTTM = {(〈M〉, w): Mis a TM that halts on input w} HALTTMis an undecidable language If HALTTMcan be decided, so can ATM.

27. Undecidability of halting • Suppose H decides HALTTM If HALTTMcan be decided, so can ATM. accept ifM halts on w H 〈M〉, w reject ifM loops on w • We want to use H to design D for ATM accept ifM accepts w ? 〈M〉, w reject ifM rejects or loops on w

28. Undecidability of halting accept ifM halts on w H 〈M〉, w reject ifM loops on w D H acc acc simulate M on w accept ifM accepts w 〈M〉, w rej rej reject ifM rejects or loops on w

29. More undecidable problems L1 = {〈M〉: Mis a TM that accepts input e} L2 = {〈M〉: Mis a TM that accepts some input} L3 = {〈M〉: Mis a TM that accepts all inputs} L4 = {〈M, M’〉: Mand M’ accept the same inputs} Which of these are recognizable?

30. Gödel’s incompleteness theorem • Proof due to Turing: Some mathematical statementsare true but not provable. If every true statement is provable, then ATM can be decided.

31. Proof sketch of Gödel’s theorem • Suppose every true statement has a proof • For every TM M and input w, one of the statementsis true, so it must have a proof M accepts w or M does not accept w

32. Proof sketch of Gödel’s theorem • Consider this algorithm for ATM: On input 〈M〉, w: Fork = 0 to infinity For all possible proofs p of length k If p is a proof of “M accepts w”, accept. If p is a proof of “M does not accept w”, reject. One of them is true, so it has a proof p Eventually, the algorithm will find this proof

33. Proof sketch of Godel’s theorem • Consider this algorithm for ATM: On input 〈M〉, w: Fork = 0 to infinity For all possible proofs p of length k If p is a proof of “M accepts w”, accept. If p is a proof of “M does not accept w”, reject. Recall that TMs can check if a proof p is correct