CSCI 3130: Formal languages and automata theory

1. Fall 2011 The Chinese University of Hong Kong CSCI 3130: Formal languages and automata theory NP and NP-completeness Andrej Bogdanov http://www.cse.cuhk.edu.hk/~andrejb/csc3130

2. Some more problems A clique is a subset of vertices that are all interconnected 2 1 {1, 4}, {2, 3, 4}, {1} are cliques An independent set is a subset of vertices so that no pair is connected 4 3 {1, 2}, {1, 3}, {4} are independent sets there is no independent set of size 3 Graph G A vertex cover is a set of vertices that touches (covers) all edges {2, 4}, {3, 4}, {1, 2, 3} are vertex covers

3. How to solve them CLIQUE = {〈G, k〉: G is a graph with a clique of k vertices} M: On input 〈G, k〉: For all subsets S of vertices of size k: If for every pair u, v in S (u, v) is an edge in G, accept Otherwise, reject. input: 〈G, 3〉 2 1 subsets: {123} {124} {134} {234} all edges in? NO NO YES NO 4 3

4. Running time analysis CLIQUE = {〈G, k〉: G is a graph with a clique of k vertices} M: On input 〈G, k〉: For all subsets S of vertices of size k: If for every pair u, v in S (u, v) is an edge in G, accept Otherwise, reject. ( ) ( ) n n subsets k k ≈ n2 pairs ≈ n2 ≈ 2n whenk = n/2

5. Status of these problems CLIQUE = {〈G, k〉: G is a graph with a clique of k vertices} IS = {〈G, k〉: G is a graph with an independent set of k vertices} VC = {〈G, k〉: G is a graph with a vertex cover of k vertices} problem CLIQUE VC IS running time of best-known algorithm ≈ 2n ≈ 2n ≈ 2n What do these problems have in common?

6. Checking solutions efficiently • We don’t know how to solve them efficiently • But if someone told us the solution, we would be able to verify it very quickly Example: 12 Is (G, 5) in CLIQUE? 9 1,5,9,12,14 1 14 5 13 6 15 2 7 4 10 8 3 11

7. The class NP • A verifier for L is a TM V such that • s is a potential solution for x • We say V runs in polynomial time if on every input x, it runs in time polynomial in |x| (for every s) x ∈ L V accepts 〈x, s〉 for some s NP is the class of all languages that have polynomial-time verifiers

8. Example CLIQUE is in NP: V := On input 〈G, k〉 and a set of vertices C, If C has size k and all edges between vertices of C are present in G, accept, otherwise reject. running time ≈ O(k2) ✔

9. P versus NP because the verifier can ignore the solution Conceptually, finding solutions can only be harder than checking them P is contained in NP decidable NP (efficiently checkable) IS SAT VC CLIQUE P (efficient) PATH L01

10. Millenium prize problems • Recall how in 1900, Hilbert gave 23 problems that guided mathematics in the 20th century • In 2000, the Clay Mathematical Institute gave 7 problems for the 21st century computer science 1 P versus NP 2 The Hodge conjecture 3 The Poincaré conjecture 4 The Riemann hypothesis 5 Yang–Mills existence and mass gap 6 Navier–Stokes existence and smoothness 7 The Birch and Swinnerton-Dyer conjecture Perelman 2006 Hilbert’s 8th problem $1,000,000

11. P versus NP • The answer to the questionis not known. But one reason it is believed to be negative is because, intuitively, searching is harder than verifying • For example, solving homework problems (searching for solutions) is harder than grading (verifying the solution is correct) Is P equal to NP? $1,000,000

12. Searching versus verifying Mathematician: Given a mathematical claim, come up with a proof for it. Scientist: Given a collection of data on some phenomena, find a theory explaining it. Engineer: Given a set of constraints (on cost, physical laws, etc.) come up with a design (of an engine, bridge, etc.) which meets them. Detective: Given the crime scene, find “who’s done it”.

13. P and NP P = languages that can be decided on a TM with polynomial running time (problems that admit efficient algorithms) languages whose solutions can be verified on a TM with polynomial running time NP = (solutions can be checked efficiently) decidable We believe that NP is bigger than P, but we are not 100% sure NP P

14. Evidence that NP is bigger than P • These (and many others) are in NP • Their solutions, once found, are easy to verify • But no efficient solutions are known for any of them • The fastest known programs take time ≈2n CLIQUE = {〈G, k〉: G is a graph with a clique of k vertices} IS = {〈G, k〉: G is a graph with an independent set of k vertices} VC = {〈G, k〉: G is a graph with a vertex cover of k vertices}

15. Equivalence of certain NP languages • We strongly suspect that problems like CLIQUE, SAT, etc. require time ≈2n to solve • We do not know how to prove this, but what we can prove is that If any one of them can be solved efficiently, then all of them can be solved efficiently

16. Equivalence of some NP languages • All these problems are as hard as one another • Moreover, they are at the “frontier” of NP • They are at least as hard as any problem in NP NP clique independent set vertex-cover P

17. Polynomial-time reductions • What do we mean when we say, for example, • We mean that • Or, we can convert any polynomial-time TM for IS into one for CLIQUE “IS is at least as hard as CLIQUE” If CLIQUE has no polynomial-time TM, then neither does IS

18. Polynomial-time reductions IS = {〈G, k〉: G is a graph with an independent set of k vertices} • Theorem CLIQUE = {〈G, k〉: G is a graph with a clique of k vertices} 2 1 If IS has a polynomial-time TM, so does CLIQUE 4 3 {1, 4}, {2, 3, 4}, {1} are independent sets {1, 2}, {1, 3}, {4} are cliques

19. Polynomial-time reductions • Proof: Suppose IS has an poly-time TM A • We want to use it to solve CLIQUE If IS has a polynomial-time TM, so does CLIQUE A for IS accept ifG has clique of size k 〈G’, k’ 〉 〈G, k〉 reject if not accept ifG’ has IS of size k reject if not

20. Reducing CLIQUE to IS • We look for a polynomial-time TM R that turns the question: into: R “Does G have a clique of size k?” G, k G’, k’ “Does G’ have an IS of size k’?” 2 2 1 1 flip all edges G G’ 4 4 3 3 k’ = k cliques of size k ISs of size k’

21. Reducing CLIQUE to IS On input 〈G, k〉 Construct G’ by flipping all edges of G Set k’ = k Output 〈G’, k’〉 R G, k G’, k’ cliques in G independent sets in G’ If G’ has an IS of size k, then G has a clique of size k ✓ If G’ does not have an IS of size k, then G has no clique of size k ✓

22. Reduction recap • We showed thatby converting an imaginary TM for IS into one for CLIQUE • To do this, we came up with a reduction that transforms instances of CLIQUE into ones of IS If IS has a polynomial-time TM, so does CLIQUE

23. Polynomial-time reductions • Language Lpolynomial-time reduces to L’ if there exists a polynomial-time TM R that takes an instance x of L into instance y of L’ s.t. x ∈ L if and only if y ∈ L’ L (CLIQUE) L’ (IS) R = 〈G, k〉 x y = 〈G’, k’〉 x ∈ L y ∈ L’ (G has clique of size k) (G’ has IS of size k’)

24. The meaning of reductions • Saying L reduces to L’ means L is no harder than L’ • In other words, if we can solve L’, then we can also solve L • Therefore If Lreduces toL’and L’∈P, then L∈P acc R poly-time TM for L’ x y rej x ∈ L y ∈ L’ TM accepts

25. The direction of reductions • The direction of the reduction is very important • Saying “A is easier than B” and “B is easier than A” mean different things • However, it is possible that L reduces to L’andL’ reduces to L • This means that L and L’ are as hard as one another • For example, IS and CLIQUE reduce to one another

26. Boolean formula satisfiability • A boolean formula is an expression made up of variables, ands, ors, and negations, like • The formula is satisfiable if one can assign values to the variables so the expression evaluates to true (x1∨x2 ) ∧ (x2∨x3∨x4) ∧ (x1) Above formula is satisfiable because this assignment makes it true: x1 = Fx2 = Fx3 = T x4 = T

27. 3SAT SAT = {〈f〉: fis a satisfiable Boolean formula} 3SAT = {〈f〉: fis a satisfiable Boolean formula in conjunctive normal form with 3 literals per clause} literal: xi or xi (x1∨x2∨x2 ) ∧ (x2∨x3∨x4) clause literals CNF: AND of ORs of literals (conjunctive normal form) 3CNF: CNF with 3 literals per clause (repetitions are allowed)

28. 3SAT and NP (x1∨x2 ) ∧ (x2∨x3∨x4) ∧ (x1) f = Finding a solution: Verifying a solution: FFTT Try all possible assignments FFFF TFFF substitute FTFF TTFF FFFT FTFT TFFT TTFT x1 = F x2 = Fx3 = T x4 = T FFTF FTTF TFTF TTTF FFTT FTTT TFTT TTTT evaluate formula (F ∨T) ∧ (F ∨ T ∨ F) ∧ (T) f = For n variables, there are 2n possible assignments can be done in linear time Takes exponential time

29. The Cook-Levin Theorem (x1∨x2 ) ∧ (x2∨x3∨x4) ∧ (x1) • So every problem in NP is no harder than SAT • But SAT itself is in NP, so SAT must be the “hardest problem” in NP: Every L∈NP reduces to SAT SAT = {f: fis a satisfiable Boolean formula} E.g. SAT NP P If SAT∈P, then P = NP

30. NP-completeness • A language C is NP-complete if: • Cook-Levin Theorem: 1. C is in NP, and 2.For every L in NP, L reduces to C. C NP SAT is NP-complete P

31. Our picture of NP NP-complete A B SAT A reduces to B NP IS CLIQUE P context-free PATH 0n1n

32. More NP-complete problems NP-complete A B 3SAT A reduces to B IS CLIQUE NP P PATH 0n1n In practice, most of the NP-problems are either in P (easy) or NP-complete (probably hard)

33. Interpretation of Cook-Levin Theorem • Optimistic view: • Pessimistic view: If we manage to solve SAT, then wecan also solve CLIQUE and many other things Since we do not believe P = NP, it is unlikely that we will ever have a fast algorithm for SAT

34. The ubiquity of NP-complete problems • We saw a few examples of NP-complete problems, but there are many more • A surprising fact of life is that most CS problems are either in P or NP-complete • A 1979 book by Garey and Johnsonlists 100+ NP-complete problems