Design and Analysis of Algorithms

1. Design and Analysis of Algorithms Yoram Moses Lecture 12 June 10, 2010 http://www.ee.technion.ac.il/courses/046002

2. Reductions amongNP-Complete Problems

3. Reminder: NP • NP = all polynomial-time verifiable languages. • Language: L  {0,1}* • Verification relation for L: R  {0,1}* {0,1}* • If x  L, then there is a “certificate” y s.t. (x,y)  R. • If x  L, then there is noy s.t. (x,y)  R. • R is polynomially bounded if there is a constant c > 0 s.t. |y| ≤ |x|c for every (x,y)  R. • Lis polynomial-time verifiable if it has a verification relation R that is both • polynomially bounded, and • polynomial-time decidable.

4. Reminder: P vs. NP Lemma: P  NP • Biggest open problem of th. computer science: is P = NP? • Two possibilities: • Current belief: P  NP P = NP? P = NP NP P

5. Reminder: NP Completeness • A language L is NP complete if both • L  NP and • L is NP-hard • A language L is NP hard if L’≤p L holds for all L’ NP. Lemma: If L1 is NP hard and L1≤p L2, then L2 is NP hard. Lemma: If L1≤p L2 and L2  P, then L1  P. Corollary: If one NP hard language is in P, then NP = P.

6. NP-Completeness: the Full Recipe • To show that L is NPC: • Prove L NP • Show L is polynomial-time verifiable • Select an NPH problem L’ • Show a polynomial-time reduction f from L’ to L: • Prove that x  L’ iff f(x)  L • Show a polynomial-time algorithm to compute f

7. A First NPC Problem We need a “first NPC problem” to start with. Theorem [Cook-Levin]: Circuit-SAT is NP Complete. • Proof: next lecture. Our goal: using reductions, show that many more problems are NP Complete.

8. Our Reduction Tree Circuit-SAT Homework: Partition k-Coloring, k ≥ 3 Others… SAT 3-SAT Clique Vertex Cover Hamiltonian Cycle Subset Sum TSP

9. Boolean Circuits • Boolean circuit: • A directed acyclic graph (DAG) • Nodes are also called gates • n input gates(in-degree = 0) • Each input gate is labeled by a distinct Boolean variable(denoted by x1,…,xn). • A single output line • Internal gates are labeled by AND/OR/NOT • AND,OR gates with in-degree ≥ 2 • NOT gate with in-degree = 1

10. Boolean Circuits: Example x1    x2  output   x3  internal gates input gates

11. Input Assignments • Input assignment: a vector  {0,1}n • Assigns a value (i {0,1} ) to each input variable xi in x1,…,xn. • Circuit evaluation: a mapping C: {0,1}n {0,1} • Maps every assignment to an output bit (1or 0). • C() = evaluation of the circuit (output value) on  • Evaluation is done in topological order, starting from the input gates.

12. Circuit Evaluation: Example 1 x1  1 1 1  0 1 1  0 x2 1 1 1  output  1 1 1  1 0 x3  1 0 internal gates input gates

13. Satisfying Assignments • Satisfying assignment: An input assignment  for which C() = 1. • Satisfiable circuit: A circuit that has at leastone satisfying assignment. There is a way to make it “output” 1.

14. Example of an Unsatisfiable Circuit x1    x2  output   x3  internal gates input gates

15. Circuit-SAT Circuit-SAT = language of all satisfiable circuits. Lemma: Circuit-SAT  NP. • Proof: • Define a relation R: (C,)  R iff • C is a valid encoding of a Boolean circuit, and •  is a valid encoding of a satisfying input assignment for C • Fact 1: R is a verification relation for Circuit-SAT: • If C Circuit-SAT, then it has a satisfying assignment ’, and hence (C,’) R for this particular’ • If C Circuit-SAT, then no assignment  satisfies C, and hence (C,) R for all  • Fact 2: R is polynomially bounded • If  is an assignment for C, then || ≤ |C| • Fact 3: R is polynomial-time decidable • Evaluating C on  takes linear time

16. Cook-Levin Theorem Theorem [Cook-Levin]: Circuit-SAT is NP-Hard. • Proof idea: Show that L ≤p Circuit-SAT for every language L NP. • For now, we’ll assume the theorem is true: • Circuit-SATis our “first” NPH problem

17. Boolean Formulae • Boolean formula: a logical expression with • n Boolean variables x1,…,xn • Logical operators: , , , , , etc. • Parentheses • Ex: = ((x1 x2)  ((x1  x3)  x4))  x2 • CNF formula: formula of the form:  = C1 C2  …  Cm, where each clause Cj is an OR of literals. • Literal: a variable xi or its negation xi • Example:(x1  x2  x3)  (x1  x4)  (x5) • k-CNF formula: a CNF formula in which each clause has exactly k literals. • Example of a 3-CNF formula: (x1  x2  x3)  (x1  x4  x5)

18. SAT • Truth assignment: a vector  {0,1}n • Assigns a value to each of the n Boolean variables x1,…,xn. • Formula evaluation: a mapping : {0,1}n {0,1} • Maps every assignment  to a bit (1 or 0). • () = evaluation of on  • Example: •  = ((x1 x2)  ((x1  x3)  x4))  x2 • = (0,0,1,1) • () = ((0  0)  ((0 1)  1))  0 = (1  (1  1))  1 = (1  0)  1 = 1 • Satisfying assignment: an assignment  s.t. () = 1. • Satisfiable formula: a formula that has at least one satisfying assignment. • SAT = language of all satisfiable Boolean formulae.

19. SAT is NP-Complete Theorem: SATis NP-Complete. Lemma 1: SAT  NP: • Verification relation: R= {(,):  is a satisfying assignment for }. Lemma 2: Circuit-SAT ≤pSAT • Need to find a mapping f from Boolean circuits to Boolean formulae s.t. • For every circuit C: C is satisfiable iff  = f(C) is satisfiable • f is polynomial-time computable

20. Reduction from Circuit-SAT to SAT: 1st Attempt Main idea: recursive construction of a formula  that represents C • Associate with each gate v C a unique variable xv • Start from the output line o • Set =xo • For each xv s.t. v is an internal gate, let • v1,…,vkbe the in-neighbors ofv • opbe the label ofv • Then replace each xvinby (op(xv1,…,xvk))

21. Example x1    x2    x3 

22. Example x1   x10  x2    x3  = x10

23. Example x1  x8  x10  x2 x9    x7 x3  = (x7 x8  x9)

24. Example x1  x8  x6 x10  x2 x9    x7 x3  = (x7 x8  (x6  x7))

25. Example x5 x1  x8  x6 x10  x2 x9    x7 x3  = (x7 (x5  x6)  (x6  x7))

26. Example x5 x1  x8  x6 x10  x2 x9    x7 x3  x4 = ((x1 x2  x4) (x5  x6)  (x6  (x1 x2  x4)))

27. Example x5 x1  x8  x6 x10  x2 x9    x7 x3  x4 = ((x1 x2  x3) ((x1  x2)  (x3))  ((x3)  (x1 x2  x3)))

28. Circuit-SAT ≤pSAT:1st Attempt • The above mapping is indeed a reduction • C() = () for every assignment  • so C is satisfiable iff  is satisfiable • BUT - this reduction is not polynomial-time computable • Common sub-formulae are generated again and again. • The formula may have exponential size

29. Bad Example … x1     nANDgates Corresponding formula: = (x1 x1  …  x1) 2n occurrences of x1

30. Reduction from Circuit-SAT to SAT: 2nd Attempt Main idea: associate each internal gate with one “basic” formula, and take AND over all gates • Associate with each gate vC a unique variable xvand the output withxO • For each internal gate vC let • v1,…,vkbe thein-neighbors of v • opbe the label of v • Thenv=(xv(op(xv1,…,xvk)))

31. Example x5 x1  x8  x6 x10  x2 x9    x7 x3  x4 = x10 (x4 x3)  (x5 (x1  x2))  (x6 x4) (x7 (x1  x2  x4))  (x8 (x5  x6))  (x9  (x6  x7))  (x10 (x7  x8  x9))

32. Circuit-SAT ≤p SAT: 2nd Attempt • Let C have n input gates and m internal gates. • Then  = f(C) has m+n variables and m+1 clauses. Lemma 1: f is polynomial-time computable. Proof: easy

33. Circuit-SAT ≤p SAT:2nd Attempt Lemma 2: f is a reduction from Circuit-SAT to SAT. •  {0,1}n: assignment for C g() =   {0,1}m+n: assignment for  v = evaluation of v on . •  {0,1}m+n: assignment for  h() =   {0,1}n: assignment for C v  v Lemma 3: For any circuit C, • If C() = 1, then (g()) = 1. • If () = 1, then C(h()) = 1. Conclusion: C is satisfiable iff  is satisfiable.

34. Next: 3SAT is NP-Complete Theorem: 3SATis NP-Complete. Lemma 1:3SAT  NP: • Verification relation: (y,), where  is a satisfying assignment for y. Lemma 2: SAT ≤p3SAT • Need to find a mapping f from general Boolean formulae to 3CNF Boolean formulae s.t. • For every general formula ,  is satisfiable iff  = f() is satisfiable • f is polynomial-time computable

35. Reduction from SAT to 3SAT: Step 1: Generate a binary parse tree for .  = ((x1 x2)  ((x1 x3)  x4))  x2   x2    x2 x1  x4 x1 x3

36. Reduction from SAT to 3SAT: Step 2: Add a variable for each internal node in the tree  = ((x1 x2)  ((x1 x3)  x4))  x2 y1  y2  x2 y4  y3  y5  x2 x1 y6  x4 x1 x3

37. Reduction from SAT to 3SAT: Step 3: For each internal node v, create a clause Cv: Let • opbe label(v) • zbe var(v) • v1,…,vkbe children(v) (k ≤ 2) • z1,…,zk be var(v1),…,var(vk) • Then Cv=(z op(z1,…,zk))

38. Reduction from SAT to 3SAT: • Step 3: For each internal node v, create a clause Cv • C1 = (y1 (y2 x2)) • C2 = (y2 (y3 y4)) • C3 = (y3 (x1 x2)) • C4 = (y4y5) • C5 = (y5 (y6 x4)) • C6 = (y6 (x1 x3))  = ((x1 x2)  ((x1 x3)  x4))  x2 y1  y2  x2 y4  y3  y5  x2 x1 y6  x4 x1 x3

39. Reduction from SAT to 3SAT: Step 4: Add a clause C0 = y1 and AND all clauses.  = y1 (y1 (y2 x2))  (y2 (y3 y4))  (y3 (x1 x2))  (y4y5)  (y5 (y6 x4))  (y6 (x1 x3))  = ((x1 x2)  ((x1 x3)  x4))  x2 y1  y2  x2 y4  y3  y5  x2 x1 y6  x4 x1 x3

40. Reduction from SAT to 3SAT: Step 5: Transform each clause Ci into CNF. • Write the truth table of Ci. • Note: the truth table has at most 8 rows • Write the DNF of Ci. • DNF: D1  …  Dm, where each Dj is an AND of literals • Take OR on the “0” entries of the table • Each clause has at most 3 literals • Use De-Morgan’s law to obtain CNF of Ci. • At most 8 clauses • Each clause has at most 3 variables

41. Reduction from SAT to 3SAT: Example: C1 = (y1 (y2 x2)) C1= (y1  y2  x2)  (y1  y2  x2)  (y1  y2  x2)  (y1  y2  x2) C1= (y1  y2  x2)  (y1  y2  x2)  (y1  y2  x2)  (y1  y2  x2)

42. Reduction from SAT to 3SAT: Step 6: Transform each clause Ci that has < 3 literals into a clause with 3 literals. • Use two dummy variables p and q. • If Ci = (z  w) (z,w: literals), then replace Ci by: Ci,1 Ci,2, where • Ci,1 = (z  w  p) andCi,2 = (z  w  p) • If Ci = (z) (z: a single literal), then replace Ci by: Ci,1 Ci,2 Ci,3 Ci,4 , where • Ci,1 = (z  p  q) and Ci,2 = (z  p  q) and • Ci,3 = (z  p  q) and Ci,4 = (z  p  q)

43. SAT ≤p 3SAT • Suppose that  has n variables • Parse tree has m≤2n – 2 internal nodes • has at most 4  8  (m + 1) =O(n) clauses •  has m + n = O(n) variables Lemma 1: f is polynomial-time computable Proof: easy Lemma 2: f is a reduction Proof: exercise Conclusion: If SAT is NPC, then so is 3SAT

44. End of Lecture 12