Design and Analysis of Algorithms

1. Design and Analysis of Algorithms Yoram Moses Lecture 11 June 3, 2010 http://www.ee.technion.ac.il/courses/046002

2. Nondeterministic Polynomial Time(NP)

3. Shortest Path: Search, Existence, Verification • Search problem: • Input: (G,w,s,t): a directed graph G with weight function w, a source s, and a sink t. • Goal: find a shortest path from s to t. (or reject if none exists) • Complexity: our solution runs in O(VE) = O(n2) (Notice n = size of input = O(V+E)) • Existence problem: • Input: (G,w,s,t,k): G,w,s,t are as before + a number k. • Goal: decide whether there is a path from s to t of length ≤ k. • Complexity: our solution runs in O(VE) = O(n2) • Verification problem: • Input: (G,w,s,t,k,p): G,w,s,t,k as before. p is a path in G. • Goal: decide whether p is a simple path from s to t of length ≤ k. • Complexity: O(V) = O(n).

4. Max Flow: Search, Existence, Verification • Search problem: • Input: (G,c,s,t): a directed graph G with capacity function c, a source s, and a sink t. • Goal: find a maximum flow in G. (or reject if none exists) • Complexity: O(VE2) = O(n3) • Existence problem: • Input: (G,c,s,t,k): G,c,s,t as before + a number k. • Goal: decide whether there is a flow in G with value ≥ k. • Complexity: O(VE2) = O(n3) • Verification problem: • Input: (G,c,s,t,k,f): G,c,s,t,k as before. f is a function from edges of G to real numbers. • Goal: decide whether fis a legal flow with value ≥k. • Complexity: O(E) = O(n).

5. Hamiltonian Cycle: Search, Existence, Verification • Search problem: • Input: an undirected graph G. • Goal: find a Hamiltonian cycle in G (or reject if none exists). • Complexity: O(VxV!) = O(n2n log n) • Existence problem: • Input: an undirected graph G. • Goal: decide whether G has a Hamiltonian cycle. • Complexity: O(VxV!) = O(n2n log n) • Verification problem: • Input: (G,p): an undirected graph G and a sequence of nodes p. • Goal: decide whether p is a Hamiltonian cycle in G. • Complexity: O(V) = O(n).

6. 3-Coloring: Search, Existence, Verification • Search problem: • Input: G: an undirected graph • Goal: find a 3-Coloring of G. (or reject if none exists) • Complexity: O(E 3V) = O(n2n log 3) • Existence problem: • Input: G: as before. • Goal: decide whether G has a 3-Coloring. • Complexity: O(E 3V) = O(n2n log 3) • Verification problem: • Input: (G,): G as before and  : V {1,2,3}. • Goal: decide whether  is a 3-Coloring of G. • Complexity: O(E) = O(n).

7. Search and Existence vs.Verification Conclusion: in many natural examples: • Search and existence are computationally equivalent • Verification is easier • Sometimes it’s just a little easier (Shortest Path, Max flow) • Sometimes it’s a lot easier (Hamiltonian cycle, 3-Coloring)

8. Verification Relations • Language: L  {0,1}* • Definition: A verification relation for L is a relation R  {0,1}* {0,1}* s.t. for all x  {0,1}*: • x  L  there is at least one y  {0,1}* s.t. (x,y)  R. • x  L  there is noy  {0,1}* s.t. (x,y)  R. • y is called the “certificate” for x • A.k.a. its “witness” or “proof” • Remarks: • Every input x  L has at least one certificate y. • If (x,y) R, then y is a certificate for x. • An input x  L may have several certificates. • A language L has many verification relations.

9. Verification Relations: Examples • Shortest path: • x = (G,w,s,t,k), y = a path p • Language: {(G,w,s,t,k): G has an s-t path of length ≤k} • Certificate: s-t path of length ≤ k • Verification relation: {((G,w,s,t,k),p): p is an s-t path of length ≤ k in G} • Hamiltonian cycle: • x = undirected graph G, y = a path p • Language: G that has a Hamiltonian cycle • Certificate: a Hamiltonian cycle in G • Verification relation: {(G,p): p is a Hamiltonian cycle in G}

10. Nondeterministic Polynomial Time Definition: A binary relation R is polynomially bounded, if there exists some c > 0 s.t. for every (x,y)  R, |y| ≤ |x|c. Definition: Lis polynomial-time verifiable, if it has a verification relation R, which satisfies both: • R is polynomially bounded, and • R is polynomial-time decidable. Definition: The class NP (Nondeterministic Polynomial Time) is the set of all polynomial-time verifiable languages.

11. NP: Examples • Examples of languages in NP: • Decision Shortest Path, Decision Max Flow, Decision LP • Hamiltonian Cycle, TSP, 3-Coloring, SAT, k-SAT, Clique • Examples of languages not known to be in NP: • HC-complement: given a graph G, decide whether G has no Hamiltonian cycles.

12. Nondeterministic Algorithms • Definition: A nondeterministic algorithm is an algorithm Nthat, on input x, • First,N “nondeterministically” guesses a “witness” y. • Then, N runs a deterministic “verification” algorithm on (x,y). • Note: Nmay make different nondeterministic guesses in different runs on the same input x. Nondeterministic Algorithm N Verification y Nondeterministic guess x (x,y) yes/no

13. Decision by Nondeterministic Algorithms • Definition: A nondeterministic algorithm N is said to decide language L if: • For every input x  L, there is at least one guess y s.t. Naccepts (x,y). • For every input x  L, the verification algorithm Nrejects (x,y), for all guesses y. • A polynomial-time nondeterministic algorithm is one in which • The guesses (y’s) are of polynomial size (in |x|), and • The verification algorithm runs in polynomial time. • Lemma: L  NP iff L is decidable by a polynomial-time nondeterministic algorithm.

14. An NP Algorithm for Clique Nondeterministic guess (input: x = (G,k)) • for i = 1,…,k • vi nondeterministic guess of a node in V=V(G) • output y = (v1,…,vk) Verification algorithm (input: (x,y)) • If x is not a valid encoding of a graph G and an integer k, reject. • If y is not a valid encoding of k nodes v1,…,vk in G, reject. • If v1,…,vk are not distinct, reject. • for i 1,…,k-1 do • for j i+1,…,k do • if {vi,vj}  E reject. • accept

15. P vs. NP Lemma: P  NP • Biggest open problem of computer science: is P = NP? • Two possibilities: • Current belief: P  NP • Search & Existence strictly harder than Verification. P = NP? P = NP NP P

16. Time Hierarchy • f: N  N: a complexity measure. • Time(f(n))= all languages decidable in time O(f(n)). Lemma: Let f(n),g(n) be two complexity measures. If there exists a constant c, s.t. for all n > c, f(n) ≤ g(n), then Time(f(n))  Time(g(n)). Theorem(Time Hierarchy) Let f(n),g(n) be two complexity measures. If there exists a constant c, s.t. for all n > c, f(n) ≤ g(n)1/2, then Time(f(n))  Time(g(n)).

17. P, NP, and EXP • Definition: Lemma: P  EXP but P  EXP Lemma: NP  EXP (exercise) Open problem: is NP =EXP? • 3 Possibilities: EXP EXP EXP = NP NP = P P NP P P

18. NP-Completeness (NPC) • Problems in NP not known to be in P: Hamiltonian Cycle, Clique, SAT, k-SAT (k ≥ 3), k-Coloring (k ≥ 3), TSP, …. (many others) • All of these are “NP-Complete” • NP-Complete Problems: • Belong to NP • If any of them belongs to P, then NP = P. • Two possibilities: NP NP = NPC = P NPC P

19. NP-Hardness (NPH) Definition: A language L is NP-hard if L’≤p L holds for all L’ NP. • NPH = class of all NP-hard problems. Lemma: If any NP-hard problem belongs to P, then NP = P. • If one NPH problem is easy, then all of NP is easy. Lemma: If L  NPH and L≤p L’, then L’  NPH.

20. NP-Completeness Definition: A language L is NP-complete if both • L  NP and • L is NP-hard • NPC = class of NP-complete problems • NPC = NP  NPH Theorem: • If some NPC language is in P, then P = NP. (P  NPC    NP = P = NPC). • If some NPC language is not in P, then no NPC language is in P. (NPC  P  P  NPC =   NP  P).

21. NP-Completeness • NPC: “hardest” problems in NP • Behave as a “single block”: either all in P or all outside P Lemma: If L1,L2 NPC, then both L1≤p L2 and L2≤p L1.

22. Proving NP-Completeness • How to prove that a given language L is NPC? • Show that L  NP, and • Show that L’ ≤p L holds for every L’  NP. • Easier alternative: • Show that L  NP, and • Find some NPC problem L’ and show L’ ≤p L. • How do we obtain the first NPC problem? • Using the first alternative • Cook-Levin theorem: Circuit-SAT is NP-complete.

23. NP-Completeness: the Full Recipe • To show that L is NPC: • Prove L is in NP • Show a polynomial time nondeterministic algorithm for L • Select an NPC problem L’ • Show a polynomial-time reduction f from L’ to L • Prove that x L’ iff f(x)  L • Show a polynomial-time algorithm to compute f

24. Example: Clique is NPC • Clique is in NP (seen today) • 3-SAT is NPC (will show this later on) • 3-SAT ≤pClique(seen in previous lecture) • Therefore: Clique is also NP-Complete!

25. End of Lecture 11