1 / 52

In The Name Of God Algorithms Design

In The Name Of God Algorithms Design. Shahed University Dr. Shahriar Bijani May. 2014. Slide References. George Bebis, NP-Completenes , Analysis of Algorithms CS 477/677, University of Nevada . The Theory of NP-Completeness , National Sun Yat-sen University . NP-Completeness.

forest
Télécharger la présentation

In The Name Of God Algorithms Design

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. In The Name Of GodAlgorithms Design Shahed University Dr. Shahriar Bijani May. 2014

  2. Slide References • George Bebis, NP-Completenes, Analysis of Algorithms CS 477/677, University of Nevada . • The Theory of NP-Completeness, National Sun Yat-sen University. Algorithms Design

  3. NP-Completeness • So far we’ve seen a lot of good news! • Problem X can be solved quickly (i.e., in close to linear time, or at least a time that is some small polynomial function of the input size) • NP-completeness is a form of bad news! • Evidence that many important problems can not be solved quickly. • NP-complete problems really come up all the time!

  4. Why should we care? • Knowing that they are hard lets you stop beating your head against a wall trying to solve them… • Use a heuristic: come up with a method for solving a reasonable fraction of the common cases. • Solve approximately: come up with a solution that you can prove that is close to right. • Use an exponential time solution: if you really have to solve the problem exactly and stop worrying about finding a better solution.

  5. Optimization & Decision Problems • Decision problems • Given an input and a question regarding a problem, determine if the answer is yes or no • Optimization problems • Find a solution with the “best” value • Optimization problems can be cast as decision problems that are easier to study • E.g.: Shortest path: G = unweighted directed graph • Find a path between u and v that uses the fewest edges • Does a path exist from u to v consisting of at most k edges?

  6. Optimization & Decision Problems • Decision Problem: computational problem with intended output of “yes” or “no”, 1 or 0 • Optimization Problem: computational problem where we try to maximize or minimize some value • Introduce parameter k and ask if the optimal value for the problem is at most or at least k. Turn optimization into decision

  7. Solving an optimization problem by a decision algorithm : • Solving TSP optimization problem by a decision algorithm : • Give c1 and test (decision algorithm) • Give c2 and test (decision algorithm) •  • Give cn and test (decision algorithm) • We can easily find the smallest ci

  8. Algorithmic vs Problem Complexity • The algorithmic complexity of a computation is some measure of how difficult is to perform the computation (i.e., specific to an algorithm) • The complexity of a computational problem or task is the complexity of the algorithm with the lowest order of growth of complexity for solving that problem or performing that task. • e.g. the problem of searching an ordered list has at most lgn time complexity. • Computational Complexity: deals with classifying problems by how hard they are.

  9. Complexity Class P • Deterministic in nature • Solved by conventional computers in polynomial time • O(1) Constant • O(log n) Sub-linear • O(n) Linear • O(n log n) Nearly Linear • O(n2) Quadratic • Polynomial upper and lower bounds

  10. Class of “P” Problems • Class P consists of (decision) problems that are solvable in polynomial time • Polynomial-time algorithms • Worst-case running time is O(nk), for some constant k • Examples of polynomial time: • O(n2), O(n3), O(1), O(n lg n) • Examples of non-polynomial time: • O(2n), O(nn), O(n!)

  11. Tractable/Intractable Problems • Problems in P are also called tractable • Problems not in P are intractable or unsolvable • Can be solved in reasonable time only for small inputs • Or, can not be solved at all • Are non-polynomial algorithms always worst than polynomial algorithms? - n1,000,000 is technically tractable, but really impossible - nlog log log n is technically intractable, but easy

  12. Example of Unsolvable Problem • Turing discovered in the 1930’s that there are problems unsolvable by any algorithm. • The most famous of them is the halting problem

  13. Example of Unsolvable Problem • Halting problem • Given an arbitrary algorithm and its input, will that algorithm eventually halt, or will it continue forever in an “infinite loop?” • Input: a procedure P (described by a programming language) and its input I • Output: true if executing P on I halts (finishes execution), false otherwise. • Proof: assuming that the Halting Problem has a Turing Machine (i.e. algorithmic) solution and demonstrates that this leads to a contradiction

  14. Undecidable Problems • If solving a problem P would allow us to solve the halting problem, then P is undecidable – there is no solution to P, since we have proved there is no solution to the halting problem! • There are lots of practical problems like this. CS 200 Spring 2004

  15. Examples of Intractable Problems

  16. Intractable Problems • Can be classified in various categories based on their degree of difficulty, e.g., • NP • NP-complete • NP-hard • Let’s define NP algorithms and NP problems …

  17. Nondeterministic and NP Algorithms Nondeterministic algorithm = two stage procedure: • Nondeterministic (“guessing”) stage: generate randomly an arbitrary string that can be thought of as a candidate solution (“certificate”) • Deterministic (“verification”) stage: take the certificate and the instance to the problem and returns YES if the certificate represents a solution NP algorithms (Nondeterministic polynomial) verification stage is polynomial

  18. Class of “NP” Problems • Class NP consists of problems that could be solved by NP algorithms • i.e., verifiable in polynomial time • If we were given a “certificate” of a solution, we could verify that the certificate is correct in time polynomial to the size of the input • Warning: NP does not mean “non-polynomial”

  19. E.g.: Hamiltonian Cycle • Given: a directed graph G = (V, E), determine a simple cycle that contains each vertex in V • Each vertex can only be visited once • Certificate: • Sequence: v1, v2, v3, …, v|V| hamiltonian not hamiltonian

  20. Is P = NP? • Any problem in P is also in NP: P  NP • The big (and open question) is whether NP  P or P = NP • i.e., if it is always easy to check a solution, should it also be easy to find a solution? • Most computer scientists believe that this is false but we do not have a proof … P NP

  21. NP-Completeness (informally) • NP-complete problems are defined as the hardest problems in NP • Most practical problems turn out to be either P or NP-complete. P NP-complete NP

  22. yes yes   f Problem B no no Problem A Reductions • Reduction is a way of saying that one problem is “easier” than another. • We say that problem A is easier than problem B, (i.e., we write “A  B”) if we can solve A using the algorithm that solves B. • Idea:transform the inputs of A to inputs of B

  23. Polynomial Reductions • Given two problems A, B, we say that A is polynomially reducible to B (A p B) if: • There exists a function f that converts the input of A to inputs of B in polynomial time • A(i) = YES  B(f(i)) = YES

  24. NP-Completeness (formally) • A problem B is NP-complete if: (1) B NP (2) A p B for all A NP • If B satisfies only property (2) we say that B is NP-hard • No polynomial time algorithm has been discovered for an NP-Complete problem • No one has ever proven that no polynomial time algorithm can exist for any NP-Complete problem P NP-complete NP

  25. P: the class of problems which can be solved by a deterministic polynomial algorithm. • NP: the class of decision problem which can be solved by a non-deterministic polynomial algorithm. • NP-hard: the class of problems to which every NP problem reduces. • NP-complete (NPC): the class of problems which are NP-hard and belong to NP.

  26. yes yes   f Problem B no no Problem A Implications of Reduction - If A pB and B  P, then A  P - if A pB and A  P, then B  P

  27. yes Polynomial time algorithm to decide A yes   f Polynomial time algorithm to decide B no no Proving Polynomial Time • Use a polynomial time reduction algorithm to transform A into B • Run a known polynomial time algorithm for B • Use the answer for B as the answer for A

  28. Proving NP-Completeness In Practice • Prove that the problem B is in NP • A randomly generated string can be checked in polynomial time to determine if it represents a solution • Show that one known NP-Complete problem can be transformed to B in polynomial time • No need to check that allNP-Complete problems are reducible to B

  29. Revisit “Is P = NP?” Theorem: If any NP-Complete problem can be solved in polynomial time  then P = NP. P NP-complete NP

  30. P & NP-Complete Problems • Shortest simple path • Given a graph G = (V, E) find a shortest path from a source to all other vertices • Polynomial solution: O(VE) • Longest simple path • Given a graph G = (V, E) find a longest path from a source to all other vertices • NP-complete

  31. P & NP-Complete Problems • Euler tour • G = (V, E) a connected, directed graph find a cycle that traverses each edge of G exactly once (may visit a vertex multiple times) • Polynomial solution O(E) • Hamiltonian cycle • G = (V, E) a connected, directed graph find a cycle that visits each vertex of G exactly once • NP-complete

  32. Satisfiability Problem (SAT) • Satisfiability problem: given a logical expression , find an assignment of values (F, T) to variables xi that causes  to evaluate to T  =x1   x2  x3   x4 • SAT was the first problem shown to be NP-complete!

  33. CFN Satisfiability • CFN is a special case of SAT •  is in “Conjuctive Normal Form” (CNF) • “AND” of expressions (i.e., clauses) • Each clause contains only “OR”s of the variables and their complements E.g.: = (x1  x2)  (x1   x2)  ( x1   x2) clauses

  34. 3-CNF Satisfiability A subcase of CNF problem: • Contains three clauses • E.g.:  = (x1  x1  x2)  (x3  x2  x4)  (x1  x3   x4) • 3-CNF is NP-Complete • Interestingly enough, 2-CNF is in P!

  35. NP-naming convention • NP-complete - means problems that are 'complete' in NP, i.e. the most difficult to solve in NP • NP-hard - stands for 'at least' as hard as NP (but not necessarily in NP); • NP-easy - stands for 'at most' as hard as NP (but not necessarily in NP); • NP-equivalent - means equally difficult as NP, (but not necessarily in NP);

  36. Examples NP-complete and NP-hard problems NP-complete NP-hard

  37. P: the class of problems which can be solved by a deterministic polynomial algorithm. • NP: the class of decision problem which can be solved by a non-deterministic polynomial algorithm. • NP-hard: the class of problems to which every NP problem reduces. • NP-complete (NPC): the class of problems which are NP-hard and belong to NP.

  38. Some concepts of NPC • Definition of reduction: Problem A reduces to problem B (A  B) iff A can be solved by a deterministic polynomial time algorithm using a deterministic algorithm that solves B in polynomialtime. • Up to now, none of the NPC problems can be solved by a deterministic polynomial time algorithm in the worst case. • It does notseem to have any polynomial time algorithm to solve the NPC problems.

  39. The theory of NP-completeness always considers the worst case. • The lower bound of any NPC problem seemsto be in the order of an exponential function. • Not all NP problems are difficult. (e.g. the MST problem is an NP problem.) • If A, B  NPC, then A  B and B  A. • Theory of NP-completeness: If any NPC problem can be solved in polynomial time, then all NP problems can be solved in polynomial time. (NP = P)

  40. Decision problems • The solution is simply “Yes” or “No”. • Optimization problems are more difficult. • e.g. the traveling salesperson problem • Optimization version: Find the shortest tour • Decision version: Is there a tour whose total length is less than or equal to a constant c ?

  41. Solving an optimization problem by a decision algorithm : • Solving TSP optimization problem by a decision algorithm : • Give c1 and test (decision algorithm) • Give c2 and test (decision algorithm) •  • Give cn and test (decision algorithm) • We can easily find the smallest ci

  42. The satisfiabilityproblem (SAT) • The satisfiability problem • The logical formula : x1 v x2 v x3 & - x1 & - x2 the assignment : x1 ← F , x2 ← F , x3 ← T will make the above formula true . (-x1, -x2 , x3) represents x1 ← F , x2 ← F , x3 ← T

  43. If there is at least one assignment which satisfies a formula, then we say that this formula is satisfiable; otherwise, it is unsatisfiable. • An unsatisfiable formula : x1 v x2 & x1 v -x2 & -x1 v x2 & -x1 v -x2

  44. Definition of the satisfiability problem: Given a Boolean formula, determine whether this formula is satisfiable or not. • A literal : xi or -xi • A clause : x1 v x2 v -x3 Ci • A formula : conjunctive normal form (CNF) C1& C2 & … & Cm

  45. The resolution principle • Resolution principle C1 : x1 v x2 C2 : -x1 v x3  C3 :x2 v x3 • From C1 &C2, we can obtain C3, and C3 can be added into the formula. • The formula becomes: C1 &C2 &C3

  46. Another example of resolution principle C1 : -x1 v -x2 v x3 C2 : x1 v x4  C3 :-x2 v x3 v x4 • If no new clauses can be deduced, then it is satisfiable. -x1 v -x2 v x3 (1) x1 (2) x2 (3) (1) & (2) -x2 v x3 (4) (4) & (3) x3 (5) (1) & (3) -x1 v x3 (6)

  47. If an empty clause is deduced, then it is unsatisfiable. - x1 v -x2 v x3 (1) x1 v -x2 (2) x2 (3) - x3 (4)  deduce (1) & (2) -x2 v x3 (5) (4) & (5) -x2 (6) (6) & (3) □ (7)

  48. Nondeterministic operations and functions [Horowitz 1998] • Choice(S) : arbitrarily chooses one of the elements in set S • Failure : an unsuccessful completion • Success : a successful completion • Nonderministic searching algorithm: j ← choice(1 : n) /* guessing */ if A(j) = x then success /* checking */ else failure

  49. A nondeterministic algorithm terminates unsuccessfully iff there does not exist a set of choices leading to a success signal. • The time required for choice(1 : n) is O(1). • A deterministic interpretation of a non-deterministic algorithm can be made by allowing unbounded parallelism in computation.

  50. Nondeterministic sorting B ← 0 /* guessing */ for i = 1 to n do j ← choice(1 : n) if B[j] ≠ 0 then failure B[j] = A[i] /* checking */ for i = 1 to n-1 do if B[i] > B[i+1] then failure success

More Related