Approximation Algorithms for Combinatorial Problems

1. Presented by Dajiang Zhu 09/20/2011 Approximation Algorithms for Combinatorial Problems David S. Johnson 1973

2. OUTLINE • Motivation • Introduction & Conclusion • Pre – Definition • Approximation Algorithms • Two problems as examples • SUBSET-SUM • MAXIMUM SATISFIABLITY

3. Motivation • Can we do a classification of polynomial complete (NP complete?) optimization problems? • Metric: Worst Case Behavior • Worst solution value / optimal value • Related publications • The complexity of theorem-proving procedures , S. A. Cook, 1971 • Reducibility among combinatorial problems , R. M. Karp, 1972 • Bounds on multiprocessing anomalies and related packing algorithms , R. L. Graham, 1972

4. Introduction – Problem 1 • SUBSET-SUM Given a finite set of positive numbers and another positive number called the “goal”, find that subset whose sum is closest to, without exceeding, the goal.

5. Introduction – Problem 2 • BIN-PACKING Given a finite list of numbers between 0 and 1 and a sequence of unit-capacity bins, find a packing of the numbers into the bins such that no bin contains a total exceeding 1 and the number of nonempty bins is minimized.

6. Introduction – Problem 3 • MAXIMUM SATISFIABILITY Given a set S of disjunctive form clauses, all of whose disjuncts are either literals or their negations, find a truth assignment to the literals which satisfies(make true) the most clauses.

7. Introduction – Problem 4 • SET CONVERING I Given a finite cover of a finite set, find that sub cover which uses the fewest sets.

8. Introduction – Problem 5 • SET CONVERING II Given a finite cover of a finite set, find that sub cover which has the least overlapping.

9. Introduction – Problem 6 • GRAPH COLORING Given a graph, find a coloring of the nodes so that no two adjacent nodes have the same color, and the total number of colors used is minimized.

10. Introduction – Problem 7 • MAXIMUM CLIQUE Given a graph, find the maximum sub graph all of whose points are mutually adjacent.

11. Conclusion – Class I • Some algorithm that they guarantee the solution they generate will be no worse than a constant times the optimal and the constant can be quite close to 1. • FIRST FIT DECREASING (11/9) • SUBSET-SUM • MAXIMUM SATISFIABLITY

12. Conclusion – Class II • Some algorithm that there is no fixed bound on the ratio of the worst case results to the optimal, but the ratio can grow no faster than the log of the problem size. • SET COVERING

13. Conclusion – Class III • Some algorithm that the ratio grows at least as O() when n is the problem size and Ɛ>0 depends on the algorithm. • GRAPH COLORING • MAXIMUM CLIQUE

14. Pre – Definition I • An optimization problem P consists of • A set of possible inputs, • A mapwhich maps each u∈ to a finite set approximate solutions, • A measure : () → defined for all possible approximate solutions.

15. Pre – Definition II • The optimal measure is defined by = BEST{(x) : x∈(u)}, • The performance is defined by = WORST{(x) : x∈(u) and x is choosable by A on input u},

16. Pre – Definition III • As a measure of the relative worst case behavior of algorithm A on input u, we use the ratio: (A,u)= Note: (A , u) ≥ 1 • Overall worst case behavior of algorithm A with respect to problem P will be a function: R[A, P](n) = MAX{(A,u):u∈ and |u|≤n}.

17. Example I – SUBSET SUM • SUBSET – SUM (SS) ={<T,s,b>: T is a finite set, s: T → is a map which assign to each x∈ T a “size” s(x), and b>0 is a single rational number}. (<T,s,b>)={⊆T: ≤b}. ()=, The optimal measure for input <T,s,b> is : =MAX{m(): ⊆T and m() ≤b}.

18. Example I – SUBSET SUM • a series of approximation algorithms {: k≥1} and is defined as: • Let SUB be that subset of { x ∈ T: s(x)>b/(k+1)} whose measure is closest to without exceeding b. Let SUM be this measure, and set LEFT=T-SUB. • If for all x ∈ LEFT, s(x) + SUB > b, halt and return SUB. • Let y be an element of LEFT for which s(y) + SUB is closest to, without exceeding, b. • Set LEFT = LEFT - {y}. SUB = SUB ∪ {y}, SUM = SUM + s(y). • Go to 2.

19. Example I – SUBSET SUM • Some observation • Algorithm A1 is closely related to the FIRST FIT DECREASING algorithm for BIN-PACKING. • The majority of the effort is concentrated in the first step. – O()

20. Example I – SUBSET SUM • Claim and Proof • Claim: for k ≥ 1 and n > 0, R[](n) ≤ (k+1)/k, = (k+1)/k

21. Example I – SUBSET SUM • Proof - R[](n) ≤ (k+1)/k • ⊆ T where is choosable by algorithm . be an optimal solution for the given input. • Partition for i ∈ {0,1} as followed: = U , where = {x∈: s(x)>b/(k+1)}, and = - . so, for i ∈ {0,1} we have m() = m() + m() • By step 1 of , we have m() ≥ m(). • If ⊒ , we have m() ≥ m(), so = . • If some x ∈ were excluded from , then by step2 of , we have s(x) + m() >b , so m()>b-s(x)≥b-b/(k+1)=kb/(k+1)≥[k/(k+1)]

22. Example I – SUBSET SUM • Proof - = (k+1)/k We consider the input <T, s, b> where T = {, ,…, }, S() = b=k+1. Clearly, = k+1 and (<T, s, b>) = k+Ɛ. Thus r(,<T, s, b>) = (k+1)/(k+Ɛ). Note: the above result shows that the SUBSET-SUM optimization problem has the desirable property: for any Ɛ>0, there is a polynomial-time algorithm s.t. R[] ≤ 1 + Ɛ

23. Example II – MAXIMUM SATISFIABILITY • MAXIMUM SATISFIABILITY(MS) ={S: S is a finite set {, ,…, } of clauses}, (T)={⊆S: there exists a truth assignment T which satisfies every clause C∈}, ()=||.

24. Example II – MAXIMUM SATISFIABILITY • algorithm B1: • Set SUB = ∅, TRUE = ∅, LEFT = S, LIT = L. • If no literal in LIT is contained in any clause of LEFT, halt and return SUB. • Let y be the literal in LIT which is contained in the most clauses of LEFT, and YT be the set of clauses in LEFT which contain y. • Set SUB = SUB U YT, LEFT = LEFT – YT, TRUE = TRUE U {y}, and LIT = LIT - {y,} , • Go to 2.

25. Example II – MAXIMUM SATISFIABILITY • Claim and Proof • Claim: for all k ≥ 1 and n > 0, R[B1, MS(k)](n) ≤ (k+1)/k,with equality for all sufficiently large n.

26. Example II – MAXIMUM SATISFIABILITY • Proof - R[B1, MS(k)](n) ≤ (k+1)/k • Each time a literal is added to TRUE, the number of clauses added to SUB at least as large as the number of clauses remaining in LEFT which are wounded (have one of their literals removed from LIT without being added to TRUE). • When the algorithm halts, the only clauses left in LEFT are those which have been wounded as many times as they contain literals. That means when the algorithm halts there are at least k*|LEFT| wounds, so |SUB| ≥ k*|LEFT|. Thus S* ≤ |S| = |SUB|+|LEFT| ≤[(k+1)/k] |SUB|

27. Example II – MAXIMUM SATISFIABILITY • algorithm B2: • Assign to each clause c ∈ S a weight w(c) = . Set SUB = TRUE = ∅, LIT = L, and LEFT = S. • If no literal in any clause of LEFT is in LIT, halt and return SUB. • Let y be any literal occurring in both LIT and a clause of LEFT. Let YT be the set of clauses in LEFT and containing y, YF the set of clauses in LEFT containing . • If ≥ , set TRUE = TRUE U {y}, SUB = SUB U YT, LEFT = LEFT – YT, and for each , set w(c) = 2w(c). Otherwise, set TRUE = TRUE U {}, SUB = SUB U YF, LEFT = LEFT – YF, and for each , set w(c) = 2w(c). • Set LIT = LIT – {y,}, and go to 2.

28. Example II – MAXIMUM SATISFIABILITY • Claim and Proof • Claim: for all k ≥ 1 and n > 0, R[B2, MS(k)](n) ≤ /(-1),with equality for all sufficiently large n.

29. Example II – MAXIMUM SATISFIABILITY • Proof - R[B2, MS(k)](n) ≤ /(-1) • At begging, the total weight of all the clauses in LEFT cannot exceed |S|/. • During each iteration, the weight of clauses removed from LEFT is at least as large as the weight added to those remaining clauses which receive new wounds. Thus the total weight can never increase even the algorithm halts. • When algorithm halts, each clauses in LEFT have been wounded as many times as it had literals, hence must have has its weight doubled that many times, and so must have a final weight of 1. • Therefore |LEFT| ≤ |S|/. So |SUB| ≥ |S| (1-1/).

30. Questions and discussion • Questions? Thanks!