1. UMass Lowell Computer Science 91.404Analysis of AlgorithmsProf. Karen DanielsFall, 2002 Monday, 12/2/02 Design Patterns for Optimization Problems Greedy Algorithms

2. Algorithmic Paradigm Context

3. Greedy Algorithms

4. What is a Greedy Algorithm? • Solves an optimization problem • Optimal Substructure: • optimal solution contains in it optimal solutions to subproblems • Greedy Strategy: • At each decision point, do what looks best “locally” • Choice does not depend on evaluating potential future choices or solving subproblems • Top-down algorithmic structure • With each step, reduce problem to a smaller problem • Greedy Choice Property: • “locally best” = globally best

5. Examples • Minimum Spanning Tree • Dijkstra Shortest Path • Huffman Codes • Fractional Knapsack • Activity Selection

6. Minimum Spanning Tree

7. Invariant: Minimum weight spanning forest Becomes single tree at end Invariant: Minimum weight tree 2 A B 4 3 G 6 Spans all vertices at end 5 1 1 E 7 6 F 8 4 D C 2 Minimum Spanning Tree Time: O(|E|lg|E|) given fast FIND-SET, UNION • Produces minimum weight tree of edges that includes every vertex. Time: O(|E|lg|V|) = O(|E|lg|E|) slightly faster with fast priority queue • forUndirected, Connected, Weighted Graph • G=(V,E) source: 91.503 textbook Cormen et al.

8. Dijkstra Shortest Path

9. 2 A 4 3 6 5 1 1 7 B 6 F 8 G 4 C 2 E D Single Source Shortest Paths: Dijkstra’s Algorithm for (nonnegative)weighted,directed graph G=(V,E) source: 91.503 textbook Cormen et al.

10. Huffman Codes

11. Huffman Codes

12. Fractional Knapsack

13. Fractional Knapsack 50 Value: $60 $100 $120 30 20 10 “knapsack” item1 item2 item3

14. Activity Selection

15. Example Optimization Problem: Activity Selection • Problem Instance: • Set S = {1,2,...,n} of n activities • Each activity i has: • start time: si • finish time : fi • Activities i, j are compatible iff non-overlapping: • Objective: • select a maximum-sized set of mutually compatible activities source: 91.503 textbook Cormen, et al.

16. source: 91.503 textbook Cormen, et al.

17. Algorithmic Progression • “Brute-Force” • Dynamic Programming #1 • Exponential number of subproblems • Dynamic Programming #2 • Quadratic number of subproblems • Greedy Algorithm

18. Running time? Greedy Algorithm source: 91.503 textbook Cormen, et al. • Algorithm: • S’ = presort activities in S by nondecreasing finish time • and renumber • GREEDY-ACTIVITY-SELECTOR(S’) • n length[S’] • A {1} • j 1 • for i 2 to n • do if • then • j i • return A

19. Greedy Heuristic • If optimization problem does not have “greedy choice property”, greedy approach may still be useful as a heuristic in bounding the optimal solution • Example: minimization problem Upper Bound (heuristic) Solution Values Optimal (unknown value) Lower Bound