130 likes | 135 Vues
Theory of Algorithms: Greedy Techniques. James Gain and Edwin Blake {jgain | edwin} @cs.uct.ac.za Department of Computer Science University of Cape Town August - October 2004. Objectives. To introduce the mind set of greedy techniques To show some examples of greedy algorithms:
E N D
Theory of Algorithms:Greedy Techniques James Gain and Edwin Blake {jgain | edwin} @cs.uct.ac.za Department of Computer Science University of Cape Town August - October 2004
Objectives • To introduce the mind set of greedy techniques • To show some examples of greedy algorithms: • Change Making • Huffman Coding • To discuss the strengths and weaknesses of greedy techniques • “Greed, for lack of a better word, is good! Greed is right! Greed works!” Gordon Grecko (Michael Douglas) in Wall Street
Greedy Algorithms • Optimization problems solved through a sequence of choices that are: • Feasible - satisfy problem constraints • Locally Optimal - best choice among all feasible options for that step • Irrevocable- no backing out • A Greedy grab of the best alternative, hoping that a sequence of locally optimal steps will lead to a globally optimal solution • Even if not optimal, sometimes an approximation is acceptable • Not all optimization problems can be approached in this manner!
Applications of the Greedy Strategy • Optimal solutions: • Change making • Minimum Spanning Trees (MST) - Prim’s and Kruskal’s Algorithms • Single-source shortest paths - Dijkstra’s Algorithm • Simple scheduling problems • Huffman codes • Approximations: • Traveling Salesman Problem (TSP) • Knapsack problem • Other combinatorial optimization problems
Change Making • Problem: give change for a specific amount n, with the least number of coins of the denominations d1 > d2 > … > dm • Example: • Smallest change for R2.54 using R5, R2, R1, 50c, 20c, 10c, 5c, 2c, 1c • R2 + 50c + 2c + 2c = R2.54 • Algorithm: • At any step choose the coin of the largest denomination that doesn’t exceed the remaining total; Repeat • Note: Optimal for reasonable sets of coins
Minimum Spanning Tree Problem • Problem: Given n points, connect them in the cheapest way so that there is a path between any pair of points • Spanning Tree (of a connected graph G ): • A connected acyclic subgraph of G that includes all of G’s vertices. • Minimum Spanning Tree (of a weighted graph G ): • A spanning tree of G with minimum total weight • Algorithms: Prim’s and Kruskal’s Algorithms • Example: 4 3 3 1 1 1 1 6 2 2 4 4 2 3 2 3
Shortest Paths Problem • Single Source Shortest Paths Problem: • Given a weighted graph G, find the shortest paths from a source vertex sto each of the other vertices • Solution: • Dijkstra’s Algorithm (a relative of Prim’s MST) for positive weights 1 4 3 2 1 2 1 4 3 6 1 2 1 4 2 3 2 4 2 3
Text Compression • Variable length encoding: • Compresses text by assigning codes of different length to characters based on their probability of occurrence • Used by Samuel Morse in constructing telegraph codes • Prefix-free Codes: • Codes are unique in that no code is a prefix for a code of another character • Tree for Binary Codes: • Leaves are characters • Left edge codes 0, right edge codes 1 • The code of a character is a simple walk from root to leaf
Algorithm for Huffman Coding • Invented by David Huffman as part of a class assignment while he was an undergraduate • Algorithm: • Construct a Huffman Tree that assigns shorter strings to higher frequencies. This defines a Huffman Code • Initialize n one-node trees labeled with the characters of the alphabet. Record the frequency (weight) of each character in the root • REPEAT until a single tree is obtained: Find two trees with the smallest weight Make them the left and right sub-tree of a new tree and record the sum of their weights in the root
Example: Huffman Coding 0.55 B 0.45 C 0.15 A 0.4 B 0.45 C 0.15 A 0.4 1.0 0.55 B 0.45 C 0.15 A 0.4
Notes on Huffman Coding • Compression Ratio: • Standard measure of compression • CR = (x - y) / y * 100%, where x is compressed and y is uncompressed • Typically, 20-80% in the case of Huffman Coding • Yields optimal compression provided: • The probabilities of character occurrences are independent • Probabilities are known in advance
Strengths and Weaknesses of Greedy Techniques • Strengths: • Intuitively simple and appealing • Weaknesses: • Only applicable to optimization problems • Doesn’t always produce an optimal solution