UMass Lowell Computer Science 91.404 Analysis of Algorithms Prof. Karen Daniels Spring, 2001

1. UMass Lowell Computer Science 91.404Analysis of AlgorithmsProf. Karen DanielsSpring, 2001 Lectures 3/4 Chapters 1 & 2 Wed. 2/7/01 and Thursday 2/8/01 or “How to Make an Algorithm Sandwich” adapted from the fall, 2000 talk Finding the Largest Area Axis-Parallel Rectangle in a Polygon in O(n log2 n) Time given at UMass Lowell MATHEMATICAL SCIENCES COLLOQUIUM

2. Distance-Based Subdivision Maximum Rectangle Geometric Restriction Ordered Containment Limited Gaps Containment Minimal Enclosure Core Algorithms Maximal Cover Application-Based Algorithms Lattice Packing Two-Phase Layout Column-Based Layout Context: Taxonomy of ProblemsSupporting Apparel Manufacturing

3. A Common (sub)Problem Find a Good Approximation Outer Inner

4. What’s the Problem? • Given a 2D polygon that: • does not intersect itself • may have holes • has n vertices • Find the Largest-Area Axis-Parallel Rectangle • How “hard” is it? • How “fast” can we find it?

5. n4 n5 n 2n n3 1 n2 Approach • Explore the problem to gain intuition: • Describe it: What are the assumptions? (model of computation, etc...) • Has it already been solved? • Have similar problems been solved? (more on this later) • What does best-case input look like? • What does worst-case input look like? • Establish worst-case upper bound on the problem using an algorithm • Design a (simple) algorithm and find an upper bound on its worst-case asymptotic running time; this tells us problem can be solved in a certain amount of time. Algorithms taking more than this amount of time may exist, but won’t help us. • Establish worst-case lower bound on the problem • Tighten each bound to form a worst-case “sandwich” increasing worst-case asymptotic running time as a function of n

6. Upper Bound First Attempt

7. Designing an Algorithmto Provide Upper Bound • First attempt uses a straightforward approach: • “brute-force” • “naïve” • Will most likely produce a “loose” upper bound • Characterize rectangle based on how it can “grow” Contacts reduce degrees of freedom

8. O(n5) O(n) O(n5) LR Algorithm Find_LR(Polygon P) area0 Find_LR_0_RC(P) area1 Find_LR_1_RC(P) area2 Find_LR_2_RC(P) area3 Find_LR_3_RC(P) area4 Find_LR_4_RC(P) return maximum(area0, area1, area2, area3, area4) Find_LR_0_RC(P) for i 1 to n [for each edge of P] for j 1 to n for k 1 to n for l 1 to n area area of LR for 0-RC determining set for (i,j,k,l) if LR is empty, then update maximum area return maximum area this test has a O(n) loop down inside it

9. First Upper Bound: What can we really conclude? • Algorithm’s worst-case running time is in O(n5). • Problemcan be solved in O(n5) time, even for worst-case inputs. • Note: there might exist algorithms for this problem that take more than n5 time, but they aren’t useful to us! • If a worst-case input exists that causes algorithm to actually use time proportional to n5, then algorithm’s worst-case running time is also in W(n5). • In this case,we can say algorithm’s worst-case running time is in Q(n5). An inefficient algorithm for the problem might exist that takes this much time, but would not help us. n 2n 1 n5 n2 increasing worst-case asymptotic running time as a function of n

10. Lower Bound First Attempt An inefficient algorithm for the problem might exist that takes this much time, but would not help us. n 2n 1 n5 n2 increasing worst-case asymptotic running time as a function of n

11. 2n 1 n5 No algorithm for the problem exists that can solve it for worst-case inputs in less than linear time . An inefficient algorithm for the problem might exist that takes this much time, but that would not help us. n worst-case bounds on problem First Attempt • First attempt will most likely produce a “loose” lower bound that can be improved later. • W(1) and W(n) are not hard to achieve: • Rationale for W(n) is that we must examine every vertex at least once in order to solve the problem. This holds for every algorithm that solves the problem, so it gives us a lower bound on the problem in an even stronger sense than our upper bound. • Remember that our upper bound does not guarantee that no algorithm exists for this problem that takes more than O(n5) time, but if one exists it will not help us!

12. 2n 1 n5 No algorithm for the problem exists that can solve it for worst-case inputs in less than linear time . An inefficient algorithm for the problem might exist that takes this much time, but would not help us. n worst-case bounds on problem Know the Difference! Strong Bound: This worst-case lower bound on the problem holds for every algorithm that solves the problem and abides by our problem’s assumptions. Weak Bound: This worst-case upper bound on the problem comes from just considering one algorithm. Other, less efficient algorithms that solve this problem might exist, but we don’t care about them! Both the upper and lower bounds are probably loose (i.e. probably can be tightened later on).

13. Upper Bound Tightening It

14. Approach • Think harder… • Design an algorithm that takes only O(n2) time for worst-case inputs tighter upper bound 2n 1 n5 n2 n worst-case bounds on problem Now we no longer care about this one!

15. worst-case bounds on problem 2n 1 n5 n2 n Approach (continued) • Think even harder… • Run into brick wall! • Characterize the brick wall: this type of case • Go around brick wall by looking at similar problems. • This requires ability to compare functions!

16. Some Related Problems n a(n) n n log2(n) n log(n) n5 2n n a(n) log(n) 1 This slide and the next are included just to show a variety of functions. Don’t worry about the details! a(n) is the veryslowly growing inverse of Ackermann’s function

17. Some Related Problems (continued) n a(n) n n log2(n) n log(n) n a(n) log(n) n5 2n 1

18. Approach (continued) • Reduce the O(n2) bound to O(n log2 n): • Adapt a technique that worked for a similar problem in order to develop a general framework for the problematic case worst-case bounds on problem n5 2n 1 n2 n log2 n n

19. Lower Bound Tightening It

20. Approach • Go around lower bound brick wall by: • examining strong lower bounds for some similar problems • transforming a similar problem to our problem [this process is similar to what we do when we prove problems NP-complete] worst-case bounds on problem n5 2n 1 n2 n log2 n n

21. point set, polygon point set, polygon SmallestOuterCircle: Q(n) LargestInnerRectangle: O(n log2(n)) polygon SmallestOuterRectangle: Q(n) 2n 1 n n log(n) n log2(n) n5 Lower Bounds for Related Problems LargestInnerRectangle: W(n) polygon LargestInnerCircle: Q(n log n) point set worst-case bounds on our problem

22. x2 x4 x1 x3 Lower Bound of W(n log n) by Transforming a (seemingly unrelated) Problem MAX-GAP instance: given n real numbers { x1, x2, ... xn } find the maximum difference between 2 consecutive numbers in the sorted list. O(n) time transformation Rectangle area is a solution to the MAX-GAP instance specialized polygon Rectangle algorithm must take as least as much time as MAX-GAP. MAX-GAP is known to be in W(n log n). Rectangle algorithm must take W(n log n) time for specialized polygons. [Transforming yet another different problem yields bound for unspecialized polygons.]

23. 2n 1 n n log(n) n log2(n) n5 Summary • First attempt: • Establish weak (and loose) worst-case upper bound on the problem using an algorithm • Establish strong (and loose) worst-case lower bound on the problem • Tighten bounds to make an “algorithm sandwich” • Establish weak (but tighter) worst-case O(n log2 n) upper bound on the problem using an algorithm • Establish strong (and tighter) worst-case lower bound on the problem by transforming a problem with known lower bound bounds on problem

24. For More Information • Computational Geometry: • Graduate CS course in Computational Geometry to be offered at UMass Lowell in Spring ‘01 • Introductory texts: • Computational Geometry in C(O’Rourke) • Computational Geometry: An Introduction(Preparata & Shamos) • Bibliography: ftp://ftp.cs.usask.ca/pub/geometry/ • Software:http://www.geom.umn.edu/software/cglist/ • My research: • http://www.cs.uml.edu/~kdaniels • Journal paper: “Finding the largest area axis-parallel rectangle in a polygon” • (Computational Geometry: Theory and Applications) • Prof. Victor Milenkovic: Frequent co-author and former PhD advisor • http://www.cs.miami.edu/~vjm

25. Remainder of 2/8 Lecture Homework 1 Clarifications & Hints

26. HW1 Clarifications & Hints Printed copy is missing some text from problem 6(d). It should read: (d) Can you conclude from your answers to (b) and (c) that Mystery’s worst-case running time is = Q(f(n))? Why or why not? Web copy: view it in Page Layout mode so you see arrows in problem 6 pseudocode and other special (graphical) symbols (like square root!)

27. HW1 Clarifications & Hints (continued) • General Hint: • Remember the definitions for O, W and Q: • f(n) is in the set of functions O(g(n)) if (and only if) 0 <= f(n) <= cg(n) for all n >= n0 where c and n0 are positive constants • f(n) is in the set of functions W(g(n)) if (and only if) f(n) >= cg(n) >= 0 for all n >= n0 where c and n0 are positive constants • f(n) is in the set of functions Q(g(n)) if (and only if) f(n) is in O(g(n)) and also in W(g(n)). This means: • 0 <= f(n) <= c1g(n) for all n >= n0 where c1 and n0 are positive constants and • f(n) >= c2g(n) >= 0 for all n >= n0 where c2 and n0 are positive constants • Important notes: • c is a constant means that c cannot depend on n! • In the Q definition, the constants c1 and c2 can be different from each other. But, the upper and lower bounds must work for the same n0! • For each constant, you only need to find 1 value that works!

28. HW1 Clarifications & Hints (continued) • Problems 1 & 2 & 3: • The definitions for O and W use inequalities. • This means that: • If f(n) is in O(g(n)), then f(n) is also in O(any function larger than g(n)) • If f(n) is in W(g(n)), then f(n) is also in W(any function smaller than g(n))

29. HW1 Clarifications & Hints (continued) • Problem 1: • Be aware of the difference between a statement about a bound on the running time of an algorithm vs. a statement about a bound on the running time of a problem. Make sure you understand slides 5,9,11,12 of this lecture, which talk about this. • You can assume that in Problem 1 the statements “Let R be a problem. Suppose that the worst-case asymptotic time complexity of R is in O(n2) and also in W(n lg n)” mean that: • The upper bound is a weak bound that means someone has found an algorithm whose running time is quadratic in n ; this shows the problem can be solved in quadratic time. It does not mean that there does not exist a very inefficient algorithm for this problem that takes more than quadratic time. • The lower bound is a strong bound that means someone has proven that there cannot exist any algorithm for this problem that can handle worst-case inputs in less than time proportional to nlgn.

30. HW1 Clarifications & Hints (continued) • Problem 1 (continued): • Also, be sure you understand what is necessary to conclude that an algorithm has a matching upper and lower bound so that the Q notation can be used (see the discussion on slide 9 and think back to our discussion in class on loops that cannot exit early…)

31. HW1 Clarifications & Hints (continued) • Problem 3: • There are 5 functions, so there are 5! possible orders. To find the right one without examining all possible orders, let transitivity help you (see slide 28). • Here’s an example of how to do this for 3 functions: 2n n n3 • First pick one pair of the functions (we’ll pick 2n and n3, but any pair will do) • You need to figure out whether n3 is in O(2n) or n3 is in W(2n) • That is, you need to solve: n3 ? c2nfor all n >= n0 where ? is either >= or <= • In some cases it is easy to isolate c and pick a value for c that satisfies the “n” part of the expression whenever n is greater than some value. This current case is a challenge because of the power of n. One approach is to use logarithms. Taking lg of both sides yields: • 3lgn ? lgc + n 3lgn – n ? lgc • Now, think about the function 3lgn – n and how it behaves for small vs. large n • For large n, n dominates and makes the function negative. When the function is negative, any value of c >= 1 works. When does it become (and remain) negative? • Trying n=2,4,8,16 is interesting because we see that 3lgn – n increases from n=2 to n=4 and then decreases. By the time n=16, it is negative and stays negative. • If we pick c=1 and n0=16, then 3lgn <= lgc + n so n3 <= c2nand n3 is in O(2n). • This means that in the ordering of our 3 functions, n3 must come before 2n.

32. HW1 Clarifications & Hints (continued) • Problem 3 (continued): • Now consider the remaining function: n. We need to know whether n belongs before n3, between n3 and 2n or after 2n. • Let’s compare n with n3. If we can show that n goes before n3, then, by transitivity, we’ll have the ordering: n n3 2n • So, let’s try to prove that n is in O(n3). To do this, we must find a value of c and a value of n0 that work so that: n <= c n3 • To isolate c, divide both sides by n3 (n is positive, so this does not change the sense of the inequality). n/ n3 <= c n3 1/ n2 <= c • Since 1/ n2 is <= 1 for all n >=1, there are many possibilities for c when n0 = 1. Let’s pick c=1. Having found c=1 and n0 = 1 that make n <= c n3 we can conclude that n is in O(n3). • Because n is in O(n3) and n3 is in O(2n), the ordering is: n n3 2n • Thus, g1 = n g2 = n3 and g3 = 2n

33. HW1 Clarifications & Hints (continued) • Problem 4: • Your answer should be a value of n that is greater than 0. • You might want to write a short program that finds the value for you. • Problem 5: • To explain why your pseudocode is correct, briefly say why you’re sure that: • Your pseudocode terminates • If recursive, your base case must correctly stop the recursion • If iterative, your loops must stop (no infinite loops)! • When your pseudocode terminates, it returns the right answer • Your pseudocode has the proper running time: Q(lg n) worst-case time in this case. Since you’re justifying upper and lower bounds on the worst-case running time, be sure you say why: • a worst-case input for this problem won’t cause your algorithm to take more than time proportional to lgn • there exists a worst-case input for this problem that actually causes your algorithm to take time proportional to lgn

34. HW1 Clarifications & Hints (continued) • Problem 6: • You can assume that each print statement in the Mystery pseudocode takes one unit of time = one step.