CS420 lecture three Lowerbounds

1. CS420 lecture threeLowerbounds wim bohm, cs CSU

2. Big O, big omega, big theta • f(x) = O(g(x)) iff there are positive integers c and n0 such that f(x) < c.g(x) for all x > n0 • f(x) = Ω(g(x)) iff there are positive integers c and n0 such that f(x) > c.g(x) for all x > n0 • f(x) = Θ(g(x)) iff f(x) = O(g(x)) and f(x) = Ω(g(x))

3. Big O etc. • Big O used in upper bounds, ie the worst or average case complexity of an algorithm. Big theta is a tight bound, eg stronger than upper bound. • Big omega used in lower bounds, ie the complexity of a problem.

4. lower bounds • An easy lower bound on a problem is the size of the output it needs to produce, or the number of inputs it has to access • Generate all permutations of size n, lower bound? • Towers of Hanoi, lower bound? • Sum n input integers, lower bound? but... sum integers 1 to n?

5. some Ω(n) problems • Given an unsorted array A and a number p, rearrange A such that all A[i]<x occur before all A[i]>x

6. some Ω(n) problems • Given an unsorted array A and a number x, rearrange A such that all A[i]<=x occur before all A[i]>=x L=1; R=n; while(L<R){ while((A[L]<=x)&&(L<R)L++ while((A[R]>=x)&&(L<R))R- - if(L<R)swap(A,L,R) }

7. Dutch National Flag • Red, blue and white tulips, rearrange them so that all reds precede all whites precede all blues r=1; b=n; i=1; while(i<=b){ case S[i] red: swap(S,i,r);r++;i++; white: i++; blue: swap(S,i,b); b- -; } //How does it work?

8. flag while(i<=b){ case S[i] red: swap(S,i,r);r++;i++; white: i++; blue: swap(S,i,b); b- -; }

9. flag while(i<=b){ case S[i] red: swap(S,i,r);r++;i++; white: i++; blue: swap(S,i,b); b- -; } but does it terminate? what happens if S[i] = S[b] = blue

10. Comparison problems • We have seen search.

11. Comparison problems • We have seen search. Why is it closed?

12. Comparison problems • We have seen search. Why is it Ω(logn)?

13. Comparison problems • We have seen search. What are the allowed operations?

14. Comparison problems • We have seen search. Why is the decision tree binary?

15. Comparison problems • We have seen search. At least how many leaves?

16. Comparison problems • We have seen search. At least how high? Why?

17. Comparison problems • Let's try the same for sort we have an O(nlogn) upper bound what would be the allowed steps?

18. Comparison problems • Let's try the same for sort we have an O(nlogn) upper bound comparison, array element assignment, index arithmetic, establish sorted

19. Comparison problems • Let's try the same for sort we have an O(nlogn) upper bound is the decision tree binary again?

20. Comparison problems • Let's try the same for sort we have an O(nlogn) upper bound how many leaves this time?

21. Comparison problems • Let's try the same for sort we have an O(nlogn) upper bound every leaf represents a specific permutation of the array

22. Comparison problems • Let's try the same for sort we have an O(nlogn) upper bound how many different permutations?

23. Comparison problems • Let's try the same for sort we have an O(nlogn) upper bound n! how high will the decision tree at least be?

24. Comparison problems • Let's try the same for sort we have an O(nlogn) upper bound the decision tree will be at least log(n!) high log(n!) = O(nlogn) so sort is Ω(nlogn), and thus closed.

25. Tournaments • Selecting the maximum element of an array. View it as a tournament. Comparison is a match, winning is transitive: • A>B and B>C implies A>C, so A does not need to play C, and B and C can never be the winner of the tournament • Many possible O(n) algorithms tennis(S): if (|S|==1) S1 else max(tennis(topHalf(S)), tennis(bottomHalf(S))) sweep(S): winner= S1 for i=2 to n winner = max(winner,Si)

26. Runner up which gives more info about runner up? a a b b c c d e d f e g h f g sweep h tennis

27. Assumptions about initial input • Transitive and total ordering • transitive: a>b and b>c implies a>c (no roshambo) • total: for all a and b: either a>b or b>a • ie the elements can be ranked • Any assignment of ranks to names is feasible • in previous example we could assign any of the 8! permutations of rankings 1 to 8 to a to h • Because we look for the winner, anyone who loses stops playing. Therefore there will be n-1 (n is number of inputs) games / comparisons for any tournament • notice: both sweep and tennis have 7 matches

28. Simpler case: n=4 • Any of the 24 assignments of 1..4 to a..d is valid • Build a decision tree for sweep (for n = 4), which is sometimes called "caterpillar"

29. Decision tree for sweep • Any of the 24 assignments of 1..4 to a..d is valid • Build a decision tree for sweep (for n = 4) a:b a b a:c b:c a b c c b:d c:d a:d c:d a c d c d d d b

30. Decision tree for sweep • Any of the 24 assignments of 1..4 to a..d is valid • Build a decision tree for sweep (for n = 4) a:b a b a:c b:c a b c c b:d c:d a:d c:d c d a c d d d b what do we know about the runner up for this outcome?

31. Decision tree for sweep • Any of the 24 assignments of 1..4 to a..d is valid • Build a decision tree for sweep (for n = 4) a:b a b a:c b:c a b c c b:d c:d a:d c:d a c d c d d d b what do we know about the runner up for this outcome?

32. tennis • Build a decision tree for tennis (for n = 4)

33. Decision tree for tennis • Build a decision tree for tennis(for n = 4) a:b a b c:d c:d c c d d b:c b:d a:c a:d a a c b d d c b

34. Decision tree fortennis • Build a decision tree fortennis(for n = 4) a:b a b c:d c:d c c d d b:c b:d a:c a:d a a c b d d c b what do we know about the runner up for this outcome?

35. Decision tree fortennis • Build a decision tree fortennis(for n = 4) a:b a b c:d c:d c c d d b:c b:d a:c a:d a a c b d d c b what do we know about the runner up for this outcome?

36. Runner up • In “The Art of Computer Programming, vol 3”, Donald Knuth proved that the tennis tournament algorithm provides the lower bound to determine the runner up: In the case of tennis we need lgn more comparisons to find the runner up