Asymptotic Analysis of Algorithms (how to evaluate your programming power tools)

1. Honors CS 102 Sept. 4, 2007 Asymptotic Analysis of Algorithms(how to evaluate your programming power tools) based on presentation material by Mike Scott

2. Algorithmic Analysis • A technique used to characterize the execution behavior of algorithms in a manner independent of a particular platform, compiler, or language. • Abstract away the minor variations and describe the performance of algorithms in a more theoretical, processor independent fashion. • A method to compare speed of algorithms against one another.

3. Different Algorithms • Why are the words in a dictionary in alphabetical order? • A brute force approach • linear search • worst case is (d x N) • Another way • divide and conquer • worst case is (c x log N) • Constants are unknown and largely irrelevant.

4. Big O • The most common method and notation for discussing the execution time of algorithms is "Big O”. • For the alphabetized dictionary the algorithm requires O(log N) steps. • For the unsorted list the algorithm requires O(N) steps. • Big O is the asymptotic execution time of the algorithm.

5. Formal Definition of Big O • T(N) is O( F(N) ) if there are positive constants c and N0 such that T(N) < cF(N) when N > N0 • There is a point N0 such that for all values of N that are past this point, T(N) is bounded by some multiple of F(N). • Thus if T(N) of the algorithm is O( N2 ) then, ignoring constants, at some point we can bound the running time by a quadratic function of the input size. • Given a linear algorithm, it is technically correct to say the running time is O(N2). O(N) is a more precise answer as to the Big O bound of a linear algorithm.

6. Big O Examples • 3n3 = O(n3) • 3n3 + 8 = O(n3) • 8n2 + 10n * log(n) + 100n + 1020 = O(n2) • 3log(n) + 2n1/2 = O(n1/2) • 2100 = O(1) • TlinearSearch(n) = O(n) • TbinarySearch(n) = O(log(n))

7. Other Algorithmic Analysis Tools • Big Omega T(N) is ( F(N) ) if there are positive constants c and N0 such that T(N) > cF( N )) when N > N0 • Big O is similar to less than or equal, an upper bound. • Big Omega is similar to greater than or equal, a lower bound. • Big Theta T(N) is ( F(N) ) if and only if T(N) is O( F(N) )and T( N ) is ( F(N) ). • Big Theta is similar to equals.

8. Relative Rates of Growth "In spite of the additional precision offered by Big Theta,Big O is more commonly used, except by researchersin the algorithms analysis field" - Mark Weiss

9. What it All Means • T(N) is the actual growth rate of the algorithm. • F(N) is the function that bounds the growth rate. • may be upper or lower bound • T(N) may not equal F(N). • constants and lesser terms ignored because it is a bounding function

10. Assumptions in Big O Analysis • Once found accessing the value of a primitive is constant time x = y; • mathematical operations are constant time x = y * 5 + z % 3; • if statement: constant time if test and maximum time for each alternative are constants if( iMySuit ==DIAMONDS || iMySuit == HEARTS) return RED; else return BLACK;

11. Fixed-Size Loops • Loops that perform a constant number of iteration are considered to execute in constant time. They don't depend on the size of some data set for(int suit = Card.CLUBS; suit <= Card.SPADES; suit++) { for(int value = Card.TWO; value <= Card.ACE; value++) { myCards[cardNum] = new Card(value, suit); cardNum++; } }

12. Loops That Work on a Data Set • Loops like on the previous slide are fairly rare. • Normally a loop operates on a data set which can vary is size. public double minimum(double[] values){ int n = values.length; double minValue = values[0]; for(int i = 1; i < n; i++) if(values[i] < minValue) minValue = values[i]; return minValue;} • The number of executions of the loop depends on the length of the array, values. The actual number of executions is (length - 1). • The run time is O(N).

13. Nested Loops • Number of executions? public void bubbleSort(double[] data){ int n = data.length; for(int i = n - 1; i > 0; i--) for(int j = 0; j < i; j++) if(data[j] > data[j+1]) { double temp = data[j]; data[j] = data[j + 1]; data[j + 1] = temp; }}

14. Summing Execution Times • If an algorithm’s execution time is N2 + N then it is said to have O(N2) execution time, not O(N2 + N). • When adding algorithmic complexities the larger value dominates. • Formally, a function f(N) dominates a function g(N) if there exists a constant value n0 such that for all values N > N0 it is the case that g(N) < f(N).

15. Example of Dominance • Suppose we go for precision and determine how fast an algorithm executes based on the number of items in the data set. • x2/10000 + 2x log10 x + 100000 • Is it plausible to say the x2 term dominates even though it is divided by 10000? • What if we separate the equation into (x2/10000 ) and (2x log x + 100000)?

16. Summing Execution Times • For large values the x2 term dominates so the algorithm is O(N2).

17. Ranking of Algorithmic Behaviors

18. Running Times • Assume N = 100,000 and processor speed is 1,000,000 operations per second

19. Graphical Results

20. Determining Big O • A DNA problem • Is a number prime? • Hand actions • inserting card • determining if card is in Hand • combining Hands • copying Hand • retrieving Card

21. Putting it in Context • Why worst-case analysis? Wouldn’t average-case analysis be more useful? • A1: No. E.g., doesn’t matter if a cashier’s terminal handles 2 transactions per second or 3, as long as it never takes more than 20. • A2: Well, OK, sometimes, yes. However, this often requires much more sophisticated mathematics. In fact, for some common algorithms, nobody has been able to do an average case analysis.