Algorithm Analysis

Algorithm Analysis

Question • Suppose you have 2 programs that will sort a list of student records and allow you to search for student information.Question: How do you judge which is a better program?Answer: • By running programs or • Algorithm Analysis

Algorithm Analysis • Is a methodology for estimating the resource (time and space) consumption of an algorithm. • Allows us to compare the relative costs of two or more algorithms for solving the same problem.

How to Measure “Betterness” • Critical resources in computer • Time • Memory space • For most algorithms, running time depends on size n of data. • Notation: T(n) • Time T is a function of data size n.

Two approaches • Two approaches to obtaining running time: • Measuring under standard benchmark conditions (running programs) • Estimating the algorithms performance (analyzing algorithms)

Estimation Assumptions • Estimation of running time is based on: • Size of the input • Number of basic operations • The time to complete a basic operation does not depend on the value of its operands.

Example: largest() // Return position of largest value // in array int largest(int array[], int n) { int posBig = 0; // 1 for (int i = 1; i < n; i++) if (array[posBig] < array[i]){ // 2 posBig = i; // 3 } return posBig; }

Example: largest() • Step 1: T(n) = c1 • Step 2: It takes a fixed amount of time to do one comparison T(n) = c2n • Step 3: T(n) = c3n, at most • Total time: T(n) = c1 + c2n+ c3n ≈ c1 + (c2 + c3)n ≈ cn

Example: Assignment int a = array[0]; • The running time is T(n) = c • This is called constant running time.

Example: total() sum = 0; for (i = 1; i <= n; i++) for (j = 1; j < n; j++) sum++; } • What is the running time for this code? • Analysis: • The basic operation is sum++ • The cost of a sum operation can be bundled into some constant time c. • Inner loop takes c1n; outer loop takes c2n. • The total running time is T(n) = c1n * c2n = (c1 + c2)n = c3n2

Growth Rate of Algorithm—Big-O Natation • Growth Rate for an algorithm • is the rate at which the cost of the algorithm grows as the data size n grows. • Constant: T(n) = c  O(1) • Linear Growth: T(n) = n  O(n) • Quadratic Growth: T(n) = n2  O(n2) • Exponential Growth: T(n) = 2n  O(2n) • Assumptions • Growth rates are estimates. • They have meaning when n is large.

Growth Rate Graph

c Growth Rate Graph Log n T(n) n

Characteristics of Growth Rates • Growth Rates • Constant—T(n) = c • Independent of n • Linear—T(n) = cn • Constant slope • Logarithmic—T(n) = c log n • Slope flattens out quickly • Quadratic—T(n) = cn2 • Increasing slope • Cubic—T(n) = cn3 • Exponential—T(n) = c2n

Growth Rate—Array Operations bool search(int list[], int n, int key) found = false Loop (for i = 0 to i < n) If (list[i] == key) Then found = true End If End loop return found End // search

Growth Rate—Array Operations • Linear Search • Insert at Back • Insert at Front • Remove from Back • Remove from Font

Linear Search bool search(int list[], int n, int key) found = false // c1 Loop (for i = 0 to i < n) If (list[i] == key) Then // nc2 found = true (at most) End If End loop return found // c3 End // search T(n) = c1 + nc2+c3 = c + nc2 ≈ nc2∴ O(n)

Insert at Back void insertAtBack(int list[], int &n, item) If (!isFull) Then list[n] = item n++ End If End // insertAtBack

Insert at Back void insertAtBack(int list[], int &n, item) If (!isFull) Then list[n] = item // c1 n++ // c2 End If End // insertAtBack T(n) = c1 + c2= c ∴ O(1)

Insert At Front void insertAtFront(int list[], int &n, int item) If (!isFull) Then // shift to right Loop (for i = n downto 1) list[i] = list[i – 1] End loop // assign list[0] = item count++ End If End // insertAtFront

Insert At Front—Growth Rate? void insertAtFront(int list[], int &n, int item) If (!isFull) Then Loop (for i = n downto 1) list[i] = list[i – 1] // c1n (at most) End loop list[0] = item // c2 count++ // c3 End If End // insertAtFront T(n) = c1n + c2+c3 = nc1+c ≈ nc1∴ O(n)

Remove From Back int removeFromBack(list[], int &n) result = NIL If (!isEmpty) Then result = list[n – 1] n = n - 1 End If return result End // removeFromBack

Remove From Front intremoveFromFront(list[], int &n) result = NIL If (!isEmpty) Then result = list[0] // shift left Loop (for i = 0 to n – 2) list[i] = list[i + 1] End Loop n = n - 1 End If return result End // removeFromFront

Growth Rate—Linked-List Operations • Linear Search • Insert at Back • Insert at Front • Remove from Back • Remove from Font

Linear Search boollinearSearch(int key){boolfound = false; c1 if (head != NULL){ node* temp = head; c2 while (temp != NULL){ if (temp=>getData() != key) found = true; else temp = temp->getNext(); c3n } } return found; } T(n) = c1 + c2 + c3n ≈ cn∴ Growth Rate: O(n)

insertAtBack() void insertAtBack(int item){ node *newP = new node(item, NULL); node *temp = head; c1 if (head == NULL) head = newP; else while (temp->getNext() != NULL){ temp = temp->getNext(); c2n } temp->setNext(newP); c3 count++; c4} T(n) = c1+ c2n + c3 + c4 ≈ c + c2n ∴ G.R. = O(n)

insertAtFront() void insertAtFront(int item){ node *newP = new node(item, NULL); c1newP->setNext(head); c2 head = newP; Next(newP); c3 count++; c4} T(n) = c1 + c2 + c3 + c3= c ∴ G.R. = O(1)

removeFromBack() void removeFromBack(){ node *temp = head; c1 node *tail = NULL; c2 while (temp != NULL){ trail = temp; nc3 temp = temp->getNext(); nc4 } count++; c5} T(n) = c1 + c2 + nc3+ nc4 + c5 = c6 + n(c3+ c4) ≈cn ∴ G.R. = O(n)

removeFromFront() void removeFromBack(){ node *newP = new node(item, NULL); c1newP->setNext(head); c2 head = newP; Next(newP); c3 count++; c4} T(n) = c1 + c2 + c3 + c3= c ∴ G.R. = O(1)

Best, Worst, Average Cases • For an algorithm with a given growth rate, we consider • Best case • Worst case • Average case • For example:Sequential search for K in an array of n integers • Best case: The first item of the array equals K • Worst case: The last position of the array equals K • Average case: Match at n/2

Which Analysis to Use • The Worst Case. • Useful in many real-time applications. • Advantage: • PredictabilityYou know for certain that the algorithm must perform at least that well. • Disadvantage: • Might not be a representative measure of the behavior of the algorithm on inputs of size n.

Which Analysis to Use? • The average case. • Often we prefer to know the average-case running time. • The average case reveals the typical behavior of the algorithm on inputs of size n. • Average case estimation is not always possible. • For the sequential search example, it assumes that the key value K is equally likely to appear in any position of the array. This assumption is not always correct.

Which Analysis to Use • The Best Case • Normally, we are not interested in the best case, because: • It is too optimistic • Not a fair characterization of the algorithms’ running time • Useful in some rare cases, where the best case has high probability of occurring.

The moral of the story • If we know enough about the distribution of our input we prefer the average-case analysis. • If we do not know the distribution, then we must resort to worst-case analysis. • For real-time applications, the worst-case analysis is the preferred method.

Your Turn (Stack w/array) Given Stack Class: …private:elemType data[MAX];int top;}

Stack (w/array) What is the growth rate (big O) of the push operation? void push(elemType item){ if (!isFull()){ top++; data[top] = item; }}

Stack (w/array) What is the growth rate (big O) of the pop operation? elemType pop (){elemType result = NIL; if (!isEmpty()){ result = data[top]; top--; }}

Stack (w/array) What is the growth rate (big O) of the clear operation? void clear(){ top = -1;}

Stack (w/linked list) Given: Stack class with linked list …private: Node *top;}

Stack (w/linked list) What is the growth rate (big O) of the push operation? void push(elemType item){ Node *temp = new Node(item, NULL); top->setNext(temp); top = temp;}

Stack (w/linked list) What is the growth rate (big O) of the pop operation? elemType pop () {elemType result = NIL; Node *temp = top; if (!isEmpty()){ result = top-<getData(); top = top->getNext(); delete temp; } return result;}

Stack (w/linked list) What is the growth rate (big O) of the clear operation? Void clear () { Node *temp = top; while (top != NULL) { top = top->getNext(); delete temp; temp = top; }}

Growth Rate of Binary Search (w/array) Comparisons Number left0 n1 n(1/2)2 n(1/2)2 3 n(1/2)3 4 n(1/2)4 … … k-1 n(1/2)k-1 k n(1/2)k Solution n(1k) n n(1/2)k = ------- = ---- = 1 (2k) (2k) 2k = n log(2k) = log(n)k log(2) = log(n) k = log(n)/log(2)k = c1 log(n) T(n) = c1 c2 log(n)O(log n)

Analysis of Bubble Sort • Suppose: count = 10maximum pass = 9passcomparisons1 92 83 74 65 56 47 38 29 1 void bubbleSort(int a[], int count) pass  count – 1last  pass – 1 Loop (for i from 1 to pass) Loop (for j from 0 to last) If (a[j] > a[j + 1]) Then swap a[j] and a[j = 1] End If End Loop last  last – 1End Loop

Analysis of Bubble Sort • (n-1) + (n-2) + (n-3)… + 3 + 2 + 1 1 + 2 + 3 …+ (n-3) + (n-2) + (n-1) n + n + n … + n + n + n • = n(n-1) • Therefore, ? = n(n – 1)/2 = cn2 – cn ≈ cn2 • Growth Rate: O(n2) • Total number of comparisons 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 = ? 9 + 8 + 7 + 6 + 5 + 4 + 3 + 2 + 1 = ?--------------------------------------------------10+10 +10+……………………+10 +10 = 9x10

Algorithm Analysis