Download
algorithm design and analysis ada n.
Skip this Video
Loading SlideShow in 5 Seconds..
Algorithm Design and Analysis (ADA) PowerPoint Presentation
Download Presentation
Algorithm Design and Analysis (ADA)

Algorithm Design and Analysis (ADA)

186 Vues Download Presentation
Télécharger la présentation

Algorithm Design and Analysis (ADA)

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Algorithm Design and Analysis (ADA) 242-535, Semester 1 2013-2014 • Objective • to introduce the Big-Oh notation for estimating the worst case running time of programs 2. Running Time of Programs

  2. Overview • Running Time: T() • Big-Oh and Approximate Running Time • Calculating Big-Oh Directly • Analyzing Function Calls • Analyzing Recursive Functions • Towers of Hanoi

  3. 1. Running Time: T() • What is the running time of this program? void main(){ int i, n; scanf("%d", &n); for(i=0; i<n; i++) printf("%d"\n", i);} continued

  4. Counting Instructions • Assume 1 instruction takes 1 ms

  5. There is no single answer! • the running time depends on the size of the n value • Instead of a time answer in seconds, we want a time answer which is related to the size of the input. continued

  6. For example: • running time T(n) = constant * n • this means that as n gets bigger, so does the program time • running time islinearly related to the input size running time constant * n size of n

  7. A simple way of writing the running time is: T(n) = 3n

  8. Running Time Theory • A program/algorithm has a running time T(n) • n is some measure of the input size • T(n) is the largest amount of time the program takes on any input of size n • T(n) is the worst running time • not always accurate, but easy to calculate • Time units are left unspecified.

  9. 1.1. Kinds of Running Time Worst-case:(we use this usually) • T(n) = maximum time of algorithm on any input of size n. - one possible value Average-case:(we sometimes use this) • T(n) = expected time of algorithm over all inputs of size n. - this approach needs info about the statistical distribution (probability) of the inputs. - e.g. uniform spread of data (i.e. all data is equally likely) Best-case:(don't use this, it's misleading) • e.g. write a slow algorithm that works fast on specially selectedinput.

  10. 1.2. T(n) Example • Loop fragment for finding the product of all the positive numbers in the A[] array of size n: (2)int prod = 1;(3) for(j = 0; j < n; j++)(4) if (A[j] > 0)(5) prod = prod * A[j]; • Count each assignment and test as 1 “time unit”.

  11. Convert 'for' to 'while' • The while-loop is easier to count (and equivalent to the for-loop): int prod = 1; // 1 int j = 0; // 1 while (j < n) { // 1 for the test if (A[j] > 0) // 1 prod = prod*A[j]; // 1 j++; // 1 } What about counting the loop? We assume that 1 instruction takes 1 "time unit"

  12. Calculation • The for loop executes n times • each loop carries out (in the worse case) 5 ops • test of j < n, if test, multiply, assign, j increment • total loop time = 5n • plus 3 ops at start and end • small assign (line 2), init of j (line 3), final j < n test • Total time T(n) = 5n + 3 • running time is linear with the size of the array

  13. 1.3. Comparing Different T()’s TB(n) = 2n2 • If input size < 50, program B takes less time. • But for large n’s, which are more common in real code, program B is worse and worse (slower). T(n) TA(n) = 100n input size n

  14. 1.4. Common Growth Formulae & Names • Formula (n = input size) Name n linear n2 quadratic n3 cubic nm polynomial, e.g. n10 mn ( m >= 2) exponential, e.g. 5n n! factorial 1 constant log n logarithmic n log n log log n

  15. Growth Examples

  16. 1.5. Execution Times Assume 1 instruction takes 1 microsec (10-6 secs) to execute. How long will n instructions take? • 3 9 50 100 1000 106n 3 9 50 100 1ms 1secn2 9 81 2.5ms 10ms 1sec 12 daysn3 27 729 125ms 1sec 16.7 min 31,710yr2n 8 512 36yr 4*1016 yr 3*10287 yr 3*10301016yrlog n 2 3 6 7 10 20 n (no. of instructions) growth formula T() if n is 50, you will wait 36 years for an answer!

  17. Notes • Logarithmic running times are best. • Polynomial running times are acceptable, if the power isn’t too big • e.g. n2 is ok, n100 is terrible • Exponential times mean sloooooooow code. • some size problems may take longer to finish than the lifetime of the universe!

  18. 1.6. Why use T(n)? • T() can guide our choice of which algorithm to implement, or program to use • e.g. selection sort or merge sort? • T() helps us look for better algorithms in our own code, without expensive implementation, testing, and measurement.

  19. (Wrong) Arguments against T(n) • Algorithms often perform much better on average than the worst case used in T() • quicksort is n log n on a “random” array, but n2 in the worse case • but for most algorithms, the worst case is a good predictor of its running time • average case analyses can be done, but they are harder mathematically continued

  20. Some people say: • “Who cares about running time? In a few years, machines will be so fast that even bad algorithms will be fast.” • History shows this argument to be wrong. As machines get faster, problem sizes get bigger. • Most interesting problems (e.g. computer vision, natural language processing) always require more resources • fast algorithms will always be needed continued

  21. Some people say: • "Benchmarking (running programs on a standard set of test cases) is easier." • This is sometimes true, but the benchmarks only give numbers for that particular machine, OS, compiler, computer language.

  22. T() is too Machine Dependent • We want T() to be the same for an algorithm independent of the machine where it is running. • This is not true since different machines (and OSes) execute instructions at different speeds. • Consider the loop example (slide 11) • on machine A, every instruction takes 1 "time unit" • the result is TA(n) = 5n + 3

  23. On machine B, every instruction takes 1 "time unit" except for multiplication, which takes 5 "time units". • The for loop executes n times • each loop carries out (in the worse case) 5 ops • test of j < n, if test, multiply, assign, j increment • total loop time = 9n • plus 3 ops at start and end • small assign (line 2), init of j (line 3), final j < n test • Total time TB(n) = 9n + 3 • running time is linear with the size of the array

  24. TA() = 4n + 3 and TB() = 9n +3 • These are both linear equations (which is good), but the constants are different (which is bad) • We want a T() notation that is independent of machines.

  25. 2. Big-Oh and Running Time • Big-Oh notation for T(n) ignores constant factors which depend on compiler/machine behaviour • that's good • Big-Oh simplifies the process of estimating the running time of programs • we don't need to count every code line • that's also good

  26. The Big-Oh value specifies running time independent of: • machine architecture • e.g. don’t consider the running speed of individual machine operations • machine load (usage) • e.g. time delays due to other users • compiler design effects • e.g. gcc versus Visual C

  27. Example • When we counted instructions for the loop example, we found: • TA() = 4n + 3 • TB() = 9n + 3 • The Big-Oh equation, O(), is based on the T(n) equation but ignores constants (which vary from machine to machine). This means for both machine A and B: T(n) is O(n)we say "T(n) is order n"

  28. More Examples • T(n) value Big Oh : O() • 10n2+ 50n+100 O(n2) • (n+1)2 O(n2) • n10 O(2n) • 5n3 + 1 O(n3) • These simplifications have a mathematical reason, which is explained in section 2.2. hard to understand

  29. 2.1. Is Big-Oh Useful? • O() ignores constant factors, which means it is a more general measure of running time for algorithms across different platforms/compilers. • It can be comparedwith Big-Oh values for other algorithms. • i.e. linear is better than polynomial and exponential, but worse than logarithmic • i.e. O(log n) < O(n) < O(n2) < O(2n)

  30. 2.2. Definition of Big-Oh • The connection between T() and O() is: • the T() = f(n) can be written as T(n) is O( g(n) ) • this means that g(n) is the most important thing in T()'s f(n) function when n is large • Example 1: • T(n) = 4n + 3 // the f() function is 4n + 3 • write as T(n) is O(n) // the g() function is n • Example 2: • T(n) = 9n + 3 // the f() function is 9n + 3 • write as T(n) is O(n) // the g() function is n continued

  31. More Formally • n0 and c are called witnesses to the relationship: T(n) = f(n) and T(n) is O(g(n) ) • In some textbooks this is written as: • f(n) is O(g(n)) // leave out the T(n) We write T(n) = f(n) as T(n) is O(g(n)) if there exist constants c > 0, n0 > 0 such that 0 f(n) c*g(n) for all nn0.

  32. O-notation as a Graph • O-notation gives an upper bound for a function to within a constant factor. We write T(n)=f(n) as T(n) is O(g(n)) if there are positive constants n0 and c such that at and to the right of n0, the value of f(n) always lies on or below c*g(n). above is

  33. Asymptotic Analysis • The fancy name for calculating big-Oh (and the other bounds mentioned later) is asymptotic analysis. • Asymptotic means "a curve whose distance to another curve tends to zero as the curves travel off to infinity" • this is seen by the limit of f(n)/c*g(n) curve, as n ∞: • 0 ≤ f(n) ≤ c*g(n) for all n ≥ n0 • so 0 ≤ f(n)/c*g(n) ≤ 1, such that f(n)/c*g(n)  a constant

  34. Set definition of O-notation O(g(n))= { f(n) : there exist constants c > 0, n0 > 0 such that 0 f(n) c*g(n) for all nn0} f(n) is in a set of functions that are less than or equal to g(n) This means that the c*g(n) curve can be an upper bound for many different f(n) curves

  35. Example 1 • T(n) = 10n2 + 50n + 100 • can be written asT(n) is O(n2) • Why? • f(n) = 10n2 + 50n + 100; g(n) = n2 • Witnesses: n0 = 1, c = 160 • then f(n)<= c*g(n), n >= 1so 10n2 + 50n + 100 <= 160 n2since 10n2 + 50n + 100 <= 10n2 + 50n2 + 100n2 <= 160 n2 Informally, the n2part isthe most important thing in the T() function

  36. T() and O() Graphed http://dlippman.imathas.com/ graphcalc/graphcalc.html T(n) T(n) is O(n2) (c g(n) == 160n2) above T(n) = 10n2 + 50n + 10 ( f(n) == 10n2 + 50n + 10) n0 == 1 n

  37. f(n)/c g(n)  constant as n  ∞ the equation will distance to the y 'curve' will approach 0.Why? (10n2 + 50n + 10) / (160n2) y = 1/16 = 0.0625 n

  38. Example 2 • T(n) = (n+1)2 • can be wriiten asT(n) is O(n2) • Why? • f(n) = (n+1)2; g(n) = n2 • Witnesses: n0 = 1, c = 4 • then f(n) <= c*g(n), n >= 1so (n+1)2 <= 4n2since n2 + 2n + 1 <= n2 + 2n2 + n2 <= 4n2

  39. T() and O() Graphed T(n) T(n) is O(n2) (c g(n) == 4n2) above T(n) = (n+1)2 (f(n) == (n+1)2 n0 == 1 n

  40. Example 3 • T(n) = n10 • can be written asT(n) is O(2n) • Why? • f(n) = n10 ; g(n) = 2n • Witnesses: n0 = 64, c = 1 • then f(n) <= c*g(n), n >= 64so n10 <= 2nsince10*log2 n <= n (by taking logs of both sides) which is true when n >= 64 (10*log2 64 == 10*6; 60 <= 64)

  41. n10 and 2n Graphed T(n) T(n) is O(2n) (c g(n) == 2n above T(n) = n10 f(n) == n10 (58.770, 4.915E17) n

  42. 2.4. Some Observations about O() • When choosing an O() approximation to T(), remember that: • constant factors do not matter • e.g. T(n) = (n+1)2 is O(n2) • low-order terms do not matter • e.g. T(n) = 10n2 + 50n + 100 is O(n2) • there are many possible witnesses because there are usually many O() graphs that are above the T() equation

  43. 2.5. Simplifying O() Expressions • Inside an O() expressions, always drop constant factors and low-order terms. • For example: • T(n) = 3n5+ 10n4 + n • T(n) is O(3n5) • but, T(n) is O(n5) is simpler and tighter • this means that the O() is closer to the T() curve

  44. 3. Calculating Big-Oh Directly • Up to now, I have calculated T(n) = f(n) by counting instructions (e.g. see the loop example), and then simplified T(n) to becomes T(n) is O(g(n)) • We can calculate the big-oh function, g(), directly, without counting instructions • easier and faster

  45. 3. Big-Oh for Programs • First decide on a size measure for the data in the program. This will become the n. • Data Type Possible Size Measure integer its valuestring its lengtharray its length

  46. 3.1. Building a Big-Oh Result • The Big-Oh value for a program is built up in stages by: • 1) Calculate the Big-Oh’s for all the simple statements in the program • e.g. assignment, arithmetic • 2) Then use those value to obtain the Big-Oh’s for the complex statements • e.g. blocks, for loops, if-statements

  47. Simple Statements (in C) • We assume that simple statements always take a constant amount of time to execute • written as O(1) • this is not a time unit (not 1 ms, not 1 microsec) • O(1) means a running time independent of the input size n • Kinds of simple statements: • assignment, break, continue, return, all library functions (e.g. putchar(),scanf()), arithmetic, boolean tests, array indexing

  48. Complex Statements • The Big-Oh value for a complex statement is a combination of the Big-Oh values of its component simple statements. • Kinds of complex statements: • blocks { ... } • conditionals: if-then-else, switch • loops: for, while, do-while continued

  49. 3.2. Structure Trees • The easiest way to see how complex statement timings are based on simple statements (and other complex statements) is by drawing a structure tree for the program.

  50. Example: binary conversion void main() { int i;(1) scanf(“%d”, &i);(2) while (i > 0) {(3) putchar(‘0’ + i%2);(4) i = i/2; }(5) putchar(‘\n’); }