Algorithm Efficiency and Sorting

1. Algorithm Efficiency and Sorting Data Structure & Algorithm

2. Measuring the Efficiency of Algorithms • Analysis of algorithms • Provides tools for contrasting the efficiency of different methods of solution • Time efficiency, space efficiency

3. The Execution Time of Algorithms • Counting an algorithm's operations is a way to assess its time efficiency • An algorithm’s execution time is related to the number of operations it requires • Example: Traversal of a linked list of n nodes need n steps for reading the nodes or writing to the output screen • Example: For loop with n data?

4. Algorithm Growth Rates • An algorithm’s time requirements can be measured as a function of the problem size • Number of nodes in a linked list • Size of an array • Number of items in a stack • Algorithm efficiency is typically a concern for large problems only

5. Algorithm Growth Rates Figure 9-1 Time requirements as a function of the problem size n • Algorithm A requires time proportional to n2 • Algorithm B requires time proportional to n

6. Algorithm Growth Rates • An algorithm’s growth rate • Enables the comparison of one algorithm with another • Algorithm A requires time proportional to n2 • Algorithm B requires time proportional to n • Algorithm B is faster than Algorithm A • n2 and n are growth-rate functions • Algorithm A is O(n2) - order n2 • Algorithm B is O(n) - order n • Big O notation • Also called complexity time

7. Big O Notation • Big ‘O’ notation is denoted as O(acc)O - order acc - class of algorithm complexity that may consist of 1, logxn, n, n logxn, n2, …

8. Order-of-Magnitude Analysis and Big O Notation • Order of growth of some common functions • O(1) < O(log2n) < O(n) < O(n * log2n) < O(n2) < O(n3) < O(2n) • Properties of growth-rate functions • O(n3 + 3n) is O(n3): ignore low-order terms • O(5 f(n)) = O(f(n)): ignore multiplicative constant in the high-order term

9. Order-of-Magnitude Analysis and Big O Notation • Worst-case analysis • A determination of the maximum amount of time that an algorithm requires to solve problems of size n • Average-case analysis • A determination of the average amount of time that an algorithm requires to solve problems of size n • Best-case analysis • A determination of the minimum amount of time that an algorithm requires to solve problems of size n

10. Keeping Your Perspective • Frequency of operations • When choosing an ADT’s implementation, consider how frequently particular ADT operations occur in a given application • Bad big O: • Frequent use - but smaller data – OK! • Frequent use – bigger amount of data – Not OK! • Bigger amount of data, not frequent use – OK!

11. Keeping Your Perspective • If the problem size is always small, you can probably ignore an algorithm’s efficiency • Order-of-magnitude analysis focuses on large problems • Weigh the trade-offs between an algorithm’s time requirements and its memory requirements • Compare algorithms for both style and efficiency

12. Big O Notation

13. Big O Notation

14. Big O Notation

15. Big O Notation

16. Determine the complexity time of algorithm • can be determined - theoretically – by calculation - practically – by experiment /implementation

17. Determine the complexity time of algorithm - practically • Implement the algorithms in any programming language and run the programs • Depend on the compiler, computer, data input and programming style.

18. Determine the complexity time of algorithm - theoretically • The complexity time is related to the number of steps /operations. • Complexity time can be determined by 1. Count the number of steps and then find the class of complexity. Or 2. Find the complexity time for each steps and then count the total.

19. Determine the number of steps • The following algorithm is categorized as O(n). int counter = 1; int i = 0; for (i = 1; i <= n; i++) { cout << "Arahan cout kali ke " << counter << "\n"; counter++; }

20. Determine the number of steps

21. Determine the number of steps • Statement 3, 4 & 5 are the loop control and can be assumed as one statement.

22. Determine the number of steps- summation series • statement 3, 6 & 7 are in the repetition structure. • It can be expressed by summation series n  = f(1) + f(2) + . . . + f(n) = n f(i) i = 1 • f(i) – statement executed in the loop

23. Determine the number of steps- summation series • example:- if n = 5, i = 1 5  = f(1) + f(2) + f(3) + f(4) + f(5) = 5 f(i) i = 1 The statement that represented by f(i) will be repeated 5 time = n times = O (n)

24. Determine the number of steps- summation series 1  = 1 f(i) i=1 1  = 1 f(i) i=1 n  = n f(i) i=1 n 1  .  = n . 1 = n f(i) f(i) i=1 i=1 n 1  . f(i)  = n . 1 = n f(i) i=1 i=1

25. Determine the number of steps- summation series • Total steps:1 + 1 + n + n + n = 2 + 3n • Consider the largest factor. • Algorithm complexity can be categorized as O(n)

26. Determine the number of steps- through actual counting

27. Determine the number of steps- through actual counting • Statements 1 and 2 executed only once, 0 (1) • Statement 3 linear execution, 0 (n) • Statements 4, 5, 6  executed only once, 0 (1) HOWEVER because those statements is in a for loop, execution is linear, 0 (n) • Algorithm complexity time:- T(n) = 1 + 1 + n + n + n = 2 + 3n = O (n)

28. Shortcut formula for loop total steps Total Steps = b-a+l b = loop final conditional value a = loop initial conditional value l = constant value for loop increment Example – using the statement 3 (slide 38th) b = n a = 1 l = 1 Total steps = b – a + 1 = n – 1 + 1 = n

29. Jum bil langkah =1(nilai malar) ; masa kerumitan = O(1)

30. Samb… Jum bil langkah =10(nilai malar) ; masa kerumitan = O(1) Jum bil langkah =2n(nilai yg b’gantung pd n) ; masa kerumitan = O(n)

31. Jum bil langkah =2(n-1)(nilai yg b’gantung pd n) ; masa kerumitan = O(n)

32. Samb… Jum bil langkah =2(n-1)(nilai yg b’gantung pd n) ; masa kerumitan = O(n) Jum bil langkah =n+2n2(nilai yg b’gantung pd n2) ; masa kerumitan = O(n2)

33. Determine the number of steps - exercise • Count the number of steps and find the Big ‘O’ notation for the following algorithm int counter = 1; int i = 0; int j = 1; for (i = 3; i <= n; i = i * 3) { while (j <= n) { cout << "Arahan cout kali ke " << counter << "\n"; counter++; j++; } }

34. Determine the number of steps - solution 1  = 1 f(i) i=1 1  = 1 f(i) i=1 1  = 1 f(i) i=1 n  = f(3) + f(9) + f(27) + … + f(n) = log3n f(i) i=3 n n  .  = log3n . n f(i) f(i) i=3 j=1

35. Determine the number of steps - solution 1 n n   .  . = log3n . n . 1 f(i) f(i) f(i) i=1 i=3 j=1 1 n n   .  . = log3n . n . 1 f(i) f(i) f(i) i=1 i=3 j=1 1 n n   .  . = log3n . n . 1 f(i) f(i) f(i) i=1 i=3 j=1

36. Determine the number of steps - solution Total steps: => 1 + 1+ 1 + log3n + log3n . n + log3n . n . 1 + log3n . n . 1 + log3n . n . 1 => 3 + log3n + log3n . n + log3n . n + log3n . n + log3n . n => 3 + log3n + 4n log3n

37. Determine the number of steps - solution 3 + log3n + 4n log3n • Consider the largest factor (4n log3n) • and remove the coefficient (n log3n) • In asymptotic classification, the base of the log can be omitted as shown in this formula: logan = logbn / logba • Thus, log3n = log2n / log23 = log2n / 1.58… • Remove the coefficient 1/1.58.. • So we get the complexity time of the algorithm is O(n log2n)