TK3043 Analysis and Design of Algorithms

1. Introduction to Algorithms TK3043 Analysis and Design of Algorithms

2. WHAT IS AN ALGORITHM? • An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time. • As a computer professional, it is useful to know a standard set of important algorithms and to be able to design new algorithms as well as to analyze their efficiency.

3. HISTORICAL PERSPECTIVE • Euclid’s algorithm for finding the greatest common divisor • One of the oldest algorithms known (300 BC). • Muhammad ibn Musa al-Khwarizmi – 9th century mathematician • The Golden Principle of Al-Khwarizmi states that all complex problems of science must be and can be solved by means of five simple steps.

4. THE GOLDEN PRINCIPLE OF AL-KHWARIZMI 1. Break down each problem into a number of elemental or ‘atomic’ steps. An elemental or atomic step is one, which cannot be simplified any further. 2. Arrange all the elements or steps of the problem in an order or sequence, such that each element can be taken up and solved one at a time, without affecting other parts of the problem.

5. 3. Find a way to solve each of the elements separately. Because each element has been simplified to the maximum, it is very likely that the solution of an elemental step will itself be extremely simple or elemental making it available with relative ease. 4. Proceed to solve each element of the problem, either one at a time or several at a time, but in the correct order. 5. When all steps are solved, the original problem itself has also been solved.

6. NOTION OF ALGORITHM problem algorithm “computer” input output Algorithmic solution

7. EXAMPLE: SORTING • Statement of problem: • Input: A sequence of n numbers <a1, a2, …, an> • Output: A reordering of the input sequence <b1, b2, …, bn> so that bi ≤ bj whenever i < j

8. Sorting algorithms: • Selection sort? • Insertion sort? • Merge sort? • … problem algorithm “computer” output input <5, 3, 2, 8, 3> <2, 3, 3, 5, 8> • Example:

9. SELECTION SORT • Input: array a[0], …, a[n-1] • Output: array a sorted in non-decreasing order • Algorithm: for i=0 to n-1 swap a[i] with smallest of a[i],…a[n-1]

10. SOME WELL-KNOWN COMPUTATIONAL PROBLEMS • Sorting • Searching • Shortest paths in a graph • Minimum spanning tree • Primality testing • Traveling salesman problem • Knapsack problem • Chess • Towers of Hanoi • Program termination

11. BASIC ISSUES RELATED TO ALGORITHMS • How to design algorithms • How to express algorithms • Proving correctness of algorithms • Efficiency • Theoretical analysis • Empirical analysis • Optimality

12. EXAMPLE • Refer to map. • How to get from Petronas Taman Samudra to Sekolah Kebangsaan Usaha Ehsan?

13. ALGORITHM DESIGN STRATEGIES • Brute force • Divide and conquer • Decrease and conquer • Transform and conquer • Greedy approach • Dynamic programming • Backtracking and Branch and bound • Space and time tradeoffs

14. ANALYSIS OF ALGORITHMS • How good is the algorithm? • Correctness • Time efficiency • Space efficiency • Simplicity • Generality • Does there exist a better algorithm? • Lower bounds • Optimality

15. WHAT IS AN ALGORITHM? • Recipe, process, method, technique, procedure, routine,… with following requirements: 1. Finiteness terminates after a finite number of steps 2. Definiteness rigorously and unambiguously specified 3. Input valid inputs are clearly specified 4. Output can be proved to produce the correct output given a valid input 5. Effectiveness steps are sufficiently simple and basic

16. WHY STUDY ALGORITHMS • Theoretical importance • The core of computer science • Practical importance • A practitioner’s toolkit of known algorithms • Framework for designing and analyzing algorithms for new problems

17. EUCLID’S ALGORITHM • Problem: Find gcd(m,n), the greatest common divisor of two nonnegative, not both zero integers m and n • Examples: gcd(60,24) = 12 gcd(60,0) = 60 gcd(0,0) = ?

18. EUCLID’S ALGORITHM • Euclid’s algorithm is based on repeated application of equality gcd(m,n) = gcd(n, m mod n) until the second number becomes 0, which makes the problem trivial. • Example: gcd(60,24) = gcd(24,12) = gcd(12,0) = 12

19. TWO DESCRIPTIONS OF EUCLID’S ALGORITHM • Step 1 If n = 0, return m and stop; otherwise go to Step 2 • Step 2 Divide m by n and assign the value fo the remainder to r • Step 3 Assign the value of n to m and the value of r to n. Go to Step 1. while n ≠ 0 { r ← m mod n m← n n ← r } return m

20. OTHER METHODS FOR COMPUTING gcd(m,n) • Consecutive integer checking algorithm • Step 1 Assign the value of min{m,n} to t • Step 2 Divide m by t. If the remainder is 0, go to Step 3; otherwise, go to Step 4 • Step 3 Divide n by t. If the remainder is 0, return t and stop; otherwise, go to Step 4 • Step 4 Decrease t by 1 and go to Step 2

21. OTHER METHODS FOR COMPUTING gcd(m,n) (CONT.) • Middle-school procedure • Step 1 Find the prime factorization of m • Step 2 Find the prime factorization of n • Step 3 Find all the common prime factors • Step 4 Compute the product of all the common prime factorsand return it as gcd(m,n) Is this an algorithm?

22. IMPORTANT PROBLEM TYPES • sorting • searching • string processing • graph problems • combinatorial problems • geometric problems • numerical problems

23. FUNDAMENTAL DATA STRUCTURES • list • array • linked list • string • stack • queue • priority queue • graph • tree • set and dictionary