Introduction to Algorithms

1. Introduction to Algorithms Rabie A. Ramadan rabie@rabieramadan.org http://www. rabieramadan.org Some of the sides are exported from different sources to clarify the topic

2. About my self Rabie A. Ramadan My website and publications http://www.rabieramadan.org

3. ClassRules Attendance is a vey important Assignments must be delivered on time All assignments are individual assignments unless it is clearly stated that you can work on groups. Assignments or part of them that are copied (including the programming assignments) will be punished by -2 assignments.

4. ClassRules Projects There will be a term project The details of the project will be announced on the class website http://rabieramadan.org/classes/2012/algorithms/    Only 4 or less students per group    Please no exception at all.

5. ClassRules You can bring anything to drink but NO FOOD PLEASE  When you come in , DO NOT knock on the door as well as when you leave I do not take attendance every class but if you miss one , it might greatly affect your grade

6. Text Book Introduction to the Design and Analysis of Algorithms, 2nd edition, 2007 by A. Levitin. Other sources Books Research papers …

7. Introduction

8. What is an algorithm? An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time. problem algorithm “computer” input output

9. What is an algorithm? Procedure for getting answers to a specific problem Problem solving strategy even if computers are not involved Scope is limited  I can not have a procedure for you to have very happy life or becoming rich and famous

10. Notion/Concept of algorithm and problem Nonambiguity for each step Range of inputs is specified carefully The same algorithm can be represented in several ways Several algorithms for the same problem may exist Several algorithms with different ideas can solve the same problem with different speed. problem algorithm “computer” input output

11. Example of computational problem: sorting • Statement of problem: • Input: A sequence of n numbers <a1, a2, …, an> • Output: A reordering of the input sequence <a´1, a´2, …, a´n> so that a´i≤a´j whenever i < j • Instance: The sequence <5, 3, 2, 8, 3> • Algorithms: • Selection sort • Insertion sort • Merge sort • (many others)

12. Sorting Searching Shortest paths in a graph Minimum spanning tree Primality testing Traveling salesman problem Knapsack problem Chess Towers of Hanoi Some Well-known Computational Problems Some of problems don’t have efficient algorithms, or algorithms at all!

13. How to design algorithms How to express algorithms Proving correctness Efficiency (or complexity) analysis Theoretical analysis Empirical/ experimental analysis Optimality Basic Issues Related to Algorithms

14. Brute force Try all possible combinations Divide and conquer breaking down a problem into two or more sub-problems of the same (or related) type, until these become simple enough to be solved directly. Decrease and conquer Change an instance into one smaller instance of the problem. Solve the smaller instance. Convert the solution of the smaller instance into a solution for the larger instance. Algorithm design strategies • Transform and conquer • a simpler instance of the same problem, or • a different representation of the same problem, or • an instance of a different problem • Greedy approach • The problem could be solved in iterations • Dynamic programming • An instance is solved using the solutions for smaller instances. • The solution for a smaller instance might be needed multiple times. • The solutions to smaller instances are stored in a table, so that each smaller instance is solved only once. • Additional space is used to save time. • Backtracking and branch-and-bound

15. How good is the algorithm? Correctness Time efficiency Space efficiency Does there exist a better algorithm? Lower bounds Optimality Analysis of Algorithms

16. Recipe, process, method, technique, procedure, routine,… with the following requirements: Finiteness terminates after a finite number of steps Definiteness rigorously and unambiguously specified Clearlyspecifiedinput valid inputs are clearly specified Clearlyspecified/expectedoutput can be proved to produce the correct output given a valid input Effectiveness steps are sufficiently simple and basic What is an algorithm?

17. Algorithms Examples

18. Addition and Subtraction

19. Add the following (try it from left to right): Simply , think of 32 add ( 30+2) ; then add it to 47 ; then add 2 to the results 2- Digit Addition

20. Add the following (try it from left to right): 2- Digit Addition (cont. )

21. Try this on your own: 2- Digit Addition (cont. )

22. Add the following : Simply : Add the hundreds digit of the second number to the first number Add the tens digit to the result Add the unit digit to the result 3- Digit Addition

23. Try this by yourself : 3- Digit Addition

24. Try this one : Try it this way (subtraction instead of addition) 3- Digit Addition

25. Find and write the best algorithm to sum up the numbers from 1 to 100 ? What is the complexity of your algorithm ? Write an algorithm to add 3-digit numbers (take the sign in your account- Allow the addition of positive or negative numbers)? Compare your algorithm to the computer does in similar cases? Part of your first assignment

26. 2-Digit Subtraction

27. 3-Digit Subtraction

28. More about numbersEuclid’s Algorithm

29. Problem: Find gcd(m,n), the greatest common divisor of two nonnegative, not both zero integers m and n Examples: gcd(60,24) = 12, gcd(60,0) = 60, gcd(0,0) = ? Euclid’s algorithm is based on repeated application of equality gcd(m,n) = gcd(n, m mod n) until the second number becomes 0, which makes the problem trivial. Example: gcd(60,24) = gcd(24,12) = gcd(12,0) = 12 Euclid’s Algorithm

30. Descriptions of Euclid’s algorithm Step 1 If n = 0, return m and stop; otherwise go to Step 2 Step 2 Divide m by n and assign the value of the remainder to r Step 3 Assign the value of n to m and the value of r to n. Go to Step 1. while n ≠ 0 do r ← m mod n m← n n ← r return m

31. Other methods for gcd(m , n) [cont.] Middle-school procedure Step 1 Find the prime factorization of m Step 2 Find the prime factorization of n Step 3 Find all the common prime factors Step 4 Compute the product of all the common prime factors and return it as gcd(m,n) E.g. find gcd(60, 24) Step 1: 60 = 2 . 2 . 3 . 5 Step 2: 25 = 2 . 2 . 2 . 3 Step 3: 2 . 2 . 3 Step 4: 2 . 2 . 3 = 12 • Is this an algorithm? • not qualified  how do you get the prime factorization? How do you get common elements in two stored lists? • How efficient is it? • Time complexity: O(sqrt(n))

32. More about numbersSieve of Eratosthenes

33. Sieve of Eratosthenes To find all the prime numbers less than or equal to a given integer n by Eratosthenes' method: Create a list of consecutive integers from two to n: (2, 3, 4, ..., n). Initially, let p equal 2, the first prime number. Strike from the list all multiples of p less than or equal to n. (2p, 3p, 4p, etc.) Find the first number remaining on the list greater than p (this number is the next prime); replace p with this number. Repeat steps 3 and 4 until p2 is greater than n. All the remaining numbers on the list are prime.

34. Input: Integer n ≥ 2 Output: List of primes less than or equal to n for p ← 2 to n do A[p] ← p for p ← 2 to n do if A[p]  0 //p hasn’t been previously eliminated from the listj ← p*p while j ≤ n do A[j] ← 0 //mark element as eliminated j ← j+ p Example: 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Sieve of Eratosthenes

35. Steps of Designing and Analyzing an Algorithm Understand the Problem Decide on the machine to be used (sequential vs. parallel) Design an algorithm: select a specific techniques such as divide and conquer . Also, decide on how do you represent the algorithm for instance pseudocode , flowchart , etc.. Prove correctness may be by example or mathematically Analyze the algorithm for instance in space and time efficiency

36. Sorting Searching String processing Graph problems Combinatorial problems Geometric problems Numerical problems Important Problem Types

37. Rearrange the items of a given list in ascending order. Input: A sequence of n numbers <a1, a2, …, an> Output: A reordering <a´1, a´2, …, a´n> of the input sequence such that a´1≤ a´2 ≤ … ≤a´n. Why sorting? Help searching Algorithms often use sorting as a key subroutine. Sorting key A specially chosen piece of information used to guide sorting. E.g., sort student records by names. Sorting (I)

38. Examples of sorting algorithms Selection sort Bubble sort Insertion sort Merge sort ….. Evaluate sorting algorithm complexity: the number of key comparisons. Two properties Stability: A sorting algorithm is called stable if it preserves the relative order of any two equal elements in its input. In place: A sorting algorithm is in place if it does not require extra memory, except, possibly for a few memory units. Sorting (II)

39. Algorithm SelectionSort(A[0..n-1]) //The algorithm sorts a given array by selection sort //Input: An array A[0..n-1] of orderable elements //Output: Array A[0..n-1] sorted in ascending order for i  0 to n – 2 do min  i for j  i + 1 to n – 1 do if A[j] < A[min] min  j swap A[i] and A[min] Selection Sort

40. Find a given value, called a search key, in a given set. Examples of searching algorithms Sequential search Binary search … Searching Input: sorted array a_i < … < a_j and key x; m (i+j)/2; while i < j and x != a_m do if x < a_m then j  m-1 else i  m+1; if x = a_m then output a_m; Time: O(log n)

41. A string is a sequence of characters from an alphabet. Text strings: letters, numbers, and special characters. String matching: searching for a given word/pattern in a text. StringProcessing • Examples: • searching for a word or phrase on WWW or in a Word document • searching for a short read in the reference genomic sequence

42. Given a text stringT of length n and a pattern stringP of length m, the exact string matching problem is to find all occurrences of P in T. Example: T=“AGCTTGA” P=“GCT” Applications: Searchingkeywords in a file Searching engines (like Google and Openfind) Database searching (GenBank) More string matching algorithms (with source codes): http://www-igm.univ-mlv.fr/~lecroq/string/ StringMatching

43. Brute Force algorithm The brute force algorithm consists in checking, at all positions in the text between 0 and n-m, whether an occurrence of the pattern starts there or not. Then, after each attempt, it shifts the pattern by exactly one position to the right. StringMatching Time: O(mn) where m=|P| and n=|T|.

44. No preprocessing phase; Constant extra space needed; Always shifts the window by exactly 1 position to the right; Comparisons can be done in any order; Searching phase in O(mn) time complexity; Main Features

45. Report three different methods for string matching and write a program to test them? Your Second part of assignment

46. Informal definition A graph is a collection of points called vertices, some of which are connected by line segments called edges. Modeling Real-life Problems Modeling WWW Communication networks Project scheduling … Examples of Graph Algorithms Graph traversal algorithms Shortest-path algorithms …….. GraphProblems

47. list array linked list string stack queue priorityqueue Fundamental data structures • graph • tree and binary tree

48. Arrays A sequence of n items of the same data type that are stored contiguously in computer memory and made accessible by specifying a value of the array’s index. LinkedList A sequence of zero or more nodes each containing two kinds of information: some data and one or more links called pointers to other nodes of the linked list. Singly linked list (next pointer) Doubly linked list (next + previous pointers) … . a1 a2 an LinearDataStructures • Arrays • fixed length (need preliminary reservation of memory) • contiguous memory locations • direct access • Insert/delete • LinkedLists • dynamic length • arbitrary memory locations • access by following links • Insert/delete

49. Stacks A stack of plates insertion/deletion can be done only at the top. LIFO Two operations (push and pop) Queues A queue of customers waiting for services Insertion/enqueue from the rear and deletion/dequeue from the front. FIFO Twooperations (enqueue and dequeue) StacksandQueues

50. PriorityQueueandHeap • Priorityqueues (implemented using heaps) • A data structure for maintaining a set of elements, each associated with a key/priority, with the following operations: • Finding the element with the highest priority • Deleting the element with the highest priority • Inserting a new element • Scheduling jobs on a shared computer