Download Presentation
## Algorithms

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Algorithms**• Readings: [SG] Ch. 2 & 3 • Chapter Outline: • Chapter Goals • What are Algorithms • Real Life Examples (origami, recipes) • Definition • Example: A = B + C • Expressing Algorithms – Pseudo-Code • Simple Algorithms • Recursive Algorithms • Time Complexity of Algorithms**Algorithms**• Computing devices are dumb • How to explain to a dumb mechanical / computing device how to solve a problem • How to solve a problem using “a small, basic set of primitive instructions”. • Complexity of Solving Problems.**1. Goals of Algorithm Study**• To develop framework for instructing computer to perform tasks • To introduce notion of algorithm as means of specifying how to solve a problem • To introduce and appreciate approaches for defining and solving very complex tasks in terms of simpler tasks;**Chapter Outline:**• Chapter Goals • What are Algorithms • Real Life Examples (origami, recipes) • Definition of Algorithm • Example: A = B + C • Expressing Algorithms – Pseudo-Code • Simple Algorithms • Recursive Algorithms • Time Complexity of Algorithms**2. Computer Science and Algorithms…**• Computer Science can be considered… as study of algorithms including • their formal properties • Their hardware and software realisations • Their applications to diverse areas**Algorithms: Real Life Examples**• Many Real-Life Examples • Cooking: Recipe for preparing a dish • Origami: The Art of Paper Folding • Directions: How to go to Changi Airport • Remember: • Framework: How to give instructions; • Alg: The actual step-by-step instructions; • Abstraction: Decomposing / Simplifying**Real Life Example: Cooking**• Framework: “Cooking or Recipe language” • Algorithm: “Recipe for a Dish” (step by step instructions on how to cook a dish) • Problem Decomposition • Preparing Ingredients; • Preparing Sauce;**Real Life Example: Origami**• Framework: “Origami or Paper-Folding language” • Algorithm: “Sequence of Paper-Folding Instructions” (step by step instructions for each fold) • Problem Decomposition • Start with a Bird Base; … • Finish the Head; • Finish the Legs; Finish the Tail;**Real Life Examples: Issues**• Problems/Difficulties: • Imprecise Instructions; • Job can often be done even if instructions are not followed precisely • Modifications may be done by the person following the instructions; • But, NOT for a Computer • Needs to told PRECISELY what to do; • Instructions must be PRECISE; • Cannot be vague or ambiguous**Definition of Algorithm:**• An algorithm for solving a problem “a finite sequence of unambiguous, executable steps or instructions, which, if followed would ultimately terminate and give the solution of the problem”. • Note the keywords: • Finite set of steps; • Unambiguous; • Executable; • Terminates;**Algorithm Examples?**• Problem 1: What is the largest integer INPUT: All the integers { … -2, -1, 0, 1, 2, … } OUTPUT: The largest integer Algorithm: • Arrange all the integers in a list in decreasing order; • MAX = first number in the list; • Print out MAX; • WHY is the above NOT an Algorithm? • (Hint: How many integers are there?) • Problem 2: Who is the tallest women in the world? • Algorithm: Tutorial...**Example: Adding two (n-digit) numbers**• Input: Two positive m-digit decimal numbers (a and b) am, am-1, …., a1 bm, bm-1, …., b1 • Output: The sum c = a + b cm+1, cm, cm-1, …., c1 (Note: In the textbook, it was am-1,…,a1,a0 …)**How to “derive” the algorithm**• Adding is something we all know • done it a thousand times, know it “by heart” • How do we give the algorithm? • A step-by-step instruction • to a dumb machine • Try an example: 3 4 9 2 8 1 5 7 “Imagine you looking at yourself solving it”**Step 1: Set the value of carry to 0**Step 2: Set the value of i to 1. Step 3: Repeat steps 4 through 6 until the value of i is > m. Step 4: Add ai and bi to the current value of carry, to get xi. Step 5: If xi < 10 then let ci=xi and reset carry to 0. Else (* i.e. xi 10 *) let ci=xi - 10 and reset carry to 1. Step 6: Increase the value of i by 1. Step 7: Set cm+1 to the value of carry. Step 8: Print the final answer cm+1, cm, …., c1 Step 9: Stop. Algorithm: Finding sum of A & B**Chapter Outline:**• Chapter Goals • What are Algorithms • Expressing Algorithms – Pseudo-Code • Communicating Alg to computer • Pseudo-Code • Primitive Operations and examples • Variables and Arrays • Algorithm C=A+B in pseudo-code • Simple Algorithms • Recursive Algorithms • Time Complexity of Algorithms**3. Expressing Algorithms to Computer**• To communicate algorithm to computer • Need way to “represent” the algorithm • Cannot use English • Can use computer language • machine language and • programming languages (Java, Pascal, C) • But, these are too tedious (&technical) • Use Pseudo-Code instead…**Pseudo-Code to express Algorithms**• Pseudo-Code • Mixture of computer language and English • Somewhere in between • precise enough to describe what is meant without being too tediuos • Examples: • Let c be 0; • Sort the list of numbers in increasing order; • Need to know both • syntax – representation • semantics – meaning**Primitive Operations**• To describe an algorithm, we need some well-defined programming primitives • Assignment primitive: • assignment statement, input/output statements • Conditional primitive: • if statement • case statement • Looping (iterative) primitive: • for loop, • while loop, • Statements are executed one-by-one**true**false condition? Step A Step B Conditional Primitives… • if statement • to take different actions based on condition • Syntax if (condition) then (Step A) else (Step B) endif if (condition) then (Step A) endif • Semantics**Examples -- (if statement)…**Let mark be the total-mark obtained if (mark < 40) then (print “Student fail”) else (print “Student pass”) endif … read in mark (*from the terminal*) if (mark < 40) then (Grade “F”) else if (mark < 50) then (Grade “D”) else if (mark < 60) then (Grade “C”) else if (mark < 70) then (Grade “B”) else if (mark < 80) then (Grade “A”); endif print “Student grade is”, Grade …**the while-loop**loop a “variable” number of times Syntax while (condition) do (some sequence of statements) endwhile Semantics… false condition? true Some sequence of statements; Looping Primitive – while-loop**First, the for-loop**loop a “fixed” or (pre-determined) number of times Syntax for j a to b do (some sequence of statements) endfor Semantics… j a; false (j <= b)? true Some sequence of statements; j j+1; Looping Primitive – for-loop**“Exercising the alg”: for and while**for j 1 to 4 do print 2*j; endfor print “--- Done ---” j 1; while (j <= 4) do print 2*j; j j + 1; endwhile print “--- Done ---” Output: 2 4 6 8 --- Done --- Output: 2 4 6 8 --- Done ---**Variables and Arrays…**• In the computer, each variable is assigned a storage “box” • can store one number at any time • eg: sum, j, carry • Arrays: • Often deal with many numbers • Such as A1, A2, A3, … , A100 • Store as an “array” A[1], A[2], … , A[100] • we treat each of them as a variable, • each is assigned a storage “box”**We can re-write the C=A+B algorithm as follows:**Alg. to Compute C = A + B: (*sum two big numbers*) carry 0; for i 1 to m do x[i] a[i] + b[i] + carry ; if (x[i] < 10) then ( c[i] x[i]; carry 0; ) else ( c[i] x[i] – 10; carry 1; ) endfor; c[m+1] carry; Print c[m+1], c[m], …., c[1] Algorithm: A = B + C (in pseudo-code)**Chapter Outline:**• Chapter Goals • What are Algorithms • Expressing Algorithms – Pseudo-Code • Simple Algorithms • Simple iterative algorithms Computing Sum, Find Max/Min • Modular Program Design • Divisibility and Prime Numbers • Recursive Algorithms • Time Complexity of Algorithms**Simple iterative algorithm: Sum**• Given: List of numbers: A1, A2, A3, …., An • Output: To compute the sum of the numbers Note: Store numbers in array A[1], A[2], … , A[n] Sum(A, n); begin Sum_sf 0; k 1; while (k <= n) do Sum_sf Sum_sf + A[k]; k k + 1; endwhile Sum Sum_sf; Print “Sum is”, Sum end;**Exercising Algorithm Sum:**A[1] A[2] A[3] A[4] A[5] A[6] n=6 2 5 10 3 12 24 Input: k Sum-sf Sum ? 0 ? 1 2 ? 2 7 ? 3 17 ? 4 20 ? 5 32 ? 6 56 ? 6 56 56 Processing: Output: Sum is 56**Algorithm for Sum (with for-loop)**• We can also use a while-loop instead of a for loop. Sum(A, n); (* Find the sum of A1, A2,…, An. *) begin Sum_sf 0; for k 1 to n do Sum_sf Sum_sf + A[k]; endfor Sum Sum_sf; Print “Sum is”, Sum end; • HW: (a) Note the differences… (b) Modify it to compute the average?**Remarks about the iterative algorithm…**• Note the three stages: • Initialization • Set some values at the beginning • Iteration • This is the KEY STEP • Where most of work is done • Post-Processing or Cleanup • Can use this setup for other problems • Calculating average, sum-of-squares • Finding max, min; Searching for a number,**Modular Program Design**• Software are complex • HUGE (millions of lines of code) eg: Linux, Outlook • COMPLEX; eg: Flight simulator • Idea: Divide-and-Conquer Method • Complex tasks can be divided and each part solved separately and combined later. • Modular Program Design • Divide big programs into smaller modules • The smaller parts are • called modules, subroutines, or procedures • Design, implement, and test separately • Modularity, Abstraction, Division of Labour • Simplifies process of writing alg/programs**Simple Algorithms: Prime Numbers**• Algorithm to determine if n is prime Given: A positive integer n Question: Is n a prime number? • What do we know? “A number n is prime iffn has no positive factors except 1 and n” • Idea: Express it “algorithmically” “n is not divisible by any positive number k such that 1 < k < n.” or k=2,3,4,…,(n-1) • What we already have: • A module Divisible(m,n) to check divisibility • We can use it as a primitive**Pseudo-Code for Prime:**• Note: Prime uses the module Divisible(k,n) • Exercise it with: • Prime (5); Prime (4); • Prime (55); Prime (41); Prime(n) (* To determine if n is prime *) begin for k 2 to (n-1) do if Divisible(k,n) then (Print “False” & exit) else (* Do nothing *) endfor Print “True” & Stop end;**A Simple Module: Divisibility**• A module (procedure) for divisibility Given: Two positive integers m and n Question: Is n divisible by m? or Algorithm to compute Divisible(m,n) • Algorithm Idea: “A positive integer n is divisible by m iff for some positive integer k n, m*k = n.”**Module for Divisibility:**• Algorithm (in pseudo-code) • Exercise it with: • Divisible (3, 9); Divisible (3, 5); Divisible (3,101); Divisible(m,n) (* To compute if n is divisible by m *) begin D false; (* assume false, first *) for k 1 to n do if (m*k = n) then (Dtrue; exit-loop) else (* Do nothing *) endfor Divisible D; (* true or false *) end;**Chapter Outline:**• Chapter Goals • What are Algorithms • Expressing Algorithms – Pseudo-Code • Simple Algorithms • Recursive Algorithms • Time Complexity of Algorithms • Sequential search algorithm • Binary search algorithm • Analysis of Time Complexity • Summary…**Search: sequential search algorithm**• Given: List of numbers: A1, A2, A3, …., An and a query number x; • Question: Search for x in the list; Sequential-Search(A, n, x); (* Search for x in A1, A2,…, An. *) begin for k 1 to n do if (x = A[k]) then (Print “Yes”; Stop) else (* do nothing *) endfor Print “No”; Stop end;**Remarks on Sequential Search Alg**• Analogy: House-to-house search… • How fast is it to search for x? • How many comparisons do we need? • Example: 13, 38, 19, 74, 76, 14, 12, 38, 22, 55 • When x=14, need 6 comparisons • When x=13, need only 1 comparison BEST CASE • When x=55, need 10 comparisons WORST CASE • When x=5, need 10 comparisons WORST CASE • In general, given n numbers, A1,…,An • Best Case: 1 comparison • Worst Case: n comparisons • Average Case: (n+1)/2 comparisons**Binary Search**• If the List is sorted, that is A1 A2 A3 …. An • Then, we can do better, • actually a lot better….**1, 4, 9, 11, 14, 43, 78**Binary Search**Binary Search (How fast is it?)**• Imagine n=100 • After 1 step, size is 50 • After 2 steps, size is 25 • After 3 steps, size is 12 • After 4 steps, size is 6 • After 5 steps, size is 3 • After 6 steps, size is 1 • DONE!! • What if n=1000? 1,000,000?**Input: Sorted List A1 A2 A3 …. An**A number x. Question: Is x in the list? Binary-Search(A,n,x); 1. First 1 Last n 2. While ( First Last ) do mid (first+last) / 2 If ( x = A[mid] ) then (output Yes and Stop) else If ( x < A[mid] ) then Last mid-1 else If ( x > A[mid] ) then First mid+1 EndWhile 3. Output False and Stop 4. End Binary Search Algorithm**Time Complexity Analysis**• Sequential Search (Alg): • Worst Case: n comparisons • Best Case: 1 comparison • Avg Case: n/2 comparisons • Binary Search (Alg): • Worst Case: log2 n comparisons • Best Case: 1 comparison • Avg Case: about log2 n comparisons • How to get the Average Case? • using mathematical analysis…**Complexity of Algorithm…**• Logarithmic Time Algorithm • Binary Search • A Linear Time Algorithm • Algorithm Sum(A,n) -- O(n) time • Algorithm Sequential-Search(A,n,x) – O(n) time • A Quadratic Time Algorithm • Simple Median-Find (T2-Q3) • An Exponential Time Algorithm • All-Subsets(A,n)**Chapter Outline:**• Chapter Goals • What are Algorithms • Expressing Algorithms – Pseudo-Code • Simple Algorithms • Recursive Algorithms • Recursion – the idea • Fibonacci sequence • Tower of Hanoi • Time Complexity of Algorithms**5. Recursion**• A problem solving method of “decomposing bigger problems into smaller sub-problems that are identical to itself.” • General Idea: • Solve simplest (smallest) cases DIRECTLY • usually these are very easy to solve • Solve bigger problems using smaller sub-problems • that are identical to itself (but smaller and simpler) • Abstraction: • To solve a given problem, we first assume that we ALREADY know how to solve it for smaller instances!! • Dictionary definition: recursion see recursion**Example: Fibonacci Numbers…**• Definition of Fibonacci numbers • F1 = 1, • F2 = 1, • for n>2, Fn = Fn-1 + Fn-2 • Problem: Compute Fn for any n. • The above is a recursive definition. • Fn is computed in-terms of itself • actually, smaller copies of itself – Fn-1 and Fn-2 • Actually, Not difficult: F3 = 1 + 1 = 2 F6 = 5 + 3 = 8 F9 = 21 + 13 = 34 F4 = 2 + 1 = 3 F7 = 8 + 5 = 13 F10 = 34 + 21 = 55 F5 = 3 + 2 = 5 F8 = 13 + 8 = 21 F11 = 55 + 34 = 89 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, …**Fibonacci Numbers: Recursive alg**• The above is a recursive algorithm • It is simple to understand and elegant! • But, very SLOW Fibonacci(n) (* Recursive, SLOW *) begin if (n=1) or (n=2) then Fibonacci(n) 1 (*simple case*) else Fibonacci(n) Fibonacci(n-1) + Fibonacci(n-2) endif end;**F(6)**F(5) F(4) F(4) F(3) F(3) F(2) F(2) F(3) F(2) F(2) F(1) F(1) F(2) F(1) Recursive Fibonacci Alg -- Remarks • How slow is it? • Eg: To compute F(6)… • HW: Can we compute it faster?**Given: Three Pegs A, B and C**Peg A initially has n disks, different size, stacked up, larger disks are below smaller disks Problem: to move the n disks to Peg C, subject to Can move only one disk at a time Smaller disk should be above larger disk Can use other peg as intermediate B B A A C C Example: Tower of Hanoi