Create Presentation
Download Presentation

Download Presentation
## Design and Analysis of Algorithms

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Design and Analysis of Algorithms**Dr. Aarij Mahmood Hussaan**Course Objective**The last ½ century has witnessed the development of a beautiful and elegant new scientific field: the design and analysis of algorithms. This course will teach you how to “design” and “analyze” algorithms.**History**• The name of “algorithm” comes from al-Khowarizmi, 9th century Persian mathematician and astronomer who wrote a book, eventually translated into Latin in the 12th century as Algoritmi de numeroIndorum ("al-Khowarizmi's book on Indian numbers"), that gave our modern pencil-and-paper algorithms for addition, subtraction, multiplication, and division. • Every algorithm should have: • Input, • Output, • Deterministic • Effective, • Finite.**Why study algorithms**• Many aspects of “programming” are more important than algorithms: • user-friendliness, correctness, extensibility, maintainability, simplicity, scalability, cost, … • But good algorithm is fundamental to all of above.**Algorithm**• A procedure for solving a (mathematical) problem in a finite number of steps that frequently involves repetition of an operation; broadly: a step-by-step method for accomplishing some task.**Algorithm: formal definition──Intro CS**• A well-ordered collection of unambiguous and effectively computable operations that, when, executed, produces a result and halts in a finite amount of time.**A person does not really understand something until after**teaching it to a computer Donald Knuth**Algorithms**• Algorithm: Any well-defined computation procedure that takes some value, or set of values, as input and produces some value, or set of values, as output. • Or: tool for solving well specific computational problem. • Example: Sorting problem**Problem**• Example: Sorting problem • Input: A sequence of n numbers • Output: A permutation of the input sequence such that . • Sorting algorithm: • The procedure that produces the sorted outputs for all possible inputs.**Properties**• Aninstance of a problem consists of all inputs needed to compute a solution to the problem. • An algorithm is said to be correct if, for every input instance, it halts with the correct output. • A correct algorithm solves the given computational problem. An incorrect algorithm might not halt at all on some input instance, or it might halt with other than the desired answer.**What kind of problem can be solved by algorithm?**• The Human Genome Project • The Internet Applications • Electronic Commerce with Public-key cryptography and digital signatures • Manufacturing and other commercial settings**Interesting problems**• How to solve these problems • Shortest path (Ch. 24) • Longest common subsequence (Ch. 15) • Convex hull (CH. 33) • Characteristics of interesting problem • Find the best solutions from the candidate ones. • Practical application.**Hard Problems**• Efficiency is speed, i.e., how long an algorithm takes to produce its result. • NP-Complete • no efficient algorithm for an NP-complete problem has ever been found, • NP-complete problems has the remarkable property that if an efficient algorithm exists for any one of them**Algorithm choice**• There are a large number of good sorting algorithms. • Which algorithm is best for a given application • • It depends on: • The number of items to be sorted. • The extent to which the items are already sorted. • Possible restrictions on item values. • The kind of storage device to be used. • An algorithm is said to be correct if, for every input instance, it gives the correct output.**Algorithm efficiency**• Algorithms devised to solve the same problem often differ in their efficiency. • These differences can be much more significant than differences due to hardware and software. Example: • Insertion sort algorithm takes time = c1n2 to sort n numbers, where c1 is a constant. • Merge sort algorithm takes time = c2 n lg n to sort n numbers, where c2 is a constant.**Algorithm efficiency**• c1 < c2, but constants less significant in the running time than the input size. • lgn < n, but insertion sort is usually faster than merge sort for small input sizes. • Once the input size n becomes large enough, merge sort will become faster than insertion sort.**Algorithm efficiency**• • Example: • Computer A executes one billion instruction per second (1 GHz) and executes insertion sort (c1n2). • Computer B executes one million instruction per second (1 MHz) and executes merge sort (c2 n lg n). • Assume c1=2 and c2=50. • Assume we need to sort 1 million numbers.**Algorithm efficiency**• Computer A takes= 2*(106)2 instructions = 2000 s 109instructions/second • Computer B takes= 50*106* lg106 instructions = 100 s 107instructions/second**Algorithm efficiency**• Imagine that we need to sort 10 million numbers; • Insertion sort takes 2.3 days. • Merge sort takes 20 minutes. • We analyze algorithms to improve them and choose the best one for a given problem.**Algorithm Questions**• How to devise algorithms? • there are various design techniques. • example: divide and conquer. • How to analyze algorithms? • the task of determining how much computing time and storage and algorithm requires. • it needs mathematical skills. • best case, average case, and worst case.**Algorithm Questions**• How to validate algorithms? • validation: showing that the algorithm computes the correct answer for all legal inputs. • after validation, verification process takes place.**Insertion sort**• solves the sorting problem • Input: A sequence of n numbers • Output: A permutation of the input sequence such that .**Insertion sort – pseudo code**• InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} }**An Example: Insertion Sort**i = j = key = A[j] = A[j+1] = 30 10 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 David Luebke 269/17/2013**An Example: Insertion Sort**i = 2 j = 1 key = 10A[j] = 30 A[j+1] = 10 30 10 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 David Luebke 279/17/2013**An Example: Insertion Sort**i = 2 j = 1 key = 10A[j] = 30 A[j+1] = 30 30 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 David Luebke 289/17/2013**An Example: Insertion Sort**i = 2 j = 1 key = 10A[j] = 30 A[j+1] = 30 30 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 David Luebke 299/17/2013**An Example: Insertion Sort**i = 2 j = 0 key = 10A[j] = A[j+1] = 30 30 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 David Luebke 309/17/2013**An Example: Insertion Sort**i = 2 j = 0 key = 10A[j] = A[j+1] = 30 30 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 David Luebke 319/17/2013**An Example: Insertion Sort**i = 2 j = 0 key = 10A[j] = A[j+1] = 10 10 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 David Luebke 329/17/2013**An Example: Insertion Sort**i = 3 j = 0 key = 10A[j] = A[j+1] = 10 10 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 David Luebke 339/17/2013**An Example: Insertion Sort**i = 3 j = 0 key = 40A[j] = A[j+1] = 10 10 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 David Luebke 349/17/2013**An Example: Insertion Sort**i = 3 j = 0 key = 40A[j] = A[j+1] = 10 10 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 David Luebke 359/17/2013**An Example: Insertion Sort**i = 3 j = 2 key = 40A[j] = 30 A[j+1] = 40 10 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 David Luebke 369/17/2013**An Example: Insertion Sort**i = 3 j = 2 key = 40A[j] = 30 A[j+1] = 40 10 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 David Luebke 379/17/2013**An Example: Insertion Sort**i = 3 j = 2 key = 40A[j] = 30 A[j+1] = 40 10 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 David Luebke 389/17/2013**An Example: Insertion Sort**i = 4 j = 2 key = 40A[j] = 30 A[j+1] = 40 10 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 David Luebke 399/17/2013**An Example: Insertion Sort**i = 4 j = 2 key = 20A[j] = 30 A[j+1] = 40 10 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 David Luebke 409/17/2013**An Example: Insertion Sort**i = 4 j = 2 key = 20A[j] = 30 A[j+1] = 40 10 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 David Luebke 419/17/2013**An Example: Insertion Sort**i = 4 j = 3 key = 20A[j] = 40 A[j+1] = 20 10 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 David Luebke 429/17/2013**An Example: Insertion Sort**i = 4 j = 3 key = 20A[j] = 40 A[j+1] = 20 10 30 40 20 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 David Luebke 439/17/2013**An Example: Insertion Sort**i = 4 j = 3 key = 20A[j] = 40 A[j+1] = 40 10 30 40 40 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 David Luebke 449/17/2013**An Example: Insertion Sort**i = 4 j = 3 key = 20A[j] = 40 A[j+1] = 40 10 30 40 40 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 David Luebke 459/17/2013**An Example: Insertion Sort**i = 4 j = 3 key = 20A[j] = 40 A[j+1] = 40 10 30 40 40 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 David Luebke 469/17/2013**An Example: Insertion Sort**i = 4 j = 2 key = 20A[j] = 30 A[j+1] = 40 10 30 40 40 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 David Luebke 479/17/2013**An Example: Insertion Sort**i = 4 j = 2 key = 20A[j] = 30 A[j+1] = 40 10 30 40 40 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 David Luebke 489/17/2013**An Example: Insertion Sort**i = 4 j = 2 key = 20A[j] = 30 A[j+1] = 30 10 30 30 40 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 David Luebke 499/17/2013**An Example: Insertion Sort**i = 4 j = 2 key = 20A[j] = 30 A[j+1] = 30 10 30 30 40 InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key} } 1 2 3 4 David Luebke 509/17/2013