Algorithms

1. Algorithms FurqanMajeed

2. output Input Algorithm What is Algorithm? • A computer algorithm is a detailed step-by-step method for solving a problem by using a computer. • An algorithm is a sequence of unambiguous instructions for solving a problem in a finite amount of time. • More generally, an Algorithm is any well defined computational procedure that takes collection of elements as input and produces a collection of elements as output.

3. Popular Algorithms, Factors of Dependence • Most basic and popular algorithms are • Sorting algorithms • Searching algorithms Which algorithm is best? • Mainly, it depends upon various factors, for example in case of sorting • The number of items to be sorted • The extent to which the items are already sorted • Possible restrictions on the item values • The kind of storage device to be used etc.

4. One Problem, Many Algorithms Problem • The statement of the problem specifies, in general terms, the desired input/output relationship. Algorithm • The algorithm describes a specific computational procedure for achieving input/output relationship. Example • One might need to sort a sequence of numbers into non-decreasing order. Algorithms • Various algorithms e.g. merge sort, quick sort, heap sorts, radix sort, counting sort etc.

5. Important Designing Techniques • Brute Force • Straightforward, naïve/simple approach • Mostly expensive • Divide-and-Conquer • Divide into smaller sub-problems • Iterative Improvement • Improve one change at a time • Decrease-and-Conquer • Decrease instance size • Transform-and-Conquer • Modify problem first and then solve it • Space and Time Tradeoffs • Use more space now to save time later

6. Some of the Important Designing Techniques • Greedy Approach • Locally optimal decisions, can not change once made. • Efficient • Easy to implement • The solution is expected to be optimal • Every problem may not have greedy solution • Dynamic programming • Decompose into sub-problems like divide and conquer • Sub-problems are dependant • Record results of smaller sub-problems • Re-use it for further occurrence • Mostly reduces complexity exponential to polynomial

7. Problem Solving Process • Problem • Strategy • Algorithm • Input • Output • Steps • Analysis • Correctness • Time & Space • Optimality • Implementation • Verification

8. Model of Computation (Assumptions) • Design assumption • Level of abstraction which meets our requirements • Neither more nor less e.g. [0, 1] infinite continuous interval • Analysis independent of the variations in • Machine • Operating system • Programming languages • Compiler etc. • Low-level details will not be considered • Our model will be an abstraction of a standard generic single-processor machine, called a random access machine or RAM.

9. Model of Computation (Assumptions) • A RAM is assumed to be an idealized machine • Infinitely large random-access memory • Instructions execute sequentially • Every instruction is in fact a basic operation on two values in the machines memory which takes unit time. • These might be characters or integers. • Example of basic operations include • Assigning a value to a variable • Arithmetic operation (+, - , × , /) on integers • Performing any comparison e.g. a < b • Boolean operations • Accessing an element of an array.

10. Model of Computation (Assumptions) • In theoretical analysis, computational complexity • Estimated in asymptotic sense, i.e. • Estimating for large inputs • Big O, Omega, Theta etc. notations are used to compute the complexity • Asymptotic notations are used because different implementations of algorithm may differ in efficiency • Efficiencies of two given algorithm are related • By a constant multiplicative factor • Called hidden constant.

11. Drawbacks in Model of Computation Firstpoor assumption • We assumed that each basic operation takes constant time, i.e. model allows • Adding • Multiplying • Comparing etc. two numbers of any length in constant time • Addition of two numbers takes a unit time! • Not good because numbers may be arbitrarily • Addition and multiplication both take unit time! • Again very bad assumption

12. Model of Computation not so Bad Finally what about Our Model? • But with all these weaknesses, our model is not so bad because we have to give the • Comparison not the absolute analysis of any algorithm. • We have to deal with large inputs not with the small size • Model seems to work well describing computational power of modern nonparallel machines Can we do Exact Measure of Efficiency ? • Exact, not asymptotic, measure of efficiency can be sometimes computed but it usually requires certain assumptions concerning implementation

13. Summary : Computational Model • Analysis will be performed with respect to this computational model for comparison of algorithms • We will give asymptotic analysis not detailed comparison i.e. for large inputs • We will use generic uniprocessor random-access machine (RAM) in analysis • All memory equally expensive to access • No concurrent operations • All reasonable instructions take unit time, except, of course, function calls

14. Design And Analysis • In order to understand design and analysis procedure we will do an example

15. Selection Problem • Suppose you want to purchase a Laptop • You want to pick fastest Laptop • But fast Laptops are expensive • You cannot decide which one is more important price or speed

16. Selection Problem • Definitely do not want a Laptop if there is a another Laptop that is both fast and cheaper • We say that fast cheap Laptops“dominates” the slow expensive • So given a list of Laptops we want those that are not dominated by the other

17. Criterion for selection Two criterion for selection: • Speed • Price

18. Mathematical Model • P is in two dimensional space and its coordinates are P=(p.x , p.y) (x,y) • x is speed of Laptop • y is negation of price We can say about y • High value of y mean cheap Laptop and low y means Laptop is expensive • A point pis said to be dominated by a point q if p.x <= q.x and p.y <= q.y • In an array of points (Laptops) P={p1,p2,p3,….,pn} a maximal point is not dominated by any other point

19. Example • (700,15) (500,20) (200,1) • (1000,50) (900,35) (800,30) • (1400,70) (1500,80) (1200,60)

20. Example • Given a set of point P={p1,p2,p3,….,pn } output the maximal points • Those points of P such that pi is not dominated by any other point

21. Brute-Force Algorithm Maximal(intn, Point p[1……n]) • for i = 1 to n • maximal = true • for j = 1 to n • if( i != j ) and (p[i].x <= p [j]. x) and (p [i] .y <= p [j].y) • maximal = false; • break; • if (maximal == true) • then output p [i].x , p[i]. y

22. Running Time Analysis We can observe that: • The running time depends upon the input size. • Different input of the same size may result different time

23. Running Time Analysis • Count the number of steps of the pseudo code that are executed • Or count the number of times an element of p is accessed • Or number of comparisons that are performed

24. Analysis of selection Problem • Input size is n • We will count how number of steps

25. Brute-Force Algorithm Maximal(int n, Point p[1……n]) • for i = 1 to n • maximal = true • for j = 1 to n • if( i != j ) and (p[i].x <= p [j]. x) and (p [i] .y <= p [j].y) • maximal = false; • break; • if (maximal == true) • output p [i].x , p[i]. y n times 1 time Worst Case Analysis n times 5 times 2 times

26. Analysis of selection Problem • The outer loop runs n times • For each iteration. The inner loop runs n times in the worst case • P is accessed four times in the if statement • The output statement accesses p 2 times

27. n times 1 time Maximal(int n, Point p[1……n]) • for i = 1 to n • maximal = true • for j = 1 to n • if( i != j ) and (p[i].x <= p [j]. x) and (p [i] .y <= p [j].y) • maximal = false; • break; • if (maximal == true) • output p [i].x , p[i]. y n times 5 times 2 times

28. Analysis of selection Problem • In the worst case every point is maximal so every point is output • Worst-case time is:

29. Analysis of selection Problem

30. Analysis of selection Problem • Suppose n=50 and • We use a computer whose speed is 10000000 instructions per second • Then this computer requires

31. Complexity of the algorithms Algorithm complexity is the work done by the algorithm there are following complexities: • Worst case time • Average case time • Best case time • Expected time

32. Growth of Functions

33. Important • We will almost always work with worst case time. • Average case time is most difficult to compute • Worst case refer to the upper limit on the running time

34. Analysis of Selection Problem T(n) is Asymptotically equivalent to n2

35. Divide & Conquer A strategy to solve large number of problems. It has following stages: • Divide: divide the problem into small number of pieces • Conquer: Solve each piece by applying divide and conquer recursively • Combine: Rearrange the pieces • Examples: MergeSort, QuickSort