1 / 26

Algorithms ( and Datastructures )

Algorithms ( and Datastructures ). Lecture 1 MAS 714 part 2 Hartmut Klauck. Algorithms. An algorithm is a procedure for performing a computation Algorithms consist of elementary steps/instructions Elementary steps depend on the model of computation

havyn
Télécharger la présentation

Algorithms ( and Datastructures )

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Algorithms (andDatastructures) Lecture 1 MAS 714 part 2 Hartmut Klauck

  2. Algorithms • An algorithm is a procedure for performing a computation • Algorithmsconsist of elementary steps/instructions • Elementarystepsdepend on the model ofcomputation • Example: Turing machine: transitionfunction • Example: C++ commands

  3. Algorithms • Algorithmsarenamed after Al-Khwārizmī(AbūʿAbdallāhMuḥammadibnMūsā al-Khwārizmī)c. 780-850 cePersianmathematicianandastronomer • (Algebra is also named after hiswork) • His worksbroughtthepositionalsystemofnumberstotheattentionofEuropeans

  4. Algorithms: an example • Addition via theschoolmethod: • Write numbersundereachother • Add numberpositionbypositionmoving a „carry“ forward • Elementaryoperations: • Add twonumbersbetween 0 and 9(memorized) • Read and Write • Can deal witharbitrarilylongnumbers!

  5. Datastructure • The addition algorithm uses (implicitly) an array as datastructure • An array is a fixed length vector of cells that can each store a number • Note thatwhenweadd x and y thenx+yisatmost 1 digitlongerthanmax{x,y} • So theresult will bestored in an arrayoflength n+1

  6. Multiplication • The schoolmultiplicationalgorithmis an exampleof a reduction • First we learn how to add n numbers with n digits each • To multiply x and y we generate n numbers xi¢ y¢2i and add them up • Reduction from Multiplication to Addition

  7. Complexity • Weusuallyanalyzealgorithmsto grade theperformance • The mostimportant (but not theonly) parametersare time andspace • Time referstothenumberofelementarysteps • Space referstothestorageneededduringthecomputation

  8. Example: addition • Assumeweaddtwonumbersx,ywith n decimaldigits • Clearlythenumberofelementarysteps (addingdigitsetc) growslinearlywith n • Space is also O(n) • Typical: asymptoticanalysis

  9. Example: multiplication • Wegenerate n numberswithatmost 2n digits, addthem • Number of steps is O(n2) • Space is O(n2) • Much fasteralgorithmsexist • (but not easy to do withpenandpaper) • Question: Is multiplicationharderthanaddition? • Answer: wedon‘tknow...

  10. Typesof Analysis • Usuallyweuseasymptoticanalysis • Reason: nextyear‘scomputer will befaster, so constantfactorsdon‘t matter (usually) • Understand theinherentcomplexityof a problem (canyoumultiply in linear time?) • Usuallygivestherightanswer in practice • Worstcase • The running time of an algorithmisthemaximum time over all inputs • On thesafesidefor all inputs…

  11. Typesofanalysis • Averagecase: • Averageunderwhichdistribution? • Oftenthe uniform distribution, but maybeunrealistic • Amortizedanalysis • Sometimes after somecostlypreparationswecansolvemanyprobleminstancescheaply • Count theaveragecostof an instance (preparationcostsarespreadbetweeninstances) • Often used in datastructure analysis

  12. Our model ofcomputation • In princplewecoulduse Turing machines... • We will consideralgorithms in a richer model, formally a RAM • RAM: • randomaccessmachine • Basicallywe will just usepseudocode/informal language

  13. RAM • Random Access Machine • Storage is made of registers that can hold a number (an unlimited amount of registers is available) • The machine is controlled by a finite program • Instructions are from a finite set that includes • Fetching data from a register into a special register • Arithmetic operations on registers • Writing into a register • Indirect addressing

  14. RAM • Random Access Machines and Turing Machines can simulate each other • There is a universal RAM • RAM’s are very similar to actual computers • machine language

  15. Computation Costs • The time cost of a RAM step involving a register is the logarithm of the number stored • logarithmic cost measure • adding numbers with n bits takes time n etc. • The time cost of a RAM program is the sum of the time costs of its steps • Space is the sum (over all registers ever used) of the logarithms of the maximum numbers stored in the register

  16. Sorting • Computers spend a lot of time sorting! • Assume we have a list of numbers x1,…,xn from a universe U • For simplicity assume the xi are distinct • The goal is to compute a permutation ¼ such thatx ¼(1)< x¼(2) <  < x¼(n) • Think of a deck ofcards

  17. Insertion Sort • An intuitive sorting algorithm • The input is provided in A[1…n] • Code:for i = 2:n, for (k = i; k > 1 and A[k] < A[k-1]; k--) swap A[k,k-1]→ invariant: A[1..i] is sortedend • Clear: O(n2) comparisons and O(n2) swaps • Algorithm works in place, i.e., uses linear space

  18. Correctness • By induction • Base case: n=2:We have one conditional swap operation, hence the output is sorted • Induction Hypothesis:After iteration i the elements A[1]…A[i] are sorted • Induction Step:Consider Step i+1. A[1]…A[i] are sorted already. Inner loop starts from A[i+1] and moves it to the correct position. After this A[1]…A[i+1] are sorted.

  19. Best Case • In the worst case Insertion Sort takes time (n2) • If the input sequence is already sorted the algorithms takes time O(n) • The same istrueiftheinputisalmostsorted • important in practice • Algorithmis simple and fast forsmall n

  20. Worst Case • On someinputs Insertion Sorttakes(n2) steps • Proof: consider a sequencethatisdecreasing, e.g., n,n-1,n-2,…,2,1 • Eachelementismovedfromposition i toposition 1 • Hencetherunning time isat leasti=1,…,n i = (n2)

  21. Can we do better? • Attempt 1:Searchingforthepositiontoinsertthenextelementisinefficient, employbinarysearch • Orderedsearch: • Given an array A with n numbers in a sorted sequence, and a number x, find the smallest i such that A[i]¸ x • Use A[n+1]=1

  22. Linear Search • Simplestwaytosearch • Run through A (from 1 to n) andcompare A[i] with x until A[i]¸ x is found, output i • Time: £(n) • Can also be used to search unsorted Arrays

  23. Binary Search • If the array is sorted already, we can find an item much faster! • Assume we search for a x among A[1]<...<A[n] • Algorithm (to be called with l=1 and r=n): • BinSearch(x,A,l,r] • If r-l=0 test if A[l]=x, end • Compare A[(r-l)/2+l] with x • If A[(r-l)/2+l]=x output (r-l)/2+l, end • If A[(r-l)/2+l]> x BinSearch(x,A,l,(r-l)/2+l) • If A[(r-l)/2+l]<x Bin Search(x,a,(r-l)/2+l,r)

  24. Time of Binary Search • Define T(n) as the time/number of comparisons needed on Arrays of length n • T(2)=1 • T(n)=T(n/2)+1 • Solution: T(n)=log(n) • All logs have basis 2

  25. Recursion • We just described an algorithm via recursion:a procedure that calls itself • This is often convenient but we must make sure that the recursion eventually terminates • Have a base case (here r=l) • Reduce some parameter in each call (here r-l)

  26. Binary Insertion Sort • Using binary search in InsertionSort reduces the number of comparisons to O(n log n) • The outer loop is executed n times, each inner loop now uses log n comparisons • Unfortunatelythenumberofswapsdoes not decrease: • To insert an element we need to shift the remaining array to the right!

More Related