1 / 21

Discrete Mathematics: Complexity of Algorithms

Discrete Mathematics: Complexity of Algorithms. Section Summary. Time Complexity Worst-Case Complexity Algorithmic Paradigms Understanding the Complexity of Algorithms. The Complexity of Algorithms.

xena-cabrera
Télécharger la présentation

Discrete Mathematics: Complexity of Algorithms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Discrete Mathematics: Complexity of Algorithms

  2. Section Summary • Time Complexity • Worst-Case Complexity • Algorithmic Paradigms • Understanding the Complexity of Algorithms

  3. The Complexity of Algorithms • Given an algorithm, how efficient is it for solving a problem given input of a particular size? to answer this, we ask: • How much time does it take to solve a problem? • How much memory does it use to solve a problem? • When we analyze the time that an algorithm uses to solve a problem given input of a particular size, we are studying the time complexityof the algorithm. • When we analyze the memory that an algorithm uses to solve a problem given input of a particular size, we are studying the space complexityof the algorithm.

  4. The Complexity of Algorithms • We will measure time complexity in terms of the number of operations an algorithm uses and we will use big-O and big-Theta notations to estimate the time complexity. • We can use this analysis to see whether it is practical to use this algorithm to solve problems with input of a particular size. • We can also compare the efficiency of different algorithms for solving the same problem. • We ignore implementation details (including the data structures used and the hardware and software platforms) because it is extremely complicated to consider them.

  5. Time Complexity • To analyze the time complexity of algorithms, we determine the number of operations such as comparisons and arithmetic operations (for instance,  and ). • We ignore minor details such as the variables’ initialization. • Worst-case time complexity (easier to compute): provides an upper bound on the number of operations an algorithm uses to solve a problem with input of a particular size. • Average case time complexity (harder to compute):is the average number of operations an algorithm uses to solve a problem over all inputs of a particular size.

  6. Complexity Analysis of Algorithms Example: describe the time complexity of the algorithm for finding the maximum element in a finite sequence. Solution: only count the number of comparisons: • n− 1: the max < ai comparison is made n− 1 times. • n− 1: each time i is incremented, a test is made to see if i ≤ n. • 1: one last comparison determines thati > n. 2n − 1 comparisons are made. Hence, the time complexity is O(n). proceduremax (a1, a2, …., an: integers) max := a1 • fori := 2ton {each time i ≤ n is done to exit the loop} • ifmax < aithenmax := ai • returnmax

  7. Worst-Case Complexity of Linear Search procedurelinearsearch (x: integer,a1, a2, …,an: distinct integers) i := 1 while (i≤n and x ≠ ai) i := i + 1 • ifi≤nthenlocation := i elselocation := 0 returnlocation Example: Solution: only count the number of comparisons: • 2n or 2i: at each step two comparisons are made; i≤n and x ≠ ai. • 1: to end the loop when x is not found, one comparison i≤n is made. • 1: after the loop, one morei≤n comparison is made. If x = ai, 2i + 1 comparisons are used. If x is not on the list, 2n + 1 comparisons are made and then an additional comparison is used to exit the loop, i.e., 2n + 2 comparisons are made. Hence, the complexity is O(n).

  8. Average-Case Complexity of Linear Search Example: describe the average case performance of the linear search algorithm. Although usually it is very difficult to determine average-case complexity, it is easy for linear search. Solution: assume the element is in the list and that the possible positions are equally likely. We know that, if x = ai, the number of comparisons is 2i + 1. We compute this for all i= 1, 2, …, n : Hence, the average-case complexity of linear search is Θ(n). - n + n = -1 - 1 … -1 + n

  9. Worst-Case Complexity of Binary Search Example: describe the time complexity of binary search. Solution: for the sake of simplicity assume n= 2k. Note that k = log n. • 2: two comparisons are made at each stage; i < j, and x > am . • k: at the first iteration the size of the list is 2k and after that it is 2k-1. Then 2k-2 and so on until the size of the list is 21 = 2. • 2: at the last step, a comparison tells us that the size of the list is 20= 1 and the element is compared with the single remaining element x = ai. Hence, at most 2k + 2 = 2 log n + 2comparisons are made. Therefore, the time complexity is O(log n), better than linear search. procedure binarysearch (x: integer, a1,a2,…, an: increasing integers) i := 1 {i is the left endpoint of interval} j := n {j is right endpoint of interval} whilei < j • m := ⌊(i + j)/2⌋ ifx > am then i := m + 1 elsej := m if x = aithenlocation := i else location := 0 returnlocation

  10. Worst-Case Complexity of Bubble Sort Example: what is the worst-case complexity of bubble sort in terms of the number of comparisons made? Solution: a sequence of n−1 passes is made through the list. On each pass n−i comparisons are made. Therefore, the worst-case complexity is O(n2). The result will be the same if we count i ≤ n-1 and j≤ n-1 for exiting the loops. procedurebubblesort(a1,…,an: real numbers with n≥2) • fori:= 1 to n− 1 • for j := 1 to n− i ifaj > aj+1then interchange aj and aj+1

  11. Worst-Case Complexity of Insertion Sort Example: what is the worst-case complexity of insertion sort in terms of the number of comparisons made? Solution: the total number of comparisons are: Hence, the complexity is O(n2). procedureinsertionsort (a1,…,an: real numbers with n≥2) for j := 2 to n i := 1 whileaj > ai i := i + 1 temp := aj • fork := 0 to j−i− 1 aj-k := aj-k-1 ai := temp 3 2 4 1 5 2 34 1 5 2 341 5 1 2 345 1 2 345

  12. Matrix Multiplication Algorithm • The definition for matrix multiplication can be expressed as an algorithm; C = A B where C is an mn matrix that is the product of the mk matrix A and the k n matrix B. • This algorithm carries out matrix multiplication based on its definition. procedurematrix multiplication (A,B: matrices) fori:= 1 to m • for j := 1 to n cij := 0 • for q := 1 to k • cij := cij + aiqbqj • return C

  13. Complexity of Matrix Multiplication Example: how many additions of integers and multiplications of integers are used by the matrix multiplication algorithm to multiply two nn matrices. Solution: there are n2entries in the product. Finding each entry requires: • n multiplications • n− 1 additions Hence, n2(n) multiplications and n2(n− 1) additions are used. As a result, the complexity of matrix multiplication is O(n3).

  14. Boolean Product Algorithm • Boolean product of zero-one matrices can be done by an algorithm: Example: how many bit operations are used to find A ⊙B? Solution: there are n2entries in the A ⊙B. A total of (n − 1) ORs and n ANDs are used to find each entry. Hence, each entry takes (2n − 1)bit operations. A total of n2(2n − 1) operations are used. Therefore the time complexity is O(n3). procedureBoolean product (A,B: zero-onematrices) fori:= 1 to m • for j := 1 to n cij := 0 • for q := 1 to k • cij:= cij∨ (aiq ∧ bqj) • return C

  15. Matrix-Chain Multiplication • How should the matrix-chainA1 A2∙ ∙ ∙ An be computed using the fewest multiplications of integers, where A1 ,A2 , … , An are m1m2 , m2m3 , … , mnmn+1 integer matrices. Example: in which order should the integer matrices A1 A2 A3, where A1 is 3020 , A2 is 2040,A3 is 4010, be multiplied to use the least number of multiplications. Solution: there are two possible ways to compute A1A2A3. • A1(A2A3): A2A3 takes 20  40  10 = 8000 multiplications. Then multiplying A1 by the matrix A2A3 takes 30 ∙ 20 ∙ 10 = 6000 multiplications. So the total number is 8000 + 6000 = 14,000. • (A1A2)A3: A1A2 takes 30  20  40 = 24,000 multiplications. Then multiplying the matrix A1A2 by A3 takes 30 ∙ 40 ∙ 10 = 12,000 multiplications. So the total number is 24,000 + 12,000 = 36,000.

  16. Algorithmic Paradigms • An algorithmic paradigmis a general approach based on a particular concept for constructing algorithms to solve a variety of problems. • The greedy algorithms were introduced in Chapter 3. • We also discuss the brute-force algorithms now: • It is solved in the most straightforward manner without taking advantage of any ideas to make the algorithm more efficient, e.g., sequential search as opposed to binary search. • You will see the divide-and-conquer algorithms in Chapter 8, dynamic programming in Chapter 8, backtracking in Chapter 11, and probabilistic algorithmsin Chapter 7.

  17. Computing the Closest Pair of Points by Brute-Force Example: construct a brute-force algorithm for finding the closest pair of points in a set of n points in the plane and provide a worst-case estimate of the number of arithmetic operations. Solution: recall that the distance between (xi , yi) and (xj, yj) is: A brute-force algorithm simply computes the distance between all pairs of points and picks the pair with the smallest distance. Note: there is no need to compute the square root, since the square of the distance is smallest when the distance is smallest.

  18. Computing the Closest Pair of Points by Brute-Force • Algorithm for finding the closest pair in a set of n points: • The algorithm loops through ((n−1)n/2) pairs of points, computes the value (xj−xi)2 + (yj−yi)2 and compares it with the minimum. Therefore, the algorithm uses O(n2) arithmetic and comparison operations. • We will see an algorithm with O(n log n) time complexity in Section 8.3. • procedureclosest pair ((x1, y1),(x2, y2), … ,(xn, yn): xi, yi real numbers) • min = ∞ fori:= 2 to n • for j := 1 to i− 1 • if (xj−xi)2+ (yj−yi)2 < minthen • min := (xj−xi)2+ (yj−yi)2 • closest pair := ( (xi, yi) ,(xj, yj) ) • return closest pair n = 5 i= 2 j = 1 i = 3 j = 1, 2 i = 4 j= 1, 2, 3 i = 5 j= 1, 2, 3, 4 (5-1) 5 / 2 = 10 comparison

  19. Complexity of Algorithms

  20. Understanding the Complexity of Algorithms Times of more than 10100 years are indicated with an *.

  21. Complexity of Problems • P or Polynomial Time or Tractable Problem: there exists a polynomial time algorithm to solve this problem. • NP or Nondeterministic Polynomial Time: a solution can be checked in polynomial time. But a polynomial time algorithm has not been found for finding a solution to problems in this class. • NP Complete: if you find a polynomial time algorithm for one member of the class, it can be used to solve all the problems in the class. • Intractable Problem: there does not exist a polynomial time algorithm to solve this problem. It requires an extremely large amount of time. • Unsolvable Problem: no algorithm exists to solve this problem. The Clay Math Institute has offered $1,000,000 for a solution to P = NP?

More Related