Download Presentation
## Algorithms

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Algorithms**Joe Meehean**Data Structures vs. Algorithms**• Data structures are great at… • storing items • providing a simple access interface • Algorithms … • operate over data structures • instructions to do more complicated work • may or may not want embedded in data structure**Algorithms in STL**• Found in <algorithm> header • From simple • for_each • find • swap • To the complex • sort • set_intersection • random_shuffle • next_permutation**Brute Force**• straight forward approach • usually based directly on • problem statement • definitions of concepts • e.g., an • an = a * a * …. * a (n times)**Brute Force**• Advantages • applicable to a wide variety of problems • no limitation on problem size for some important problems • simple to design • designing better algo not always worth it • if problem small or algo will run infrequently • provides a comparison for more complex algos**Brute Force**• Disadvantages • slow • may be so slow as to make impossible to complete in human lifetime**Sorting**• Problem • arrange comparable items in list into sorted order • Most sorting algorithms involve comparing item values • We assume items define • < operator • > operator • == operator**Selection Sort Idea**Find the smallest value in vector A and put in it in A[0] Find 2nd smallest value and put it in A[1] Etc…**Selection Sort Approach**• Use a nested loop • Outer loop index k • indicates position to fill • Inner loop index j • from k+1 to A.length – 1 • indicates value to compare to min next • Swap A[k] with A[min] • A[min] is min value in range k+1 to A.length–1**Selection Sort Approach**min 8 2 9 0 5 6 3 4 1 7 k**Selection Sort Approach**min 8 2 9 0 5 6 3 4 1 7 k j**Selection Sort Approach**min 8 2 9 0 5 6 3 4 1 7 k j**Selection Sort Approach**min 8 2 9 0 5 6 3 4 1 7 k j**Selection Sort Approach**min 8 2 9 0 5 6 3 4 1 7 k j**Selection Sort Approach**min 8 2 9 0 5 6 3 4 1 7 k j**Selection Sort Approach**min 8 2 9 0 5 6 3 4 1 7 k j**Selection Sort Approach**min 8 2 9 0 5 6 3 4 1 7 k j**Selection Sort Approach**min 8 2 9 0 5 6 3 4 1 7 k j**Selection Sort Approach**min 8 2 9 0 5 6 3 4 1 7 k j**Selection Sort Approach**swap min 8 2 9 0 5 6 3 4 1 7 k j**Selection Sort Approach**swap min 0 2 9 8 5 6 3 4 1 7 k j**Selection Sort Approach**min 0 2 9 8 5 6 3 4 1 7 k j**Selection Sort**Item value Position in the Array**Section Sort Discussion**• After i outer loop iterations • A[0] through A[i-1] contain their final values**Time for Selection Sort**• Outer loop executes N times • Inner loop executes a different number of times depending on outer loop • 1st outer = N – 1 inner • 2nd outer = N – 2 inner • … • Nth outer = 0 inner • (N-1) + (N-2) +…+ 2 + 1 + 0 = O(N2) • Always O(N2)**Exhaustive Search**• Combinatorial problems • problems where the answer is an ordered combination of items from a set • Exhaustive search • brute force approach to combinatorial problems • generate all possible combinations • check each combination to see if it is a possible solution • may need to find the best solution • e.g., try all possible combinations to a safe**0-1 Knapsack Problem**• Have a set of items • each has a weight: wi • each has a monetary value: vi • Have knapsack • can only hold a total weight of W • Fill the knapsack to maximize its value**0-1 Knapsack Problem**• Exhaustive search • try every combination of items • throw out combinations that are too heavy • keep the combination with the largest value • Complexity • dominated by generating all combinations • each item may be in the knapsack or not • for N items: 2N possible combinations • O(2N): very, very bad**Decrease & Conquer**Exploit relationship between solution to a problem and solution to a smaller instance Reduce problem to a smaller problem Solve smaller problem Use smaller problem solution to solve original problem**Decrease & Conquer**• Top-down • reduce larger problem into successively smaller problems • recursive approach • Bottom-up • solve smallest version of problem • use to solve next larger problem • incremental approach**Decrease & Conquer**• 3 variations • Decrease-by-constant • compute an for positive integer n • an = an-1 * a • f(n) = an • f(n) = f(n-1) * a, if n >0 • f(n) = 1, if n == 0 • recursive definition**Decrease & Conquer**• 3 variations • Decrease-by-constant-factor • an = (an/2)2 • n/2 is not an integer if n is odd, so… • an = (an/2)2, if n is even • an = (a(n-1)/2)2 * a, if n is odd • an= 1, if n = 0 • O(logN) number of multiplications**Decrease & Conquer**• 3 variations • Variable-size-decrease • lookup in BST • BST is unbalanced • at each node going left or right removes a variable number of nodes**Insertion Sort Idea**Incremental approach Reduce list size to trivial, sort and solve Then increase list size Put 1st 2 items in correct order Insert the 3rd item in the correct place relative to the first 2 Insert the 4th item in the correct place relative to the first 3 etc…**Insertion Sort Approach**• Nested loop • Outer loop • index k from 1 to A.length – 1 • Item to put into correct place • Inner loop • index j from k – 1 to 0 • Items to compare to A[k] to find its correct place**Insertion Sort**8 2 9 5 6 3 4 1 0 7 j k**Insertion Sort**temp 5 8 2 9 6 3 4 1 0 7 j k**Insertion Sort**temp 5 2 9 6 3 4 8 1 0 7 j k**Insertion Sort**2 9 6 3 4 5 8 1 0 7 j k**Insertion Sort**5 2 9 8 6 3 4 1 0 7 j k**Insertion Sort**temp 2 5 9 8 6 3 4 1 0 7 j k**Insertion Sort**temp 2 5 9 6 3 4 1 0 8 7 j k**Insertion Sort**temp 2 5 9 6 3 4 1 0 8 7 j k**Insertion Sort**5 9 6 3 4 1 0 8 2 7 j k**Insertion Sort**5 9 6 3 4 1 0 8 2 7 j k**Insertion Sort**temp 6 5 9 3 4 1 0 8 2 7 j k**Insertion Sort**temp 6 5 9 3 4 1 0 8 2 7 j k**Insertion Sort**5 9 3 4 1 0 6 8 2 7 j k