Dynamic Programming (DP)

Dynamic Programming (DP) Presented by Omar Haider Chowdhury

Overview • Background on DP • Example of some DP Problems • Memoization • Conclusion.

Background on DP • Why use DP ? • A technique for avoiding recomputation • Can make exponential running times…. • … become polynomial 

Background on DP • When to apply ? • Overlapping subproblems. • Optimal substructure. • How to do DP ? • Break the problem into subproblems • Recursively define the value of an optimal solution • Maintain a Topological order to fill the table.

Overlapping subproblems • A problem is said to have overlapping subproblems if it can be broken down into subproblems which are reused multiple times. • This is closely related to RECURSION

Overlapping subproblems • Let us see how to calculate factorial of a number recursively . int factorial( int n ) { if( n <= 1 ) return 1 ; return n * factorial ( n - 1 ) ; } • Thus the problem of calculating factorial(n) depends on calculating the subproblem factorial(n-1). • This problem does not exhibit overlapping subproblems since factorial is called exactly once for each positive integer less than n.

Overlapping subproblems • The problem of calculating the nth Fibonacci number does, however, exhibit overlapping subproblems. The naïve recursive implementation would be int fib( int n ){ if( n == 0 ) return 0 ; if( n == 1 ) return 1 ; return fib ( n -1 ) + fib ( n - 2 ) ; }

Overlapping subproblems • The problem of calculating fib(n) thus depends on both fib(n-1) and fib(n-2). To see how these subproblems overlap look at how many times fib is called and with what arguments when we try to calculate fib(6):

Optimal Substructure • A problem is said to have optimal substructure if the globally optimal solution can be constructed from locally optimal solutions to subproblems We will look into this property when we get to the examples.

Fibonacci Number • Run time: T(n) = T(n-1) + T(n-2) • This function grows as n grows • The run time doubles as n grows and is order O(2n). • This is a bad algorithm as there are numerous repetitive calculations.

Fibonacci Number • What is the solution ? Use Dynamic Programming ( cache the once computed values for later use to avoid repetitive calculation)

Fibonacci Number int fib( int n ){ int *f = new int[n+1] ; f[0] = 0 ; f[1] = 1 ; int i ; for(i=2;i<=n;++i) f[i] = f[i-1] + f[i-2] ; return f[n] ; } • The time complexity is now O(n) compared to O(2n)

0-1 Knapsack problem • Given a knapsack with maximum capacity W, and a set S consisting of n items • Each item i has some weight wi and benefit value bi(all wi , biand W are integer values) • Problem: How to pack the knapsack to achieve maximum total value of packed items?

W = 20 0-1 Knapsack problem:a picture Benefit value Weight bi wi Items 3 2 This is a knapsack Max weight: W = 20 4 3 5 4 8 5 10 9

0-1 Knapsack problem • Problem, in other words, is to find • The problem is called a “0-1” problem, because each item must be entirely accepted or rejected. • Just another version of this problem is the “Fractional Knapsack Problem”, where we can take fractions of items.

0-1 Knapsack problem: brute-force approach Let’s first solve this problem with a straightforward algorithm • Since there are n items, there are 2n possible combinations of items. • We go through all combinations and find the one with the most total value and with total weight less or equal to W • Running time will be O(2n)

0-1 Knapsack problem: brute-force approach • Can we do better? • Yes, with an algorithm based on dynamic programming • We need to carefully identify the subproblems Let’s try this: If items are labeled 1..n, then a subproblem would be to find an optimal solution for Sk = {items labeled 1, 2, .. k}

Defining a Subproblem If items are labeled 1..n, then a subproblem would be to find an optimal solution for Sk = {items labeled 1, 2, .. k} • This is a valid subproblem definition. • The question is: can we describe the final solution (Sn) in terms of subproblems (Sk)? • Unfortunately, we can’t do that. Explanation follows….

w1 =2 b1 =3 w2 =4 b2 =5 w3 =5 b3 =8 w4 =3 b4 =4 w1 =2 b1 =3 w2 =4 b2 =5 w3 =5 b3 =8 w4 =9 b4 =10 Defining a Subproblem Weight Benefit wi bi Item # ? 1 2 3 Max weight: W = 20 For S4: Total weight: 14; total benefit: 20 S4 2 3 4 S5 3 4 5 4 5 8 5 9 10 Solution for S4 is not part of the solution for S5!!! For S5: Total weight: 20 total benefit: 26

Defining a Subproblem (continued) • As we have seen, the solution for S4 is not part of the solution for S5 • So our definition of a subproblem is flawed and we need another one! • Let’s add another parameter: w, which will represent the exact weight for each subset of items • The subproblem then will be to compute B[k,w]

Recursive Formula for subproblems • Recursive formula for subproblems: • It means, that the best subset of Sk that has total weight w is one of the two: 1) the best subset of Sk-1 that has total weight w, or 2) the best subset of Sk-1 that has total weight w-wk plus the item k

Recursive Formula • The best subset of Sk that has the total weight w, either contains item k or not. • First case: wk>w. Item k can’t be part of the solution, since if it was, the total weight would be > w, which is unacceptable • Second case: wk <=w. Then the item kcan be in the solution, and we choose the case with greater value

0-1 Knapsack Algorithm for w = 0 to W B[0,w] = 0 for i = 0 to n B[i,0] = 0 for w = 0 to W if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w

Running time O(W) for w = 0 to W B[0,w] = 0 for i = 0 to n B[i,0] = 0 for w = 0 to W < the rest of the code > Repeatn times O(W) What is the running time of this algorithm? O(n*W) Remember that the brute-force algorithm takes O(2n)

Example Let’s run our algorithm on the following data: n = 4 (# of elements) W = 5 (max weight) Elements (weight, benefit): (2,3), (3,4), (4,5), (5,6)

Example (2) i 0 1 2 3 4 W 0 0 1 0 2 0 3 0 4 0 5 0 for w = 0 to W B[0,w] = 0

Example (3) i 0 1 2 3 4 W 0 0 0 0 0 0 1 0 2 0 3 0 4 0 5 0 for i = 0 to n B[i,0] = 0

Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Example (4) i 0 1 2 3 4 W 0 0 0 0 0 0 i=1 bi=3 wi=2 w=1 w-wi =-1 1 0 0 2 0 3 0 4 0 5 0 if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w]// wi > w

Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Example (5) i 0 1 2 3 4 W 0 0 0 0 0 0 i=1 bi=3 wi=2 w=2 w-wi =0 1 0 0 2 0 3 3 0 4 0 5 0 if wi <= w// item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w

Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Example (6) i 0 1 2 3 4 W 0 0 0 0 0 0 i=1 bi=3 wi=2 w=3 w-wi=1 1 0 0 2 0 3 3 0 3 4 0 5 0 if wi <= w// item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w

Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Example (7) i 0 1 2 3 4 W 0 0 0 0 0 0 i=1 bi=3 wi=2 w=4 w-wi=2 1 0 0 2 0 3 3 0 3 4 0 3 5 0 if wi <= w// item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w

Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Example (8) i 0 1 2 3 4 W 0 0 0 0 0 0 i=1 bi=3 wi=2 w=5 w-wi=2 1 0 0 2 0 3 3 0 3 4 0 3 5 0 3 if wi <= w// item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w

Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Example (9) i 0 1 2 3 4 W 0 0 0 0 0 0 i=2 bi=4 wi=3 w=1 w-wi=-2 1 0 0 0 2 0 3 3 0 3 4 0 3 5 0 3 if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] elseB[i,w] = B[i-1,w]// wi > w

Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Example (10) i 0 1 2 3 4 W 0 0 0 0 0 0 i=2 bi=4 wi=3 w=2 w-wi=-1 1 0 0 0 2 0 3 3 3 0 3 4 0 3 5 0 3 if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] elseB[i,w] = B[i-1,w]// wi > w

Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Example (11) i 0 1 2 3 4 W 0 0 0 0 0 0 i=2 bi=4 wi=3 w=3 w-wi=0 1 0 0 0 2 0 3 3 3 0 3 4 4 0 3 5 0 3 if wi <= w// item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w

Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Example (12) i 0 1 2 3 4 W 0 0 0 0 0 0 i=2 bi=4 wi=3 w=4 w-wi=1 1 0 0 0 2 0 3 3 3 0 3 4 4 0 3 4 5 0 3 if wi <= w// item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w

Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Example (13) i 0 1 2 3 4 W 0 0 0 0 0 0 i=2 bi=4 wi=3 w=5 w-wi=2 1 0 0 0 2 0 3 3 3 0 3 4 4 0 3 4 5 0 3 7 if wi <= w// item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w

Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Example (14) i 0 1 2 3 4 W 0 0 0 0 0 0 i=3 bi=5 wi=4 w=1..3 1 0 0 0 0 0 2 0 3 3 3 3 0 3 4 4 4 0 3 4 5 0 3 7 if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] elseB[i,w] = B[i-1,w]// wi > w

Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Example (15) i 0 1 2 3 4 W 0 0 0 0 0 0 i=3 bi=5 wi=4 w=4 w- wi=0 1 0 0 0 0 0 2 0 3 3 3 3 0 3 4 4 4 0 3 4 5 5 0 3 7 if wi <= w// item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w

Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Example (15) i 0 1 2 3 4 W 0 0 0 0 0 0 i=3 bi=5 wi=4 w=5 w- wi=1 1 0 0 0 0 0 2 0 3 3 3 3 0 3 4 4 4 0 3 4 5 5 0 3 7 7 if wi <= w// item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w

Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Example (16) i 0 1 2 3 4 W 0 0 0 0 0 0 i=3 bi=5 wi=4 w=1..4 1 0 0 0 0 0 0 2 0 3 3 3 3 3 0 3 4 4 4 4 0 3 4 5 5 5 0 3 7 7 if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w]// wi > w

Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Example (17) i 0 1 2 3 4 W 0 0 0 0 0 0 i=3 bi=5 wi=4 w=5 1 0 0 0 0 0 0 2 0 3 3 3 3 3 0 3 4 4 4 4 0 3 4 5 5 5 0 3 7 7 7 if wi <= w// item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w

Longest Common Subsequence Example. X=<A,B,C,B,D>, Y<B,A,D,C>, then <B,C> is a common subsequence of X and Y. The problem has applications in bioinformatics and communications.

Longest Common Subsequence • Given a sequence X=<x1,x2,…,xm>, another sequence Z=<z1,z2,…,zk> is a subsequence of X if there exists a strictly increasing sequence <i1,i2,…,ik> such that xij=zj for j =1,2,…,k. • Example. X=<A,B,C,B,D>, then Z=<A,D> is a subsequence of X. • Choose i1=1,i2=5, then xi1=z1=A, xi2=z2=D.

Longest Common Subsequence • Given a sequence X=<x1,x2,…,xm>, another sequence Z=<z1,z2,…,zk> is a subsequence of X if there exists a strictly increasing sequence <i1,i2,…,ik> such that xij=zj for j =1,2,…,k. • Example. X=<A,B,C,B,D>, then Z=<A,D> is a subsequence of X. • Choose i1=1,i2=5, then xi1=z1=A, xi2=z2=D. • So by name, the LCS of X and Y is a common subsequence of X and Y whose length is the largest.

Longest Common Subsequence Example. X=<A,B,C,B,D>, Y=<B,A,C,A,D> then Z=<A,C,D> is a LCS of X and Y. Can we find another LCS in this case?

Longest Common Subsequence Example. X=<A,B,C,B,D>, Y=<B,A,C,A,D> then Z=<A,C,D> is a LCS of X and Y. Can we find another LCS in this case? Z’=<B,C,D> is another LCS, so longest common subsequence is not unique!

Longest Common Subsequence • Let us try a brute force way. • Assume the lengths of the string be n and m. All possible subsequence of the first string will be of O(2^n). Checking if the subsequence of string 1 is common with string 2 is of O(m). The value of LCS would be the longest of the subsequence of string 1 which is common with string 2. Thus the time complexity of the algorithm is O(2^n * m ). • Is there any better algorithm with better running time ? YesObviouslyDP

Longest Common Subsequence Dynamic solution. (For simplicity, let’s say we are only interested in finding the length.) -Define table c[-,-] -c[i,j] stores the length of an LCS of the sequences X[1..i] and Y[1..j]. c[i,j] = 0, if i=0 or j=0 c[i,j] = c[i-1,j-1] + 1, if i,j>0 and xi = yj c[i,j] = max{ c[i-1,j], c[i,j-1] }, if i,j>0 and xi ≠ yj

Related to LCS - Edit distance Edit distance problem For given sequences A and B find the minimal number of Insert, Delete and Replace operations that converts A to B. A = abxxz B = abyxy Edit distance between A and B is 2

Dynamic Programming (DP)