Download Presentation
## Dynamic Programming

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**A typical infinite horizon problem**(1) (2) (Intertemporal constraint.) (3) (Initial condition.) State xt Control ut**Value function**Finite time horizon example**Hamilton-Jacobi-Bellman equation**Solution Method (1) Backward induction (2) Optimality principle (HJB equation)**Optimality principle**Relationship between value functions in adjacent periods Bellman equation)**Solving dynamic programming problem using HJB equation**• Assume a functional form of V(x) (which has free parameters • to be determined later). (2) Obtain the first order condition from (3) Use HJB equation to determine free parameters. (4) Ensure solutions satisfy initial and transversality conditions.**Variational Approach**Euler equation as the optimality condition Choosing u(t) is equivalent to choosing x(t+1).**Derivation of the Euler equation**First order condition Second order difference equation.**Solving dynamic programming problem using variational method**(1) Obtain the first order condition (Euler equation) (2) Solve for the general solutions to the Euler equation. (3) Use initial and transversality conditions to determine the unique solution.**Examples**Value function (1) Bellman equation (2)**HJB approach**(i) Postulate the form of the value function (ii) Obtain the FOC**(iii) Determine parameters**(a) Obtain Riccardi equations using HJB Equating coefficients Solving Riccardi equations**Optimal policy and resulting dynamics**(b) Boundary conditions Transversality Initial condition**Variational approach**(i) Obtain the Euler equation Period t and t+1 payoffs Euler equation (ii) Solving the Euler equation**(iii) Use boundary conditions to get the unique solution**Transversality Initial condition (iv) Final results