MA409: Continous-time Optimization Adam Ostaszewski
Lecture times for 2011/12 • Lectures: Monday in TW1.U208 : 4-5 pm and Thursday in NAB.1.14 : 12-1pm • Classes: Wednesday in TW1.U101: 6-7pm … starting Week 1 all in Michaelmas Term • Plus: Revision in Summer Term!
Q: What is this course about?A: How to … • Select a curve x(t) for 0 < t < T • … subject to constraints holding over some or over all time t • To minimize (maximize) some performance index
About the course…geodesics &other performance index • E.g. … subject to x(0) being given, and also x(T) • …or x(t) must lie on some fixed surface • E.g. is a geodesic = minimizes distance along the given fixed surface between initial and terminal positions
About the course … time optimality • E.g. … subject to x(0) being given and maybe x(T) only to lie on a specified surface • …or x(t) must lie on some varying surface • E.g. minimize time taken to reach the prescribed surface: T
Notes: • The variable x(t) may be in Rn • The constraint may be f(x(t)) = 0 • … but could be a differential equation like this ‘canonical’ one: x′(t)=a(t,x(t)) • Where dash is defined by: x′=dx/dt • …so dx = a(t,x(t)) dt
State equation in differential form (first order) • … Or even like x′(t)=a(t,x(t),u(t)) • here u(t) is to be selected and is known as a control (costly, usually!) • (Think: selecting car speed with car constrained to a particular road.)
A state equation in vector differential form – first order • …indeed … despite Newton’s law … • x′′ = u (accn = force) • Rewritten, with x1=x, and with x2=x1′, • …as the linear matrix first-order system x1′ = x2, x2′ = u
…or in stochastic form • … could be a ‘stochastic’ differential equation • dx(t)=a(t,x(t),u(t)) dt + b(t,x(t),u(t)) dWt • ..where for each t the term Wt represents a continuously varying standard `random variable’, modelling uncertainty
… risky assets • Model of asset price evolution St ‘Instantaneous’ return on the asset over the time interval dt (from t to t+dt ) is defined to be: • (St+dt - St)/St , or in compact form = dSt/ St
… Black-Scholes model for asset price St • Return on the asset modelled by dSt / St = dt + dWt • …anticipated ‘rate’ of return • …PLUS ‘volatility factor’ times a standard volatility term, namely dWt … • nb dWt = (Wt+dt - Wt) the increments are independent, normal, with variance dt
Format of Performance Index:Deterministic Case • Bequest/gift: g(x(T)) – a Meyer problem • Running cost format: 0T L(x(t),…)dt – a Lagrange problem • Mix of above: g(x(T)) + 0T L(x(t),…) dt – a Bolza problem
…Actually they’re equivalent • …proved by fiddling with the number,n, of variables, e.g. introduce xn+1 and adjoin • … a diff. eqn.: dxn+1 = L(x(t),…)dt • … initial condition: xn+1(0) = 0 • Then g(x(T))= xn+1 (T) = 0T L(x(t),…)dt
.. Stochastic caseExpected performance • Thus might be • E[g(x(T))] – a Meyer problem … • So we can solve the investor’s optimal consumption problem
Expected performance • At what rate to consume wealth, at what rate to invest in a risky asset and how much to salt away in a safe bank deposit • Max Ew[0T exp(-rt)U(u(t))]dt • maximize expected utility of lifetime consumption, where u(t) =fraction of wealth x(t) consumed and x(0)=w.
Note the implicit ‘stopping time’ • …in the upper limit of integration • Max Ew[0T exp(-rt)U(u(t))] dt • T denotes the first time t when x(t) = 0, so T may be a special sort of random variable
Solution technique • Optimality as with real-variables where • Max f(x) • is solved using the differential condition: df(x) = 0, the ‘first order condition’ (f.o.c.) • We develop a higher level calculus, which converts df(x) = 0 to …a differential eqn.
…f.o.c. leads to a …differential equation …satisfied by the optimal trajectory (curve) • And so the technique amounts to setting up the problem (modelling) and then solving the d.e.
Reading List • To begin with we use A.O.’s “Advanced Mathematical Methods” • Notes will be provided throughout the course by way of Moodle (& the maths web-site)
Literature – Mathematical Basics • (♥) Troutman, Variational Calculus, Springer • (♥) Pinch, Optimal Control and the Calculus of Variations, OUP • Fleming & Rishel, Deterministic & Stochastic Control, Springer • (♥♥) Burghes & Graham, Control & Optimal Control Theories with Applications, Horwood.
Mathematical Literature - Advanced • Fleming & Soner, Controlled Markov Processes and Viscosity Solutions, Springer • Oksendal & Sulem, Applied Stochastic Control of Jump Diffusions,Springer • Vinter, Optimal Control, Birkhauser
Literature – Very Advanced • Rigorous Mathematical Background …well beyond our course: • Rogers & Williams, Diffusions, Markov Processes and Martingales: Volumes 1& 2, Itô Calculus, (Cambridge Mathematical Library) • But you should know it’s there
P.S. My secret place: http://www.maths.lse.ac.uk/Courses/MA409/ This also yields access to my course materials
