Brian DO Anderson The Australian National University and National ICT Australia Limited

1. Two Decades of Adaptive Control Pitfalls KUL System Identification and Data ModellingLennart Ljung Symposium Brian DO Anderson The Australian National University and National ICT Australia Limited

2. Outline • Adaptive Control • MIT Rule • Bursting • Good Models, Bad Models and Changing the Controller • Multiple Model Approach to Adaptive Control • Conclusions Lennart Ljung Symposium Oct 2004

3. Adaptive Control Disturbance d Controller Plant + Output Input Reference r y u — ∫ • Plant is initially unknown or partially known, or is slowly varying. • There is an underlying performance index, eg Lennart Ljung Symposium Oct 2004

4. Adaptive Control(continued) Disturbance d Controller Plant + Output Input Reference r y u — • A non-adaptive controller maps the error signal r-y into u in a causal, time-invariant way eg • An adaptive controller is one where parameters are adjusted. Lennart Ljung Symposium Oct 2004

5. One Formof Adaptive Controller Control Law Calculation Identifier Plant Parameters Controller Parameters disturbance Controller Plant u y • Other ways of doing this exist • Often 3 time scales: • Underlying plant dynamics (with fixed parameters) • Time scale for identifying plant • Time scale of plant parameter variation Lennart Ljung Symposium Oct 2004

6. Outline • Adaptive Control • MIT Rule • Bursting • Good Models, Bad Models and Changing the Controller • Multiple Model Approach to Adaptive Control • Conclusions Lennart Ljung Symposium Oct 2004

7. MIT Rule Problem kc(t) kpZp(s) yp(t) r(t) e(t) kmZp(s) ym(t) + - k m(t) • Zp(s) is known, kmis known, k pis positive and unknown, but kc(t) is known and adjustable • Problem is to find a rule using e(t) to adjust kc(t) to cause e(t) to go to zero • Problem source: k pdepends on dynamic air pressure for aircraft. Lennart Ljung Symposium Oct 2004

8. MIT Rule Intuition and Performance • Equivalently, • Use gradient descent to try to drive e(t) to 0: with g a gain constant • Sometimes this worked, sometimes it did not work. • Why? Lennart Ljung Symposium Oct 2004

9. Example of performance g • Unshaded region • is stable • Sine wave input • at frequency  • Plant is (s+1)-1  Lennart Ljung Symposium Oct 2004

10. Explaining Instability • High (adaptive) gain instability for some Zp(s) : consider a constant input R to display phenomenon. The MIT rule leads to a characteristic equation; high g may give RHP zero zero: From Plant Dynamics s + gkmkpR2Zp(s) = 0 From derivative of kc • Underlying differential equation for kc is Mathieu equation. Solution regions of this equation are depicted. • One instability mechanism is interaction of excited plant dynamics with adaptive dynamics, made worse at high gain g Lennart Ljung Symposium Oct 2004

11. Explaining Instability II kc(t) kpZp(s) yp(t) r(t) e(t) ym(t) + - kmZm(s) k m(t) • Zm(s) is known, kmis known, k pis positive and unknown, but kc(t) is known and adjustable • A second instability mechanism comes from modelling errors, here errors between Zp(s) and Zm(s) • Following two figures show case where plant and model are the sameand where they are different. Lennart Ljung Symposium Oct 2004

12. Example of performance g • Unshaded region • is stable • Sine wave input • at frequency  • Plant and model • are (s+1)-1  Lennart Ljung Symposium Oct 2004

13. Performance: another example g • Unshaded region • is stable • Sine wave input • at frequency  • Plant is e-s(s+1)-1 • while model is still • (s+1)-1  Lennart Ljung Symposium Oct 2004

14. Rescuing the MIT Rule: Averaging Zp(s)kpkc(t)r(t) • Averaging theory is the general analysis tool usable given separation of time scales of the plant dynamics and the learning/adaptation rate • Averaging theory treats kc slowly-varying : kc* is approximately kc for small g where . kc*=-g{Zp(s)kp r(t)}{Zm(s)kmr(t)} kc* + terms indep of kc* • Stability is ensured if the average value of {Zp(s)kp r(t)}{Zm(s)kmr(t)} is positive--and if Zp is like Zmat frequencies where r(t) is concentrated, then stability is achieved. Lennart Ljung Symposium Oct 2004

15. Performance: another example g • Unshaded region • is stable • Sine wave input • at frequency  • Plant is e-s(s+1)-1 is • while model is still • (s+1)-1  Lennart Ljung Symposium Oct 2004

16. Rescuing the MIT Rule: Averaging • Lennart Ljung used averaging when he explained how to analyse the behaviour of a discrete time adaptive algorithm with the aid of an ordinary differential equation • The adaptation rate became slower and slower as time evolved--achieving the time scale separation. Lennart Ljung Symposium Oct 2004

17. Outline • Adaptive Control • MIT Rule • Bursting • Good Models, Bad Models and Changing the Controller • Multiple Model Approach to Adaptive Control • Conclusions Lennart Ljung Symposium Oct 2004

18. Bursting Phenomenon Lennart Ljung Symposium Oct 2004

19. Bursting Phenomenon (continued) u(t) y(t) • Bursting phenomena were seen in an experimental adaptive control system - sometimes after 1 week of successful operation • Why do they occur? How could they be stopped? • From measurements of u(•), y(•), one should be able to identify b and c • If u = constant, can only identify b/c-the DC gain • Adaptive controllers contain adaptive identifier of b and c Lennart Ljung Symposium Oct 2004

20. Bursting Phenomenon (Explanation) • Identification process is robust if T such that for all s and some positive 1, 2, regression vector (t) obeys:  • Control law is designed based on estimates of b,c. Hence could accidentally implement unstable closed loop. • Instability then enriches the signals, giving improved identification. •  normally involves inputs and outputs. Need to convert to input-only condition Lennart Ljung Symposium Oct 2004

21. Rich Excitation • If there are p scalar parameters to be identified, input needs to have a complexity related to p: (p/2 sinusoidal frequencies). • Practical issue: unless adaptation is turned off, must drive the system with “rich” input. [Some algorithms turn adaptation off at 1/t rate] Lennart Ljung Symposium Oct 2004

22. Outline • Adaptive Control • MIT Rule • Bursting • Good Models, Bad Models and Changing the Controller • Multiple Model Approach to Adaptive Control • Conclusions Lennart Ljung Symposium Oct 2004

23. Good Models, Bad Models and Changing the Controller • In adaptive control, at each time instant • There is a model of the plant (which may be a good model) • There is a certain controller attached to the plant • If the plant model is a good one, a simulation of the model and controller will perform like the actual plant and controller • In adaptive control • The controller may be changed to better reflect a control objective • The calculation of the new controller is based on the current model--applying with the current controller Lennart Ljung Symposium Oct 2004

24. Good Models, Bad Models and Changing the Controller (continued) • This presents a fundamental challenge in adaptive control • Consider: True plant: Model: [Transfer functions are and ] Lennart Ljung Symposium Oct 2004

25. Good Models, Bad Models and Changing the Controller (continued) Similar open-loop behaviours: and open-loop closed-loop Lennart Ljung Symposium Oct 2004

26. Good Models, Bad Models and Changing the Controller (continued) Different open-loop behaviours: and Plants in open-loop Plants in closed-loop with Lennart Ljung Symposium Oct 2004

27. Good Models, Bad Models and Changing the Controller (continued) • Moral: changing the controller may turn a good model into a bad one, or vice versa • Changing the controller is like changing the experimental condition--and Lennart Ljung always told us to watch the experimental conditions! • “Goodness of fit of a model” is a term which only makes sense for a particular set of experimental conditions OR Don’t overgeneralise what you have learnt • If you change the controller significantly, you might produce instability with the real plant, while it works fine with the model (=estimate of plant) Lennart Ljung Symposium Oct 2004

28. Iterative Identification and Controller Redesign • A frequently advanced approach to adaptive control design is iterative identification and controller redesign. • One iteration comprises • (re) identifying the plant with the current controller • redesigning the controller to achieve the design objective on the basis of the identified model, and implementing it on the real plant This can lead to instability! • One needs algorithms which will move performance with the model and the new controller towards the design objective--but not change the controller too much. • Same issue for IFT, VRFT  Safe adaptive control. Lennart Ljung Symposium Oct 2004

29. Outline • Adaptive Control • MIT Rule • Bursting • Good Models, Bad Models and Changing the Controller • Multiple Model Approach to Adaptive Control • Conclusions Lennart Ljung Symposium Oct 2004

30. Multiple Model Adaptive Control • Imagine a bus on a city street. The equations of motion of the bus have parameters depending on • the load • The friction between tyres and road • Many plants have equations in which a (frequently small) number of physically-originating parameters are changeable/unknown. Call such a plant p(). Here  = physical parameter vector • Learning  from measurements with an equation of the form may be too hard, especially for nonlinear plants Lennart Ljung Symposium Oct 2004

31. Multiple Model Adaptive Control (continued) • An alternative approach (MMAC) is as follows: • Suppose that the unknown parameter  lies in a bounded simply connected region. Call the unknown plant . • Choose a set of values in this region, with associated plants P1,.......,PN. • Design (in advance) nice controllers for P1,.......,PN. • Call them C1,......,CN . • Run an algorithm which at any instant of time estimates (via the measurements) the particular Pi which is the best model to explain the measurements from . Call the associated parameter • Connect up Lennart Ljung Symposium Oct 2004

32. Multiple Model Adaptive Control(continued) Supervisor Supervisor noise reference + + Controller i u Unknown or + y partially known input - Plant P • Supervisor studies effect of using present controller and decides whether or not to switch controller • Desirable outcome: after a finite number of switchings, the best controller for the plant is obtained. Controller i Unknown or Partially known Plant P Lennart Ljung Symposium Oct 2004

33. Why the name “multiple model”? • Underlying precept is that the plant coincides with or is near one of N nominal plants P1,.......,PN • Controller i, denoted Ci, is a good controller for Pi(and possibly plants “near”Pi) Lennart Ljung Symposium Oct 2004

34. Deciding the Best Model Pi for P Multi- estimator u y1 Multi- y estimator yN + r + Controller k Plant P - • Multiestimator is a device which produces N outputs • if (and only if with complicated signals) (The controller is irrelevant) _ Plant P Controller k Lennart Ljung Symposium Oct 2004

35. Early Approach to Supervision: Using Multiestimator P I C I u Multi- estimator Multi- y1 estimator y yN r + Controller J Plant P - • Idea of algorithm: study for some small a > 0, and k=1,…,N. If the smallest occurs for k = I, say that P is best modelled by Switch in Controller J _ Plant P ∫  This may lead to switching in a destabilising controller! Lennart Ljung Symposium Oct 2004

36. Example • Plant is 3rd order, stable, with non-minimum phase zero in [1,10] and DC gain in [.2,2]. • Control objective is to extend bandwidth beyond open loop plant, with closed loop transfer function close to 1 in magnitude. Non-minimum phase zero is a limiter. • 441 plant models chosen, with DC gain and non-minimum phase zero each in 20 logarithmically space intervals • Reference signal is wideband noise • Measurement noise and process (input) noise are present Lennart Ljung Symposium Oct 2004

37. Example of Temporary Instability Figure 7a: Example of Temporary Instability (without safety) Lennart Ljung Symposium Oct 2004

38. Example of Temporary Instability Figure 7b: Example of Temporary Instability (without safety) Lennart Ljung Symposium Oct 2004

39. Multiple Model Adaptive Control-Difficulties • How can one avoid the instabilities? • Should there be 7, 70 or 7000 models? How should one actually choose the models? These questions are actually linked. Lennart Ljung Symposium Oct 2004

40. Choosing the Multiple Models • How does one choose N? How does one choose 1,.........., N?  Idea of solution: Pick 1 . Design C1 for P( 1). Figure the plant set P() around P1 = P( 1) such that C1 is a good controller. Pick 2 near the boundary. Figure the plant set P() around P2 = P( 2) such that C2 is a good controller. Pick 3 near the boundary of union of these two sets, etc. • The set is then covered by a set of balls indexed by 1,.........., N and this determines N. Metrics (Vinnicombe) help with this in the linear case Lennart Ljung Symposium Oct 2004

41. Safe Switching • The index of the best model of P (out of P1,........PN) with controller CJ connected is NOT NECESSARILY the index of the best controller to connect to P. • Even if PI is the best model of P when CJ is connected, CI may not be in the safe set of {CK } . Only switch if it is safe. • Difficulty is that P may be best modelled by PI when CJ is connected, but may be best modelled by PKwhen CI is connected. PI may be a terrible model of P when CI is connected. • Nontrivial fact: using crude estimation techniques one can obtain set of controllers {CK } which, when used to replace CJ, are guaranteed to retain stability (and even retain similarity of performance). Vinnicombe metric is used. Lennart Ljung Symposium Oct 2004

42. OVERVIEW OF RESULTS • With safety constraint, controller switching is less frequent, convergence to the “best” controller was slower. • With safety constraint, performance could be poor but never unstable. • Without safety constraint, most runs exhibited poor performance, some yielded instability • With use of possibly more nominal plants and controllers, one can probably get “performance safety” as well as “stability safety” Lennart Ljung Symposium Oct 2004

43. Safe Controller Switching Figure 3a:(Safe) Controller Switching Lennart Ljung Symposium Oct 2004

44. Safe Controller Switching Figure 3b:(Safe) Controller Switching Lennart Ljung Symposium Oct 2004

45. Outline • Adaptive Control • MIT Rule • Bursting • Good Models, Bad Models and Changing the Controller • Multiple Model Approach to Adaptive Control • Conclusions Lennart Ljung Symposium Oct 2004

46. Conclusions • Keeping adaptation and plant time scales different is good practice • Modelling as well as you can is a good idea--even with an adaptation capability. • Having lots more parameters than you need could be dangerous • Bursting • Satisfactory learning occurs only for a limited set of experimental conditions • If you want to be able to keep learning (accurately) , you need to continue excitation Lennart Ljung Symposium Oct 2004

47. Conclusions (continued) • A good model is only good for a particular set of experimental conditions. If you change the controller, it may cease to be good. • Picking representative models from an infinite set can often be done scientifically • Abrupt changes of a controller can introduce instability -even if on the basis of having a good model, the new controller looks good. • Safe adaptive control should be contemplated--to avoid temporary connection of a controller which can destabilise the (unknown) plant • Need Vinnicombe metric ideas for nonlinear problems Lennart Ljung Symposium Oct 2004