Extremum seeking control

1. Extremum seeking control Dragan Nešić The University of Melbourne Acknowledgements: Y. Tan, I. Mareels, A. Astolfi, G. Bastin, C. Manzie; A. Mohammadi; W. Moas Australian Research Council. TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAAA

2. Outline • Motivating examples • Background • Ad hoc designs • Black box: • Problem formulation • Systematic design • Gray box: • Problem formulation • Systematic design • Conclusions & future directions

3. A Prelude • This is an approach for online optimisation of the steady-state system behaviour. • A standing assumption is that the plant model or the cost is not known. • The controller finds the extremum in closed-loop fashion.

4. Motivating examples

5. Continuously Stirred Tank (CST) Reactor u=Vol. flow rate Performance output y: Productivity JP Yield JY Product Substrate Overall JT Inflow Outflow

6. Single enzymatic reactionMichaelis-Menten Kinetics Productivity and yield Total cost is typically unknown!! In steady-state, we would typically want to operate around G. Bastin, D. Nešić, Y. Tan and I. Mareels, “On extremum seeking in bioprocesses with multivalued cost functions”, Biotechnology Progress, 2009.

7. Raman amplifiers p Fibre span ¸ u=laser power • Performance output y: • Spectral flatness • (equalization) • Desired power Power sensors Pump lasers Cost P.M. Dower, P. Farrell and D. Nešić, “Extremum seeking control of cascaded optical Raman amplifiers”, IEEE Trans. Contr. Syst. Tech., 2008.

8. Other engineering examples

9. Examples from biology • E. Coli bacteria search for food in a similar manner to an extremum seeking algorithm (M. Krstic et al). • Some fish search for food in a similar manner to extremum seeking (M. Krstic et al).

10. Background

11. Classification of approaches Stochastic Deterministic Adaptive ESC [Krstić, Manzie,…] Adaptive ESC [Krstić, Ariyur, Guay, Tan, Nešić,…] NLP based ESC [Spall,..] NLP based ESC [Popović, Teel,…] Also continuous-time versus discrete-time.

12. Brief history (deterministic): Åström & Wittenmark: “one of the most promising adaptive control techniques”. First local stability result for adaptive ESC Vibrant research area Many new schemes proposed Especially Adaptive Systematic design discrete-time NLP. Systematic design adaptive Schemes. First ESC? 2000 1922 1950 1960 1970 2009 1995 Beginning Ad-hoc designs Rigorous analysis and design

13. Ad hoc adaptive designs

14. Adaptive ESC [Krstić & Wang 2000] Parameters: + x Extremum seeking controller

15. Static scalar case (gradient descent) Parameters: + x Extremum seeking controller Y. Tan, D. Nešić and I. Mareels, “On non-local stability properties of extremum seeking control”, Automatica, 2006.

16. Comments • Many similar adaptive algorithms proposed. • Case-by-case convergence analysis. • No clear relationship with optimization. • A unifying design approach is unavailable. • A unifying convergence analysis is missing. • A unifying approach exists for another class of schemes [Teel and Popovic, 2000].

17. Black Box Approach

18. Problem formulation (black box) • Assumption 1: • - Q(.) has an extremum (max) • - Q(.) is unknown Dynamic case: Extremum Seeking Controller Problem: Design ESC so that Assumption:

19. Systematic design(derivatives estimation) D. Nešić, Y. Tan, W. Moas and C. Manzie, “A unifying approach to extremum seeking: adaptive schemes based on derivatives estimation”, IEEE Conf. Dec. Contr. 2010.

20. Continuous optimization (offline) • No inputs & outputs • Q(.) is known, so all • derivatives of Q(.) • are known

21. Examples • Gradient method • Continuous Newton method

22. Extremum seeking (online) • Inputs & outputs • available • Q(.) is unknown • a, !L, ² are • positive controller • parameters Derivatives estimator

23. Systematic design (use the previous block diagram) • Step 1: Choose an optimization scheme. • Step 2: Use an estimator for DN Q(¢). • Step 3: Adjust the controller parameters.

24. Estimator design

25. Estimating DQ(µ)

26. Analysis Model of the system: where µ is assumed constant. Average the right hand side of the model.

27. Estimating D2Q(µ)

28. Higher order derivatives

29. Convergence analysis

30. Model of the overall system Slow: Fast: !L, ² and a are controller parameters that need to be tuned to achieve appropriate convergence properties.

31. Assumption 1 (global max) • There exists a global maximum

32. Assumption 2 (robust optimizer) • The solutions of satisfy for sufficiently small w(t).

33. Theorem • Suppose Assumptions 1-2 hold. Then Tuning guidelines

34. Geometrical interpretation For any ¢, º Fast transient (estimator) Exist !L, ², a Slow transient optimization

35. Comments • A systematic design approach proposed. • Rigorous convergence analysis provided. • Controller tuning proposed in general. • Dynamic plants treated in the same way. • Multi-input case is treated in a similar way. • Averaging and singular perturbations used. • Tradeoffs between the domain of attraction, accuracy and speed of convergence!

36. Bioreactor example All our assumptions hold – gradient method used.

37. Gray Box Approach

38. Problem formulation (gray box) • Assumption 1: • - Q(.,p) has an extremum (max) • - Q(.,.) is known; p is unknown Dynamic case: Extremum Seeking Controller Problem: Design ESC so that Assumption:

39. Systematic design(parameter estimation) D. Nešić, A. Mohamadi and C. Manzie, “A unifying approach to extremum seeking: adaptive schemes based on parameter estimation”, IEEE Conf. Dec. Contr. 2010.

40. Extremum seeking (online) • Inputs & outputs • available • p is unknown • a, !L, ² are • controller • parameters Parameter estimator

41. Comments • Similar systematic framework in this case. • Similar convergence analysis holds. • Classical adaptive parameter estimation schemes can be used. • Dynamic plants dealt with in the same way. • Persistence of excitation is crucial for convergence. • Tradeoffs between domain of attraction, accuracy and convergence speed.

42. Example • Consider the static plant: • We used the continuous Newton method. • Classical parameter estimation used. • Values p1=9 and p2=8 used in simulations.

43. Simulations Performance output Control inputs Parameters

44. Final remarks • Several tradeoffs exist; convergence slow. • Many degrees of freedom: dither shape, controller parameters, optimization algorithm, estimators. • Some global convergence results available (similar to simulated annealing).

45. Summary • A systematic design framework presented for two classes of adaptive control schemes. • Precise convergence analysis provided. • Controller tuning and various tradeoffs understood well. • Applicable to a range of engineering and non-engineering fields.

46. Future directions • Tradeoffs: convergence speed, domain of attraction and accuracy. • Various extensions: non-compact sets, global results, non-smooth systems, multi-valued cost functions. • Schemes robust although no formal proofs. • Tailor the tools to specific problems. • Exciting research area.

47. Thank you!