Non Linear Regression Y i = f( b, x i ) + e i

1. Non Linear RegressionYi = f(b,xi) + ei Marco Lattuada Swiss Federal Institute of Technology - ETH Institut für Chemie und Bioingenieurwissenschaften ETH Hönggerberg/ HCI F135 – Zürich (Switzerland) E-mail: lattuada@chem.ethz.ch http://www.morbidelli-group.ethz.ch/education/index

2. Puromycin Description: Puromycin is an antibiotic used by scientists in bio-research to select cells modified by genetic engineering. Mechanism of action: This is described by the Michaelis-Menten model for enzyme kinetics, which relates the initial velocity on an enzymatic reaction to the substrate concentration x trough the equation: Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 2

3. The model: Puromycin Kinetics Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 3

4. Model Linearization • Puromycin Kinetics: • Model Rearrangement: • Linearized Model: Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 4

5. Regression Line b1 = 0.0051072 b2 = 0.00024722 Model Linearization Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 5

6. Model Linearization Regression from linearized model q1 = 195.8 q2 = 0.048407 Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 6

7. q1 The model: Linearized model is needed to estimate q2 q1 Puromycin Kinetics Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 7

8. Nonlinear Regression • Object • To minimize the objective function • where n is the number of observations, yi the responses, xi is the vector of the observations, q the vector of the parameters and f(xi,q) the nonlinear model function. • It is possible to plot the objective function S(q) as a function of the parameter values, in order to reveal the presence of a minimum. Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 8

9. Minimum Estimated value of q from linearization Objective Function S(q) • Contour plot of S(q) Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 9

10. Minimization of S(q) • Model linearization: • where: • so the residuals are: • Search for minimum with Gauss-Newton method: J0 = Jacobian Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 10

11. Gauss-Newton Method Applied to S(q) Convergence path of Gauss-Newton Method (q1)opt = 212.66 (q2)opt = 0.064091 Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 11

12. Nonlinear Regression Nonlinear Regression Regression from linearized model Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 12

13. Ellipsoidal Confidence Region • The ellipsoidal confidence region can be evaluate from the linearized model around the point , which is the vector of the parameters for which the objective function has a minimum. • In practice, every vector of the parameters q which satisfies the following condition: • is within the confidence interval, where n is the number of observations, p the number of parameters and s the standard deviation: Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 13

14. a Ellipsoidal Confidence Region Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 14

15. True Confidence Region for Parameters • The real confidence region can be estimated by plotting the region of space for which: Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 15

16. Matlab Nonlinear Regression Routine • First, create a function providing the residuals for the n observation as a function of the parameter values: • Then, use the routine 'nlinfit'; Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 16

17. Tukey-Ancombe Plot Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 17

18. Normal Plot >> normplot(r) Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 18

19. Matlab Estimation of Parameter CI • Parameter confidence interval can be estimated by Matlab as follows: • The confidence interval can be estimated using the following Matlab GUI: Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 19

20. t-Profiles Goal: verify weather one parameter can have a certain value, as compared to the optimum value Method: let be qk the parameter to be investigated. Let us fix one value of qk, and then optimize for all the other parameters. Let us then compute: where Qk is the set of Q values with qkkept constant. The probability distribution of t(qk) is a t-Student distribution with n-p degrees of freedom Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 20

21. t-Profiles Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 21

22. t-Profiles Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 22

23. Combined t-Profiles Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 23

24. BOD Regression • Biochemical Oxygen Demand (BOD) refers to the amount of oxygen that would be consumed if all the organics in one liter of water were oxidized by bacteria and protozoa (ReVelle and ReVelle, 1988). • The model considered here is: • The response variable (y) is biochemical oxygen demand in mg/l. The predictor variable (x) is incubation time in days Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 24

25. Expected behavior Raw Data Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 25

26. Minimum Estimated value of q Objective Function S(q) • Contour plot of S(q) Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 26

27. Objective Function S(q) • Contour plot of S(q) Path followed by the Gauss-Newton method Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 27

28. Regressed Model Regressed Experimental Points Experimental Points Regressed Model q1 = 19.1; q2 = 0.531 Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 28

29. Estimated Minimum of S Confidence Region for the Parameters True Confidence Region Confidence Region from Model Linearization Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 29

30. Confidence Region for Response 1-a confidence region for the response Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 30

31. Matlab Regression Regressed Experimental Points by Matlab q1 = 19.1; q2 = 0.531 Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 31

32. Tukey-Ancombe Plot Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 32

33. Normal Probability Plot Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 33

34. t-Profiles Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions – Page # 34