Bruce Mayer, PE Registered Electrical & Mechanical Engineer BMayer@ChabotCollege

1. Engr/Math/Physics 25 Chp6 MATLABRegression Bruce Mayer, PE Registered Electrical & Mechanical EngineerBMayer@ChabotCollege.edu

2. Learning Goals cont • Use Regression Analysis as quantified by the “Least Squares” Method • Calculate • Sum-of-Squared Errors (SSE or J) • The Squared Errors are Called “Residuals” • “Best Fit” Coefficients • Sum-of-Squares About the Mean (SSM or S) • Coefficient of Determination (r2) • Scale Data if Needed • Creates more meaningful spacing

3. Learning Goals cont • Build Math Models for Physical Data using “nth” Degree Polynomials • Use MATLAB’s “Basic Fitting” Utility to find Math models for Plotted Data

4. Scatter on plots on XY-Plane • A scatter plot usually shows how a EXPLANATORY, or independent, variable affects a RESPONSE, or Dependent Variable • Sometimes the SHAPE of the scatter reveals a relationship • Shown Below is a Conceptual Scatter plot that could Relate the RESPONSE to some EXCITITATION

5. Linear Fit by Guessing • The previous plot looks sort of Linear • We could use a Ruler to draw a y = mx+bline thru the data • But • which Line is BETTER? • and WHY?

6. In a Previous Example polyfit(x,y,1) returned the Values of m & b How does PolyFitMake these Calcs? How Good is the fitted Line Compared to the Data? polyfit, as do most other curve fitters, uses the “Least Squares” Criterion Least Squares Curve Fitting

7. To make a Good Fit, MINIMIZE the |GUESS − data|distance by one of Least Squares Best Guess data

8. MATLAB polyfit Minimizes the VERTICAL distances; i.e.: Least Squares cont • Note that The Function J contains two Variables; m & b • Recall from MTH1 that to MINIMIZE a Function set the 1st (partial) Derivative(s) to Zero

9. To Minimize J, take Least Squares cont • The Two Partials yield Two LINEAR Eqns in m & b • The two eqns can be solved EXACTLY for m & b • the Book on pg 271 gives a good example • Remember, at this point m & b are UNKNOWN

10. In This Case Least Squares cont • Solving theseEqns for m & b yields • m = 9/10 • b = 11/6 • This produces the best fit Line • Taking ∂J/∂m = 0, and ∂J/∂b = 0 yields

11. The Distance from The Best-Fit Line to the Actual Data Point is called the RESIDUAL For the Vertical Distance the Residual is just δy Goodness of Fit • If the Sum of the Residuals were ZERO, then the Line would Fit Perfectly • Thus J, afterfinding m & b, is an Indication of the Goodness of Fit

12. Now J is an indication of Fit, but we Might want to SCALE it relative to the MAGNITUDE of the Data For example consider DataSet1 with x&y values in the MILLIONS DataSet2 with x&y values in the single digits Goodness of Fit cont • In this case we would expect J1 >> J2 • To remove the affect of Absolute Magnitude, Scale J against the Data Set mean; e.g • mean1 = 730 000 • mean2 = 4.91

13. The Mean-Scaling Quantity is the Actual-Data Relative to the Actual-Mean Goodness of Fit cont • Finally the Scaled Fit-Metric, “r-squared’ • As before the Squaring Ensures that all Terms in the sum are POSITIVE

14. The r2 Value is Also Called the COEFFICIENT OF DETERMINATION r2 = Coeff of Determination • J  Sum of Residual (errors) • May be Zero or Positive • S  Data-to-Mean Scaling Factor • Always Positive if >1 Data-Pt and data not “perfectly Horizontal” • If J = 0, then there is NO Distance Between the calculated Line and Data • Thus if J = 0, then r2 = 1; so r2 = 1 (or 100%)indicates a PERFECT FIT

15. The COEFFICIENT OF DETERMINATION Meaning of r2 • Has This Meaning The coefficient of determination tells you what proportion of the variation between the data points is explained or accounted for by the best line fitted to the points. It indicates how close the points are to the line.

16. Norm of Residuals • MATLAB uses the Norm of Residuals as a Measure of Goodness of Fit • The Norm of Residuals, NR, is simply the SqRt of J: • Thus r2 in Terms of NR: • As a Measure of Goodness of Fit as the FIT Approaches PerfectionJ→0 so:

17. NR vs r2 → • Notice that r2 is a RELATIVE Measure → it’s NORMALIZED to the WORST CASE Value of J which is S • Thus r2 can be expressed at PERCENTAGE withOUT Units • NR is an ABSOLUTE measure that Technically Requires that it be stated with SAME UNITS as the dependent variable, y

18. Data-Scaling is a SubTopic of DIMENSIONAL ANALYIS (DA) DA is Covered in 3rd Yr ME/CE Courses I Learned it in a Fluid Mechanics Course For Our Purposes we will cover only SCALING Sometimes we Collect Data with a SMALL Variation Relative to the Magnitude of the MEAN Leads to a SENSITIVE Analysis; e.g. This Data is Noisy During Analysis Data Scaling

19. The Significance of ANY Data Set Can be Improved by Normalizing Normalize  Scale Data such that the Values run: 0 →1 0% → 100% Steps to Normalization Find the MAX & MIN values in the Data Set; e.g., zmax & zmin Calculate the Data Range, RD RD = (zmax – zmin) Calc the Individual Data Differences Relative to the MIN Δzk = zk - zmin Data Scaling - Normalization

20. Finally, Scale the Δzk relative to RD Ψk = Δzk /RD Scale the corresponding “y” values in the Same Manner to produce say, Φk Plot Φk vs Ψk on x & y scales that Run from 0→1 Example– Do Frogs Croak More on WARM Nites? Data Scaling – Normailzation cont

21. Normalize T → Θ CPH → Ω Normalization Example • Now Compare Plots • CPH vs T • Ω vs Θ

22. T-CPH Plot Ω-Θ Plot Plots Compared • The Θ-Ω Plot Fully Utilizes Both Axes

23. Use MATLAB’s AutoMatic Fitting Utility to Find The Best Line for the the Frog Croaking Data Basic Fitting Time For Live Demo SEE: Demo_Chp5_Croak_0507.m

24. All Done for Today CroakingFrog

25. Engr/Math/Physics 25 Appendix Time For Live Demo Bruce Mayer, PE Licensed Electrical & Mechanical EngineerBMayer@ChabotCollege.edu

26. Linear Regression Tutorial • Minimize Height Error δy • See File ENGR-25_Linear_Regression_Tutorial_1003.ppt

27. The Area of RIGHT Triangle Altitude of Right Triangle • The Area of an ARBITRARY Triangle • By Pythagoras for Rt-Triangle

28. Altitude of Right Triangle cont • Equating the A=½·Base·Hgt noting that • Solving for h Q.E.D.

29. Normalized Plot >> T = [69.4, 69.7, 71.6, 75.2, 76.3, 78.6, 80.6, 80.6, 82, 82.6, 83.3, 83.5, 84.3, 88.6, 93.3]; >> CPH = [15.4, 14.7, 16, 15.5, 14.1, 15, 17.1, 16, 17.1, 17.2, 16.2, 17, 18.4, 20, 19.8]; >> Tmax = max(T); >> Tmin = min(T); >> CPHmax = max(CPH); >> CPHmin = min(CPH); >> Rtemp = Tmax - Tmin; >> Rcroak = CPHmax - CPHmin; >> DelT = T - Tmin; >> DelCPH = CPH - CPHmin; >> Theta = DelT/Rtemp;DelCPH = CPH - CPHmin; >> Omega = DelCPH/Rcroak; >> plot(T, CPH,), grid >> plot(Theta,Omega), grid

30. FIRST → Plot the DATA Start Basic Fitting Interface 1

31. Start Basic Fitting Interface 2 Goodness of Fit; smaller is Better Expand Dialog Box

32. Result Chk by polyfit Start Basic Fitting Interface 3 >> p = polyfit(Theta,Omega,1) p = 0.8737 0.0429 

33. Caveat

34. Greek Letters in Plots

35. Plot “Discoverables”

36. % "Discoverable" Functions Displayed % Bruce Mayer, PE • ENGR25 • 15Jul09 % x = linspace(-5, 5); ye = exp(x); ypp = x.^2; ypm = x.^(-2); % plot all 3 on a single graphe plot(x,ye, x,ypp, x,ypm),grid,legend('ye', 'ypp', 'ypm') disp('Showing MultiGraph Plot - Hit ANY KEY to continue') pause % % PLot Side-by-Side subplot(1,3,1) plot(x,ye), grid subplot(1,3,2) plot(x,ypp), grid subplot(1,3,3) plot(x,ypm), grid