1 / 53

Week 11 November 10-14

Week 11 November 10-14. Four Mini-Lectures QMM 510 Fall 2014 . Chapter Learning Objectives LO12-1: Calculate and test a correlation coefficient for significance . LO12-2: Interpret the slope and intercept of a regression equation.

janine
Télécharger la présentation

Week 11 November 10-14

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Week 11 November 10-14 Four Mini-Lectures QMM 510 Fall 2014

  2. Chapter Learning Objectives LO12-1: Calculate and test a correlation coefficient for significance. LO12-2: Interpret the slope and intercept of a regression equation. LO12-3: Make a prediction for a given x value using a regression equation. LO12-4: Fit a simple regression on an Excel scatter plot. LO12-5: Calculate and interpret confidence intervals for regression coefficients. LO12-6: Test hypotheses about the slope and intercept by using t tests. LO12-7: Perform regression analysis with Excel or other software. LO12-8: Interpret the standard error, R2, ANOVA table, and F test. LO12-9: Distinguish between confidence and prediction intervals for Y. LO12-10:Test residuals for violations of regression assumptions. LO12-11:Identify unusual residuals and high-leverage observations. Chapter 12: Correlation and Regression Chapter 12 Too much?

  3. Correlation Analysis ML 11.1 Chapter 12 Visual Displays • Begin the analysis of bivariate data (i.e., two variables) with a scatter plot. • A scatter plot: - displays each observed data pair (xi, yi) as a dot on an X / Y grid.- indicates visually the strength of the relationship between X and Y Sample Scatter Plot

  4. Correlation Analysis Chapter 12 Strong Positive Correlation Weak Positive Correlation Weak Negative Correlation Strong Negative Correlation Nonlinear Relation No Correlation Note: r is an estimate of the population correlation coefficient r (rho).

  5. Correlation Analysis Chapter 12 Steps in Testing if  = 0 (population correlation = 0) • Step 1: State the HypothesesH0: = 0H1: ≠ 0 • Step 2: Specify the Decision RuleFor degrees of freedom d.f. = n 2, look up the critical value ta in Appendix D or Excel =T.INV.2T(α,df).  for a 2-tailed test • Step 3: Calculate the Test Statistic • Step 4: Make the DecisionIf using the t statistic method, reject H0 if t > ta or if the p-value a. 1 ≤ r ≤ +1 r = 0 indicates no linear relationship

  6. Correlation Analysis Chapter 12 Alternative Method to Test for = 0 • Equivalently, you can calculate the critical value for the correlation coefficient using • This method gives a benchmark for the correlation coefficient. • However, there is no p-value and is inflexible if you change your mind about a. • MegaStatuses this method, giving two-tail critical values for  = 0.05and  = 0.01. Critical values of r for various sample sizes

  7. Simple Regression ML 11.2 Chapter 12 What is Simple Regression? • Simple regression analyzes the relationship between two variables. • It specifies one dependent (response) variable and one independent (predictor) variable. • This hypothesized relationship (in this chapter) will be linear.

  8. Simple Regression Chapter 12 Interpreting an Estimated Regression Equation

  9. Simple Regression Chapter 12 Prediction Using Regression: Examples

  10. Simple Regression Chapter 12 Cause-and-Effect? Can We Make Predictions?

  11. Regression Terminology Chapter 12 Model and Parameters • The assumed model for a linear relationship is y = b0 + b1x + e . • The relationship holds for all pairs (xi, yi). • The error term  is not observable; it is assumed to be independently normally distributed with mean of 0 and standard deviation s. • The unknown parameters are:b0 Interceptb1 Slope

  12. Regression Terminology Chapter 12 Model and Parameters • The fitted model orregression model used to predict the expected value of Y for a given value of X is • The fitted coefficients areb0 Estimated interceptb1 Estimated slope

  13. Regression Terminology Chapter 12 A more precise method is to let Excel calculate the estimates. Enter observations on the independent variable x1, x2, . . ., xn and the dependent variable y1, y2, . . ., yninto separate columns, and let Excel fit the regression equation. Excel will choose the regression coefficients so as to produce a good fit.

  14. Regression Terminology Chapter 12 Scatter plot shows a sample of miles per gallon and horsepower for 15 vehicles. • Slope Interpretation: The slope of 0.0785 says that for each additional unit of engine horsepower, the miles per gallon decreases by 0.0785 mile. This estimated slope is a statistic because a different sample might yield a different estimate of the slope. • Intercept Interpretation: The intercept value of 49.216 suggests that when the engine has no horsepower, the fuel efficiency would be quite high. However, the intercept has little meaning in this case, not only because zero horsepower makes no logical sense, but also because extrapolating to x = 0 is beyond the range of the observed data.

  15. Ordinary Least Squares (OLS) Formulas Chapter 12 OLS Method • The ordinary least squares method (OLS) estimates the slope and intercept of the regression line so that the sum of squared residuals is minimized. • The sum of the residuals = 0. • The sum of the squared residuals is SSE.

  16. Ordinary Least Squares (OLS) Formulas Chapter 12 Slope and Intercept • The OLS estimator for the slope is: • The OLS estimator for the intercept is: Excel function: =SLOPE(YData, XData) Excel function: =INTERCEPT(YData, XData) These formulas are built into Excel.

  17. Ordinary Least Squares (OLS) Formulas Chapter 12 Example: Achievement Test Scores 20 high school students’ achievement exam scores. Note that verbal scores average higher than quant scores (slope exceeds 1 and intercept shifts the line up almost 20 points).

  18. Ordinary Least Squares (OLS) Formulas Chapter 12 Slope and Intercept

  19. Assessing Fit Chapter 12 Assessing Fit • We want to explain the total variation in Y around its mean (SST for total sums of squares). • The regression sum of squares (SSR) is the explained variation in Y.

  20. Assessing Fit Chapter 12 Assessing Fit • The error sum of squares (SSE) is the unexplained variation in Y. • If the fit is good, SSE will be relatively small compared to SST. • A perfect fit is indicated by an SSE = 0. • The magnitude of SSE depends on n and on the units of measurement.

  21. Assessing Fit Chapter 12 Coefficient of Determination • R2 is a measure of relative fit based on a comparison of SSR(explained variation) and SST (total variation). 0 R2 1 • Often expressed as a percent, an R2 = 1 (i.e., 100%) indicates perfect fit. • In simple regression, R2 = r2 where r2 is the squared correlation coefficient).

  22. Assessing Fit Chapter 12 Example: Achievement Test Scores Strong relationship between quant score and verbal score (68 percent of variation explained) R2 = SSR / SST = 387771 / 567053 = .6838 SSR Excel shows the sums needed to calculate R2. SST

  23. Tests for Significance Chapter 12 Standard Error of Regression • The standard error (se) is an overall measure of model fit. Excel’s Data Analysis > Regression calculates se. • If the fitted model’s predictions are perfect (SSE = 0), then se= 0. Thus, a small seindicates a better fit. • Used to construct confidence intervals. • Magnitude of sedepends on the units of measurement of Y and on data magnitude.

  24. Tests for Significance Chapter 12 Confidence Intervals for Slope and Intercept • Standard error of the slope and intercept: Excel’s Data Analysis > Regression constructs confidence intervals for the slope and intercept. • Confidence interval for the true slope and intercept:

  25. Tests for Significance Chapter 12 Hypothesis Tests • If b1 = 0, then the regression model collapses to a constant b0 plus random error. • The hypotheses to be tested are: Excel ‘s Data Analysis > Regression performs these tests. d.f.= n 2 Reject H0 if tcalc > ta/2 or if p-value  α.

  26. Analysis of Variance: Overall Fit Chapter 12 Example: Achievement Test Scores 20 high school students’ achievement exam scores. Excel shows 95% confidence intervals and t test statistics MegaStat is similar but rounds off and highlights p-values to show significance (light yellow .05, bright yellow .01)

  27. To test a regression for overall significance, we use an F test to compare the explained (SSR) and unexplained (SSE) sums of squares. Analysis of Variance: Overall Fit Chapter 12 F Test for Overall Fit

  28. Analysis of Variance: Overall Fit Chapter 12 Example: Achievement Test Scores 20 high school students’ achievement exam scores. Excel shows the ANOVA sums, the F test statistic , and its p-value. MegaStat is similar, but also highlights p-values to indicate significance (light yellow .05, bright yellow .01)

  29. Confidence and Prediction Intervals for Y Confidence interval for the conditional mean of Y is shown below. Prediction intervals are wider than confidence intervals for the mean because individual Y values vary more than the mean of Y. Chapter 12 How to Construct an Interval Estimate for Y Excel does not do these CIs!

  30. Tests of Assumptions 11.3 Chapter 12 Three Important Assumptions • The errors are normally distributed. • The errors have constant variance (i.e., they are homoscedastic). • The errors are independent (i.e., they are nonautocorrelated). Non-normal Errors • Non-normality of errors is a mild violation since the regression parameter estimates b0 and b1 and their variances remain unbiased and consistent. • Confidence intervals for the parameters may be untrustworthy because the normality assumption is used to justify using Student’s t distribution.

  31. Residual Tests Chapter 12 Non-normal Errors • A large sample size would compensate. • Outliers could pose serious problems. Normal Probability Plot • The normal probability plot tests the assumptionH0: Errors are normally distributedH1: Errors are not normally distributed • If H0 is true, the residual probability plot should be linear, as shown in the example.

  32. Residual Tests Chapter 12 What to Do about Non-normality? • Trim outliers only if they clearly are mistakes. • Increase the sample size if possible. • If data are totals, try a logarithmic transformation of both X and Y.

  33. Residual Tests Chapter 12 Heteroscedastic Errors (Nonconstant Variance) • The ideal condition is if the error magnitude is constant (i.e., errors are homoscedastic). • Heteroscedasticerrors increase or decrease with X. • In the most common form of heteroscedasticity, the variances of the estimators are likely to be understated. • This results in overstated t statistics and artificially narrow confidence intervals. Tests for Heteroscedasticity • Plot the residuals against X. Ideally, there is no pattern in the residuals moving from left to right.

  34. Residual Tests Chapter 12 Tests for Heteroscedasticity • The “fan-out” pattern of increasing residual variance is the most common pattern indicating heteroscedasticity.

  35. Residual Tests Chapter 12 What to Do about Heteroscedasticity? • Transform both X and Y, for example, by taking logs. • Although it can widen the confidence intervals for the coefficients, heteroscedasticity does not bias the estimates. Autocorrelated Errors • Autocorrelation is a pattern of non-independent errors. • In a first-order autocorrelation, et is correlated with et  1. • The estimated variances of the OLS estimators are biased, resulting in confidence intervals that are too narrow, overstating the model’s fit.

  36. Residual Tests Chapter 12 Runs Test for Autocorrelation • In the runs test, count the number of the residuals’ sign reversals (i.e., how often does the residual cross the zero centerline?). • If the pattern is random, the number of sign changes should be n/2. • Fewer than n/2 would suggest positive autocorrelation. • More than n/2 would suggest negative autocorrelation. Durbin-Watson (DW) Test • Tests for autocorrelation under the hypothesesH0: Errors are nonautocorrelatedH1: Errors are autocorrelated • The DW statistic will range from 0 to 4.DW < 2 suggests positive autocorrelationDW = 2 suggests no autocorrelation (ideal)DW > 2 suggests negative autocorrelation

  37. Residual Tests Chapter 12 What to Do about Autocorrelation? • Transform both variables using the method of first differences in which both variables are redefined as changes. Then we regress Y against X. • Although it can widen the confidence interval for the coefficients, autocorrelation does not bias the estimates. • Don’t worry about it at this stage of your training. Just learn to detect whether it exists.

  38. Residual Tests Chapter 12 Example: Excel’s Tests of Assumptions Excel’s Data Analysis > Regression does residual plots and gives the DW test statistic. Its standardized residuals are done in a strange way, but usually they are not misleading. Warning: Excel offers normal probability plots for residuals, but they are done incorrectly.

  39. Residual Tests Chapter 12 Example: MegaStat’s Tests of Assumptions MegaStat will do all three tests (if you check the boxes). Its runs plot (residuals by observation) is a visual test for autocorrelation, which Excel does not offer.

  40. Residual Tests Chapter 12 Example: MegaStat’s Tests of Assumptions near-linear plot - indicates normal errors no pattern - suggests homoscedastic errors no pattern - suggests homoscedastic errors DW near 2 - suggests no autocorrelation

  41. Unusual Observations Chapter 12 Standardized Residuals • Use Excel, MINITAB, MegaStat or other software to compute standardized residuals. • If the absolute value of any standardized residual is at least 2, then it is classified as unusual. Leverage and Influence • A high leverage statistic indicates the observation is far from the mean of X. • These observations are influential because they are at the “end of the lever.” • The leverage for observation i is denoted hi.

  42. Unusual Observations Chapter 12 Leverage • A leverage that exceeds 4/n is unusual.

  43. Unusual Observations Example: Achievement Test Scores Chapter 12 • If the absolute value of any standardized residual is at least 2, then it is classified as unusual. • Leveragethat exceeds 4/nindicates an influential X value(far from mean of X).

  44. Other Regression Problems Chapter 12 Outliers • Outliers may be caused by • an error in recording data. • impossible data (can be omitted). • an observation that has been influenced by an unspecified “lurking” variable that should have been controlled but wasn’t. • To fix the problem • delete the observation(s) if you are sure they are actually wrong. • formulate a multiple regression model that includes the lurking variable. 12B-44

  45. Other Regression Problems Chapter 12 Model Misspecification • If a relevant predictor has been omitted, then the model is misspecified. • For example, Height depends on Gender as well as Age. • Use multiple regression instead of bivariate regression. Ill-Conditioned Data • Well-conditioned data values are of the same general order of magnitude. • Ill-conditioned data have unusually large or small data values and can cause loss of regression accuracy or awkward estimates.

  46. Other Regression Problems Chapter 12 Ill-Conditioned Data • Avoid mixing magnitudes by adjusting the magnitude of your data before running the regression. • For example, Revenue= 139,405,377 mixed with ROI = .037. Spurious Correlation • In a spurious correlation two variables appear related because of the way they are defined. • This problem is called the size effect or problem of totals. • Expressing variables as per capita or per cent may be helpful.

  47. Other Regression Problems Chapter 12 Model Form and Variable Transforms • Sometimes a nonlinear model is a better fit than a linear model. Excel offers other model forms for simple regression (one X and one Y) • Variables may be transformed (e.g., logarithmic or exponential functions) in order to provide a better fit. • Log transformations reduce heteroscedasticity. • Nonlinear models may be difficult to interpret.

  48. AssignmentsML 11.4 • Connect C-8 (covers chapter 12) • You get three attempts • Feedback is given if requested • Printable if you wish • Deadline is midnight each Monday • Project P-3 (data, tasks, questions) • Review instructions • Look at the data • Your task is to write a nice, readable report (not a spreadsheet) • Length is up to you

  49. Projects: General Instructions General Instructions For each team project, submit a short (5-10 page) report (using Microsoft Word or equivalent) that answers the questions posed. Strive for effective writing (see textbook Appendix I). Creativity and initiative will be rewarded. Avoid careless spelling and grammar. Paste graphs and computer tables or output into your written report (it may be easier to format tables in Excel and then use Paste Special > Picture to avoid weird formatting and permit sizing within Word). Allocate tasks among team members as you see fit, but all should review and proofread the report (submit only one report).

  50. Project P-3 You will be assigned team members and a dependent variable (see Moodle) from the 2010 state database Big Dataset 09 - US States. The team may change the assigned dependent variable (instructor assigned one just to give you a quick start). Delegate tasks and collaborate as seems appropriate, based on your various skills. Submit one report. Data:Choose an interesting dependent variable (non-binary) from the 2010 state database posted on Moodle.Analysis:(a). Propose a reasonable model of the form Y = f(X1, X2, ... , Xk) using not more than 12 predictors. (b) Use regression to investigate the hypothesized relationship. (c) Try deleting poor predictors until you feel that you have a parsimonious model, based on the t-values, p-values, standard error, and R2adj. (d) For the preferred model only, obtain a list of residuals and request residual tests and VIFs. (e) List the states with high leverage and/or unusual residuals. (f) Make a histogram and/or probability plot of the residuals. Are the residuals normal? (g) For the predictors that were retained, analyze the correlation matrix and/or VIFs. Is multicollinearity a problem? If so, what could be done? (h) If you had more time, what might you do? Watch the instructor’s video walkthrough using Assault as an example (posted on Moodle)

More Related