MULTIPLE REGRESSION

1. MULTIPLE REGRESSION

2. OVERVIEW • What Makes it Multiple? • Additional Assumptions • Methods of Entering Variables • Adjusted R2 • Using z-Scores

3. WHAT MAKES IT MULTIPLE? • Predict from a combination of two or more predictor (X) variables. • The regression model may account for more variance with more predictors. • Look for predictor variables with low inter-correlations.

4. Multiple Regression Equation • Like simple regression, use a linear equation to predict Y scores. • Use the least squares solution.

5. Assumptions for Regression • Quantitative data (or dichotomous) • Independent observations • Predict for same population that was sampled • Linear relationship

6. Assumptions for Regression • Homoscedasticity • Independent errors • Normality of errors

7. ADDITIONAL ASSUMPTIONS • Sufficient sample size • For medium effect, minimum N of 100 for up to six predictors; N of 160 for up to 20 predictors • Predictor variables are not strongly intercorrelated (no multicollinearity) • Examine VIF – should be close to 1

8. Multicollinearity • When predictor variables are highly intercorrelated with each other, prediction accuracy is not as good. • Be cautious about determining which predictor variable is predicting the best when there is high collinearity among the predictors.

9. Review! Write out the multiple regression equation for predicting Statistics Grade from Undergrad GPA, GRE, and Neuroticism. (Use variable names instead of X and Y).

10. Review! In predicting Statistics Grade from Undergrad GPA, GRE, and Neuroticism, what minimum sample size would you recommend?

11. Review! TRUE OR FALSE It is better to have predictor variables that are highly correlated with both each other and with the outcome variable.

12. METHODS OF ENTERING VARIABLES • Simultaneous • Hierarchical/Block Entry • Stepwise • Forward • Backward • Stepwise

13. Simultaneous Multiple Regression • All predictor variables are entered into the regression at the same time • Allows you to determine portion of variance explained by each predictor with the others statistically controlled (part correlation)

14. Hierarchical Multiple Regression • Enter variables in a particular order based on a theory or on prior research • Can be done with blocks of variables

15. Stepwise Multiple Regression • Enter or remove predictor variables one at a time based on explaining significant portions of variance in the criterion • Forward • Backward • Stepwise

16. Forward Stepwise • begin with no predictor variables • add predictors one at a time according to which one will result in the largest increase in R2 • stop when R2 will not be significantly increased

17. Backward Stepwise • begin with all predictor variables • remove predictors one at a time according to which one will result in the smallest decrease in R2 • stop when R2 would be significantly decreased • may uncover suppressor variables

18. Suppressor Variable • Predictor variable which, when entered into the equation, increases the amount of variance explained by another predictor variable • In backward regression, removing the suppressor would likely result in a significant decrease in R2, so it will be left in the equation

19. Suppressor Variable Example • Y = Job Performance Rating • X1 = College GPA • X2 = Writing Test Score

20. Suppressor Variable Example • Let’s say Writing Score is not correlated with Job Performance, because the job doesn’t require much writing • Let’s say GPA is only a weak predictor of Job Performance, but it seems like it should be a good predictor

21. Suppressor Variable Example • Let’s say GPA is “contaminated” by differences in writing ability – really good writers can fake and get higher grades • So, if Writing Score is in the equation, the contamination is removed, and we get a better picture of the GPA-Job Performance relationship

22. Stepwise • begin with no predictor variables • add predictors one at a time according to which one will result in the largest increase in R2 • at each step remove any variable that does not explain a significant portion of variance • stop when R2 will not be significantly increased

23. Choosing a Stepwise Method • Forward • Easier to conceptualize • Provides efficient model for predicting Y • Backward • Can uncover suppressor effects • Stepwise • Can uncover suppressor effects • Tends to be unstable with smaller N’s

24. ADJUSTED R2 • R2 may overestimate the true amount of variance explained. • Adjusted R2 compensates by reducing the R2 according to the ratio of subjects per predictor variable.

25. BETA WEIGHTS • The regression weights can be standardized into beta weights. • Beta weights do not depend on the scales of the variables. • A beta weight indicates the amount of change in Y in units of SD for each SD change in the predictor.

26. Example of Reporting Results of Multiple Regression We performed a simultaneous multiple regression with vocabulary score, abstraction score, and age as predictors and preference for intense music as the dependent variable. The equation accounted for a significant portion of variance, F(3,66) = 4.47, p = .006, Adjusted R2 = .17. As shown in Table 1, the only significant predictor was abstraction score.

28. Choosing Stats Participants who have recently experienced a relationship breakup complete a measure of subjective well-being as well as a personality inventory and a questionnaire asking about the length and intensity of the relationship. The researcher hypothesizes that neuroticism will explain additional variance in subjective well-being on top of that variance accounted for by length and intensity of the relationship.