1 / 130

Alternatives to Difference Scores: Polynomial Regression and Response Surface Methodology

Alternatives to Difference Scores: Polynomial Regression and Response Surface Methodology. Jeffrey R. Edwards University of North Carolina. Outline. Types of Difference Scores Questions Difference Scores Are Intended To Address Problems With Difference Scores An Alternative Procedure

xiang
Télécharger la présentation

Alternatives to Difference Scores: Polynomial Regression and Response Surface Methodology

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Alternatives to Difference Scores:Polynomial Regression and Response Surface Methodology Jeffrey R. Edwards University of North Carolina

  2. Outline • Types of Difference Scores • Questions Difference Scores Are Intended To Address • Problems With Difference Scores • An Alternative Procedure • Analyzing Quadratic Regression Equations Using Response Surface Methodology • Moderated Polynomial Regression • Mediated Polynomial Regression • Difference Scores As Dependent Variables • Answers to Frequently Asked Questions

  3. Types of Difference Scores • Univariate: • Algebraic difference: (X – Y) • Absolute difference: |X – Y| • Squared difference: (X – Y)2 • Multivariate: • Sum of algebraic differences: Σ(Xi – Yi) = D1 • Sum of absolute differences: Σ|Xi – Yi| = |D| • Sum of squared differences: Σ(Xi – Yi)2 = D2 • Euclidean distance: (Σ(Xi – Yi)2)½ = D • Profile correlation: C(Xi,Yi)/S(X)S(Y) = rXi,Yi = Q

  4. Questions Difference Scores are Intended to Address • How well do characteristics of the job fit the needs or desires of the employee? • To what extent do job demands exceed or fall short of the abilities of the person? • Are prior expectations of the employee met by actual job experiences? • What is the degree of similarity between perceptions or beliefs of supervisors and subordinates? • Do the values of the person match the culture of the organization? • Can novices provide performance evaluations that agree with expert ratings?

  5. Problems with Difference Scores:Reliability • When component measures are positively correlated, difference scores are often less reliable than either component. • The reliability of an algebraic difference is: • To illustrate, if X and Y have unit variances, have reliabilities of .75, and are correlated .50, the reliability of X – Y equals .50.

  6. Problems with Difference Scores:Conceptual Ambiguity • It might seem that component variables are reflected equally in a difference score, given that the components are implicitly assigned the same weight when the difference score is constructed. • However, the variance of a difference score depends on the variances and covariances of the component measures, which are sample dependent. • When one component is a constant, the variance of a difference score is solely due to the other component, i.e., the one that varies. For instance, when P-O fit is assessed in a single organization, the P-O difference solely represents variation in the person scores.

  7. Problems with Difference Scores:Confounded Effects • Difference scores confound the effects of the components of the difference. • For example, an equation using an algebraic difference as a predictor can be written as: Z = b0 + b1(X – Y) + e • In this equation, b1 can reflect a positive relationship for X, an negative relationship for Y, or some combination thereof.

  8. Problems with Difference Scores:Untested Constraints • Difference scores constrain the coefficients relating X and Y to Z without testing these constraints. • The constraints imposed by an algebraic difference can be seen with the following equations: Z = b0 + b1(X – Y) + e • Expansion yields: Z = b0 + b1X – b1Y + e

  9. Problems with Difference Scores:Untested Constraints • Now, consider an equation that uses X and Y as separate predictors: Z = b0 + b1X + b2Y + e • Using (X – Y) as a predictor constrains the coefficients on X and Y to be equal in magnitude but opposite in sign (i.e., b1 = –b2). • This constraint should not be simply imposed on the data but instead should be treated as a hypothesis to be tested.

  10. Problems with Difference Scores:Untested Constraints • The constraints imposed by a squared difference can be seen with the following equations: Z = b0 + b1(X – Y)2 + e • Expansion yields: Z = b0 + b1X2 – 2b1XY + b1Y2 + e • Thus, a squared difference implicitly treats Z as a function of X2, XY, and Y2.

  11. Problems with Difference Scores:Untested Constraints • Now, consider a quadratic equation using X and Y: Z = b0 + b1X + b2Y + b3X2 + b4XY + b5Y2 + e • Comparing this equation to the previous equation shows that (X – Y)2 imposes four constraints: • b1 = 0 • b2 = 0 • b3 = b5, or b3 – b5 = 0 • b3 + b4 + b5 = 0 • Again, these constraints should be treated as hypotheses to be tested empirically, not simply imposed on the data.

  12. Problems with Difference Scores:Dimensional Reduction • Difference scores reduce the three-dimensional relationship of X and Y with Z to two dimensions. • The linear algebraic difference function represents a symmetric plane with equal but opposite slopes with respect to the X-axis and Y-axis. • The V-shaped absolute difference function represents a symmetric V-shaped surface with its minimum (or maximum) running along the X = Y line. • The U-shaped squared difference function represents a symmetric U-shaped surface with its minimum (or maximum) running along the X = Y line.

  13. Two-Dimensional Algebraic Difference Function

  14. Three-Dimensional Algebraic Difference Function

  15. Two-Dimensional Absolute Difference Function

  16. Three-Dimensional Absolute Difference Function

  17. Two-Dimensional Squared Difference Function

  18. Three-Dimensional Squared Difference Function

  19. Problems with Difference Scores:Dimensional Reduction • These surfaces represent only three of the many possible surfaces depicting how X and Y may be related to Z. • This problem is compounded by the use of profile similarity indices, which collapse a series of three-dimensional surfaces into a single two-dimensional function.

  20. An Alternative Procedure • The relationship of X and Y with Z should be viewed in three dimensions, with X and Y constituting the two horizontal axes and Z constituting the vertical axis. • Analyses should focus not on two‑dimensional functions relating the difference between X and Y to Z, but instead on three‑dimensional surfaces depicting the joint relationship of X and Y with Z. • Constraints should not be simply imposed on the data, but instead should be viewed as hypotheses that, if confirmed, lend support to the conceptual model upon which the difference score is based.

  21. Data Used for Illustration • Data were collected from 373 MBA students who were engaged in the recruiting process. • Respondents rated the actual and desired amounts of various job attributes and the anticipated satisfaction concerning a job for which they had recently interviewed. • Actual and desired measured had three items and used 7-point response scales ranging from “none at all” to “a very great amount.” The satisfaction measured had three items and used a 7-point response scale ranging from “strongly disagree” to “strongly agree.” • The job attributes used for illustration are autonomy, prestige, span of control, and travel.

  22. Confirmatory Approach • When a difference scores represents a hypothesis that is predicted a priori, the alternative procedure should be applied using the confirmatory approach. • The R2 for the unconstrained equation should be significant. • The coefficients in the unconstrained equation should follow the pattern indicated by the difference score. • The constraints implied by the difference score should not be rejected. • The set of terms one order higher than those in the unconstrained equation should not be significant.

  23. Confirmatory Approach Applied to the Algebraic Difference • The unconstrained equation is: Z = b0 + b1X + b2Y + e • The constrained equation used to evaluate the third condition is: Z = b0 + b1 (X – Y) + e • The equation that adds higher-order terms used to evaluate the fourth condition is: Z = b0 + b1X + b2Y + b3X2 + b4XY + b5Y2 + e

  24. Example: Confirmatory Test of Algebraic Difference for Autonomy • Unconstrained equation: Dep Var: SAT N: 360 Multiple R: 0.356 Squared multiple R: 0.127 Adjusted squared multiple R: 0.122 Standard error of estimate: 1.077 Effect Coefficient Std Error Std Coef Tolerance t P(2 Tail) CONSTANT 5.835 0.077 0.000 . 75.874 0.000 AUTCA 0.445 0.062 0.413 0.737 7.172 0.000 AUTCD -0.301 0.071 -0.244 0.737 -4.235 0.000 Analysis of Variance Source Sum-of-Squares df Mean-Square F-ratio P Regression 60.133 2 30.067 25.930 0.000 Residual 413.953 357 1.160

  25. Example: Confirmatory Test of Algebraic Difference for Autonomy • Unconstrained surface:

  26. Example: Confirmatory Test of Algebraic Difference for Autonomy • The first condition is met, because the R2 from the unconstrained equation is significant. • The second condition is met, because the coefficients on X and Y are significant and in the expected direction. • For the third condition, testing the constraints imposed by the algebraic difference is the same as testing the difference in R2 between the constrained and unconstrained equations.

  27. Example: Confirmatory Test of Algebraic Difference for Autonomy • Constrained equation: Dep Var: SAT N: 360 Multiple R: 0.339 Squared multiple R: 0.115 Adjusted squared multiple R: 0.113 Standard error of estimate: 1.082 Effect Coefficient Std Error Std Coef Tolerance t P(2 Tail) CONSTANT 5.937 0.061 0.0 . 97.007 0.000 AUTALD 0.393 0.058 0.339 1.000 6.825 0.000 Analysis of Variance Source Sum-of-Squares df Mean-Square F-ratio P Regression 54.589 1 54.589 46.586 0.000 Residual 419.498 358 1.172

  28. Example: Confirmatory Test of Algebraic Difference for Autonomy • Constrained surface:

  29. Example: Confirmatory Test of Algebraic Difference for Autonomy • The general formula for the difference in R2 between two regression equations is: • The test of the constraint imposed by the algebraic difference for autonomy is: • The constraint is rejected, so the third condition is not satisfied.

  30. Example: Confirmatory Test of Algebraic Difference for Autonomy • For the fourth condition, the unconstrained equation for the algebraic equation is linear, so the higher-order terms are the three quadratic terms X2, XY, and Y2. • Testing the three quadratic terms as a set is the same as testing the difference in R2 between the linear and quadratic equations.

  31. Example: Confirmatory Test of Algebraic Difference for Autonomy • Quadratic equation: Dep Var: SAT N: 360 Multiple R: 0.411 Squared multiple R: 0.169 Adjusted squared multiple R: 0.157 Standard error of estimate: 1.055 Effect Coefficient Std Error Std Coef Tolerance t P(2 Tail) CONSTANT 5.825 0.083 0.000 . 70.161 0.000 AUTCA 0.197 0.100 0.182 0.273 1.966 0.050 AUTCD -0.293 0.106 -0.238 0.315 -2.754 0.006 AUTCA2 -0.056 0.047 -0.086 0.444 -1.177 0.240 AUTCAD 0.276 0.080 0.396 0.178 3.453 0.001 AUTCD2 -0.035 0.063 -0.054 0.242 -0.553 0.581 Analysis of Variance Source Sum-of-Squares df Mean-Square F-ratio P Regression 79.951 5 15.990 14.362 0.000 Residual 394.135 354 1.113

  32. Example: Confirmatory Test of Algebraic Difference for Autonomy • The test of the higher-order terms associated with the algebraic difference for autonomy: • The higher-order terms are significant, so the fourth condition is not satisfied.

  33. Confirmatory Approach Applied to the Absolute Difference • The unconstrained equation is: Z = b0 + b1X + b2Y + b3W + b4WX + b5WY + e • The constrained equation used to evaluate the third condition is: Z = b0 + b1 |X – Y| + e • The equation that adds higher-order terms used to evaluate the fourth condition is: Z = b0 + b1X + b2Y + b3W + b4WX + b5WY + b6X2 + b7XY + b8Y2 + b9WX2 + b10WXY + b10WY2 + e

  34. Example: Confirmatory Test of Absolute Difference for Autonomy • Unconstrained equation: Dep Var: SAT N: 360 Multiple R: 0.399 Squared multiple R: 0.159 Adjusted squared multiple R: 0.147 Standard error of estimate: 1.061 Effect Coefficient Std Error Std Coef Tolerance t P(2 Tail) CONSTANT 6.233 0.152 0.000 . 41.136 0.000 AUTCA -0.150 0.184 -0.139 0.082 -0.818 0.414 AUTCD 0.183 0.188 0.148 0.102 0.970 0.333 AUTW -0.349 0.201 -0.148 0.329 -1.737 0.083 AUTCAW 0.752 0.209 0.490 0.129 3.605 0.000 AUTCDW -0.554 0.219 -0.406 0.093 -2.537 0.012 Analysis of Variance Source Sum-of-Squares df Mean-Square F-ratio P Regression 75.381 5 15.076 13.386 0.000 Residual 398.705 354 1.126

  35. Example: Confirmatory Test of Absolute Difference for Autonomy • Unconstrained surface:

  36. Example: Confirmatory Test of Absolute Difference for Autonomy • The first condition is met, because the R2 from the unconstrained equation is significant. • The second condition is not met, because the coefficients on X and Y are not significant, and in the expected direction. • For the third condition, testing the constraints imposed by the absolute difference is the same as testing the difference in R2 between the constrained and unconstrained equations.

  37. Example: Confirmatory Test of Absolute Difference for Autonomy • Constrained equation: Dep Var: SAT N: 360 Multiple R: 0.323 Squared multiple R: 0.105 Adjusted squared multiple R: 0.102 Standard error of estimate: 1.089 Effect Coefficient Std Error Std Coef Tolerance t P(2 Tail) CONSTANT 6.212 0.087 0.000 . 71.122 0.000 AUTABD -0.531 0.082 -0.323 1.000 -6.464 0.000 Analysis of Variance Source Sum-of-Squares df Mean-Square F-ratio P Regression 49.555 1 49.555 41.788 0.000 Residual 424.532 358 1.186

  38. Example: Confirmatory Test of Absolute Difference for Autonomy • Constrained surface:

  39. Example: Confirmatory Test of Absolute Difference for Autonomy • The test of the constraints imposed by the absolute difference for autonomy is: • The constraints are rejected, so the third condition is not satisfied.

  40. Example: Confirmatory Test of Absolute Difference for Autonomy • For the fourth condition, the unconstrained equation for the absolute equation is piecewise linear, so the higher-order terms are the six quadratic terms X2, XY, Y2, WX2, WXY, and WY2. • Testing the six quadratic terms as a set is the same as testing the difference in R2 between the piecewise linear and piecewise quadratic equations.

  41. Example: Confirmatory Test of Absolute Difference for Autonomy • Piecewise quadratic equation: Dep Var: SAT N: 360 Multiple R: 0.431 Squared multiple R: 0.185 Adjusted squared multiple R: 0.160 Standard error of estimate: 1.053 Effect Coefficient Std Error Std Coef Tolerance t P(2 Tail) CONSTANT 6.193 0.206 0.000 . 30.124 0.000 AUTCA -0.438 0.548 -0.407 0.009 -0.799 0.425 AUTCD 0.256 0.505 0.207 0.014 0.506 0.613 AUTW -0.534 0.276 -0.225 0.172 -1.931 0.054 AUTCAW 0.672 0.608 0.438 0.015 1.105 0.270 AUTCDW -0.373 0.592 -0.273 0.013 -0.631 0.529 AUTCA2 0.146 0.312 0.225 0.010 0.468 0.640 AUTCAD -0.092 0.618 -0.133 0.003 -0.150 0.881 AUTCD2 0.107 0.350 0.169 0.008 0.307 0.759 AUTCA2W -0.088 0.325 -0.082 0.026 -0.272 0.786 AUTCADW 0.325 0.641 0.368 0.004 0.507 0.613 AUTCD2W -0.219 0.371 -0.342 0.007 -0.589 0.556 Analysis of Variance Source Sum-of-Squares df Mean-Square F-ratio P Regression 87.940 11 7.995 7.205 0.000 Residual 386.146 348 1.110

  42. Example: Confirmatory Test of Absolute Difference for Autonomy • The test of the higher-order terms associated with the absolute difference for autonomy is: • The higher-order terms are not significant, so the fourth condition is satisfied.

  43. Confirmatory Approach Applied to the Squared Difference • The unconstrained equation is: Z = b0 + b1X + b2Y + b3X2 + b4XY + b5Y2 + e • The constrained equation used to evaluate the third condition is: Z = b0 + b1 (X – Y)2 + e • The equation that adds higher-order terms used to evaluate the fourth condition is: Z = b0 + b1X + b2Y + b3X2 + b4XY + b5Y2 + b6X3 + b7X2Y + b8XY2 + b9Y3 + e

  44. Example: Confirmatory Test of Squared Difference for Autonomy • Unconstrained equation: Dep Var: SAT N: 360 Multiple R: 0.411 Squared multiple R: 0.169 Adjusted squared multiple R: 0.157 Standard error of estimate: 1.055 Effect Coefficient Std Error Std Coef Tolerance t P(2 Tail) CONSTANT 5.825 0.083 0.000 . 70.161 0.000 AUTCA 0.197 0.100 0.182 0.273 1.966 0.050 AUTCD -0.293 0.106 -0.238 0.315 -2.754 0.006 AUTCA2 -0.056 0.047 -0.086 0.444 -1.177 0.240 AUTCAD 0.276 0.080 0.396 0.178 3.453 0.001 AUTCD2 -0.035 0.063 -0.054 0.242 -0.553 0.581 Analysis of Variance Source Sum-of-Squares df Mean-Square F-ratio P Regression 79.951 5 15.990 14.362 0.000 Residual 394.135 354 1.113

  45. Example: Confirmatory Test of Squared Difference for Autonomy • Unconstrained surface:

  46. Example: Confirmatory Test of Squared Difference for Autonomy • The first condition is met, because the R2 from the unconstrained equation is significant. • The second condition is not met, because the coefficients on X and Y are significant, and the coefficients on X2 and Y2 are not significant. • For the third condition, testing the constraints imposed by the squared difference is the same as testing the difference in R2 between the constrained and unconstrained equations.

  47. Example: Confirmatory Test of Squared Difference for Autonomy • Constrained equation: Dep Var: SAT N: 360 Multiple R: 0.310 Squared multiple R: 0.096 Adjusted squared multiple R: 0.093 Standard error of estimate: 1.094 Effect Coefficient Std Error Std Coef Tolerance t P(2 Tail) CONSTANT 5.993 0.067 0.000 . 89.830 0.000 AUTSQD -0.183 0.030 -0.310 1.000 -6.162 0.000 Analysis of Variance Source Sum-of-Squares df Mean-Square F-ratio P Regression 45.463 1 45.463 37.972 0.000 Residual 428.623 358 1.197

  48. Example: Confirmatory Test of Squared Difference for Autonomy • Constrained surface:

  49. Example: Confirmatory Test of Squared Difference for Autonomy • The test of the constraint imposed by the squared difference for autonomy is: • The constraint is rejected, so the third condition is not satisfied.

  50. Example: Confirmatory Test of Squared Difference for Autonomy • For the fourth condition, the unconstrained equation for the squared equation is quadratic, so the higher-order terms are the four cubic terms X3, X2Y, XY2, and Y3. • Testing the four cubic terms as a set is the same as testing the difference in R2 between the quadratic and cubic equations.

More Related