Interpreting Research Results in Behavioral Science: Key Concepts and Inferences

1. Chapter 13: Interpreting Research Results • Describing Results • Inferences in Behavioral Science Research • Null Results • Integrating Results of Research • Summary

2. Describing Results • Nature of relationships • Types of Relationships • Linear v. Curvilinear • Mediators and Moderators (partial corr or MR) • Interaction (factorial experiments) • Predicted and Observed Relationships • Cf results (observed) to expected (hypothesis) • Table 13-1, p. 428, Table 13-4, p. 429

3. Real v. Chance Relationships • Inferential Stats (what alpha level? Why p <.05?) • Type I and Type II error trade offs • Testing the Proper Statistical Hypothesis • Multiple tests (what effect on alpha level?) • Omnibus (MANOVA) v. Planned comparisons • What’s the benefit of Planned comparisons? • Effect Size and Importance of Effect Size • Effect size (always include effect size) • http://web.uccs.edu/lbecker/Psy590/es.htm#II.%20independent • Pearson r; Cohen’s d • Practical Significance (small, medium, large?)

4. Effect size • When effect size is small) • Prentice & Miller (’92) – “Minimal group effect” (Tajefel et al. ’71) • What is important about this small effect size? • Weak manipulation -> any effect, important • Hatfield & Sprecther, (’86) – physical attractiveness • What is important about this small effect size? • When everything else is equal, it may play an important role

5. Practical SignificanceSmall effect sizes • Clinical significance (a value judgment) • Abelson (’85) skill and batting average (r = .06) • Important over a whole season • Fishbein & Ajzen (’75) religiosity and religious behavior • Small effect size and large populations • Framington study (Rosenthal & Rosnow, ’91) • Asprin and avoid heart attack (r = .03) • Population of 750k people = decrease of 3.4% heart attack rate • Theory testing v. Applied research • Which is effect size more important for? (Chow, ’88) • Applied research

6. Inference in Behavioral Science Research • Knowledge as a Social Construction • Constructionist viewpoint • Do we build our own reality? Or • Is logical positivism a real possibility? • How do we view the cause of racial prejudice now? • What zeitgeist are we in now? • Blank slate? Or biological evolution (cognitive)? • Bias in Interpreting Data • Theoretical bias (e.g. Mony & Ehrhardt, ’72) • Which interpretation is correct? • E.O. Wilson (’78) sociobiologist or • Mackie (’83) cultural influence to explain results

7. Inferences: Bias • Personal Bias (tenacity) • Sherwood & Nataupsky (’68) study of 82 psychologists’ beliefs about racial differences in IQ • Environmentalists • Hereditarians • Middle-of-the-roaders (inconclusive) • Statistical sig differences (Bias shows up) • Larry Summers (What happened to him? Why?) • Assuming group differences are biological / environmental • Correlational data make it hard to decide • “Victim blame” (look beyond the group for theory) • Behavior labeling (aggressive v. assertive)

8. Inferences:Making Valid Ones • Measurement and Statistics • Know the level of measure • Recognize the “fallacy of the mean” • E.g. distributions overlap • State correlational results and group means appropriately • Corr: state direction and strength • E.g. “positively related” • “high scores on X were associated with high scores on Y” • Group means: • “mean for group A was significantly higher than the mean for group B” • Don’t forget to show group means (ANOVA table doesn’t)

9. Valid Inferences • Empiricism • Stay close to the actual statistical findings, don’t speculate until the discussion • Clarify (or qualify) the relationship between the hypothetical construct and op definition • E.g. how is race (hypothetical construct) defined operationally? • Describe, avoid unwarranted evalutations • E.g. do women underestimate the credit they deserve or do men overestimate? (you know the truth!) • Causality • Don’t infer causality from correlational findings • Generalization • Theory and or findings

10. Inferences:3 Uses of the Null & Prejudice against Null • Testing hypotheses • Research validity • Testing generalizability • Null findings don’t get published (despite the fact they may be well done) • If the null is, in fact true, What does this imply about the published studies? • They may be Type I errors! • Researchers unlikely to test the null directly • Why?

11. Possible Sources of type II Errors • IV • Construct valid? • Manipulation effective? • strong enough? • DV • Construct valid? • Sensitive enough? • Unrestricted range? • Design • Curvilinear relationship? (inspect the distribution) • Extraneous vars controlled? • Moderators or mediators operating? • Large enough sample (power test)

12. Accepting the Null • Common criteria • Proper design and Sufficient power • Predicted null results • Based on good theory • Unexpected null results • Theory could be wrong! (believe it or not) • Suppose it is a Type II error? • Cold Fusion: Another chance. Does theory matter? Cost of Type II error

13. Integrating Results • Identifying Implications for Theory • Comparison with prior research • Comparison with theoretical prediction • Identifying Implications for Research • Research procedures • New research questions • Identifying Implications for Application

14. Chapter 13: Interpreting Research ResultsSummary • Describing Results • Inferences in Behavioral Science Research • Null Results • Integrating Results of Research • Summary