Topics: Research Design

1. Topics: Research Design Basic issues of research design Role of statistics in behavioral research Classification of variables Quantification of variables (scales of measurement) Validity of interpretations of research studies Types of research designs

2. Research Design Decisions What kinds of subjects/participants and how many? What will subjects be asked to do? How many comparison groups if any? What dependent/independent variables to focus on? How and when subjects will be measured? Where study will be conducted?

3. Design Issues: Subjects Where did subjects come from? What kinds of samples? How many of intended subjects actually supplied data? Were in final analysis? If comparison groups, how were they formed? How motivated were subjects?

4. Design Issues: Data Instrument quality Question/data match Independence of observations Person/people collecting data

5. Design Issues: Study Context Physical setting Pretest sensitization Treatment conditions Subjects thoughts about the study

6. Descriptive and Inferential Statistics Descriptive Statistics: Methods used to obtain indices that characterize or summarize data collected Inferential Statistics: Methods that allow the researcher to make inferences from a set of data collected from a sample to a larger population.

7. Review of Terms Research: a systematic approach to finding answers to questions. Research Design: a plan for gathering data for answering specific research questions. Statistics: the methods used on the data collected to answer the research questions at hand.

8. Basic Elements: Hypotheses Hypothesis: a tentative statement (�educated guess�) about the expected relationship between two or more variables. State expected relationship or difference between 2 variables Be worthy of being tested Be testable Be brief and clear

9. Basic Elements: Variables Variable: what is measured or varied. An attribute or characteristic of a person (or object) that can change from person to person. Independent Dependent Control Intervening (mediator) Moderator

10. Classification of Variables Independent Variable: a variable that is manipulated, measured or selected by the researcher in order to observe its relation to the subject's "response� on another variable. An antecedent condition. Dependent Variable: the variable that is observed and measured in response to an independent variable.

11. Classification of Variables (Con�t) Control Variable: any variable that is held constant in a research study by observing only one if its instances or levels. Intervening Variable: a hypothetical variable that is not observed directly in the research study, but is inferred from the relationship between the independent and dependent variable. Intervening variables are the things we can�t see which might influence outcome. Intervening are inferred, not concrete. Gender, race, SES - the things we can get a handle on - are moderator variables.Intervening variables are the things we can�t see which might influence outcome. Intervening are inferred, not concrete. Gender, race, SES - the things we can get a handle on - are moderator variables.

12. Quantification of Variables Measurement: the application of rules in assigning numbers to cases so as to represent the presence or absence of quantity of an attribute possessed by each case. Four (4) scales of measurement We have to define how we will measure and then go out and measure it. Quantitatively, that means somehow assigning a numeric value to the attributeWe have to define how we will measure and then go out and measure it. Quantitatively, that means somehow assigning a numeric value to the attribute

13. Scales of Measurement Nominal Scale Measurement (Lowest) Ordinal Scale Measurement Interval Scale Measurement Ratio Scale Measurement (Highest) Variables measured at higher levels can be scaled down to lower levels Nominal is just a descriptive name - no value, no ranking - like colors (red, blue, yellow) or gender (male, female) and are mutually exclusive (for the purposes of the study). �Categorical� Race (African American, Asian American, Latino/a, Caucasian) Marital status (singlenever married, married, divorced, etc) Ordinal - ranked highest to lowest. Cities ranked by size of population. Baseball players ranked by batting average. Interval scale of measurement - �Cpntinuous variable� in which a number is assigned to the amount of an attribute and IN WHICH ZERO DOES NOT MEAN THE ABSENCE OF THE ATTRIBUTE - like temperature - 0 does not mean there is not temperature. Test scores - zero doesn�t mean the absence of intelligence/knowledge. Ratio scale - �Cpntinuous variable� in which a number is assigned to the amount of an attribute and IN WHICH ZERO HAS A MEANING - like in the amount of money in your pocket - you can�t go below zero. Number of children, amount of food you have, height, weight, age� �Variables measured at higher levels can be scaled down to lower levels� : you can rank or name ratio or interval scales, making them ordinal or nominal, but not viceversa. You can do more statistical processes with the �higher� level variables. Nominal is just a descriptive name - no value, no ranking - like colors (red, blue, yellow) or gender (male, female) and are mutually exclusive (for the purposes of the study). �Categorical� Race (African American, Asian American, Latino/a, Caucasian) Marital status (singlenever married, married, divorced, etc) Ordinal - ranked highest to lowest. Cities ranked by size of population. Baseball players ranked by batting average. Interval scale of measurement - �Cpntinuous variable� in which a number is assigned to the amount of an attribute and IN WHICH ZERO DOES NOT MEAN THE ABSENCE OF THE ATTRIBUTE - like temperature - 0 does not mean there is not temperature. Test scores - zero doesn�t mean the absence of intelligence/knowledge. Ratio scale - �Cpntinuous variable� in which a number is assigned to the amount of an attribute and IN WHICH ZERO HAS A MEANING - like in the amount of money in your pocket - you can�t go below zero. Number of children, amount of food you have, height, weight, age� �Variables measured at higher levels can be scaled down to lower levels� : you can rank or name ratio or interval scales, making them ordinal or nominal, but not viceversa. You can do more statistical processes with the �higher� level variables.

14. Likert Scale: Disagree/Agree(5 Question/Item Survey) Likert scale is not an ordinal ranking (low to high) - the response categories, not the scale of measurement. In this case, the total score is not ordinal (it could be but isn�t) - it is an interval scale, because 0 doesn�t indicate the absence of creativity Likert scale is not an ordinal ranking (low to high) - the response categories, not the scale of measurement. In this case, the total score is not ordinal (it could be but isn�t) - it is an interval scale, because 0 doesn�t indicate the absence of creativity

15. Validity of the Study Can you trust the conclusions of the study? Internal Validity: The extent to which the outcomes of the study result from the variables manipulated,measured or selected rather than from other variables not systematically managed. External Validity: the extent to which the findings of a particular study can be generalized to people and/or situations other than those observed in the study. Do our interpretations come safely from the study, or are there other possible alternative explanations?Do our interpretations come safely from the study, or are there other possible alternative explanations?

16. Are Conclusions Trustworthy? Latch-Key Study Study of the effects of having young children spend part of their day without an adult. The issue of �latch key� or �self-care� children. Two groups of 50 children each One group who have never spent after school time without adult present (adult supervised) One group who spend at least one hour a day without adult present (self care) Dependent variables Level of anxiety Rate of delinquency School achievement

17. Results Latch-Key Study It doesn�t look significant, but might come out to mean something Depends on sample size what is significant and what is not.It doesn�t look significant, but might come out to mean something Depends on sample size what is significant and what is not.

18. Conclusions of Latch-key Study The researchers concluded: The effects of self-care on anxiety are negligible Self-care results in increased rates of delinquency among young children Self-care results in a small average loss in reading comprehension of young children Do you agree? Are the results trustworthy? Not generalizable One hour isn�t much There are a lot of other factors that could cause this effect - location, SES Any time you find a way the groups differed that might have had an impact on the outcome then you have to question. You can easily say there was a difference in outcome between the two groups - but anytime you use causal language you have to be more careful.Not generalizable One hour isn�t much There are a lot of other factors that could cause this effect - location, SES Any time you find a way the groups differed that might have had an impact on the outcome then you have to question. You can easily say there was a difference in outcome between the two groups - but anytime you use causal language you have to be more careful.

19. Potential Threats to the Internal Validity: �Counter-Interpretations� History--Occurrence of events that take place in the course of the study that might affect the dependent variable. Maturation--Developmental (physical or mental) changes in the participants which account for results Testing--If participants given pre-test, they may learn from pre-test and hence do better on the post-test. Instrumentation--If measuring instruments are not reliable or valid, then their scores could be inaccurate. Selection--Initial differences among groups being compared might account for results and lead to misinterpretations. Statistical Regression--If sample selected on basis of extreme scores, their scores will move toward the mean on repeated testing Mortality--Drop out of subjects who might share a common characteristic. Stability--The possibility that the results are a fluke, a chance occurrence. This list - think of all of these when you ask, is anything else happening that might explain the results.This list - think of all of these when you ask, is anything else happening that might explain the results.

20. Potential Threats to the Internal Validity: (Cont�d) Diffusion of treatments--comparison groups learn about other treatment Experimenter effects -- deliberate or unintentional influence of researchers Statistical conclusions--if statistical assumptions are violated Subject effects - changes in participating subjects

21. Counteracting Potential Threats to Internal Validity Control Group: a group of subjects whose selection and treatment are exactly the same as those of the experimental group except that the control group does not receive the experimental treatment. Note, that doesn't mean "no treatment� Random Assignment: a method for assigning subjects to control and experimental groups. Not to be confused with random selection (a method for selecting a sample of subjects from a population). Pretests:When random assignment is impossible or undesirable, pretests can be used to examine the possibility or prior existing differences between groups and to statistically adjust for these differences.

22. Potential Threats to External Validity: �Counter Interpretations� Reactive Effects of Subject Selection--If sample is not representative of the population, the results can not be generalized to that population Reactive Effects of Testing--If pretest is given and somehow affects the outcomes of the study, results can�t be generalized to population unless a pre-test will also be given to that population. Reactive Effects of Treatment Selection--If treatments of study can not be replicated outside of study, then results can�t be generalized to that population. Multiple Treatment Interference--When subjects in one treatment are exposed to another treatment condition, then can�t isolate results to a given treament, but only to the interaction of the two treatments. So results cannot be generalized to population for which treatment interaction does not occur. Pretest sensitization Contextual (setting) issues Multiple treatment interference - one group talks to other group about treatment, compare or combine treatments. If two roommates get different questionnaires, or siblings get separate treatmentPretest sensitization Contextual (setting) issues Multiple treatment interference - one group talks to other group about treatment, compare or combine treatments. If two roommates get different questionnaires, or siblings get separate treatment

23. Major Types of Research Studies Experimental: A type of research used to establish cause-and-effect relationships by manipulating variables/treatments Observational/Correlational: A type of research that measures two or more variables and looks to see how the variables are related to each other.

24. Classes of Research Design Pre-experimental Experimental Quasi-experimental Ex Post Facto

25. Pre-Experimental Designs:No Control Group and/or Randomization One-shot case study One-group pretest-posttest design Intact-group comparison

26. True Experimental Designs:Control Group & Randomization Posttest-only control-group design Pretest-posttest control-group design Factorial experimental design

27. Quasi-Experimental Designs:Control Group But No Randomization Non-equivalent control group design Time-series designs Others

28. Ex-Post Facto Designs:Researcher Arrives After Treatment Is Given Correlational designs -- Simple predictive -- Causal modeling Criterion-group designs