1 / 51

Exploring Small-N Designs in Research: Validity, Replication, and Error Analysis

This tutorial provides an in-depth review of small-n designs (or single-subject designs) in psychological research. It covers key concepts such as external and ecological validity, baseline characteristics, and the importance of intrasubject and intersubject replication. The assessment of reliability and the management of error variance are also discussed. Specific examples are provided to illustrate the application of these designs, along with an analysis of time-series methodologies. This essential guide will enhance your understanding of single-subject research methodologies.

hachi
Télécharger la présentation

Exploring Small-N Designs in Research: Validity, Replication, and Error Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Research Design & Analysis 2: Class 14 • Small n-designs (continued) • Review of and more on: External and Ecological Validity • Using animals in research

  2. 2-Way ANOVA Output From Lab Note: F-ratios are the Mean square terms for the effect being tested divided by the mean square error. Psyc2023 Class#14 (c) Peter McLeod

  3. Characteristics of single-subject baseline (from table 10-1 of text) • Individuals are observed under each of several phases with multiple records under each phase • Extensive baseline observations are made • Each subject is observed under all phases with each treatment repeated at least twice (intrasubject replication) • A stability criterion is often used • Multiple subjects may be used (intersubject replication) Psyc2023 Class#14 (c) Peter McLeod

  4. Functions of the Baseline 1) establishes level of the DV prior to intervention 2) allows assessment of variability in DV • May involve a stability criterion to minimize error variance so that any effect of the intervention will be more apparent Psyc2023 Class#14 (c) Peter McLeod

  5. Assessing Generalizabiliy • Intersubject Replication • Present data from each subject individually Psyc2023 Class#14 (c) Peter McLeod

  6. Fig 10-1: Psyc2023 Class#13 (c) Peter McLeod

  7. Fig 10-3 Psyc2023 Class#14 (c) Peter McLeod

  8. Assessing Reliability With single-subject designs, reliability is assessed through intrasubjectreplication • (e.g., ABAB design) in contrast with: Group designs in which inferential statistics are used to assess reliability after a fairly complex chain of logical inferences Note that in both cases, the key is degree of overlap of distributions Psyc2023 Class#14 (c) Peter McLeod

  9. Dealing with Error Variance Philosophical difference between single-subject approach and group approach in dealing with error variance. With group approach, error variance is controlled somewhat, either experimentally (through randomization) OR statistically - if effects are significant (can reject the null) little further effort is made to determine the sources of error variance. With Small-n designs, control as much as possible experimentally Psyc2023 Class#14 (c) Peter McLeod

  10. Time-Series (Small-N) Designs • Don’t involve random assignment • control and manipulate variables sequentially (not simultaneously as in analytic experiments) • anything else that occurred in this time is therefore a potential confound. • Remember: with experimentation we look after these things with simultaneous manipulations (between groups designs) or by counterbalancing (within groups designs) Psyc2023 Class#14 (c) Peter McLeod

  11. Types of Replication Intrasubject - to assess reliability (given time confound) Intersubject - to assess generalizability also: Distinction between systematic replication (introduces some extension or variation on the original research - often to assess generalizability of the phenomenon) and direct replication Psyc2023 Class#14 (c) Peter McLeod

  12. Variation in baseline Variability can be due to: 1) chance variation 2) carryover effects Psyc2023 Class#14 (c) Peter McLeod

  13. Problematic Baselines • Drifting baselines • Unrecoverable baselines (due to carryover effects) • Unequal baselines between subjects • Inappropriate baselines -- floor and ceiling effects Psyc2023 Class#14 (c) Peter McLeod

  14. Drifting Baseline Psyc2023 Class#14 (c) Peter McLeod

  15. Small-n Design: Terminology Not standardized but: • Single-factor • e.g., AB, ABA, ABAB (reversal design), multilevel (ABACA..) • Multi-factor designs (> 1 IV) • Multiple baseline design (>1 DV) Psyc2023 Class#14 (c) Peter McLeod

  16. Multivalent Single-Factor Small-n Designs A-baseline B-Placebo cookie C- artificially coloured cookie Psyc2023 Class#14 (c) Peter McLeod

  17. AB (ABA) Type Non-Experimental Studies Example: Drunk driving in Michigan • legal age 21  18 in 1972, then 18  21 in 1978 • % alcohol related traffic accidents: 15%  22% in ‘72, decreased again in ‘78 • Does a lower drinking age cause alcohol related traffic deaths? Confounds? • Wider alcohol availability in ‘72 • oil crisis in ‘78 -- 55 mph limit imposed in 1973/74 Psyc2023 Class#14 (c) Peter McLeod

  18. Other Examples of AB Studies TV effects on children: Tannis MacBeth-Williams David Phillips Research: • Motor vehicle accidents after publicized suicides • Airplane accidents after publicized murder-suicides (non-equivalent control groups) • Homicides after heavyweight prize fights Psyc2023 Class#14 (c) Peter McLeod

  19. Berkowitz ‘70 Psyc2023 Class#14 (c) Peter McLeod

  20. Phillips:Car suicides? Psyc2023 Class#14 (c) Peter McLeod

  21. Phillips:Noncommercial Plane suicides? A: all plane crash fatalities B: multifatality crashes C: Single fatality crashes Psyc2023 Class#14 (c) Peter McLeod

  22. Phillips:Converging evidence? Psyc2023 Class#14 (c) Peter McLeod

  23. Phillips;prizefights Highest peak in homicides on day 3 Psyc2023 Class#14 (c) Peter McLeod

  24. Multiple baselines Psyc2023 Class#14 (c) Peter McLeod

  25. Liberman & Smith 1972 Psyc2023 Class#13 (c) Peter McLeod

  26. Non-Equivalent Control Groups Different than AB type non-experimental studies in that a comparison “individual” is also studied e.g., Mining Safety study: Lucky Star mine… (Overhead) Psyc2023 Class#14 (c) Peter McLeod

  27. Non-Equivalent Control Groups Psyc2023 Class#14 (c) Peter McLeod

  28. External and Ecological ValidityUsing Animals in Research Psyc2023 Class#14 (c) Peter McLeod

  29. Stress and Cancer 93 lab rats randomly assigned from a defined population Yoked control Psyc2023 Class#14 (c) Peter McLeod Based on Visintainer, Volpicelli, & Seligman, 1982

  30. Results Psyc2023 Class#14 (c) Peter McLeod

  31. Factors Influencing External Validity Issue of generalizability beyond the specific experimental conditions • Population sampled • Operational definitions • Parameter values • Demand characteristics Ecological validity? Psyc2023 Class#14 (c) Peter McLeod

  32. Factors Influencing External Validity Population sampled • be careful generalizing beyond population studied • look for converging evidence that there is nothing importantly unique • External validity not crucial for basic research • Sample - to - population generalizations • Sometimes unusual populations are sought out • HIV resistant individuals, spotted hyenas Psyc2023 Class#14 (c) Peter McLeod

  33. Spotted Hyena Psyc2023 Class#14 (c) Peter McLeod

  34. Factors Influencing External Validity Operational definitions • construct validity Unavoidable shock-- unnatural but are effects unique? • Look for converging evidence Psyc2023 Class#14 (c) Peter McLeod

  35. Operational Definitions of Stress Phenomena associated with stress Operational definition of stress Overcrowding Gastric ulcers Inescapable shock Depression Sensory deprivation High blood pressure Sleep deprivation Psyc2023 Class#14 (c) Peter McLeod

  36. Factors Influencing External Validity Parameter values Values selected for each variable • both independent and controlled Psyc2023 Class#14 (c) Peter McLeod

  37. Factors Influencing External Validity Demand characteristics subtle cues in a research procedure that influence the participants serious problem in social sciences • characteristics of volunteers can influence both internal and external validity Students holding “poisonous” snakes, throwing “acid” in another’s face Psyc2023 Class#14 (c) Peter McLeod

  38. Factors Influencing External Validity Ecological validity How generalizable are the experimental results to the specific set of conditions -- those of the natural context in which the phenomena usually occurs Psyc2023 Class#14 (c) Peter McLeod

  39. Research Examples: Alcohol consumption • Alcohol consumption extremely stable in adults (r>.85) • Substantial individual variability • Four patterns of drinking Abusive binge drinking Heavy steady drinking Moderate drinking Avoidance of drinking Psyc2023 Class#14 (c) Peter McLeod

  40. Research Examples: Alcohol consumption • Abusive drinkers also prefer stronger drinks • Can be distinguished pharmacologically • lower 5-HIAA levels (serotonin metabolite) Psyc2023 Class#14 (c) Peter McLeod

  41. Abusive drinking ... Psyc2023 Class#14 (c) Peter McLeod

  42. Research Examples: Alcohol consumption • Females > Males to be moderate drinkers • Females prefer sweetened mix • Subjects Psyc2023 Class#14 (c) Peter McLeod

  43. Fruit flies & alcoholism • Dr. Vanessa Auld: Dept. of Zoology at the University of British Columbia studies alcoholism in fruit flies Drosophola spp. (with alcohol vapor) • Researchers have been studying fruit flies to see if they can understand alcoholism in humans. • Increased activity and courting, then poor coordination etc. then pass out. Quirk & quarks Jan 15th approx 19:00 in Real Audio Psyc2023 Class#14 (c) Peter McLeod

  44. Fruit flies & alcoholism • “Inebriometer”: DV: time to fall Norm approx 20min, mutant, 12 min (sensitive to alcohol) • Isolated “cheap-date” gene • had already been studied in learning and memory (“amnesiac” gene) • First evidence for a defect in the brain associated with alcohol susceptibility • Looking at the mechanism of alcohol’s long term effects on the brain Quirks & quarks Jan 15th @ approximately 19:00 in Real Audio Psyc2023 Class#14 (c) Peter McLeod

  45. Using animals in psychological research • Much of what we know in psychology comes from animal research • Models of human psychopathologies • Basic research on how the brain works, perception, learning, memory, motivation • Which animal to use, depends on: • research question, previous literature, facilities, availability/cost Psyc2023 Class#14 (c) Peter McLeod

  46. Using Animals in Psychological Research • Why? • Ethical reasons; e.g., effects of brain lesions, extreme levels of some IVs that can be studied in humans • Greater control and internal validity • Convenient • Ethical considerations and guidelines (CCAC) • Cost-benefit ratio • Generality (external validity question) Psyc2023 Class#14 (c) Peter McLeod

  47. Thalidomide Psyc2023 Class#14 (c) Peter McLeod

  48. Prenatal stress & feminized behaviour • Environmentally stressed females give birth to male rats/mice whose behaviour is feminized and demasculinized • Prenatally stressed males display less sexual behaviour • If castrated and given female hormones, more likely to show lordosis • Show more paternal behaviour • Less masculine anatomy and SDN-POA Psyc2023 Class#14 (c) Peter McLeod

  49. Using animals in psychological research • Animal rights movement • Alternatives to using animals • in vitro studies • computer simulations • …”there are no viable alternatives to using animals in behavioral research” Psyc2023 Class#14 (c) Peter McLeod

  50. Psyc2023 Class#14 (c) Peter McLeod

More Related