Measurement and Scales of Variables

1. Chapter 5 Measuring Variables & Sampling

2. Variable and Measurement • Variable – A condition/characteristic that can take on different values or categories • Measurement – the assignment of symbols or numbers to something according to a set of rules

3. Scales of Measurement • Nominal Scale • use of symbols to classify or categorize • Ordinal Scale • rank-order scale of measurement • Allows to conclude who is 1st, 2nd, but not for how much! • equal distances on scale not necessarily equal on dimension being measured • Difference between 1st & 2nd = 20 secs; difference between 2nd & 3rd = 60 secs. • Laredo #1 Most Diverse City in the US,

4. Entropy index (E): Index gauges how uniformly members of a population are spread. E = 0 = complete homogeneity or no diversity

5. Scales of Measurement (cont'd) • Interval Scale • same properties of ordinal plus equal distances between adjacent numbers • e.g., temperature on Fahrenheit scale • Ratio Scale • highest scale of measurement • same properties of other scales plus absolute zero point • e.g., weight, height

6. Psychometric Properties of Good Measurement • Reliability – Consistency or stability of the scores of your measurement instrument • Temperatures: 98.9, 98.6, 97.6. 97.8, 99.0 RELIABLE? • Temperatures: 98.0,100.3, 95.3, 103.0 RELIABLE? • Validity – Extent to which your measurement procedure is measuring what you think it is measuring and whether you have interpreted your scores correctly • A measure must be reliable in order to be valid but a reliable measure is not necessarily valid

7. Types of Reliability • Test-Retest Reliability • consistency of individual scores over time • same test administered to individuals two times • correlate scores to determine reliability (high correlation?) • how long to wait between tests? • Equivalent-Forms Reliability • consistency of scores on two versions of test • each version of test given to different groups of individuals

8. Types of Reliability (cont'd) • Internal Consistency Reliability • consistency with which items on a test measure a single construct • involves comparing individual items within a single test • coefficient alpha or Cronbach’s Alpha (> .70) is common index • eHORMONY CLAIMS that with 5 Questions you can be MATCHED • HIRES Social Psychology students.. Owner a social psychologist! • Interrater Reliability • degree of agreement between two or more observers • interobserver agreement is the percentage of times different raters agree

9. Validity • Validity refers to the accuracy of the inferences, interpretations, or actions made on the basis of test scores. • The validity of test concerns what the test measures and how well it does so. It tells us what can be inferred from test scores (Anastasia & Urbina, 1997, p. 113) • Involves the measurement of constructs (e.g., intelligence or happiness) • Do operational definitions accurately represent construct we are interested in? • LEADERSHIP. How do you measure leadership? Multiple indicators? • ULTIMATELY: PREVIOUS RESEARCH!

11. Methods Used to Collect Evidence of Validity • Content-Related Evidence (content validity): JUDGMENT of the degree to which the task(s) adequately represent the construct’s domain • validity assessed by experts • do items appear to measure construct of interest? (FACE VALIDITY: Do questions address issues related to LOVE? • were any important content areas omitted? Romantic, Friendship • were any unnecessary items included?... Animal, classroom love.. • Evidence Based on Internal Structure • how well do individual items relate to the overall test score or other items on the test: Different FACTOR LOADINGS? • factor analysis – statistical procedure used to determine the number of dimensions present in a set of items; Factors… Romantic…

12. Methods Used to Collect Evidence of Validity (cont'd) • Evidence Based on Relations to Other Variables • criterion-related validity: How good the scores of TEST predict or relate to an existing test… : • predictive validity – using scores obtained at one time to predict the scores on a criterion at a later time • concurrent validity – degree to which scores obtained at one time correctly relate to the scores on a known criterion obtained at the same time • E.G., Administering a brand-new depression task that you created and at the same time ADMINISTERING THE BECK’S DEPRESSION INVENTORY… and the results of both SHOULD BE HIGHLY CORRELATED • EYETRACKING vs. Traditional behavioral measures (RT) • Run the experiment using eye-tracking and at the same time using RT Eye-tracking highly correlated to RT measures..

13. Methods Used to Collect Evidence of Validity (cont'd) • Evidence Based on Relations to Other Variables • convergent validity – extent to which test scores relate to other measures of the same construct TRIANGULATION • Eye-tracking vs. Visual Moving window: Measure your eye movements as you read vs. pressing a word each time as you read the sentence.. • discriminant validity – extent to which your test scores do not relate to other test scores measuring different constructs: Test in Research methods…. Should not relate to therapy. • known groups validity evidence – extent to which groups that are known to different from one another actually differ on the construct being developed

14. Sampling Methods • Sample – a set of elements selected from a population • Population – the full set of elements from which the sample was selected

15. Sampling Methods (cont'd) • Sampling – process of drawing elements from population to form a sample • representative sample • equal probability method of selection method (EPSEM) • Statistic – a numerical characteristic of sample data • Parameter – a numerical characteristic of population data • Sampling error (Standard error) – difference between the value of the sample statistic and the value of the population parameter • You want a small sampling error? Use large N

16. Random Sampling Techniques • Simple Random Sampling • choosing a sample in a manner in which everyone has an equal chance of being selected • sampling “without replacement” is preferred • Random numbers generators simplify the process or =rand()*n • Stratified Random Sampling • random samples drawn from different groups or strata within the population • If gender stratification variable: Separate groups into M F • Choose at random from the two groups! • proportional stratified sampling involves insuring that each subgroup in sample is proportional to the subgroups in the population • If F population = 60%, choose 60% from F

17. Random number tablehttp://www.stattrek.com/statistics/random-number-generator.aspx

21. Random Sampling Techniques (cont'd) • Cluster Random Sampling • involves random selection of groups of individuals (clusters) • Cluster: Collective type of unit including multiple elements: neighborhoods, schools, families • e.g., select at random 30 classrooms (from sampling distribution of 50: one-stage Cluster sampling). Include all subjects • Or select at random 10 subjects from the 30 classrooms (two-stage)

22. Random Sampling Techniques (cont'd) • Systematic Sampling (similar to random sampling) • Involves three steps • (1) determine the sampling interval (symbolized by k) • (2) randomly select a number between 1 and k, and include that person in your sample: CHOOSE 5 • (3) also include each kth element in your sample 5th person + 10th (kth)= 15 So, subjects would follow the pattern: 5, 15, 25, 35, 45, and son on… (son increments of 10!) If in two (Choose 3): 3, 13, 23, 33, 43…

23. Nonrandom Sampling Techniques • Convenience Sampling – using research participants that are readily available – e.g., college students • Quota Sampling – identifying quotas for individual groups and then using convenience sampling to select participants within each group

24. Nonrandom Sampling Techniques (cont'd) • Purposive Sampling – involves identifying a group of individuals with specific characteristics – e.g., college freshmen who have been diagnosed with ADHD • Snowball Sampling – technique in which research participants identify other potential participants. • particularly useful in identifying participants from a difficult to find population

25. Random Selection and Random Assignment • Random selection involves selecting participants for research • purpose is to obtain a representative sample • Random assignment involves how participants are assigned to conditions within the research • purpose is to create equivalent groups to allow for investigation of causality • 20 Participants assigned at random to Condition A • 20 Participants assigned at random to Condition B

26. Determining Sample Size • If less than 100 use entire population • Larger sample sizes make it easier to detect an effect or relationship in the population • Compare to other research studies in area • Larger sample sizes are needed if population is • heterogeneous

27. Determining Sample Size (Power) • If N < 100 use entire population • Larger N make it easier to detect an effect/relationship in the population • Compare to other research studies in area • Larger sample sizes are needed if population is • Heterogeneous, you have multiple groups • if you want to increase precision • when you expect a small effect • for some statistical techniques • if you expect a low response rate • ***when you use less efficient methods of sampling

28. Sampling in Qualitative Research • Qualitative research focuses on in-depth study of one or a few cases. • Several different sampling methods are available. It is common to mix several different methods.