Tests and Measurements

1. Tests and Measurements Educational Research

2. Educational Measurement • technique used to measure data

4. Classifications • Cognitive versus Noncognitive • Commercial versus Teacher-made • Self-reporting versus Observation

5. Noncognitive examines personality, attitudes, values Cognitive measures what a person knows or is able to do mentally Cognitive Vs Noncognitive

6. Disadvantages costs of using/ scoring may not match objectives or needs norms may not reflect local pop Advantages comparative norms already put together results compared to others in norms known reliabilities and validities Commercial or Standardized

7. disadvantages time involvement unknown reliabilities and validities advantages use when inappropriate to use standardized test can more closely relate to objectives inexpensive costs Teacher-made

8. Self-reporting best way to obtain lots of information quickly NOT always honest do NOT always know Observation Self-reporting Versus Observation

9. Important Terms • Raw score • # of (in)correct responses • amount of time • Standard score • raw scores transformed to normalized score (z scores, t scores, GREs)

10. Grade equivalents • norm-referenced grade level performance • not interpreted well; limited meaning • Percentile ranks • describes how well an individual did on a test compared to a particular group • not interpreted well, esp. at extremes

11. Reliability • degree of consistency/dependability of a measuring instrument • increase # items on test • range from 0 to +1.0 • Validity • credibility of measuring instrument • does instrument measure what it claims to measure?

12. Referenced-based Measurement Interpretation • Norm-referenced • compare one to the many • performance reported in %iles, standard scores, etc. • items produce wide range of scores • norms refer to typical/avg performance of a group of people

13. Criterion-referenced • what the student CAN do without reference to others • in reference to a prespecified standard of performance • concerned with mastery • items selected to match instructional objectives

14. Types of Tests • Standard questions of cognitive knowledge or skills • Achievement Tests • Intelligence Tests • Aptitude Tests • Performance Assessment

15. Achievement Tests • measures mastery in different areas of knowledge • most common form of measurement in educational research

16. Intelligence Tests • look at performance in a specific area • NOT measures of innate or pure intelligence • dependent on background and schooling (more measures of scholastic aptitude • used to predict school/academic success; IVs

17. Important persons and tests: • Army Alpha and Army Beta • first group IQ tests • Stanford-Binet • first individual IQ test • David Wechsler • Wechsler intelligence scales • most popular in use today

18. Aptitude Tests • aimed at measuring a person’s future performance in a specific skill or area of achievement • very close to intelligence

19. Performance Assessment • authentic assessment/alternative assessment • approach to evaluating students by directly examining performance on tasks with intrinsic value

20. Measures of Personality • Assess individual differences in aspects of personality such as traits, needs, psychological disorders, values, and attitudes. • Often self-reporting.

21. Types • Inventories • Projective Techniques • association completion • role playing creative/construction • Attitude Scales/Opinionnaires • Likert scales Thurstone Scale • Semantic Differential Technique

22. Inventories • attempt to yield a measure of the types of activities an individual likes or has a tendency to choose.

23. Projective Techniques • persons project thoughts, feelings, attitudes, needs onto vague stimulus • Rorschach Ink Blot • Thematic Apperception Test • Association Completion • Role Playing Creative/Construction

24. Attitude Scales/Opinionnaires • measures degree to which person possesses characteristic of interest • do NOT reflect success/failure or strengths/weaknesses • Likert Scale • Thurstone Scale • Semantic Differential Technique

25. Likert Scale • R. Likert • collection of statements about a given topic • half are positively stated and half are negatively stated • arranged in random order • measured on a scale of 1 - 5 • (SA A U D SD)

26. Semantic Differential Scale • Circle one of the #s between each pair of adjectives to best indicate how closely one of the adjectives describes your attitude toward essay questions. • Good 1 2 3 4 5 6 7 Bad • Pleasant 1 2 3 4 5 6 7 Unpleasant • Fair 1 2 3 4 5 6 7 Unfair • Positive 1 2 3 4 5 6 7 Negative

27. Rating Scales • widely used measuring instrument • involves assessment by one person of another’s behavior/performance • Four types

28. Graphic Scale • uses a continuum • place a checkmark at the area to describe the performance/behavior • numerical values may be assigned to descriptive points

29. Numerical Scale • involves assignment of numbers to points on graphic scales

30. Categorical Scale • categories arranged in ordered series • rater selects category of person being rated

31. Comparative Rating Scale • make judgments compared to others • sometimes school systems use this method for references

32. Rating Errors • Halo Effect • generalized impression of person that slants the perception • (good manners, bad boy) • Generosity Error • giving subjects benefit of the doubt

33. Rating Errors • Error of Severity • tendency to rate all persons too low on all characteristics • Error of Central Tendency • avoiding the extremes of the scale, rate persons in middle

34. How to Reduce Errors • Thorough training of raters • BEFORE • making ratings

35. Sociometric Techniques • SOCIOGRAM • used to study social relationships present among various group members • study of choices made by each person in a group

36. Procedure • each member of group indicates other members with whom s/he would MOST like to engage in activity such as recess, lunch, or work time • choices will vary, depending upon the activity • choices are plotted

37. Sociogram Legend • boys=triangles • girls=circles • arrows show direction of selection

38. Sociogram Interpretation • star(s) • most frequently chosen member(s) • clique(s) • small subgroups of persons (3 or more) who mutually select each other • isolate • member(s) who receives no choices

39. Examples • Students asked to indicate two other children with whom they would MOST like to play: • Maria--Juan, Pat Juan--Maria, Pat • Pat--Maria, Juan Sue--Tony, Marco • Tony--Sue, Marco Laura--Sue, Marco • Jackson--Tony, Marco Marco-Laura,Tony

40. Examples • Students in an RSCH 7100 class were asked to specify the names of two other students with whom they would most like to work with on the midterm exam. Of the six students asked, the following are their selections.

41. Choices of students w/whom to work, on midterm exam • Herbert --- Sam, Julie • John --- Fred, Julie • Sam -- Julie, Herbert • Fred -- Cynthia, Julie • Julie -- Sam, Herbert • Cynthia -- Fred, Julie

42. Number of times selected • Herbert Twice (Julie, Sam) • John None (isolate) • Sam Twice (Herbert, Julie) • Fred Twice (John, Cynthia) • Julie 5X (all persons--star) • Cynthia Once (Fred) • Clique: Herbert, Julie, and Sam mutually selected only each other

43. Observations • Purpose: determine extent to which a particular behavior(s) is present • used to study classroom behaviors; natural or contrived settings; with infants, preschool, and elementary school children • in quantitative and qualitative research

44. Disadvantages expensive to use in terms of time and resources presence of observer may alter behavior observer bias Advantages observe persons’ behaviors in natural surroundings may use w/those who cannot communicate through language Observations

45. Interviews • form of data collection in which questions asked orally and participant’s responses are recorded

46. WE ARE DONE