Chapter 6, part I: Educational Measurement

1. Chapter 6, part I: Educational Measurement EDUC 502 October 10, 2005

2. Definition of terms • Measurement: assignment of numbers to differentiate values of a variable • Evaluation: procedures for collecting information and using it to make decisions for which some value is placed on the results • Assessment - multiple meanings • Measurement of a variable • Evaluation • Diagnosis of individual difficulties • Procedures to gather information on student performance (formative)

3. Purpose of measurement for research • Obtain information about the variables being studied • Provide a standard format for recording observations, performances, or other responses of subjects • Provide for a quantitative summary of the results from many subjects

4. Measurement scales • Nominal - categories • Race • Gender • Types of schools (e.g., public, private, parochial) • Ordinal - ordered categories, but the degree of difference between the categories is not specified. • Finishing position in a race • Ranks in the military

5. Measurement scales • Interval - equal intervals between numbers on the scale – one score can be compared directly to another in terms of the amount of difference. • Classroom Test scores • Some Achievement Test Levels • Ratio - equal intervals and an absolute zero (0) • Height • Weight • Time

6. Measurement scales • The line between interval and ordinal scales is not always clear-cut. • Example: Is a Likert scale that runs from 1-5 an example of an interval scale or an ordinal scale? • The difference matters here, because some will argue that it makes sense to calculate an average value on a Likert scale, while others argue that it does not.

7. Descriptive statistics • Definition of terms • Statistics: procedures that summarize and analyze quantitative data • Descriptive statistics: statistical procedures that summarize a set of numbers in terms of central tendency, variation, or relationships

8. Types of descriptive statistics • Frequency distributions: an organization of the data set indicating the number of times (i.e., frequency) each score was present • Types of presentations • Frequency table • Frequency polygon • Histogram • Example: Scores on a test: 20, 20, 30, 40, 50, 60, 60, 70, 70, 70, 70, 90, 90, 100.

9. Shapes of distributions • Symmetric - a set of scores that are equally distributed around a middle score. • Positively skewed - a set of scores characterized by a large number of low scores and a small number of high scores. • Negatively skewed - a set of scores characterized by a large number of high scores and a small number of low scores. • See Figure 6.2 in Chapter 6 of the text.

10. Central tendency - what is the typical score • Mode: the most frequently occurring score • Median: the score above and below which one-half of the scores occur • Mean • The arithmetic average of all scores • Statistical properties make it very useful • Concerns related to outlying scores • Determine each of these for the earlier data set

11. Excerpt from “How to Lie with Statistics” • In a classic book entitled "How to Lie with Statistics," George Huff makes the point that individuals will often choose the average (mean, median, or mode) that best supports their argument. Here is an excerpt from the beginning of his chapter that is entitled "The Well-Chosen Average". Keep in mind this book was written in 1955, when it was a big deal to make $15,000 a year:

12. Excerpt from “How to Lie with Statistics” • "You, I trust, are not a snob, and I certainly am not in the real-estate business. But let's say that you are and I am and that you are looking for property to buy along a road that is not far from the California valley in which I live.      Having sized you up, I take pains to tell you that the average income in this neighborhood is some $15,000 a year. Maybe that clinches your interest in living here; anyway, you buy and that handsome figure sticks in your mind. More than likely, since we have agreed that for the purposes of the moment you are a bit of a snob, you toss it in casually when telling your friends about where you live.

13. Excerpt from “How to Lie with Statistics” • A year or so later we meet again. As a member of some taxpayer's committee I am circulating a petition to keep the tax rate down or assessments down or bus fare down. My plea is that we cannot afford the increase: After all, the average income in this neighborhood is only $3,500 a year. Perhaps you go along with me and my committee in this-you're not only a snob, you're stingy too-but you can't help being surprised to hear that measly $3,500. Am I lying now, or was I lying last year?       You can't pin it on me either time. That is the essential beauty of lying with statistics. Both those figures are legitimate averages, legally arrived at. Both represent the same data, the same people, the same incomes. All the same it is obvious that at least one of them must be so misleading as to rival an out-and-out lie.

14. Excerpt from “How to Lie with Statistics” • My trick was to use a different kind of average each time, the word "average" having a very loose meaning. It is a trick commonly used, sometimes in innocence but often in guilt, by fellows wishing to influence public opinion or sell advertising space. When you are told that something is an average you still don't know very much about it unless you can find out which of the common kinds of average it is - mean, median, or mode.      The $15,000 figure I used when I wanted a big one is a mean, the arithmetic average of the incomes of all the families in the neighborhood. You get it by adding up all the incomes and dividing by the number there are. The smaller figure is the median, and so it tells you that half the families in question have more than $3,500 a year and half have less. I might also have used the mode, which is the most frequently met-with figure in a series. If in this neighborhood there are more families with incomes of $5,000 a year than with any other amount, $5,000 a year is the modal income" (pp. 27-29)

15. Application • Problem: Seven 100 point tests were given during the Fall Semester. Erika’s scores on the tests were: 76, 82, 82, 79, 85, 25, 83. If her grade for the semester is based completely on these tests, what grade should she receive? • Moral of the story: Quantitative claims based on measures are never objective, although they often masquerade as such.

16. Variability - how different are the scores • Range: the difference between the highest and lowest scores • Standard deviation (SD): • The average distance of the scores from the mean • Formula for calculating the SD of a population:

17. Variability Measure Exercises • Exercise 1: Calculate the range and standard deviation of Erika’s test scores. Which measure more accurately describes the variation in her scores? • Exercise 2: Suppose the instructor decided to add 5 points to each of Erika’s test scores out of the kindness of his heart. How would this impact the standard deviation? The range? Why?

18. Homework Exercises • Textbook p. 147 (4, 5, 7, 9, 10)