Choosing a comparison group

1. Choosing a comparison group Jane E. Miller, PhD

2. Overview • What is a comparison group? • Choosing comparison groups based on • Theoretical criteria • Previous literature on the topic • Writing patterns • Sample size

3. What is a comparison group or value? • For each nominal or ordinal variable, the comparison group is the one against which all other categories of that variable will be compared. • In multivariate regression, the comparison group is known as the “reference category.” • For each continuous variable, the comparison value is the value against which all other values of that variable will be compared. • Choice of a comparison group or value for each variable in your analysis should NOT be arbitrary.

4. Choosing a comparison groupbased on theoretical criteria • Your specific research question will often determine choice of comparison group. E.g., • If you are analyzing effects of a medication compared to a placebo, the placebo condition is the logical comparison group. • If you are comparing other states to your home state, your home state should be the comparison group.

5. Choosing a comparison group based on prior literature • If previous studies of your topic have standard conventions of a comparison group, often you will use it as your comparison group as well. • Doing so facilitates comparison of results across studies. • BUT, it is important to think through whether their choice fits your study. • Identify the reasons why other researchers have chosen that comparison group. • Check those reasons against your own.

6. Choosing a different comparison group than the prior literature • If you have strong reasons to use a different comparison group than a major study of your topic • In your methods section, explain the theoretical or empirical basis forwhy you chose a different comparison group. • In the discussion section, translate your results to compare against the same comparison group as other leading studies.

7. Choosing a comparison group based on writing patterns • If your sentences tend to read “compared to group X,” then group X should be your comparison group. • Doing so will ensure that your calculations are consistent with how you will write about the results. • But see • Empirical criteria for sample size • Precedent in the literature

8. Anticipating what you will write • Think ahead about what you want to write. • E.g., For a ratio, write “(the numerator) is (some amount) higher/lower than (the denominator). • If you want to word it the other way around, flip your ratio over! • E.g., • “IMR among blacks is twice that among whites.” is the same as • “IMR among whites is half that among blacks.” • Whichever way you decide to write it, make sure the number in the prose matches the number you present in a table or chart. • Reversing your comparisons from table to text is a good way to confuse your readers!

9. Choosing a comparison groupbased on sample size • Lacking some other basis for selecting a comparison group, choose the largest (modal) group. • Doing so maximizes statistical power. • Sometimes this will mesh with theoretical criteria, as when the majority racial ethnic group is chosen as the comparison group. • Sometimes, the comparison group based on theoretical criteria or prior literature includes very few cases in your data set. • In that case, you might need to pick a different group to provide stable statistical estimates.

10. Comparative writing • For every comparison (e.g., rank, difference, ratio, or percentage difference) • Specify what is being compared to what. • If all you write is “X is 20% higher”, the reader doesn’t know higher than what? • Especially if you are comparing several groups, places, or time periods, failure to specify the reference group or value can be very confusing!

11. Reference group for multiple comparisons • E.g., if you are comparing age distributions for two time periods in two regions, “The elderly age group is smaller” doesn’t tell your reader whether you mean: • Smaller than other age groups in the same region, or • Smaller than the same age group in the other region, or • Smaller than it used to be, in the same region. The Chicago Guide to Writing about Numbers, 2nd edition.

12. Summary • Choice of a comparison group or value for each variable in your analysis should NOT be arbitrary. • Consider the following criteria when selecting a comparison group for each of your variables. • Theoretical • Previous literature • Writing patterns • Sample size • Write your prose interpretation to convey • Which groups or values are being compared to one another. • The order in which the calculation was done.

13. Suggested resources • Miller, J. E. 2015. The Chicago Guide to Writing about Numbers, 2nd Edition. • Chapter 5, section on choosing a comparison group • Chapter 9 The Chicago Guide to Writing about Numbers, 2nd edition.

14. Suggested online resources • Podcasts on • Reporting one number • Comparing two numbers or series of numbers • Getting to know your variables • Types of quantitative comparisons • Writing about ratios

15. Suggested practice exercises • Study guide to The Chicago Guide to Writing about Numbers, 2nd Edition. • Problem sets for • Chapter 5, questions #3, 4 and 5 • Chapter 8, questions #5 and 7 • Suggested course extensions for chapter 5 • “Reviewing” exercise #1b • “Applying statistics” exercise #1

16. Contact information Jane E. Miller, PhD jmiller@ifh.rutgers.edu Online materials available at http://press.uchicago.edu/books/miller/numbers/index.html