Measurement Error in Relationship and Marital Status Questions Nancy Bates Senior Researcher for Survey Methodology U.S. Census Bureau COPAFS Quarterly Meeting December 2, 2011
Research problem: definitions • Societal and legal definition of “marriage” has changed • New terms (e.g., same-sex husbands and wives, partner, civil unions, domestic partnerships) • State-to-state recognition and no federal level recognition • According to 2008 ACS: ~150,000 same-sex spousal couples but… • Only ~32,000 same-sex legally married couples in U.S. in 2008 (Williams Institute) Why the discrepancy?
Classification “error” • Are cohabiting gays & lesbians selecting “husband/wife” (even if not married)? • Is primacy effect a factor (‘husband/wife’ is 1st in list, ‘unmarried partner’ is 13th)? • False positives caused by heterosexual married couples mis-marking gender? • Some combination of all the above?
Research to date • Interagency Workgroup on Measuring Relationships in Federal Household Surveys (IWMRFHS) • Focus Groups (Bates et al., 2010) • Understand terms commonly used • Interpreted as legal or something else? • Shortcomings of current questions? • Come up with alternative relationship and marital status questions to cognitively test • Cognitive Interviews (DeMaio and Bates, 2011) • Test 2 alternatives of each question among target populations • Select one set for further quantitative testing
Methodology: focus groups • 18 groups in 8 geographically varied sites (15 with gay couples; 3 straight) • Included legally married, domestic partners, and no legal status Summary Findings: • Interpreted Qs in context of a federal form - that means legal status • Legally married anywhere = “husband/wife” • Very few selected “husband/wife” or “now married’ if not legally married • Need categories to reflect new legal unions • “Functional equivalence” problem • Most same-sex couples able to make selection in Relationship Q. • Marital status had bigger problems: • No place to indicate committed relationship for many same-sex couples • Consider measuring cohabitation/relationship status and marital status separately
Methodology: Cognitive Interviews • 40 interviews - included both gays and straights • Married, domestic partnerships (DP)/civil unions (CU), no legal status • Cleveland, Charlotte, Las Vegas, DC metropolitan area • Observed as respondents completed one of two versions • Retrospective cognitive interviews • How and why respondents answered as they did? • Meaning of terms and categories? • Sensitive questions? • Showed alternate versions at end – preference?
New Relationship Questions Version 1 How is this person related to Person 1? Mark (X) ONE box. [ ] Husband/wife/spouse [ ] Parent-in-law [ ] Unmarried partner [ ] Son-in-law or daughter-in-law [ ] Biological son or daughter [ ] Other relative [ ] Adopted son or daughter [ ] Roomer or boarder [ ] Stepson or stepdaughter [ ] Housemate or roommate [ ] Brother or sister [ ] Foster child [ ] Father or mother [ ] Other nonrelative [ ] Grandchild
New Relationship Questions Version 2 How is this person related to Person 1? Mark (X) ONE box. [ ] Opposite-sex husband/wife/spouse [ ] Grandchild [ ] Same-sex husband/wife/spouse [ ] Parent-in-law [ ] Opposite-sex unmarried partner [ ] Son-in-law or daughter-in-law [ ] Same-sex unmarried partner [ ] Other relative [ ] Biological son or daughter [ ] Roomer or boarder [ ] Adopted son or daughter [ ] Housemate or roommate [ ] Stepson or stepdaughter [ ] Foster child [ ] Brother or sister [ ] Other nonrelative [ ] Father or mother
New Marital Status Questions Version 1 What is this person’s current marital status? Mark (X) ONE box. [ ] Now married [ ] In a registered domestic partnership or civil union [ ] Widowed [ ] Divorced [ ] Separated [ ] Never married
New Marital Status Questions Version 2 1. What is this person’s current marital status? Mark (X) ONE box. [ ] Now married - Skip to #4 [ ] Widowed [ ] Divorced [ ] Separated [ ] Never married 2. Is this person currently living in a registered domestic partnership or civil union? [ ] Yes - skip to #4 [ ] No
Last in series: cohabitation question 3. Is this person currently living with a boyfriend/girlfriend or partner in this household? [ ] Yes [ ] No • Allows same-sex couples without legal recognition to indicate relationship status • Allows for identification of sub-families • Currently asked in Current Population Survey
Results: Relationship Question • Most survey reports aligned with “true” legal status • Category ordering effect of Version 2 caused misreporting • Version that delineates “same-sex” / “opposite-sex” had most mentions for being potentially sensitive (but was also most preferred) Recommendation: Further test delineated Version 2 • Acts as consistency check to reduce gender misreports among straight married couples • However, new category of “same-sex husband/wife/spouse” could inflate gay couple reports (reorder categories to minimize) • Also, “same-sex / opposite-sex” labels could have negative impact on unit response (need to empirically test)
Reactions to Relationship with Same-Sex and Opposite-Sex categories “This [version with opposite-sex and same-sex labels] actually might, would offend me a little bit, because I think it’s …I feel like we’re kind of just lowering the standards..[as in] OK, we’ve got to conform to everyone. We want to be politically correct” - married straight female “...it does show how we grow as a country, that’s what the Census is, showing how we deposit ourselves all over the country and how we define our families” - married gay man “[that version] tries too hard…it’s making a point. [that version means] we’re gonna put it out there…that we’re including everybody. But, I get that..it’s fine…we’re in America” - married straight female
Results: Marital Status Question • Most survey reports were consistent with “true” legal status • Inconsistent reports involved “domestic partner/civil union” in some way • Confusion /misunderstanding of “domestic partnership/civil union” (mostly among straights) • Overwhelming preference for embedded Version 1, but… Recommendation: Further test Version 2 (separate questions) • Misunderstanding of DP/CU terms is widespread • Embedding may inflate DP/CU selection among straight couples • Reduce above by having separate question • Reorder to ask cohabitation first followed by DP/CU question
Reactions to DP/CU concept “two people who have lived together for at least 7 years” -straight married male “I might instead put registered DP or CU..if I lived with somebody in a committed relationship [and] we weren’t married. Now I have another option” -straight married female “In California, you can do a domestic partnership, which is same-sex marriage” - straight married male
Cohabitation Item • When space allows, keep cohabitation question • Re-order to follow Marital Status and precede Domestic Partnership/Civil Union question
Recommended: Relationship Q. How is this person related to Person 1? Mark (X) ONE box. [ ] Opposite-sex husband/wife/spouse [ ] Grandchild [ ] Opposite-sex unmarried partner [ ] Parent-in-law [ ] Same-sex husband/wife/spouse [ ] Son-in-law or daughter-in-law [ ] Same-sex unmarried partner [ ] Other relative [ ] Biological son or daughter [ ] Roomer or boarder [ ] Adopted son or daughter [ ] Housemate or roommate [ ] Stepson or stepdaughter [ ] Foster child [ ] Brother or sister [ ] Other nonrelative [ ] Father or mother
Recommended: Marital Status Q. 1. What is this person’s current marital status? Mark (X) ONE box. [ ] Now married - Skip to #4 [ ] Widowed [ ] Divorced [ ] Separated [ ] Never married 2. Is this person currently living with a boyfriend/girlfriend or partner in this household? [ ] Yes [ ] No - skip to #4 3. Is this person currently living in a registered domestic partnership or civil union? [ ] Yes - skip to #4 [ ] No
Next Steps • Recommended questions currently under review of interagency group • Piggybacking on the next ACS forms-length test • Mail only • Small test panel (5,000 mailout or only ~2,000 returns) • Need to test in other modes • Reinterview component would be ideal
Questions? Contact: email@example.com