Download:
pdf |
pdfInteragency Technical Working Group on Sexual
Orientation and Gender Identity Items in the Household
Pulse Survey: Report and Recommendations
OMB
4/30/21
Page
Final0
Interagency Technical Working Group on Sexual Orientation and Gender
Identity Items in the Household Pulse Survey: Report and Recommendations
TABLE OF CONTENTS
1.
BACKGROUND AND PURPOSE............................................................................................................... 1
2.
EXISTING RESEARCH ON SURVEY ITEMS ............................................................................................... 2
3.
RECOMMENDATIONS ........................................................................................................................... 3
a.
Proposed Module.............................................................................................................................. 3
b.
Recommendations for Reporting SOGI Information ........................................................................ 5
c.
Recommended Testing ..................................................................................................................... 5
4.
REFERENCES .......................................................................................................................................... 6
5.
Appendix A: QxQ – Sexual Orientation ............................................................................................... 11
6.
Appendix B: QxQ – Gender Identity .................................................................................................... 13
7.
Appendix C: SOGI Measurement – Previous Experimental Results .................................................... 17
Page 1
Interagency Technical Working Group (ITWG) on Sexual Orientation and Gender
Identity Items in the Household Pulse Survey: Report and Recommendations
1. BACKGROUND AND PURPOSE
The Household Pulse Survey (HPS), conducted by the U.S. Census Bureau, is designed to provide
researchers and decision-makers with timely, relevant, and accurate information on the coronavirus
pandemic’s impact on the American public. Survey content on the HPS is provided by several different
agency partners to reflect priority information needs across government.
Sexual and gender minority (SGM) populations have differential health and mental health access to care
and outcomes (e.g, Ivey-Stephenson et al, 2019 and Heslin and Hall, 2021). They also have differential
economic and educational experiences. As such, they are an important demographic group to measure
in the HPS to monitor the impact of the coronavirus pandemic on them; however, the HPS does not
currently have SGM items.
While some federal surveys already collect data by SGM, different agencies may ask about SGM
status in different ways. The Office of Management and Budget (OMB) convened an interagency
technical working group (ITWG) to provide a consensus recommendation to OMB and Census on how to
measure sexual orientation and gender identity (SOGI) on the HPS, taking into consideration relevant
features and analytic goals of the HPS. OMB also charged the ITWG with designing a plan for monitoring,
testing, and revising any SOGI questions added to the HPS.
OMB’s Statistical and Science Policy Office (SSP) chaired the ITWG with membership composed of
Federal employees from Census HPS partner agencies and selected on the basis of their subject matter
expertise.
This report provides recommendations to guide Census and OMB. The Census Bureau is responsible for
pre-testing, refining, and deploying any SOGI items in a manner that supports the overall success and
analytic needs of the HPS. Census agrees to include this report with their Phase 4 HPS (July through
October 2021) Information Collection Request (ICR) to OMB for review and approval.
OMB may reconvene the ITWG to assist in evaluation of the performance of the questions once tested
by Census and to evaluate the need for revisions or additional research.
Census’s Household Pulse Survey
The HPS is an online survey intended to measure how the coronavirus pandemic has impacted
households across the country from a social and economic perspective. The HPS includes questions from
partner agencies about how education, employment, food security, health, housing, social security
benefits, household spending, consumer spending associated with stimulus payments, intention to
receive a COVID-19 vaccination, and transportation have been affected by the ongoing crisis.
The HPS is designed to provide useful and expeditious information to support the nation’s recovery,
focusing specifically on identifying how the pandemic has affected people’s lives and livelihoods.
Therefore, the HPS is designed to have low respondent burden, quick turnaround on product releases,
and provide experimental data estimates. Data from this survey show the widespread effects of the
coronavirus pandemic on individuals, families, and communities across the country.
Page 1
Respondents are encouraged to participate in the HPS to provide information that will help federal and
state officials to direct aid, assistance, and support to the people and places that need it most.
Respondents are also reminded that the HPS provides information that cannot be collected elsewhere.
Below are some important points about the HPS methodology. Any items proposed for inclusion in the
HPS should match the fitness of use for the HPS, meaning quick turnaround, experimental data
estimates that meet urgent needs for information to inform policies and the public.
•
•
•
•
•
•
•
Questions will be asked only of respondents (i.e., no proxy response).
Questions will only be asked of those aged 18 years and over.
This is a self-response, online survey (no interviewer-administered option).
While Spanish language responses are a small percentage of completed cases, a viable Spanish
translation is still needed for all items.
The HPS has a very tight turnaround time for data processing. This means that Census only
produces counts for the number of responses in any open-ended items on the survey.
The sample size within cells along with Census Bureau dissemination policies determine the
granularity of published results.
There is limited time for Census to cognitively test any revisions to baseline SOGI questions.
However, there may be time and resources for Census to conduct cognitive testing of proposed
survey items during fielding of the HPS. There may also be opportunities in the future to explore
split-sample tests to examine survey item wording differences.
These considerations informed the following principles that the ITWG established:
•
•
SOGI questions on the Household Pulse Survey provide another demographic variable for use in
looking at various indicators on the survey (e.g., vaccine hesitancy, food insecurity, physical and
mental wellbeing) to better understand the impact of federal programs and improve equitable
deployment.
The SOGI items should not be used for prevalence estimates of sexual or gender minority
populations.
ITWG Deliverables
This report contains a draft module for classifying the sexual orientation and gender identity of HPS
respondents (SOGI module). It also includes a draft testing strategy for consideration.
2. EXISTING RESEARCH ON SURVEY ITEMS
The ITWG reviewed the existing literature as well as discussed results from SOGI cognitive testing and/or
data collections with survey researchers. Below are brief highlights of research findings and an
expanded list can be found in Appendix C.
• In general, respondents don’t have difficulty providing responses for SOGI items, meaning there
is low item nonresponse, especially compared to other sensitive data items. Item nonresponse
varies, though, by demographic group, with older, women, non-Hispanic African Americans,
Asians, and Hispanics, and those with less education having higher rates.
• Also, the addition of SOGI items does not lead to survey breakoffs.
• Terms used in the most commonly fielded questions on sexual identity do not exhaustively
describe respondents’ identities, primarily for youth or young adults.
Page 2
•
•
•
•
•
•
Write-in responses to “something else” provide mixed results.
Research to date has illustrated the difficulty in translating SOGI terms in a way that conveys the
intended meaning in a culturally and linguistically appropriate way as well as a lack of
terminology that conveys concepts in some languages.
Two-step questions (i.e., ask sex assigned at birth followed by current gender identity) are
recommended by many researchers.
Some studies use a third gender identity question, which if responses to the previous two
gender questions are inconsistent, respondents are asked to confirm both of their previous
gender identity answers.
Most respondents have no difficulty responding to the sex assigned at birth and gender
questions. However, they do not work well for all transgender individuals as some gender
minority groups do not see transgender as an identity distinct from male or female; others
prefer genderqueer or another term to female or male.
Spanish Translation remains a challenge for the SOGI items.
3. RECOMMENDATIONS
a. Proposed Module
Below are the recommended SOGI items for use in the HPS, given the ITWG’s deliberations. More
details about what was discussed and justification for the item wording can be found in Appendix A
(QxQ - Sexual Orientation Items) and Appendix B (QxQ - Gender Identity Items).
The SOGI questions should be placed at the end of the beginning demographic questions, with the
gender identity series appearing together (i.e., not split up) for appropriate context. These items are
based on the NHIS and NCVS surveys. Recommendations for changes to current HPS questions (including
those that are related to the SOGI questions) are outside of the scope of this working group.
Accordingly, HPS decision-makers and their OMB desk officer should reach consensus on whether the
items recommended in this document should replace or alter existing questions.
The Spanish versions of the questions should undergo expert review to determine if any changes are
needed prior to implementation; however, if meaningful changes are made, testing should be employed
(if none already exists). Expert reviewers should draw upon existing research and evaluation of
translation of these items, such as the Census Barriers, Attitudes, and Motivators (CBAMS) survey
questionnaire, to inform their expert review.1
Sexual Orientation – English
Based on 2014 NHIS question series. Q1. Which of the following best represents how you think of
yourself?
1
2
3
4
Gay or lesbian
Straight, that is not gay or lesbian
Bisexual
Something else
1
The CBAMS questionnaire is available at: https://www2.census.gov/programs-surveys/decennial/2020/programmanagement/2020_cbams_questionnaire_final.pdf
Page 3
5
I don’t know
Sexual Orientation – Spanish
Based on 2014 NHIS question series. Q1. ¿Cuál de las siguientes mejor representa su manera de
pensar en sí mismo?
1
2
3
4
5
Gay o lesbiana
Heterosexual, o sea, no gay o lesbiana
Bisexual
Otra cosa
No sé
Gender Identity – English
Based on the 2016 NCVS. Q1. What sex were you assigned at birth, on your original birth certificate?
1
2
Male
Female
Q2. Do you currently describe yourself as male, female or transgender?
1
2
3
4
Male
Female
Transgender
None of these
Q3. Just to confirm, you were assigned {FILL} at birth and now you describe yourself as {FILL}. Is that
correct?
1
2
Yes
No
Gender Identity – Spanish
Based on the 2016 NCVS. Q1. ¿Qué sexo le asignaron al nacer, en su acta de nacimiento original?
1
2
Hombre
Mujer
Q2. ¿Actualmente se describe a sí mismo(a) como hombre, mujer o transgénero?
1
2
3
4
Hombre
Mujer
Transgénero
Ninguna de las anteriores
Q3. Solo para confirmar, le asignaron el sexo {FILL} al nacer y ahora se describe como {FILL}. ¿Es esto
correcto?
1
2
Sí
No
Page 4
b. Recommendations for Reporting SOGI Information
The ITWG also discussed how Census might report SOGI information in their data releases. These
recommendations are as follows:
Gender Reporting Categories
The ITWG recommends reporting out four gender categories for ”gender” based on the following
response logic. There is not enough evidence at this time to inform classifying “none of these” into the
“gender minority” category.
1. Male – would be determined by Q1 male AND Q2 male
2. Female – would be determined by Q1 female AND Q2 female
3. Gender minority would be any of these combinations:
a. Q1 female and Q2 male
b. Q1 male and Q2 female
c. Q2 transgender
4. None of these
Sexual Orientation Reporting Categories
Evaluation of NHIS data has found that people who use “don’t know” are qualitatively different than
those that use “something else” for “sexual orientation.” Therefore, it is recommended that these
categories be reported out separately.
1.
2.
3.
4.
5.
Gay or lesbian
Straight
Bisexual
Something else
Don’t know
c. Recommended Testing
The first topic the ITWG discussed was whether a test, such as a split-sample test, was needed to
determine whether inclusion of SOGI items in the HPS would lower response rates or increase item
nonresponse rates, versus a survey without these items included. Given that research has found low
item nonresponse rates to SOGI items and no impact on survey break-off rates, the ITWG believes that
such a test is not needed prior to implementing SOGI items in the HPS.
The ITWG recommends that Census consider conducting the following research on the recommended
module, as permitted by time and resources:
•
•
Explore changes to the response categories for the sexual orientation question including using
“none of these describe me” as a replacement for the “something else” response as well as
considering additional prospective categories, such as “asexual.”
Explore more about respondents who answer “something else” or “I don’t know” on the sexual
orientation question.
Page 5
•
The current gender identity items fit the purpose and use of the HPS, but other researchers
should examine how the gender identity items could better reflect contemporary self-labeling of
gender. This includes research to incorporate the addition of “intersex” or “X” in the question
text on birth certificates, the use of “man” or “woman” as response categories for gender
identity, and possibly using a ”check all that apply” response option for gender.
4. REFERENCES
Bates, N.A., Garcia Trejo, Y.A, Vines, M. (2019). Are Sexual Minorities hard to Survey? Insights from the
2020 Census Barriers, Attitudes and Motivators (CBAMS) Study. Journal of Official Statistics, 35, 709-729.
Retrieved February 26, 2021 at: https://content.sciendo.com/view/journals/jos/35/4/articlep709.xml?language=en
Bulgar-Medina, J. 2017. ‘SOGI’ in Context: Findings from Cognitive Interviews. Paper presented at the
American Association for Public Opinion Research Annual Meeting: May 18-21, 2017, New Orleans, LA.
Cahill, S, Singal, R, Grasso, C, et al. 2014. “Do Ask, Do Tell: High Levels of Acceptability by Patients of
Routine Collection of Sexual Orientation and Gender Identity in Four Diverse American Community
Health Centers.” PLoS One 9: e107104.
Case, P, Austin, B, Hunter D, Willett, W, Malspeis, S, Manson, J, and Spiegelman, D. 2006. "Disclosure of
Sexual Orientation and Behavior in the Nurses' Health Study II." Journal of Homosexuality 51(1): 13-31.
doi:10.1300/j082v51n01_02.
Christopher, E.M. and Burns, L. 2021. Using paradata and metadata to assess effects of addition of
sensitive items to an ongoing longitudinal survey. Presentation for FedCASIC Conference, April 14, 2021.
Coffman, K, Coffman, L and Ericson, K. 2013. “The Size of the LGBT Population and the Magnitude of
Anti-gay Sentiment are Substantially Underestimated.” No. w19508. National Bureau of Economic
Research.
Dahlhamer, J., Galinsky, A., Joestl, S., and Ward, B. "Sexual Orientation in the 2013 National Health
Interview Survey: A Quality Assessment." Vital and Health Statistics. Series 2, Data Evaluation and
Methods Research 169 (2014): 1-32.
Deutsch, M, Green, J, Keatley, J, et al. 2013. “Electronic Medical Records and the Transgender Patient:
Recommendations from the World Professional Association for Transgender Health EMR Working
Group.” Journal of the American Medical Informatics Association 20 (4): 700-703.
Eliason, M. J., A. Radix, J. A. McElroy, S. Garbers, and S. G. Haynes. 2016. The “Something Else” of Sexual
Orientation: Measuring Sexual Identities of Older Lesbian and Bisexual Women Using National Health
Interview Survey Questions. Women’s Health Issues 26(Suppl 1): S71–S80.
Eliason, M. J. and C. G. Streed, Jr. 2017. Choosing “Something Else” as a Sexual Identity: Evaluating
Response Options on the National Health Interview Survey. LGBT Health 4(5): 376–379.
Ellis, R., Virgile, M., Holzberg, J., Nelson, D.V., Edgar, J., Phipps, P., Kaplan, R. (2018). Assessing the
Feasibility of Asking about Sexual Orientation and Gender Identity in the Current Population Survey:
Page 6
Results from Cognitive Interviews. Research Report Series, Survey Methodology #2018-06. Retrieved
February 26, 2021 at: https://www.census.gov/content/dam/Census/library/workingpapers/2018/adrm/rsm2018-06.pdf
Federal Interagency Working Group on Improving Measurement of Sexual Orientation and Gender
Identity in Federal Surveys. (2016). Current Measures of Sexual Orientation and Gender Identity in
Federal Surveys. Retrieved at: https://nces.ed.gov/FCSM/pdf/buda5.pdf
Federal Interagency Working Group on Improving Measurement of Sexual Orientation and Gender
Identity in Federal Surveys. (2016). Evaluations of Sexual Orientation and Gender Identity Survey
Measures: What Have We Learned. Retrieved at:
https://nces.ed.gov/FCSM/pdf/Evaluations_of_SOGI_Questions_20160923.pdf
Federal Interagency Working Group on Improving Measurement of Sexual Orientation and Gender
Identity in Federal Surveys. (2016). Toward a Research Agenda for Measuring Sexual Orientation and
Gender Identity in Federal Surveys: Findings, Recommendations, and Next Steps. Retrieved at:
https://dpcpsi.nih.gov/sites/default/files/SOGI_Research_Agenda_Final_Report_20161020_508.pdf
Fisher, S, Michaels, M, and Kahn, E. 2001. “A Needs Assessment Study of LBT Women: The Relationship
between Sexual Orientation/Gender Identity, Health-Seeking Behaviors, and Perceived Quality of Care
Issues.” Proceedings of the Annual Meeting of the American Statistical Association, August 5-9, 2001.
Galupo, M. P., S. B. Henise, and N. L. Mercer. 2016. “The Labels Don't Work Very Well”: Transgender
Individuals’ Conceptualizations of Sexual Orientation and Sexual Identity. International Journal of
Transgenderism 17(2): 93–104.
Glaze, L. 2015. “Nonsubstantive Changes to OMB #1121-0152, Survey of Prison Inmates, 2016”
Department of Justice.
Grant, D, and Jans, M. No date. “Assessing Sexual Identity & Behavior in California’s Diverse Population”
UCLA Center for Health Policy Research.
Grant, LA, Mottet, J.D., Tanis, J., Herman, J, Harrison, J, Keisling, M. 2010. “National Transgender
Discrimination Survey Report on Health and Health Care.” National LGBTQ Task Force. Retrieved at:
https://transequality.org/sites/default/files/docs/resources/NTDS_Report.pdf
Gruskin, E, Hart, S, Gordon, N, and Ackerson, L. 2001. "Patterns of Cigarette Smoking and Alcohol Use
among Lesbians and Bisexual Women Enrolled in a Large Health Maintenance Organization." American
Journal of Public Health 91 (6): 976-9.
Heslin, KC, and Hall, JE. 2021. Sexual Orientation Disparities in Risk Factors for Adverse COVID-19–
Related Outcomes, by Race/Ethnicity — Behavioral Risk Factor Surveillance System, United States,
2017–2019. Morbidity Mortality Weekly Report (MMWR) 70:149–154. Retrieved at:
http://dx.doi.org/10.15585/mmwr.mm7005a1
Holzberg, J., Ellis, R., Virgile, M., Nelson, D.V., Edgar, J., Phipps, P., Kaplan, R. (2017). Assessing the
Feasibility of Asking about Gender Identity in the Current Population Survey: Results from Focus Groups
with Members of the Transgender Population.
Page 7
Institute of Medicine (US) Committee on Lesbian, Gay, Bisexual, and Transgender Health Issues and
Research Gaps and Opportunities. (2011). The Health of Lesbian, Gay, Bisexual, and Transgender People:
Building a Foundation for Better Understanding. Retrieved at:
https://www.ncbi.nlm.nih.gov/books/NBK64806/
Institute of Medicine of the National Academies, Committee on Lesbian, Gay, Bisexual, and Transgender
Health Issues and Research Gaps and Opportunities. 2011. “The Health of Lesbian, Gay, Bisexual, and
Transgender People: Building a Foundation for Better Understanding.” National Academies Press, March
31. Retrieved at: http://www.nationalacademies.org/hmd/Reports/2011/The-Health-of-LesbianGayBisexual-and-Transgender-People.aspx
Interagency Working Group on Measuring Relationships in Federal Household Surveys (MRFHS). 2014.
Improved Measurement of Household Relationships in Federal Surveys. Retrieved at:
http://fcsm.sites.usa.gov/files/2014/04/MRFHS_StatisticalPolicyWorkingPaper201408.pdf
Ivey-Stephenson, AZ, Demissie, Z, Crosby, AE, Stone, DM, Gaylor, E, Wilkins, N, Lowry, R, and Brown, M.
2019. Suicidal Ideation and Behaviors Among High School Students — Youth Risk Behavior Survey,
United States. Morbidity and Mortality Weekly Report (MMWR), 69(Suppl-1):47–55. Retrieved at:
https://www.cdc.gov/mmwr/volumes/69/su/su6901a6.htm?s_cid=su6901a6_w
Jans, M, Viana, J, Grant, D, Cochran, S, Lee, A, and Ponce, N. 2015. "Trends in Sexual Orientation Missing
Data over a Decade of the California Health Interview Survey." American Journal of Public Health 105
(5): E43-E50
Joloza, T, Evans, J., and O’Brien, R. 2010. “Measuring Sexual Identity: An Evaluation Report.” Office for
National Statistics, United Kingdom.
Kim, H, and Fredriksen-Goldsen, K. 2013 “Nonresponse to a Question on Self-Identified Sexual
Orientation in a Public Health Survey and Its Relationship to Race and Ethnicity.” American Journal of
Public Health 103(1): 67-69
Lombardi, E, and Banik, S. 2016. “The Utility of the Two-Step Gender Measure within Trans and Cis
Populations.” Sexuality Research and Social Policy 13(3): 288–296.
Martinez, M., A. Henderson, J. Luck, and M.C. Davis. 2017. Cognitive Pretesting of the National Crime
Victimization Survey Supplemental Victimization Survey. Suitland, MD: U.S. Census Bureau, Center for
Survey Measurement. Retrieved at: https://www.census.gov/library/workingpapers/2017/adrm/rsm2017-03.html
Meyer, I. H., S. Marken, Z. Auter, B. D. M. Wilson, and K. Conron. 2019. Asking About Sexual Orientation
in a National General Population Survey: Do Expanded Response Options Improve Survey Performance
with Sexual Minority Respondents? May 16-19, 2019, Toronto, Canada.
Michaels, S., Milesi C., Stern, M., Viox ,M. H., Morrison, H., Guerino, P., Dragon, C. N., and, Haffer, S. C.
(2016a). Improving Measures of Sexual and Gender Identity in English and Spanish to Identify LGBT
Older Adults in Surveys. LGBT Health 4(6): 1–7.
Page 8
Michaels, S., Stern, M., Guerino, P., Haffer, S.C., and Reed-Gillette, D. (2016b). Results from Qualitative
Research using Cognitive Interviews. Presented at the Annual Meeting of the American Association for
Public Opinion Research.
Miller, K and Ryan, JM. 2011. "Design, Development and Testing of the NHIS Sexual Identity Question."
National Center for Health Statistics (2011): 1-33. Retrieved at:
https://wwwn.cdc.gov/qbank/report/Miller_NCHS_2011_NHIS%20Sexual%20Identity.pdf
Mohr, J and Kendra, M. 2011. "Revision and Extension of a Multidimensional Measure of Sexual
Minority Identity: The Lesbian, Gay, and Bisexual Identity Scale." Journal of Counseling Psychology 58
(2): 234.
Reisner, SL, Conron, KJ, Tardiff, LA, et al. 2014. “Monitoring the Health of Transgender and Other Gender
Minority Populations: Validity of Natal Sex and Gender Identity in a U.S. National Cohort of Young
Adults.” BMC Public Health 14: 1224.
Ridolfo, H, Miller, K, and Maitland, A. 2012. "Measuring Sexual Identity using Survey Questionnaires:
How Valid are our Measures?" Sexuality Research & Social Policy 9 (2): 113-124.
Ridolfo, H., Perez, K., and Miller, K. (2010). Testing of Sexual Identity and Health Related Questions:
Results of Interviews Conducted May-July 2005. Q-Bank. Retrieved at: http://www.cdc.gov/qbank
Ridolfo, H., and Schoua-Glusberg, A. (2009). Testing of NHANES ACASI Sexual Behavior Questions:
Results of Interviews Conducted November 2008–February 2009. Q-Bank. Retrieved at:
http://www.cdc.gov/qbank
Russell, S. T., T. J. Clarke, and J. Clary. 2009. Are Teens “Post-Gay”? Contemporary Adolescents’ Sexual
Identity Labels. Journal of Youth and Adolescence 38(7): 884–890.
Saewyc, E, Bauer, G, Skay, C, Bearinger, L, Resnick, M, Reis, E, and Murphy, A. "Measuring Sexual
Orientation in Adolescent Health Surveys: Evaluation of Eight School-based Surveys." Journal of
Adolescent Health 35, no. 4 (2004): 345-e1.
Stegier, D., L. Heaton, J. Behm, C. MacAllum, and J. Stroop. 2017. Improving the Measurement of Sexual
Orientation and Gender Identity Among Youth. Paper presented at the American Association for Public
Opinion Research Annual Meeting, May 18-21, 2017, New Orleans, LA.
Stern, MJ, Michaels, S, Milesi, C, et al. 2016. “Task 1.32.a: Develop and Test Sexual and Gender Minority
Status (LGBT) Items Final Summary Report.” (unpublished report)
Temkin, D., J. Belford, T. McDaniel, B. Stratford, and D. Parris. 2017. Improving Measurement of Sexual
Orientation and Gender Identity Among Middle and High School Students. Child Trends 22: 1–64.
The GenIUSS Group. 2014. Best Practices for Asking Questions to Identify Transgender and Other
Gender Minority Respondents on Population-Based Surveys. J.L. Herman (Ed.). Los Angeles, CA: The
Williams Institute.
Truman, J.L., Morgan, R.E., Gilbert, T., and Vaghela, P. 2019. Measuring sexual orientation and gender
identity in the national crime victimization survey. Journal of Official Statistics, 35 (4), 835–858.
Retrieved at: http://dx.doi.org/10.2478/JOS-2019-0035
Page 9
Vrangalova, Z., and Savin-Williams, R.C. 2012. “Mostly Heterosexual and Mostly Gay/Lesbian: Evidence
for New Sexual Orientation Identities.” Archives of Sexual Behavior 41(1): 85-101.
Westbrook, L., and Saperstein, A. 2015. “New Categories are Not Enough: Rethinking the Measurement
of Sex and Gender in Social Surveys.” Gender and Society 29(4): 534-560.
VanKim, N, Padilla, J, Joseph, L, and Goldstein, A. 2010. "Adding Sexual Orientation Questions to
Statewide Public Health Surveillance: New Mexico's Experience." American Journal of Public Health 100
(12): 2392-6.
Page 10
5. Appendix A: QxQ – Sexual Orientation
Baseline
item from
what survey?
Proposed item
Purpose of
item
Q1. Which of the
following best
represents how
you think of
yourself?
1. Lesbian or
Gay or
lesbian
2. Straight, that
is not gay or
lesbian
3. Bisexual
4. Something
else
5. I don’t know
Item is not
intended as a
prevalence
estimate for
SGM. It is
intended to
help policy
makers use the
HPS to
determine
inequities for
SGM during the
coronavirus
pandemic.
2014 NHIS
Q1. ¿Cuál de las
siguientes mejor
See above
2014 NHIS
Spanish
Cognitively
Question Text
Tested?
Sexual Orientation
NHIS:
No change. The team
https://wwwn.cdc. discussed having follow-up
gov/qbank/report/ questions for 4s and 5s,
Miller_NCHS_2011 much like NCHS surveys,
_NHIS%20Sexual% however, this isn’t well20Identity.pdf2
suited or aligned with the
specific interests of the
NCVS SOGI:
HPS, as this is a much
https://www.censu more burdensome
s.
construction.
gov/library/workin
gpapers/2017/adrm
/rsm2017-03.html3
NHIS:
https://wwwn.cdc.
Response Categories
Don’t Know/
Refused
Recommendation is no
change, meaning to keep
“something else” as used
in the NHIS, unless there
can be testing at some
point of possibly
changing this to “none of
these describe me”.
Response 5 was
shortened to “I
don’t know” from “I
don’t know the
answer.” The
phrase “the answer”
implies there is a
right or wrong
answer to the
question and is
simpler language.
The NHIS uses “Lesbian
or gay” as opposed to
“gay or lesbian”. While
the order hasn’t been
explicitly tested, ‘gay’
could come first since it’s
a more widely known
term than ‘lesbian’. Also
gay may be a broader
category.
See English version.
2
Miller, K., and Ryan, J.M. 2011. Design, Development and Testing of the NHIS Sexual Identity Question, Hyattsville, MD: National Center for Health Statistics.
Available at https://wwwn.cdc.gov/qbank/report/Miller_NCHS_2011_NHIS%20Sexual%20Identity.pdf
3
Martinez, M., A. Henderson, J. Luck, and M.C. Davis. 2017. Cognitive Pretesting of the National Crime Victimization Survey Supplemental Victimization Survey.
Suitland, MD: U.S. Census Bureau, Center for Survey Measurement. Available at: https://www.census.gov/library/working-papers/2017/adrm/rsm2017-
03.html
Page 11
Proposed item
representa su
manera de pensar
en sí mismo?
1. Gay
2. Heterosexual,
o sea, no gay
3. Bisexual
4. Otra cosa
5. No sé
Purpose of
item
Baseline
item from
what survey?
language
questionnair
e, with same
updates as
made to the
English
version.
Cognitively
Tested?
gov/qbank/report/
Miller_NCHS_2011
_NHIS%20Sexual%
20Identity.pdf4
Question Text
Response Categories
Don’t Know/
Refused
NCVS SOGI: None
Center for
Medicare and
Medicaid Studies:
https://www.lieber
tpub.com/doi/full/
10.1089/lgbt.2016.
01685
NOTES: SGM = Sexual and Gender Minorities
4
Miller, K., and Ryan, J.M. 2011. Design, Development and Testing of the NHIS Sexual Identity Question, Hyattsville, MD: National Center for Health Statistics.
Available at https://wwwn.cdc.gov/qbank/report/Miller_NCHS_2011_NHIS%20Sexual%20Identity.pdf
5
Micheals, S., Milesi, C., Stern, M., Viox, M.H., Morrison, H., Guerino, Pl, Dragon, C.N., and Haffer, S.C. 2017. Improving measures of sexual and gender identity
in english and spanish to identify lgbt older adults in surveys. LGBT Health, 4(6), 412-418. Available at
https://www.liebertpub.com/doi/abs/10.1089/lgbt.2016.0168.
Page 12
6. Appendix B: QxQ – Gender Identity
Proposed item
Purpose of
item
Q1. What sex were
you assigned at
birth, on your
original birth
certificate?
1. Male
2. Female
Item is not
intended as a
prevalence
estimate for
SGM. It is
intended to
help policy
makers use the
HPS to
determine
inequities for
SGM during the
coronavirus
pandemic.
Baseline
item from
what survey?
2016 NCVS
Cognitively
Question Text
Tested?
Gender Identity
NCVS SOGI:
No change. Prior testing
https://www.c found specifying the “original
ensus.
birth certificate” (as opposed
gov/library/w to “biological sex assigned at
orkingbirth”) was needed to ensure
papers/2017/ the respondent understood
adrm/rsm201 what information was
7-03.html6
requested. Use of “Intersex”
may not be widely
understood and research
would be needed to
determine if it can be
appropriately be included as
a response option, which is
beyond the scope of the
current endeavor. Also HPS
survey respondents are aged
18 and over. It is highly
unlikely that any of these
individuals would have had
“intersex” reported as a
category on their original
birth certificate.
6
Response Categories
No change, but could
explore using “man”,
“woman” if testing can
be done at some point.
Don’t Know/
Refused
Deleted “don’t
know” and
“refused” as
respondents can
skip any questions
they don’t want to
answer in the HPS.
[“Refused” is not a
response option in
the HPS.]
Martinez, M., A. Henderson, J. Luck, and M.C. Davis. 2017. Cognitive Pretesting of the National Crime Victimization Survey Supplemental Victimization Survey.
Suitland, MD: U.S. Census Bureau, Center for Survey Measurement. Available at: https://www.census.gov/library/working-papers/2017/adrm/rsm201703.html
Page 13
Proposed item
Q2. Do you current
describe yourself as
male, female or
transgender?
1. Male
2. Female
3. Transgender
4. None of these
Q3. Just to confirm,
you were assigned
{FILL} at birth and
now you describe
yourself as {FILL}. Is
that correct?
1. Yes
2. No
Q1. ¿Qué sexo le
asignaron al nacer,
en su acta de
Purpose of
item
For those
respondents
that show an
inconsistency
between Q1
and Q2, they
would also be
labeled as
“transgender”.
Baseline
item from
Cognitively
what survey?
Tested?
2016 NCVS
NCVS SOGI:
https://www.c
ensus.
gov/library/w
orkingpapers/2017/
adrm/rsm201
7-03.html7
This is
confirmation
that the
respondent did
not make a
keystroke error
in Q1 or Q2.
2016 NCVS
NCVS SOGI:
https://www.c
ensus.
gov/library/w
orkingpapers/2017/
adrm/rsm201
7-03.html8
See English
revisions
2016 NCVS
NCVS: None
Question Text
Response Categories
No change.
Prior testing and evaluation
suggests use of the term
“gender identity” may not be
widely understood. This may
be further complicated if
attempting to translate the
term into Spanish.
No change. The Group
discussed “check all that
apply” and believed it
may introduce non-SGMminority response error
and the analytic
granularity is not needed
for the HPS. Testing
would be needed to
evaluate this and is out of
scope for this application.
No change.
In a recent NCVS, 10% of the
152 respondents who
received Q3 indicated a
response was not correct, 5%
indicated they were
confused, and 2% indicated
they did not know the
answer. Therefore, the ITWG
believe this question is
needed.
See English version.
7
Don’t Know/
Refused
“Don’t know” is not
in the baseline NCVS
version. Future
testing can compare
different types of
“don’t know”s
included (see
Williams Institute
Recommendations).
N/A
Martinez, M., A. Henderson, J. Luck, and M.C. Davis. 2017. Cognitive Pretesting of the National Crime Victimization Survey Supplemental Victimization Survey.
Suitland, MD: U.S. Census Bureau, Center for Survey Measurement. Available at: https://www.census.gov/library/working-papers/2017/adrm/rsm201703.html
8
Martinez, M., A. Henderson, J. Luck, and M.C. Davis. 2017. Cognitive Pretesting of the National Crime Victimization Survey Supplemental Victimization
Survey. Suitland, MD: U.S. Census Bureau, Center for Survey Measurement. Available at: https://www.census.gov/library/working-
papers/2017/adrm/rsm2017-03.html
Page 14
Proposed item
Purpose of
item
Baseline
item from
what survey?
2016 NCVS
Cognitively
Tested?
Center for
Medicare and
Medicaid
Studies:
https://www.l
iebertpub.co
m/doi/full/10.
1089/lgbt.201
6.01689
NCVS: None
See English version.
2016 NCVS
Center for
Medicaid
Studies:
https://www.l
iebertpub.co
m/doi/full/10.
1089/lgbt.201
6.016810
NCVS: None
See English version.
nacimiento
original?
1. Hombre
2. Mujer
Q2. ¿Actualmente
se describe a sí
mismo(a) como
hombre, mujer o
transgénero?
1. Hombre
2. Mujer
3. Transgénero
4. Ninguna de las
anteriores
Q3. Solo para
confirmar, le
asignaron el sexo
{FILL} al nacer y
See English
revisions
See English
revisions
Question Text
Response Categories
Don’t Know/
Refused
Center for
Medicaid
9
Micheals, S., Milesi, C., Stern, M., Viox, M.H., Morrison, H., Guerino, Pl, Dragon, C.N., and Haffer, S.C. 2017. Improving measures of sexual and gender identity
in english and spanish to identify lgbt older adults in surveys. LGBT Health, 4(6), 412-418. Available at
https://www.liebertpub.com/doi/abs/10.1089/lgbt.2016.0168
10
Micheals, S., Milesi, C., Stern, M., Viox, M.H., Morrison, H., Guerino, Pl, Dragon, C.N., and Haffer, S.C. 2017. Improving measures of sexual and gender
identity in english and spanish to identify lgbt older adults in surveys. LGBT Health, 4(6), 412-418. Available at
https://www.liebertpub.com/doi/abs/10.1089/lgbt.2016.0168
Page 15
Proposed item
Purpose of
item
Baseline
item from
what survey?
ahora se describe
como {FILL}. ¿Es
esto correcto?
1. Sí
2. No
Cognitively
Tested?
Studies:
https://www.l
iebertpub.co
m/doi/full/10.
1089/lgbt.201
6.016811
Question Text
Response Categories
NOTES: SGM = Sexual and Gender Minorities
11
Micheals, S., Milesi, C., Stern, M., Viox, M.H., Morrison, H., Guerino, Pl, Dragon, C.N., and Haffer, S.C. 2017. Improving measures of sexual and gender
identity in english and spanish to identify lgbt older adults in surveys. LGBT Health, 4(6), 412-418. Available at
https://www.liebertpub.com/doi/abs/10.1089/lgbt.2016.0168
Page 16
Don’t Know/
Refused
7. Appendix C: SOGI Measurement – Previous Experimental Results
The ITWG carried out a targeted review of literature and consulted with experts from the Census
Bureau, Bureau of Labor Statistics, National Center for Health Statistics, National Institutes Health,
National Center for Education Statistics, Health Resources and Services Administration, and Williams
Institute to document what we know about field SOGI questions. Below are additional resources the
ITWG used during discussions:
Federal Resources
•
•
•
NHIS SOGI items: Frequently Asked Questions and NHIS Questionnaires
2016 NCVS Questionnaire (for SO + GI) – English
“All of Us” NIH SOGI items
Non-Federal Resources
•
•
•
•
•
•
Best Practices for Asking Questions to Identify Transgender and Other Gender Minority
Respondents on Population-Based Surveys (GenIUSS) – 2014
Williams Institute Sexual Orientation and Gender Identity (SOGI) Adult Measures
Recommendations FAQs – March 2020
CHIS Questionnaires (translated): Design & Methods | UCLA Center for Health Policy
Research
CHIS Spanish Questionnaire (Web): CHIS 2021 Spanish CAWI v1.02 19MARCH2021 Adult
Questionnaire.pdf (ucla.edu)
CHIS English Questionnaire (Web): CHIS 2021 CAWI v1.25 05APRIL2021 Adult
Questionnaire.pdf (ucla.edu)
CHIS report: CHIS 2019 SOGI Work Group Summary and Recommendations 20180629.pdf
(ucla.edu)
What have we learned about item non-response relative to Sexual Orientation (SO) questions from
research and evaluation?
• The majority of respondents appear to have no difficulty answering sexual identity items (Case et
al., 2006; Coffman et al., 2013; Dahlhamer et al., 2014; Fisher et al., 2001; Joloza et al., 2010; Mohr
and Kendra, 2011; Saewyc et al., 2004).
• Overall, item nonresponse appears to be relatively low across all studies that have examined
sexual identity, varying from less than 1% to just over 6% (Bates et al. 2019; Case et al., 2006;
Christopher and Burns, 2021; Dahlhamer et al., 2014; Grant and Jans, n.d.; Ridolfo et al., 2012;
Truman et al., 2019; VanKim et al., 2010).
• Item nonresponse may also vary over time. Cross-sectional data from the California Health
Interview Survey (CHIS) showed that sexual minority self-identification increased over time as
item nonresponse declined (Jans et al., 2015).
• Research has found that nonresponse to sexual identity questions tends to increase with age
(Gruskin et al., 2001). The NHIS found that there were significantly more adults ages 65 and older
Page 17
in the shortest and longest response time groups (based on quintiles), suggesting possible
comprehension problems often associated with item nonresponse (Dahlhamer et al., 2014).
• Item nonresponse has been found to be higher for women compared to men (Grant and Jans,
n.d.; Gruskin et al., 2001). In some cases, it is particularly high for women who speak a language
other than English (Grant and Jans, n.d.) In contrast, Saewyc et al., (2004) found that male
adolescents and young adults had higher nonresponse than female adolescents and young adults
in a survey of students ages 12 to 20
• In general, non-Hispanic African Americans, Asians, and Hispanics had higher nonresponse rates
than non-Hispanic whites (Grant and Jans, n.d.; Gruskin et al., 2001; Jans et al., 2015; Kim and
Fredriksen-Goldsen, 2013; Saewyc et al., 2004).
• Persons with less education (less than a high school degree or no college education) were less
likely than those with at least some college education to respond to sexual identity questions
(Dahlhamer et al., 2014; Gruskin et al., 2001).
What have we learned about Sexual Orientation and Gender Identity (SOGI) questions from
research and evaluation?
• Item nonresponse rates to SOGI items are generally lower than for other survey items.
• Truman et al. (2019) found that 2.77% of National Crime Victimization Survey (NCVS) respondents
refused to answer the sexual orientation question and less than 1% refused to answer the gender
identity question, In contrast, about 25% of the respondents refused to answer the income
question.
• Breakoffs on the NCVS at the SOGI questions were also low – 0.24% happened at the sexual
orientation question; 0.10% happened at the sex at birth question; 0.04% happened at the current
gender identity question; and no respondents broke off at the gender confirmation question
(Truman et al. 2019). For contrast, about 13% of breakoffs occurred when respondents were asked
if their house was rented or owned.
What have we learned about additional sexual identities relative to Sexual Orientation (SO)
questions from research and evaluation?
• Terms used in the most commonly fielded questions on sexual identity do not exhaustively
describe respondents’ identities, primarily for youth or young adults who might be more likely to
identify with sexual identities other than “gay,” “lesbian,” “straight,” or “bisexual.”
• In several studies for both youth (Temkin et al. 2017; Steiger et al. 2017; Russell, Clarke and Cary
2009) and adults (Bulgar-Median 2017; Meyer et al. 2019), suggestions of additional sexual
identities such as “pansexual,” “demisexual,” “asexual” and “aromantic,” “queer,” “questioning,”
“same-gender loving” were provided or chosen when available on the instrument.
• Write-in responses to “something else” provide mixed results – some identify a Sexual
Orientation category, while others use it as a place to voice an objection or misunderstanding of
the question.
• Research from the Census Barriers, Attitudes and Motivators Study (CBAMS) survey found that of
the over 200 non-blank write-ins, 16 percent represented other sexual minority labels (e.g., queer,
pansexual, or asexual); the majority were write-ins expressing objections to or misunderstanding
of the question, entering answers such as “All male” or “Normal” (Bates et al. 2019).
• Some sexual minority groups, especially teens, do not like using labels for their sexual orientation.
Eliason and Streed (2017) found that persons who reported their sexual identity as “something
Page 18
else” subsequently responded that they “haven’t figured out their sexuality,” “don’t use labels,” or
“are not straight but use another label.” Similarly, Eliason et al. (2016) reported that persons
identifying as “something else” responded that they were “not straight, but identify with another
label such as queer, trisexual, omnisexual, or pansexual.”
• Research suggests the response of “something else” was typically selected by English-speaking
respondents when they did not want to disclose their identity or when the existing categories did
not reflect their identity. Spanish-speaking respondents seemed to select “something else” due to
confusion about the terminology. (Truman et al. 2019).
• Findings for cisgender adults (Klein et al. 1985; Diamond 2003) and teens (Russell, Clarke and Clary
2009; Galupo, Henise, and Mercer 2016) show that sexual orientation is best characterized as a
spectrum or continuum that can change over time, but the terminology typically used does not
treat it as such.
What have we learned about fielding Gender Identity (GI) questions from research and evaluation?
• Two-step questions (i.e., ask sex at birth followed by current gender identity) are recommended
by the Gender Identity in the U.S. Surveillance (GenIUSS) Group, the Center of Excellence for
Transgender Health (CoE), and the World Professional Association for Transgender Health (WPATH;
Deutsch et al., 2013). Furthermore, a third (“confirmation”) item is needed so that respondents
may verify their answers to the two items and be given a chance to revise them if something was
incorrect. In a recent NCVS, 10% of the 152 respondents who received Q3 indicated a response was
not correct, 5% indicated they were confused, and 2% indicated they did not know the answer.
• Single-item measures compel respondents who identify as both “male” or “female” and
“transgender” to decide between both identities. Respondents who do not identify as transgender,
but whose gender identities are incongruent with their natal sex will not be included in the
estimate of the transgender population (resulting in an undercount).
• Most respondents have no difficulty responding to the sex at birth and gender questions, though
some cisgender respondents found the questions redundant (Cahill et al. 2014; Glaze 2015;
Lombardi and Banik 2016; Reisner et al. 2014).
Current survey measures and terminology for Gender Identity (GI) do not work well for all
transgender individuals.
•
•
Some gender minority groups do not see “transgender” as an identity distinct from “male” or
“female;” others prefer “genderqueer” to “female” or “male.” Some respondents want to be able
to mark “all that apply” when asked “Are you male, female or transgender” and suggested adding
responses such as “gender non-binary,” “trans-man,” “trans-woman,” and “something else” (Ellis
et al. 2018).
Members of the transgender population indicated that they would use “transgender” as an
umbrella term to describe members of a diverse community, even if it was not personally their
first choice of self-identification (Holzberg et al. 2017). These respondents self-identified in many
ways, including “man, woman, transgender, queer, gender-fluid, non-binary, and genderqueer.”
Some indicated that their self-identification had changed over time or that it might change in the
future, a process one respondent described as “‘fine tuning” their own self-description. Finally,
some respondents explicitly said that they thought it would be difficult for researchers to create
questions with adequate response options, given the diversity of terms used and debate within
the transgender community about terminology.
Page 19
Translation challenges associated with the Sexual Orientation (SO) and Gender Identity (GI)
questions.
• Research to date has illustrated the difficulty translating SOGI terms in a way that conveys the
intended meaning in a culturally appropriate way as well as a lack of terminology that conveys
concepts in some languages. As an example, research has found there is no comparable word for
“straight” in Spanish (Miller and Ryan 2011) and evidence of comprehension problems among
Spanish-speaking respondents (Michaels et al. 2016a, Michaels et al. 2016b). Research has also
shown that nonresponse rates to SOGI questions are higher among respondents to surveys
conducted in Spanish and other languages (Jans et al. 2015, Miller and Ryan 2011, Ridolfo et al.
2012).
• Translation remains a challenge for the gender identity items. Studies show that English-speaking
respondents have no problems comprehending the items, although a few conflated sexual identity
and gender identity. A majority of Spanish-speaking participants conflated sexual identity and
gender identity, and some expressed discomfort in being asked about their gender identity. In
addition, a number of Spanish-speaking cisgender participants expressed that their gender identity
was “normal” or “non-deviant.” Ultimately, both English and Spanish speakers were able to
respond to the questions in a manner consistent with screener questions, indicating that the
conflation of concepts did not necessarily result in measurement issues for this study (Stern et al.
2016).
Page 20
| File Type | application/pdf |
| File Modified | 2021-05-25 |
| File Created | 2021-05-25 |