2017 BRFSS SOGI Module Overview and Analysis Plan

2017 BRFSS Sexual Orientation and Gender Identity (SOGI) Module

Overview and Analysis Plan to Accompany OMB package (Control Number 0920-1061)

This overview is being provided to OMB and ASPE consistent with the terms of clearance issued by OMB for control number 0920-1061. According to those terms, by October 1, 2016, the BRFSS will provide a plan “for generating the empirical data necessary to a) calibrate the BRFSS 2017 state-level sexual identity prevalence estimates with respect to the NHIS results and b) inform BRFSS’ 2018 question development process. With respect to the latter, that would include cognitive work, preferably in conjunction with ASPE, NCHS, and the CMS Office of Minority Health on comparing the two versions of the gender identify questions featured in the GenIUSS Group report (the recommended and the promising) with respect to the version that BRFSS has been using.” The plan outlined below anticipates that during the next few months, the BRFSS will work to collect information on the questions that are used by other surveys, recommended by research and used by the current administration of the BRFSS questionnaire. Since the current BRFSS OMB approval expires in March 2018, and the 2017 questionnaire is already finalized, new question wording (if any) will be instituted on the 2018 questionnaire and included for OMB approval in midyear 2017. It is important to note that the SOGI questions are funded by ASPE, and inclusion of the SOGI optional module on future administrations of the BRFSS is contingent on that funding continuing.

Background

The Behavioral Risk Factor Surveillance System (BRFSS) is a state-based system of health-related telephone surveys that collect state data about U.S. residents regarding their health-related risk behaviors, chronic health conditions, and use of preventive services. By collecting behavioral health risk data at the state and local level, BRFSS has become a powerful tool for targeting and building health promotion activities. As a result, BRFSS users have increasingly demanded more data and asked for more questions on the survey. Currently, there is a wide sponsorship of the BRFSS survey, including most divisions in the CDC National Center for Chronic Disease Prevention and Health Promotion; other CDC centers; and federal agencies, such as the Health Resources and Services Administration, Administration on Aging, Department of Veterans Affairs, and the Substance Abuse and Mental Health Services Administration. The Population Health Surveillance Branch (PHSB) of the Centers for Disease Control and Prevention (CDC) provides technical assistance to the states and coordinates data collection processes.

The structure of the BRFSS questionnaire includes a set of core questions asked of all respondents (including some rotating core questions which appear in even- and odd-numbered years), and optional modules from which states may select based on specific informational needs. Optional modules cover a variety of topics from chronic disease management to social determinants of health.

Beginning in 2014, states were able to select an optional module that included questions on Sexual Orientation and Gender Identity (attached). The module was proposed by the Office of the Assistant Secretary for Planning and Evaluation (ASPE) of the US Department of Health and Human Services. It was presented to the states by representatives of ASPE at the BRFSS questionnaire sessions in 2013. The module appeared on the ballot of suggested optional modules and was voted on by the states in that year. Questions were included on the field test and subsequently have been made available for state use. In 2014, nineteen states opted to participate in the Sexual Orientation and Gender Identity (SOGI) module. By 2016, the number of states adopting this module has increased to twenty-five, a list of participating states by year is presented below.

States Participating in BRFSS/ASPE SOGI Module by Year (2014-2016)
2014	2015	2016
		California
	Colorado
	Connecticut	Connecticut
Delaware	Delaware	Delaware
	Georgia	Georgia
Hawaii	Hawaii	Hawaii
Idaho	Idaho	Idaho
	Illinois	Illinois
Indiana	Indiana	Indiana
Iowa	Iowa	Iowa
Kansas	Kansas
Kentucky		Kentucky
Louisiana		Louisiana
Maryland	Maryland
	Massachusetts	Massachusetts
Minnesota	Minnesota	Minnesota
		Mississippi
	Missouri	Missouri
Montana
Nevada	Nevada	Nevada
New York	New York	New York
Ohio	Ohio	Ohio
Pennsylvania	Pennsylvania	Pennsylvania
		Rhode Island
	Texas	Texas
Vermont		Vermont
Virginia	Virginia	Virginia
		Washington
	West Virginia
Wisconsin	Wisconsin	Wisconsin
Wyoming

SOGI Question Testing

Question wording of the SOGI module was mandated by ASPE Unlike other modules, where states vote on question wording, no changes were permitted to be made to the questions by the states or by the BRFSS staff at CDC. ASPE also funds the administration of the module. A request to consider changing the questions was made by PHSB to ASPE in 2015. The PHSB recommended removal of the language “consider yourself to be” and replacing it with language asking respondents “Are you….” After consideration of the suggestion, ASPE mandated that the BRFSS continue with the original wording in order to be consistent with earlier administrations of the survey.

Feedback from fielding the SOGI questions has been provided by the states to PHSB over the course of the past two years. In addition, the PHSB has monitored the fielding of the questions, as is standard practice for all BRFSS core sections and optional modules. The following is a brief listing of outcomes from that oversight:

1. SOGI questions do not result in a large percentage of refusals. In 2015, 1.7 percent of all respondents asked the questions refused to answer. However this is a larger percentage than most other demographic questions on the BRFSS (as an example, only .7% refused to answer the marital status question).

2. SOGI questions take longer to administer than some other modules, perhaps a result of interviewer providing information on their inclusion, or reading instructions to the respondents. The SOGI module takes approximately 27 seconds to administer 2 questions (an average of 11 seconds). During the 2017 field test the other demographic questions took an average of 7 seconds each.

3. Some respondents have pushed back on the language that includes the phrase “do you consider yourself to be…” These respondents have said that this language suggests that their response is based on something that they consider rather than a factual statement of their gender identity or sexual orientation. This feedback was provided to the PHSB in a debriefing of veteran interviewers in late 2015 and was noted during the 2015 field test by one of the interviewers during debriefing.

4. Older respondents are frequently more confused by the questions than other respondents.

5. There have been no reports of systematic negative reactions to the questions. However, though the break-off rate (termination of the interview by the respondents) of the BRFSS is very low, anecdotally states report that the SOGI questions are more likely to result in a respondent break off than questions on other optional modules. As a result, some states prefer to offer the module late in the questionnaire. We do not have metrics on this issue because states do not all place the module at the same point.

SOGI Question Wording on BRFSS and other Surveys

A crosswalk of SOGI questions that appear on other surveys and the BRFSS questions is provided at the end of this document.

With respect to how the respondent “consider(s)” or “thinks” of themselves, the SOGI questions are worded similarly to those used by other surveys on the question of sexual orientation including those conducted by the National Health Interview Survey (NHIS), the National Survey of Drug Use and Health (NSDUH), the Health Information National Trends Survey (HINTS), the National Adult Tobacco Survey (NATS) and the National Crime Victimization Survey (NCVS).

At present, the exact gender identity questions included in the BRFSS SOGI module are not asked on other national surveys, this represents an opportunity to obtain new information that is not available from other sources yet identified. The BRFSS SOGI questions adhere to the recommended measures for single-item transgender status and the recommended measure for LGBT identity suggested by the Report “Best Practices for Asking Questions to Identify Transgender and Other Gender Minority Respondents on Population-Based Surveys” which was published by the Williams Institute in 2014. These questions are sometimes referred to as the GenIUSS (Gender Identity in US Surveys) Group questions. Specifically the BRFSS questions are virtually identical to the secondary measures listed on page iv of the Executive Summary. The only difference is that the BRFSS item excludes from the introduction that explains, in layman’s terms, the definition of transgender; BRFSS interviewers read this text only if the respondent is confused or asks for meaning. These questions are provided in the Report as secondary measures to be used when it is not feasible to change the question on sex to ask about assigned sex at birth and current gender identity in two separate questions. As was noted earlier, the BRFSS uses questions that were provided by ASPE and could not change the question on sex.

The National Crime Victimization Survey (NCVS) is currently using questions based on the GenIUSS Group questions. The NCVS sexual orientation question (downloaded from their OMB package documentation) is the same as the BRFSS. The gender identity question on the NCVS is three-part and (according to information on the questionnaire provided online) includes skip patterns not associated with the BRFSS questions. Therefore persons who indicate that they are “straight” on the first question are not asked the second question. This is not true on the BRFSS. There are no skip patterns; all respondents are asked both questions. We believe it is appropriate to ask the second question of all persons, as a small number of individuals responding to the BRFSS (n< 50) respond that they are straight on the first question and then indicate that they are transgender on the second question.

The BRFSS is currently collecting information on the appropriate wording of the SOGI questions and will collaborate with ASPE and other survey methodologist on the most efficacious question wording. At this time the following efforts are underway or planned to review question wording:

1. The BRFSS is currently in the process of scheduling a focus group of nine people from the general public who will review the current BRFSS SOGI module questions, and the GenIUSS Group questions (both recommended and stand-alone). A report on the focus group will be available before January 2017. BRFSS will collaborate with those who recently completed testing in conjunction with the NCVS to maximize the utility of the BRFSS work without duplication of effort.

2. The BRFSS has contacted the CDC Association for Gay, Lesbian, Bisexual and Transgender Employees (GLOBE), about their input into the questions. The membership of the organization is large enough (n~ 300) for there to be a wide range of input on the questions. Details of the methods of collecting feedback (from a survey, discussion groups or other means), is not yet determined, but the GLOBE’s Board has approved the project. If we are able to go forward with the project using information from the membership of the GLOBE community, and a suitable generic ICR mechanism is identified, a request for data collection will be submitted when details are available. BRFSS will also consult with EPA as OMB has advised us that EPA is conducting a similar effort.

3. BRFSS will continue to report to ASPE on all aspects of question administration and will receive instruction on question wording from that group. Members of OMB-lead interagency working group on SOGI questions include methodologists from other surveys currently fielding SOGI questions.

4. Within the next two months, the BRFSS also will be contacting other agencies (including BLS and DOJ) which include SOGI questions to engage in information exchange regarding the administration and wording of SOGI questions. This contact will follow the currently scheduled BRFSS SOGI focus group.

As information from the scheduled focus group, and feedback from GLOBE become available, the BRFSS will share information and with the OMB Interagency Working Group. BRFSS will also continue to receive information from the Working Group and from other agencies that are including similar questions. Since the current BRFSS OMB approval expires in March 2018, it is anticipated that any changes in the wording of SOGI questions will be included in the request for renewal. This would place the new question wording (if any) on the 2018 questionnaire. The timeline below indicates the schedule for this activity.

Analysis Plan for Validity and Reliability of the BRFSS SOGI Module

Since the BRFSS SOGI module has the same or similar language for the sexual identity question as other Federal surveys, comparisons of population estimates are of interest. Some of these estimates are already available in the August 2016 SOGI IWG report, entitled “Current Measures of Sexual Orientation and Gender Identity in Federal Surveys.” Data might be presented from each survey as suggested by the table shell below:

Table Shell Of Validity Comparison For Sexual Identity Question
Sexual Orientation Categories	BRFSS	NHIS	NSDUH	HINTS-FDA	NATS	NCVS
Straight	#/% (CI)	#/% (CI)	#/% (CI)	#/% (CI)	#/% (CI)	#/% (CI)
Lesbian or gay	#/% (CI)	#/% (CI)	#/% (CI)	#/% (CI)	#/% (CI)	#/% (CI)
Bisexual	#/% (CI)	#/% (CI)	#/% (CI)	#/% (CI)	#/% (CI)	#/% (CI)
Other	#/% (CI)	#/% (CI)	#/% (CI)	#/% (CI)	#/% (CI)	#/% (CI)
DKN	#/% (CI)	#/% (CI)	#/% (CI)	#/% (CI)	#/% (CI)	#/% (CI)
Refused	#/% (CI)	#/% (CI)	#/% (CI)	#/% (CI)	#/% (CI)	#/% (CI)

Data comparison from the BRFSS SOGI module is complicated by the samples of the surveys. As was noted earlier, the BRFSS SOGI module has been administered by a maximum of 25 states in any given year. Other surveys included in the comparison have national samples. Therefore a number of approaches may be undertaken to statistically model national estimates of the sexual orientation question from available data. These may include:

1. Modeling national estimates based on demographic characteristics of individual respondents.

2. Modeling regional estimates based on demographic characteristics of individual respondents, including region as a random effect in the model, to account for state cultural differences by region of the country. This approach is dependent upon available population control totals at the regional level.

3. Imputation of individual responses in states where questions are not asked and calculation of prevalence estimates based on actual and imputed data.

4. State-to-state comparisons of estimates, where state-level data are available from other surveys, or where modeled state estimates can be derived from other surveys. This approach is dependent on the number of states which have sufficient sample sizes in other surveys for direct estimation.

In addition, as is standard practice for all BRFSS questions, year-to-year comparisons are being made, especially among those states which have administered the SOGI module consistently from 2014-2016. Wide variation in prevalence estimates from one year to the next would suggest that the questions are not reliable.

As the BRFSS moves toward the 2018 questionnaire, potential changes in the questions can be determined in part by the comparisons suggested in the table above as well as by continued funding of the SOGI module. Question wording changes will be with the approval of ASPE.

Timelines for the analyses presented above are anticipated to be relatively short. Year-to-year comparisons are currently underway and presentations on the state-level reliability of estimates could be produced within 6 months. Comparison of the sexual identity question results using other surveys may take as long as one year. A brief timeline is presented below:

Brief Timeline of SOGI Analyses

2016

2017

Item

Oct

Nov

Dec

Jan

Feb

Mar

Apr

May

Jun

July

Aug

Sept

Oct

SOGI focus group

Imputation of responses using 2014-2015 data

Statistical modeling for nationwide estimate

Consultation with the CDC GLOBE Community

Consultation with other agencies (i.e. EPA, BLS, DOD)

Reporting

Consultation with ASPE

Attending the Working Group Meetings and engaging in information exchange

State-to-state comparisons

Year-to-year comparisons

Development of BRFSS OMB package for 2018-2021

Finalization of questions for BRFSS 2018 (including SOGI) questions

Crosswalk of Sexual Orientation and Gender Identity Questions by Survey
Survey	BRFSS 2016	2016 NHIS	2016 NHANES	2016 NSDUH	HINTS-FDA (2015)	2014 NATS	YRBSS 2017	HRS 2016	2016 NCVS
Sexual Orientation Question Wording	Do you consider yourself to be:	Which of the following best represents how you think of yourself?	Which of the following best represents how you think of yourself?	Which one of the following do you consider yourself to be?	Do you think of yourself as…	Do you think of yourself as…	Which of the following best describes you?	Do you consider yourself to be gay, straight, bisexual or something else?	Which of the following best represents how you think of yourself?
Response Options	1 = Straight 2 = Lesbian or gay 3 = Bisexual 4 = Other 7 = Don't know / Not sure 9 = Refused	[Lesbian or] gay Straight, that is, not [lesbian or] gay Bisexual Something else I do not know the answer Refused	[Lesbian or] gay Straight, that is, not [lesbian or] gay Bisexual Something else I do not know the answer Refused	1 = Heterosexual, that is, straight 2 = [IF QD01=9 THEN "Lesbian or] Gay 3 = Bisexual DK/REF	1 = Heterosexual, or straight 2 = Homosexual, or gay or lesbian 3 = Bisexual 91 = Something else - Specify __________	1 = Lesbian or Gay 2 = Straight, that is not lesbian or gay 3 = Bisexual 4 = Something else 6 = Respondent does not understand responses -8 = Don't know/Not sure -7 = Refused -9 = Not ascertained	A = Heterosexual (straight) B = Gay or lesbian C = Bisexual D = Not sure	FOR MEN: 1 = Gay 2 = Straight, that is, not gay 3 = Bisexual 4 = Something else FOR WOMEN: 1 = Lesbian or Gay 2 = Straight, that is, not gay 3 = Bisexual 4 = Something else	[Lesbian or] gay Straight, that is, not [lesbian or] gay Bisexual Something else I do not know the answer Refused
Follow-up to Sexual Orientation Question
Gender Identity Question Wording	Do you consider yourself to be transgender?					What sex were you at birth?			What sex were you assigned at birth, on your original birth certificate?
Response Options	1 = Yes, Transgender, male-to-female 2 = Yes, Transgender, female to male 3 = Yes, Transgender, gender nonconforming 4 = No 7 = Don't know / Not sure 9 = Refused					Male Female
									Male Female (Refused) (Don’t know)
						Do you currently consider yourself to be:			Do you currently describe yourself as male, female or transgender?
						Male Female DK/Not sure Refused			Male Female Transgender None of these
									[If sex assigned at birth does not match current gender identity] Just to confirm, you were assigned {FILL} at birth and now describe yourself as {FILL}. Is that correct?
									Yes No (Refused) (Don’t know)