OMB Memo

Generic Information Collection Request:
Cognitive Testing of Sexual Orientation and Gender Identity (SOGI) Questions on the ­­American Community Survey (ACS)

Request: The Census Bureau, through a contract with RTI, plans to conduct additional research under the generic clearance for questionnaire pretesting research (OMB number 0607-0725). This research will cognitively pretest sexual orientation and gender identity (SOGI) Questions on the American Community Survey (ACS).

Purpose: As part of its fiscal year 2024 budget, the Census Bureau received funding to test adding SOGI questions to the ACS. The Census Bureau also received a request from the Department of Justice to add SOGI questions to the survey. Testing of SOGI questions will follow the ACS content change process, which includes pretesting and field testing. This request is for cognitive pretesting of SOGI questions for the ACS, with an emphasis on three understudied areas:

1. Wording of questions for stateside Spanish-language and Spanish Puerto Rico questionnaires. While SOGI questions have been pretested in Spanish for other surveys, there is still room for improvement in Spanish translation. One major issue is that monolingual Spanish-speaking respondents are often unfamiliar with the translations for response categories and as a result may report that they are “normal.” This problem is compounded because terms like “straight” in English are difficult to translate in a neutral and accessible manner in Spanish. We are unaware of any previous efforts by other federal agencies to pretest SOGI questions in Puerto Rico.

2. Multi-mode measurement. Respondents are invited to complete the ACS online (stateside only) or via a paper questionnaire. If they do not respond, they may be contacted by a field interviewer in person or over the phone. Most federal pretesting of SOGI questions to date has focused on a single mode (typically in person data collection). It is important to ensure SOGI questions lead to comparable data across all modes of the ACS.

3. Proxy response. Other than testing for the American Housing Survey (AHS), no other federal survey to date has asked respondents to report SOGI information for other people in their household (“proxy report”). Cognitive testing of proxy response has also been limited and conducted only in English.

Population of Interest: The ACS is sent to a nationally representative sample of households in the United States. We plan to recruit from this general population as described further below.

Timeline: Testing will be conducted from December 2023 through June 2024.

Language: Testing will be conducted in English and Spanish. Spanish will be tested both stateside and in Puerto Rico.

Sample: We plan to conduct 218 cognitive interviews. Of the 218 interviews, 64 will be in English and 154 will be in Spanish. Eighty of the 154 will be with stateside monolingual Spanish-speaking respondents. The remaining 74 will be with Spanish-speaking respondents in Puerto Rico. About half of the total respondents will be cisgender and straight. The other half of respondents will be a sexual and/or gender minority (i.e., LGBT).

Recruitment: Respondents will be recruited through a variety of methods. RTI will use methods such as posting fliers at local community organizations, putting advertisements in community newspapers, and collaborating with staff at community-based organizations to identify potential respondents. RTI will also post advertisements on Craigslist.com and social media sites such as Facebook and Reddit. Finally, broadcast messages will be distributed to Census Bureau and RTI staff to leverage personal connections.

Recruitment will primarily focus on identifying respondents who are LGBT and those who are not. For the LGBT respondents, we will recruit people with diverse types of identities. Most (80%) of the total respondents will live in a household with at least one other person since proxy response is a key research area. We will also aim for demographic diversity on the following characteristics:

• Age and education (all three interview types)

• Race/ethnicity (English interviews only)

• Hispanic origin (Spanish stateside interviews only)

• Geographic diversity in region of the country (all three interview types)

• Household composition (including diversity in types of both related and unrelated households; all three interview types)

The screening questions and advertising materials are attached (see Attachments 1 and 2).

Method: Staff from RTI will conduct cognitive interviews. Interviews will be conducted remotely using Microsoft Teams. In addition to testing in both English and Spanish, cognitive interviews will vary in two ways. First, respondents will be assigned to one of four modes. We will test the paper questionnaire, internet instrument¹, and two computer-assisted interview (CAI) versions of the questions. The purpose of including two CAI modes is to test both in-person interviewing (with flashcards) and telephone interviewing (without flashcards). Second, respondents will be assigned to receive either Treatment 1 or Treatment 2 of the SOGI questions. The treatments differ in the second step of the gender identity questions, with Treatment 1 asking respondents to select one answer and Treatment 2 asking respondents to mark all that apply (see Attachment 3). Attachment 4 provides more context on the question wording.

Protocol: During the interviews, respondents will first be asked to complete a shortened version of the ACS as they would if they received it at home or an interviewer had contacted them at home (see Attachment 5 for a rough mock-up of the paper questionnaire). Researchers will observe respondents’ interactions with the survey. Afterwards, they will be debriefed in several different ways (see Attachment 6). They will be asked generic probing questions and complete a card sort activity to learn more about the perceived difficulty of the questions compared to other ACS items. They will then be probed about their answers to the SOGI questions. Probes include asking how they came up with their answers and what certain terms and phrases mean to them. The protocol has a particular emphasis on probing about self versus proxy responses.

We will also specifically probe on two other questions. We will ask some probes on the relationship question since it was edited from the production ACS question to use gender-neutral terminology. At the end of the interview, we will ask respondents to answer a question about intersex status and probe them about their responses to this question. While the Census Bureau does not intend to add an intersex question to the ACS at this time, these results can help inform future efforts for the ACS and other federal surveys given the dearth of research in this area. The interviews will conclude by asking respondents general debriefing probes about their experience.

Consent: We will inform respondents that their response is voluntary and that the information they provide is confidential and will be accessed only by employees involved in the research project. The consent form will also indicate that the respondent agrees that the interview can be audio and/or videotaped to facilitate analysis of the results (see Attachment 7). Verbal consent will be captured on recordings. Respondents who do not consent to be video and/or audio-taped will still be allowed to participate.

Incentive: Respondents will receive $40.

Length of Interview: We estimate that each of the 218 interviews will take approximately one hour. This results in a burden of 218 hours.

The screening questions specific to this research will take approximately 5 minutes per person. We estimate that we will screen 2 people for each successful recruit for each of the 218 interviews. Therefore, for the first round of interviewing, we estimate a total of 436 people screened for a total of 37 hours (436 people at 5 minutes each).

Thus, the total estimated burden for the first round of this research is 255 hours.

Table 1. Total Estimated Burden

Category of Respondent	No. of Respondents	Participation Time	Burden
Screening	436	5 minutes	37 hours
Cognitive Interviews	218	60 minutes	218 hours
Totals			255 hours

Below is a list of materials to be used in the current study:

Attachment 1. Screening questionnaire

Attachment 2. Recruitment methodology and ads

Attachment 3. Treatments 1 and 2 SOGI questions by mode

Attachment 4. Background on SOGI wording

Attachment 5. Mock questionnaire

Attachment 6. Protocol

Attachment 7. Consent form

Contact: The contact person for questions regarding data collection and statistical aspects of the design of this research is listed below:

Jessica Holzberg

Center for Behavioral Science Methods

U.S. Census Bureau

Washington, D.C. 20233

301-763-2298

[email protected]

1 Internet will not be tested for Puerto Rico since it is not currently offered as a response mode in Puerto Rico for production.

3