Project Description and Justification (10-day letter)

DEPARTMENT OF HEALTH & HUMAN SERVICES Public Health Service

Centers for Disease Control and Prevention

National Center for Health Statistics

3311 Toledo Road

Hyattsville, Maryland 20782

November 18, 2021

Margo Schwab, Ph.D.

Office of Management and Budget

725 17th Street, N.W.

Washington, DC 20503

Dear Dr. Schwab:

The staff of the NCHS Collaborating Center for Questionnaire Design and Evaluation Research (CCQDER) (OMB No. 0920-0222, Exp. Date 09/30/2024) plans to conduct a cognitive interviewing study to examine the US passport application form with an ‘X’ gender marker.

NCHS is recognized for expertise in cognitive testing and will conduct the study on behalf of the U.S. Department of State, which is currently updating forms DS-11, DS-82, and DS-5504 to add a gender X marker in a manner compliant with the International Civil Aviation Organization (ICAO) standards to provide a gender option for non-binary, intersex and gender non-conforming applicants. This process will continue the update of Department policies regarding gender markers on U.S. passports and Consular Reports of Birth Abroad (CRBAs) to better serve all U.S. citizens, regardless of their gender identity. Canada, Australia, New Zealand, and several other countries currently offer a gender X option on passports. Additionally, 21 U.S. states and the District of Columbia also offer the category on driver’s licenses and birth certificates. However, these countries and states have varying definitions of what constitutes the gender X, and it is not known how passport applicants interpret and utilize the category as they complete the form. Due to the collaborative and exploratory nature of the project, the questions to be tested and the proposed screening questions reflect departures from current HHS best practices. These departures are inherent to the subject matter to be tested.

Findings may also be of direct interest to NCHS and other units of HHS.

We propose to start recruiting for volunteer participants as soon as we receive clearance and to begin interviews as soon as possible after that.

The methodological aim of this study is consistent with the design of most NCHS/CCQDER cognitive interviewing studies, that is, to understand the construct captured by each question, identify patterns of interpretation across respondent groups, and explore potential sources of response error. Interviews will be in-depth and semi-structured; analysis will be conducted using qualitative methodologies.

Like other CCQDER studies, this study is consistent with the socio-cultural approach to question evaluation as described by Miller and Willis (2016) and is specifically designed to address comparability. The design pertains to each stage of the research process, including, data collection, analysis and documentation, and is fully detailed in Miller et al’s book, Cognitive Interviewing Methodology: An Interpretive Approach for Survey Question Evaluation. For data collection, a purposive sample design, based on the constant comparative method, is implemented, and interviews are defined by narrativity and the collection of ‘respondent experience’ as opposed to structured probe-questions designed to identify ‘question problems.’ Furthermore, the principle of reflexivity is invoked for both interviewing procedures and analysis. Specifically, for analysis, a 5-stage process is utilized in which interpretive patterns are first identified across respondents and ultimately compared across relevant respondent groups, with a particular focus on race and ethnicity, education, income, and gender. Final conclusions are publicly documented and reveal the phenomena (including unintended constructs) captured by each survey question, variations across respondent groups, and explanations for those variations.

Proposed project: US Passport Application Form with Gender X Marker

Overview of procedure for cognitive testing

Cognitive interviews, remuneration experiment, and follow-up interviews

Cognitive interviews for this project will be one-on-one between a single interviewer and a respondent and will be no longer than 60 minutes. The Passport Application Form to be tested is shown in Attachment 1. Interviews will be conducted by CCQDER staff and contractors from Research Support Services. Given the social distancing requirements of COVID-19, interviews will take place via video conference through Skype or Zoom. These platforms have been approved by CDC and NCHS Information Security Offices. In the event that interviews cannot be conducted virtually, NCHS staff propose holding them in person in the Questionnaire Design Research Laboratory or off-site when COVID-19 restrictions allow.

Screening and scheduling procedure

The first contact with potential respondents will occur in response to the flyers or advertisements, as shown in Attachment 2. Interested persons will leave contact information (name and telephone number) on the CCQDER voice mail system or in an email. A CCQDER Recruiter will then call the potential respondent and give a brief description of the nature of the study, video conference and recording procedures, and monetary incentive. The CCQDER Recruiter will determine through a brief series of questions whether the potential respondent possesses the desired research characteristics (e.g., we ask for gender and age to avoid interviewing people with very similar demographic characteristics). The 5-minute screener used to determine eligibility of individuals responding to the newspaper advertisements/flyers is shown in Attachment 3. Note that wording of the templates has been approved and are contained within our umbrella package. Only project specific information has been added to the documents. It is anticipated that as many as 150 individuals may need to be screened in order to recruit 100 participants.

If the person does possess the desired research characteristics and would like to participate, he/she will be scheduled for an interview. Otherwise, the volunteer will be asked whether he/she would be interested in participating in future laboratory interviews. Telephone numbers and the minimal demographic information listed earlier will be obtained for all scheduled volunteers and for those who would like to be contacted in the future. For those callers who are ineligible for the study and do not want to be contacted in the future, only demographic characteristics will be maintained for future analysis of successful recruitment efforts.

Interview methodology

All interviews will be video and audio recorded to allow researchers to review the behaviors and body language of the respondents. These recordings will also allow researchers to ensure the quality of their interview notes. Recordings will be used in accordance with the CCQDER Data Storage and Access Policy (Attachment 6). Researchers from CCQDER and Research Support Services who are working on the project will use the recordings for research purposes. Recordings will remain under CCQDER staff control.

For in-person interviews, after respondents have been briefed on the purpose of the study and procedures that CCQDER routinely takes to protect human subjects, respondents will be asked to read and sign an Informed Consent document (Attachment 4). Only project specific information has been added to the document. Respondents will also be asked to fill in their demographic characteristics on the Respondent Data Collection Sheet (Attachment 5). This document is also contained in our umbrella package.

For virtual interviews, a member of the CCQDER Operations Team will begin each interview with introduction text explaining the general purpose of the survey, providing the confidentiality and Paperwork Reduction Act language, and informing the respondent that their participation is voluntary and that they may refuse to answer any question. Only project specific information has been added to the document. The Operations Team member will collect basic demographic information via the Respondent Information Sheet. This document is also contained in our umbrella package. Finally, the Operations Team member will inform the respondent about the need to video record the interview.

At this point, the interviewer will join the video conference and brief the respondent on the purpose of the study and the procedures that CCQDER routinely takes to protect human subjects. This includes strong measures to ensure that private information is not disclosed in a breach of confidentiality and storage of confidential data is protected through procedures such as an internal CCQDER LAN, passwords, and restricted access. The respondent will be asked to vocally affirm their consent to being interviewed. Because many interviews will not take place in person, and it will not be possible for respondents to read and sign the usual informed consent document, a waiver of signed informed consent has been requested from NCHS’ ERB. If a respondent does not end the call, consent will be assumed. In the rare instance that consent to record the interview is not granted, the session will not be recorded. In this case the interviewer will depend on their handwritten notes when conducting analysis. If the respondent grants consent to record the interview but changes his/her mind while the session is being recorded, the interviewer will ask for verbal consent to retain the interviewing materials and the portion already recorded. The interviewer will get verbal consent from the respondent to do so prior to turning off the recording software. If the respondent does not give consent for the CCQDER to retain the recording it will be deleted from all locations. Once deleted, the file is no longer available for use. A note will be placed in the hardcopy file and the CCQDER database indicating that particular recording (identified by the unique identification number assigned to the respondent) has been destroyed.

NCHS government issued encrypted laptops will be used to video and audio record the interviews conducted virtually. CCQDER staff and Research Support Services will also use the NCHS government issued encrypted laptops to input their interviewer notes into Q-Notes, the CCQDER data entry and analysis tool. Within 24 hours, a CCQDER staff member will review the interview notes and will delete any direct or indirect personal identifiable information (PII) if found.

Extreme care will be taken with all recordings from the interviews conducted virtually. Once the video and audio recordings are transferred to the QDRL Network, the recordings will be deleted from interviewers’ devices. Once deleted, the files are no longer available for use.

Interviews will be conducted using retrospective probing, whereby respondents are presented survey questions first, then asked to explain how and why they answered as they did. The interviewer will use probes extensively to ascertain the degree of comprehension and the recall processes involved. The interviewer may also ask the respondent to think aloud while answering.

Within 5 business days of the conclusion of virtual interviews, the respondent will receive their monetary incentive via FedEx mail, information explaining the terms of consent and contact information for the CCQDER Laboratory Manager, the NCHS ERB Chair, and the NCHS Confidentiality Officer.

As indicated above, we propose giving participants $50 incentives, the standard pre-approved amount for cognitive interviewing studies. In total, for this project, the maximum respondent burden will be 129 hours. A burden table for this project is shown below:

Form Name	Number of Participants	Number of Responses/ Participant	Average hours per response	Response Burden (in hours)
Screener (Attachment 3)	150	1	5/60	13
Questionnaire (Attachment 1)	100	1	55/60	92
Virtual Interview Zoom Set-up/Support	100	1	5/60	8
Respondent Data Collection Sheet (Attachment 5)	100	1	5/60	8
Informed Consent (Attachment 4)	100	1	5/60	8
Total	129

Attachments (6)

cc:

S. King

J. Zirger

DHHS RCO

4 | Page