Letter of request

DEPARTMENT OF HEALTH & HUMAN SERVICES Public Health Service

Centers for Disease Control and Prevention

National Center for Health Statistics

3311 Toledo Road

Hyattsville, Maryland 20782

September 30, 2020

Margo Schwab, Ph.D.

Office of Management and Budget

725 17th Street, N.W.

Washington, DC 20503

Dear Dr. Schwab:

This 10-day letter is requesting approval for the staff of the NCHS Collaborating Center for Questionnaire Design and Evaluation Research (CCQDER) (OMB No. 0920-0222, Exp. Date 08/31/2021) to evaluate questions on occupational health for the National Institute for Occupational Safety and Health (NIOSH), Division of Surveillance, Hazard Evaluations, and Field Studies (DSHEFS). This package is directly related to a previously approved protocol (ICR Reference No. 201807-0920-004) that received OMB approval in February 2020. Unfortunately, the cognitive interviews that were scheduled in March 2020 had to be canceled to comply with the Centers for Disease Control and Prevention’s (CDC) recommendations of social distancing in response to COVID-19. In order to include cognitive interviews via video conference in this protocol, changes needed to be made to nearly every form; therefore, this is package is being submitted as a new OMB package. The intent of the study (i.e. background information and questionnaire to be cognitively tested) is consistent with the previously approved protocol.

Attachment 1 contains NIOSH’s full response to the questions raised by OMB across multiple discussions related to the prior 10-day letters submitted on December 6, 2018 and February 24, 2020 (ICR Reference No. 201807-0920-004).

We propose to start recruiting for volunteer participants as soon as we receive clearance and to begin video conference interviews as soon as possible after that.

Proposed project: 2021 NIOSH Occupational Health - Virtual Cognitive Interview Study

Cognitive Interviewing Methodology

The methodological design of this proposed study is consistent with the design of most NCHS/CCQDER cognitive interviewing studies: the purpose is to identify the various patterns of interpretation that respondents consider when formulating an answer to a survey question as well as any problems experienced. Findings reveal the construct captured by each question, patterns of interpretation across respondent groups, and potential sources of response error. This project will contain iterative rounds of cognitive testing (not to exceed the requested burden detailed in the burden table on page 7 of this letter) to allow questions to be revised and re-tested; thus, creating a full set of validated questions. Interviews are in-depth and semi-structured; analysis is conducted using qualitative methodologies. Findings from all CCQDER studies are documented in a final report and made publicly accessible on a searchable website at https://wwwn.cdc.gov/QBank.

Overview of procedure

This cognitive interviewing study will evaluate a series of proposed questions for the National Health Interview Survey (NHIS) Occupational Health Supplement (OHS) is slated to be fielded in 2021. NIOSH plans to use the 2021 OHS for population-based occupational health surveillance of U.S. workers across all industries, occupations, work arrangements, and demographic subgroups. The proposed questions for the 2021 OHS build on prior NIOSH sponsored occupational health supplements to the NHIS from 2010 and 2015.

Background for the OHS: Changes in the economy have led to changes in the structure of work. The service-providing sector constitutes over 80 percent of U.S. jobs, which are now characterized by non-standard work arrangements and schedules. Healthcare and social assistance services comprise 12.2 percent of all jobs and are increasingly relying on on-call work or staffing on demand. NIOSH’s new Healthy Work Design (HWD) and Well-being research program includes a focus on research examining the relationship between work arrangements and worker safety, health, and well-being. Preliminary evidence has suggested that workers in some nonstandard arrangements are more likely to experience increased exposure to poor psychosocial work environments, safety hazards, job stress, lower pay, less access to fringe benefits (including health insurance), financial stress, and poor health-related quality of life. The list of exposures proposed to be measured (e.g. job insecurity) and associated outcomes measured by NIOSH (e.g. general health status) is provided in Attachment 1.

In developing the questions for the 2021 OHS, NIOSH’s DSHEF collaborated with a panel of subject matter experts and participated in the National Academies of Sciences Meeting in June 2019 which was led by the Contingent Work and Alternate Work Arrangements Committee. The 2021 OHS questions will concentrate on characteristics of healthy work design, focusing on detailed characterization of work schedules, work arrangements, and workplace psychosocial exposures. The proposed questions will help characterize several types of nonstandard work arrangements (e.g. self-employed, contractor or temporary agency worker), work schedules, and related workplace psychosocial exposures (e.g., job insecurity). The 2021 NHIS-OHS data will primarily be used to study associations between these job characteristics and select health outcomes in order to determine which elements of work design may be associated with poor health outcomes (e.g. general health status). Although some of the elements of interest (nonstandard work arrangements, contingent work, and precarious employment) are covered by the NHIS core or rotating core content, the 2021 NHIS-OHS will collect data that will expand our understanding of these concepts. Attachment 1 details the constructs that will be captured in the 2021 NHIS annual or rotating core, the proposed constructs for cognitive testing, as well as the potential associations with health outcomes and occupational safety.

This project is only proposing cognitive testing; however, NIOSH has noted that the analytical sample derived from the 2021 NHIS-OHS quantitative data will be described as workers from a representative sample of the civilian noninstitutionalized population of the United States of America. Additional details about the quantitative sample (e.g., impact of sampling error on small subpopulations) are discussed in Attachment 1.

Testing the 2021 OHS will help NIOSH understand the performance of questions on the OHS, including construct(s) captured by each question, patterns of interpretation across respondent groups, and potential sources of response error. Study findings will help NHIS and NIOSH revise and validate the supplement before fielding the 2021 OHS on the NHIS.

The occupational health questions we are evaluating are included as Attachment 2. The testing procedure conforms to the cognitive interviewing techniques that have been described in CCQDER’s generic OMB clearance package (No. 0920-0222, exp. 08/31/2021).

Recruiting: Twenty respondents with various experiences with work environments will be recruited. This may include respondents who are involved in standard 9 to 5 work, those in non-standard work such as shift work, temp work, self-employed, gig work (e.g., Uber/Lyft), contractors, and those who are not employed. Additionally, we aim to recruit respondents with a roughly even mix of age, race, and educational attainment. The initial goal is to recruit groups in equal proportion, to the extent possible – that is, within the constraints of those willing to participate in the study. However, because qualitative sampling is based on theoretical relevance more than equal cell sizes, on-going analysis may reveal the need to recruit more from one group than others. Recruitment will be carried out through a combination of a newspaper advertisement, flyers, word-of-mouth, and CCQDER Respondent Database. The newspaper advertisements/flyers used to recruit respondents are shown in Attachments 3a&b.

Screening and Scheduling: After potential respondents contact the recruiter by phone or email, the recruiter will follow up to screen them for eligibility for inclusion in the study. The 10-minute screener used to determine eligibility of individuals responding to the newspaper advertisements/flyers is shown in Attachment 4a. Note that wording of the template has been approved and is contained within our umbrella package. Only project specific information has been added to the document. The screener used to determine eligibility of individuals from the CCQDER Respondent Database is shown in Attachment 4b. Wording of the template has been approved and is contained within our umbrella package. Only project specific information has been added to the document. Within these constraints, we plan to recruit participants with some demographic variety (particularly in terms of gender, education, and race/ethnicity). During the screening process, recruiters will inform respondents of the consent form, video recording and remuneration procedures. For virtual interviews, recruiters will also assess respondents’ video conferencing capabilities and will work with them to set up the video conference application (Zoom or Skype). After respondents are scheduled, a confirmation email will be sent which will include information about the date and time of interview, instructions for using the video conferencing application and information about informed consent. If the videoconferencing application fails to work properly, the recruiter will use this script to cancel the interview. Script: Unfortunately, we cannot move forward if your [video conference application/Skype/Zoom] is not working properly because these interviews will be conducted over video chat. I’m sorry, we have to cancel this interview. Would it be okay if I added your name, telephone number, age, educational level, and race to our database so that I can contact you about other studies coming up in the future? If yes, add to database. If no: OK, thank you for your time. Your name and any information you gave me will not be added to our database.

It is anticipated that as many as 50 individuals may need to be screened in order to recruit 20 participants.

Interview Procedure: Interviews will be conducted by CCQDER staff members and Strategic Innovative Solutions (SIS) contractors with English speaking respondents for up to 60 minutes per interview. Interviews may be conducted in-person in the Questionnaire Design and Evaluation Research Laboratory (QDRL), off-site or virtually through either the Skype or Zoom video conferencing applications.

In-person: After respondents have been briefed on the purpose of the study and the procedures that CCQDER routinely takes to protect human subjects, respondents will be asked to read and sign an Informed Consent document (Attachment 5a). Only project specific information has been added to the document. Respondents will also be asked to fill in their demographic characteristics on the Respondent Data Collection Sheet (Attachment 6a). This document is contained in our umbrella package.

Virtual: For virtual cognitive interviews, respondents and interviewers will see and hear each other through the video conference software from their own computer, tablet or cellphone from their respective locations. A recruiter will meet the respondent on the scheduled video conferencing appointment (through Zoom or Skype). The recruiter will make sure the respondent is prepared for the interview and that the video conferencing application is working properly. The recruiter will go over the informed consent information that was previously emailed to the respondent (Attachment 5b/only project specific information has been added to the document) and remind the respondent of the remuneration procedure. If the application fails to work properly, the recruiter will use this script to cancel the interview. Script: Unfortunately, we cannot move forward if your [video conference application/Skype/Zoom] is not working properly because these interviews will be conducted over video chat. I’m sorry, we have to cancel this interview. Would it be okay if I added your name, telephone number, age, educational level, and race to our database so that I can contact you about other studies coming up in the future? If yes, add to database. If no: OK, thank you for your time. Your name and any information you gave me will not be added to our database.

Once the videoconferencing software is working properly, the recruiter will collect the information from the Respondent Data Collection Sheet. (Attachment 6b). Only project specific information has been added to the document. At this point, the interviewer will join the video conference and the recruiter will leave the meeting.

The interviewer will then ask the respondent to confirm that he/she understands the information in the Informed Consent (Attachment 5b), and then state that we would like to record the interview. The recorder will be turned on once it is clear that the procedures are understood and agreed upon.

Once the recording has started, the interviewer will confirm that the respondent has agreed to be recorded. The interviewer will confirm that the respondent has agreed to be recorded by asking, “Do you agree that the interview will be video recorded? Yes, I agree or No, I don’t agree.”

The interviewer will then orient the respondent to the cognitive interview with the following introduction:

[fill staff name] may have told you that we will be working on some questions that will eventually be added to national surveys. Before that happens, we like to test them out on a variety of people. The questions we are testing today are about your work environment. We are interested in your answers, but also in how you go about making them. I may also ask you questions about the questions—whether they make sense, what you think about when you hear certain words, and so on.

I will read each question to you, and I’d like you to answer as best you can. Please try to tell me what you are thinking as you figure out how to answer. Also, please tell me if:

there are words you don’t understand,

the question doesn’t make sense to you,

you could interpret it more than one way,

it seems out of order,

or if the answer you are looking for is not provided.

The more you can tell us, the more useful it will be to us as we try to develop better questions. Okay? Do you have any questions before we start? If yes, answer questions. If not, let’s get started.

Video/Audio Recording: Video or audio recording is required for this project. These recordings will allow researchers to review the behaviors and body language of the respondents and to ensure the quality of their interview notes. Recordings will be used and retained in accordance with the CCQDER Data Storage and Access Policy. Researchers from CCQDER, SIS contractors, NIOSH DSHEFS staff who are working on the project, will use the recordings for research purposes. SIS contractors and NIOSH DSHEFS staff viewing/listening to recordings in the QDRL under CCQDER supervision have read and signed a non-disclosure affidavit and completed confidentiality training.

In-person: All interviews conducted in the QDRL or off-site will be video and/or audio recorded. In rare cases, a study participant may initially agree to be video recorded during the telephone screening but change their mind at the time of the interview. If the respondent changes their mind in the case of an in-person interview, the participant will be asked if they agree to be audio recorded. If they decline to be audio recorded the interview will proceed without recording. In this case the interviewer will depend on their handwritten notes when conducting analysis.

Virtual: CCQDER will request permission to video record all virtual interviews conducted via Skype or Zoom. If the respondent consents to being video recorded and then changes their mind after the interview has begun, the interview will proceed with audio-only recording. In this case, the interviewer will instruct the respondent to turn off their camera and explain that only audio will be recorded as long as the respondent’s camera remains off. In the event that the respondent’s camera is turned back on, the interviewer will remind the respondent to turn the camera off if they do not want to be recorded. If the respondent does not want to be recorded but is unable to turn their camera off, the interview will be terminated.

NCHS government issued encrypted laptops will be used to video and audio record the interviews conducted off-site or virtually by both CCQDER researchers and SIS contractors. An NCHS government issued encrypted flash drive will be used to transfer the recordings from the interviewer’s laptop to the CCQDER’s secure LAN. The encrypted flash drive is FIPS 140-2 compliant and approved for use by OCISO.

Extreme care will be taken with all recordings and paperwork from the interviews conducted off-site. Recordings and identifying paperwork will be stored in a secured travel case until returned to NCHS, at which point they will be transferred to the usual secured locked storage cabinets. Once the video and audio recordings are transferred to the QDRL Network, the recordings will be deleted from encrypted flash drive. Once deleted, the files are no longer available for use.

After the interview: Respondents will be given the thank-you letter signed by the Director of NCHS (Attachment 7), a copy of the informed consent document (Attachments 5a/b), and $40. For interviews conducted virtually, the recruiter will send the respondent a “thank you” email, informing them that they will receive a hard copy “thank you” letter (Attachment 7) and $40 in cash via FedEx within 7 business days.

Deletion of information: Once the interview is complete and the remuneration has been sent, all email, phone call and calendar records for the respondent will be deleted.

Retention of Recordings: If a respondent requests that their recording be destroyed at the end of the project (virtual), the initial retention period of the video recordings is 5 years after project completion. If the respondent gives future consent, after the initial retention period, the recordings will be re-evaluated by the CCQDER Director to determine relevance, ongoing usefulness, and qualitative value for likely use in question evaluation research. If it is determined by the CCQDER Director in conjunction with CCQDER project-relevant staff that there is no valid reason to retain the recording, it will be destroyed by designated CCQDER staff. If the interview continues to be of value (defined as ongoing use by research staff, topic relevance, likely use for federal questions evaluation research), reassessment of the recording will occur again in 5 years.

Interview Notes: CCQDER staff and SIS contractors will also use the NCHS government issued encrypted laptops to input their interviewer notes into Q-Notes. Within 24 hours, a CCQDER staff member will review interview notes and will delete any direct or indirect personal identifiable information (PII) if found.

We propose giving participants $40 incentives, which is our standard incentive. In total, for this project, the maximum respondent burden will be 31 hours. A burden table for this project is shown below:

Form Name	Number of Participants	Number of Responses/ Participant	Average hours per response	Response Burden (in hours)
Screener (recruited from newspaper/flyer)	32	1	10/60	6
Screener (recruited from CCQDER respondent database)	18	1	10/60	3
Questionnaire	20	1	60/60	20
Respondent Data Collection Sheet: NCHS interviews/offsite (Attachment 6a) or virtual interviews (Attachment 6b)	20	1	5/60	2
Total	31

Attachments (7)

cc:

S King

J. Zirger

DHHS RCO

6 | Page