Project Description and Justification

DEPARTMENT OF HEALTH & HUMAN SERVICES Public Health Service

Centers for Disease Control and Prevention

National Center for Health Statistics

3311 Toledo Road

Hyattsville, Maryland 20782

June 12, 2019

Margo Schwab, Ph.D.

Office of Management and Budget

725 17th Street, N.W.

Washington, DC 20503

Dear Dr. Schwab:

The NCHS Collaborating Center for Questionnaire Design and Evaluation Research (CCQDER) (OMB No. 0920-0222, Exp. Date 08/31/2021) proposes to conduct a cognitive interviewing study to evaluate a subset of Spanish-language telephone-administered questions for the Pregnancy Risk Assessment Monitoring System (PRAMS) for the Division of Reproductive Health (DRH), National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP). The proposed PRAMS questions are the same as those that will be concurrently tested in English. The questions have not been otherwise fully evaluated, and it is not certain whether or not, and the degree to which, the questions capture the constructs required by CDC. Given that the goal of the survey is to ensure collection of high quality data that has practical application, this subset of questions will benefit from cognitive testing to improve the validity and reliability of the data. Specifically, information from the cognitive testing study will reveal interpretive patterns among Spanish language speakers and inform any question design issues related to language differences between the English and Spanish versions. For this reason, CCQDER has been asked to cognitively test these items. Recruitment of respondents and interviewing would begin as soon as approval is received.

Background: PRAMS 2019 Questions (Phone/Spanish version)

The Pregnancy Risk Assessment Monitoring System focuses on collecting data on women’s experiences before, during and shortly after pregnancy to inform programs and policies focused on improving maternal-infant outcomes. The PRAMS Opioid call-back survey expands the timeframe of typical PRAMS by gathering data later in the postpartum period (9 months postpartum, rather than to 3-4 months postpartum). Topics covered on the Opioid call-back survey are similar to PRAMS in many cases, but there is an emphasis on substance use (e.g. prescription pain relievers, alcohol, tobacco products, illicit drugs), treatment and counseling for substance misuse, as well as infant development.

The PRAMS Opioid call-back questions that are being submitted for cognitive testing by NCHS fall into three categories: modified PRAMS questions, adapted from other surveys, or new questions. The majority of questions are already available in the PRAMS database of core (i.e. required) and standard (i.e. non-required) tested questions. For the purpose of the Opioid call-back survey, some existing PRAMS questions have been modified slightly to: a) correspond to the new time frame of the survey, or b) to add response options that have relevance to opioid or other substance use. In the case where new responses have been added, or the format of the question may also have been modified and requires testing.

For topics that were not available in the PRAMS database of questions, questions were taken from other surveys (i.e. NSDUH, BRFSS, USDA food security questions, National Survey of Children’s Health, Survey of Well-Being of Young Children) to address project needs. Subject matter experts (SMEs) on the topics of interest covered by all of these surveys were consulted regarding the inclusion of questions on the PRAMS Opioid call-back survey. Some of these questions were also modified slightly to be consistent with the PRAMS formatting or the timeframe of the survey.

A few topics of interest that were new in the context of the opioid epidemic, survey questions were not available. In this case, we developed questions based on existing guidance available for health care providers (for example, care for NAS infants in the hospital, and safe plan of care for infants after hospital discharge).

All modifications and development activities have been overseen by SMEs in CDC’s Division of Reproductive Health, in collaboration with the agencies overarching Opioid Response Coordinating Unit (ORCU). For inclusivity and avoidance duplicative data collection efforts, the draft survey has also been reviewed by SMEs external to CDC, including those from other federal agencies (HRSA, SAMSHA, CMS).

Plan of Study

The methodological design of the proposed study is consistent with the design of most NCHS/CCQDER cognitive interviewing studies. The purpose is to identify various patterns of interpretation that respondents consider when formulating an answer to a survey question, as well as any difficulties, confusion, or response error that occurs during administration. These patterns will be analyzed alongside the English-language data for comparability. Interviews are in-depth and semi-structured. Unlike other PRAMS questions that are self-administered, this subset is designed to be read over the phone; therefore, interviewers will read the questions aloud to the respondent. Analysis will be conducted using the constant comparative qualitative method and will focus on the constructs captured by each question, consistency of patterns across respondent groups and languages, and potential causes of response error. Findings (like all CCQDER studies) will be documented in a final report and made publicly accessible on a searchable website at https://wwwn.cdc.gov/QBank.

Research Questions: Findings from previous CCQDER studies on the PRAMS survey show that, when formulating answers to survey questions, respondents’ interpretations are directly tied to their personal experiences and circumstances.¹ Overall, this study will explore how survey questions on neo-natal and post-natal experiences with the health care system and use of various substances since the birth of a baby influence interpretive patterns. Moreover, analysis will compare English and Spanish interviews to assess comparability across languages. As with the English version, specific research questions for the Spanish language questions include:

1. Delivery outcomes: Do women with different pregnancy delivery outcomes (such as C-sections or premature deliveries) and different experiences with health care providers and hospitals understand the questions in a similar manner? Or are there fundamental differences in question interpretation based on these varying experiences?

2. Understanding of substances: Are the different substances – prescription pain relievers, non-prescription drugs, alcohol, and tobacco products – consistently understood? Do respondents understand and differentiate between prescription pain relievers and non-prescription drugs? Do they understand and include different tobacco products? Do they have adequate recall on use of substances such as alcohol?

3. Sensitivity: Are there different levels of sensitivity associated with answering survey questions on the use of different substances? With respondent utilization of treatment or counseling? With questions on baby’s neonatal opioid withdrawal syndrome?

Study Protocol: The Spanish-language PRAMS 2019 questions to be evaluated are included as Attachment 1. The testing procedure conforms to the cognitive interviewing techniques that have been described above and in CCQDER’s generic clearance package (OMB No. 0920-0222, Exp. Date 08/31/2021).

We propose to recruit up to 20 Spanish-speaking women (aged 18 and over) who:

• Had a baby between 3-24 months ago

• Had a C-Section

• Had a premature baby, or

• Have used/taken prescription (opioids) and non-prescription drugs, alcohol, and tobacco products, since the baby was born

Recruitment will be carried out through a combination of a newspaper advertisement, flyers, and word-of-mouth by Research Support Services (RSS). The newspaper advertisement/flyer used to recruit respondents are shown in Attachments 2a&b. Flyers may be posted at locations such as Social Service centers, WIC programs, women and children shelters, churches, rehabilitation centers, health care centers, treatment centers for chronic pain, and child care centers. The 5-minute screener to be used to determine eligibility of individuals responding to the newspaper advertisements/flyers/word-of-mouth is shown in Attachment 3. Note that wording of the template has been approved and is contained within our umbrella package. Only project specific information has been added to the document. It is anticipated that as many as 40 individuals may need to be screened in order to recruit 20 participants.

Recruitment of respondents may also be done in coordination with Social Service centers, WIC programs, women and children shelters, churches, rehabilitation centers, health care centers, treatment centers for chronic pain, and childcare centers. The contacted centers will read a statement about CCQDER’s study (Attachment 2c). If the respondent is interested in participating in the PRAMS 2019 study, the potential respondent will contact CCQDER staff members [in person/over the phone] to set-up an appointment.

Spanish-language cognitive interviews averaging 90 minutes (including the completion of a Respondent Data Collection Sheet (Attachment 5)) will be conducted by RSS with Spanish-speaking respondents.

Interviews will be conducted in a private room of a community-based organization or at a mutually agreed upon location with an individual respondent and an interviewer. Interviews will be audio recorded to allow researchers to ensure the quality of their interview notes. Recordings will only be used by researchers from CCQDER, the Division of Reproductive Health, NCCDPHP, and RSS who are working on the project. There will be no external sharing of the recordings.

RSS will use NCHS government issued encrypted laptops to audio record the Spanish cognitive interviews. Due to the size of the audio recordings, the internal drive of the encrypted laptop is not sufficient for storage of the recordings. Recordings will be saved to an NCHS government issued encrypted flash drive. The encrypted flash drive is FIPS 140-2 compliant and approved for use by OCISO.

RSS staff will also use the NCHS government issued encrypted laptops to input their interviewer notes into Q-Notes. Within 24 hours, a CCQDER staff member will review RSS’s interview notes and will delete any direct or indirect personal identifiable information (PII) if found.

Extreme care will be taken with all recordings and paperwork from the interviews conducted off-site. Recordings and identifying paperwork will be stored in a secured travel case until returned to NCHS, at which point they will be transferred to the usual secured locked storage cabinets. Once the video and audio recordings are transferred to the Questionnaire Design Research Laboratory Network, the recordings will be deleted from encrypted flash drive. Once deleted, the flash drive files are no longer available for use.

Audio recording is required for this project except in the rare case that a study participant initially agrees to be audio recorded during the telephone screening, but changes their mind. If they decline to be audio recorded the interview will proceed without recording. In this case the interviewer will depend on their handwritten notes when conducting analysis. In addition, individuals who select “yes” for allowing the recording on the informed consent form, but “no” for retaining the recording for future research (final text before signatures on informed consent form), will be allowed to participate in the study.

The initial retention period of the audio recordings is 2 years after project completion. After the initial retention period, the recordings will be re-evaluated by the CCQDER Director to determine relevance, ongoing usefulness, and qualitative value for likely use in question evaluation research. If it is determined by the CCQDER Director in conjunction with CCQDER project-relevant staff that there is no valid reason to retain the recording, it will be destroyed by designated CCQDER staff. If the interview continues to be of value (defined as ongoing use by research staff, topic relevance, likely use for federal questions evaluation research), reassessment of the recording will occur again in either 2 years.

After respondents have been briefed on the purpose of the study and the procedures that CCQDER routinely takes to protect human subjects, respondents will be asked to read and sign an Informed Consent (Attachment 4). Only project specific information has been added to the document. Respondents will also be asked to fill in their demographic characteristics on the Respondent Data Collection Sheet (Attachment 5). This document is contained in our umbrella package. The burden for completion of this form is captured in the interview.

The interviewer will then ask the respondent to confirm that she understands the information in the Informed Consent, and then state that we would like to record the interview. The recorder will be turned on once it is clear that the procedures are understood and agreed upon.

The interviewer will then orient the respondent to the cognitive interview with the following introduction:

[Fill staff name] may have told you that we will be working on some questions that will eventually be added to national surveys. Before that happens, we like to test them out on a variety of people. The questions we are testing today are about you and your baby’s experience with health care providers and hospitals, as well as questions on prescription and non-prescription drugs, alcohol, and tobacco use after your baby was born.

We are interested in your answers, but also in how you go about making them. I may also ask you questions about the questions—whether they make sense, what you think about when you hear certain words, and so on.

I will read each question to you, and I’d like you to answer as best you can. Please try to tell me what you are thinking as you figure out how to answer. Also, please tell me if:

there are words you don’t understand,

the question doesn’t make sense to you,

you could interpret it more than one way,

or if the answer you are looking for is not provided.

The more you can tell us, the more useful it will be to us as we try to develop better questions. Okay? Do you have any questions before we start? [If yes, answer questions. If not,] let’s get started.

Interviewers may use some or all of these probes, depending upon the content of the interview and how much information the respondents reveal without being prompted. Most probes will be administered after the respondent has finished the questionnaire, but some may occur concurrently, for example, if the respondent asks a question or raises an issue. Examples of the sorts of probes that may be asked (either retrospectively or concurrently) at the interviewer’s discretion include:

• Could you tell me what [term] means to you?

• Why did you answer that way?

• Can you give me an example?

• In your own words, can you tell me what you think this question is asking?

• Was this question easy or hard to answer? Why?

• How sure are you about your answer?

If time prevents all items from being covered in a single interview, the interviewer will explore items in accordance with the emergent themes and topics most relevant to the respondent.  If items at the end of the instrument are consistently not being covered, emphasis will shift to those items as the project evolves.

After the interview, respondents will be given the thank-you letter (document contained in umbrella package) signed by the Acting Director of NCHS (Attachment 6), a copy of the informed consent document, and $50.

Respondents will not be asked for Special Consent for Expanded Use of Video and Audio Recordings.

We propose giving participants $50 for their participation, which is $10 over our standard payment. We hope the extra $10 above our standard $40 payment will be sufficient to entice new mothers of babies 3-24 months of age to come to participate in the 90-minute interview. In the 2014 PRAMS, 2016 PRAMS, and 2019 PRAMS Opioid cognitive interviewing studies, we found that new mothers had difficulties keeping appointments due to problems with childhood illnesses, baby sitters for older children, and transportation. In total, for this project, the maximum respondent burden will be 37 hours. A burden table for this project is shown below:

Form Name	Number of Participants	Number of Responses/ Participant	Average hours per response	Response Burden (in hours)
Screener (recruited from newspaper/flyer)	40	1	5/60	4
Spanish Questionnaire	20	1	85/60	29
Respondent Data Collection Sheet	20	1	5/60	2
Total	35

Attachments (6)

cc:

S. King

J. Zirger

DHHS RCO

1 Willson, S., 2007, Cognitive Interview Evaluation of the Pregnancy Risk Assessment Monitoring System (PRAMS) Phase 6: Results of Interviews Conducted August-October, 2007, Hyattsville, MD: National Center for Health Statistics.

5 | Page