Letter to Dr. Schwab

DEPARTMENT OF HEALTH & HUMAN SERVICES Public Health Service

Centers for Disease Control and Prevention

National Center for Health Statistics

3311 Toledo Road

Hyattsville, Maryland 20782

October 17, 2018

Margo Schwab, Ph.D.

Office of Management and Budget

725 17th Street, N.W.

Washington, DC 20503

Dear Dr. Schwab:

The NCHS Collaborating Center for Questionnaire Design and Evaluation Research (CCQDER) (OMB No. 0920-0222, Exp. Date 08/31/2021) proposes to conduct a cognitive interviewing study to evaluate questions for the Pregnancy Risk Assessment Monitoring System (PRAMS) for the Division of Reproductive Health (DRH), National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP).

The proposed PRAMS questions have not been otherwise fully evaluated, and it is not certain whether or not, and the degree to which, the questions capture the constructs required by the CDC. For this reason, CCQDER has been asked to cognitively test these items. Recruitment of respondents and interviewing would begin as soon as approval is received.

Cognitive Interviewing Methodology

The methodological design of this proposed study is consistent with the design of most NCHS/CCQDER cognitive interviewing studies: the purpose is to identify the various patterns of interpretation that respondents consider when formulating an answer to a survey question as well as any problems experienced. Findings demonstrate the construct captured by each question, consistency of patterns across respondent groups, and potential sources of response error. Interviews are in-depth and semi-structured; analysis is conducted using qualitative methodologies. Findings from all CCQDER studies are documented in a final report and made publicly accessible on a searchable website at https://wwwn.cdc.gov/QBank.

Cognitive Interviewing Study of PRAMS Opioid-Related Questions

Background: The PRAMS questionnaire, first implemented in 1988, has been revised several times over the life of the project and revisions have been made primarily to capture data on recent public health guidelines or emerging issues concerning maternal and child health and to improve respondents’ comprehension of questions. CCQDER has conducted cognitive interviewing studies for earlier revisions (1999, 2001, 2003, 2007, 2014 and 2016).

Given that the goal of the survey is to ensure collection of high quality data that has practical application, the newly proposed PRAMS opioid supplemental questions will benefit from cognitive testing to improve the validity and reliability of the data. The information from the cognitive testing will not only improve data collection and question design, but also the researchers’ ability to better understand the data.

Study Research Questions: Findings from previous CCQDER studies show that, when formulating answers to opioid impairment questions, respondents’ interpretations of such questions were directly linked to their personal experience and circumstance.¹ This study will address similar research questions in the context of respondents’ most recent pregnancy experience. Such questions include:

1. When answering questions about opioid use, what kinds of medication do respondents consider? How is it defined, and what are the parameters for these considerations? Did the pregnancy experience affect understandings?

2. How do respondents understand the concept of misuse? How do these personal explanations impact their response to questions about misuse? How consistent are these patterns across differing groups of respondents?

3. In terms of answering questions about usage, are there any cognitive tasks that are overly-burdensome to the extent that data quality is compromised? If so, what are the characteristics of those questions? Does this differ across respondents?

4. Should some types of opioid-related questions be deemed as too sensitive to ask on the PRAMS survey? What are the characteristics of those questions? Does this vary across respondents?

5. Do respondents understand opioid pain relievers differently from the other substances asked about, such as depression and anxiety medication? From marijuana? From illegal substances?

Study Protocol: The PRAMS opioid related questions to be evaluated are included as Attachment 1. The testing procedure conforms to the cognitive interviewing techniques that have been described in CCQDER’s generic clearance package (OMB No. 0920-0222, Exp. Date 08/31/2021).

We proposed to recruit up to 40 English-speaking women (aged 18 and over) who:

• Had a baby between 2-12 months ago

• Have used/taken prescription and non-prescription drugs before and during pregnancy

  • In order to cast a wider net and detect possible response error we will recruit women who have used/taken prescription and non-prescription drugs before and during pregnancy in hopes of catching unplanned pregnancies and women saying “no” when they were on opioids before they knew they were pregnant.

Recruitment will be carried out through a combination of a newspaper advertisements/flyers, and word-of-mouth. The newspaper advertisement/flyer used to recruit respondents are shown in Attachment 2a&b. Flyers may be posted at locations such as Social Service centers, WIC programs, women and children shelters, churches, rehabilitation centers, health care centers, treatment centers for chronic pain, and child care centers. The 5 minute screener used to determine eligibility of individuals responding to the newspaper advertisement/flyer is shown in Attachment 3. Note that wording of the template has been approved and is contained within our umbrella package. Only project specific information has been added to the document. It is anticipated that as many as 96 individuals may need to be screened in order to recruit 40 participants.

Recruitment of respondents may be done in coordination with Social Service centers, WIC programs, women and children shelters, churches, rehabilitation centers, health care centers, treatment centers for chronic pain, and child care centers. The contacted centers will read a statement about CCQDER’s study (Attachment 2c). If the respondent is interested in participating in the PRAMS Opioid 2018 study, the potential respondent will contact CCQDER staff members [in person/over the phone] to set-up an appointment.

Interviews averaging 90 minutes (including the completion of a Respondent Data Collection Sheet (Attachment 5)) will be conducted by CCQDER staff members with English speaking respondents.

Interviews will be conducted in the Questionnaire Design and Evaluation Research Laboratory as well as at off-site locations. All interviews conducted in the Questionnaire Design and Evaluation Research Laboratory and off-site will be video and audio recorded to allow researchers to review the behaviors and body language of the respondents. These recordings will allow researchers to ensure the quality of their interview notes. Recordings will only be used by researchers from CCQDER, National Center for Chronic Disease Prevention and Health Promotion and Research Support Services (subcontracted through Swan Solutions to conduct the Spanish-language interviews) who are working on the project. Recordings will remain under CCQDER staff control. There will be no external sharing of the recordings.

Video or audio recording is required for this project except in the rare case that a study participant initially agrees to be video recorded during the telephone screening, but changes their mind. In that case, they will be asked if they agree to be audio recorded. If they decline to be audio recorded the interview will proceed without recording. In this case the interviewer will depend on their handwritten notes when conducting analysis. In addition, individuals who select “yes” for allowing the recording on the informed consent form, but “no” for retaining the recording for future research (final text before signatures on informed consent form), will be allowed to participate in the study.

NCHS government issued encrypted laptops will be used to video and audio record the interviews conducted off-site. Due to the size of the video recordings, the internal drive of the encrypted laptop is not sufficient for storage of the recordings. Recordings will be saved to an NCHS government issued encrypted flash drive. The encrypted flash drive is FIPS 140-2 compliant and approved for use by OCISO.

CCQDER staff will also use the NCHS government issued encrypted laptops to input their interviewer notes into Q-Notes. Within 24 hours, a CCQDER staff member will review PRAMS Opioid 2018 interview notes and will delete any direct or indirect personal identifiable information (PII) if found.

Extreme care will be taken with all recordings and paperwork from the interviews conducted off-site. Recordings and identifying paperwork will be stored in a secured travel case until returned to NCHS, at which point they will be transferred to the usual secured locked storage cabinets. Once the video and audio recordings are transferred to the QDRL Network, the recordings will be deleted from encrypted flash drive. Once deleted, the files are no longer available for use.

After respondents have been briefed on the purpose of the study and the procedures that CCQDER routinely takes to protect human subjects, respondents will be asked to read and sign an Informed Consent (Attachment 4). Only project specific information has been added to the document. Respondents will also be asked to fill in their demographic characteristics on the Respondent Data Collection Sheet (Attachment 5). This document is contained in our umbrella package. The burden for completion of this form is captured in the interview.

The interviewer will then ask the respondent to confirm that she understands the information in the Informed Consent, and then state that we would like to record the interview. The recorder will be turned on once it is clear that the procedures are understood and agreed upon.

The interviewer will then orient the respondent to the cognitive interview with the following introduction:

[Fill staff name] may have told you that we will be working on some questions that will eventually be added to national surveys. Before that happens, we like to test them out on a variety of people. The questions we are testing today are about your use of prescription and non-prescription drugs during your recent pregnancy.

We are interested in your answers, but also in how you go about making them. I may also ask you questions about the questions—whether they make sense, what you think about when you hear certain words, and so on.

I would first like you to fill out this questionnaire, and then we will discuss the questions and how you formulated your answers. Please answer the best you can, and mark any questions that you don’t understand or are having difficulty answering and we will discuss them once you have finished, and I’d like you to answer as best you can. Please note on the instrument if:

there are words you don’t understand,

the question doesn’t make sense to you,

you could interpret it more than one way,

it seems out of order,

or if the answer you are looking for is not provided.

The more you can tell us, the more useful it will be to us as we try to develop better questions. Okay? Do you have any questions before we start?

The interviewer will follow the interview protocol, but will also ask emergent probes to better understand the question-response process. Examples of the sorts of probes that may be asked at the interviewer’s discretion include:

• Could you tell me what [term] means to you?

• Why did you answer that way?

• In your own words, could you tell me what you think this question is asking?

• Was this question easy or hard to answer? Why?

• How sure are you about your answer?

Interviewers may use some or all of these probes, depending upon the content of the interview and how much information the respondents reveal without being prompted. All probes will be administered after the respondent has completed the questionnaire.

After the interview, respondents will be given the thank-you letter signed by the Director of NCHS (Attachment 6), a copy of the informed consent document, and $50.

We propose giving participants $50 for their participation, which is $10 over our standard payment. We hope the extra $10 above our standard $40 payment will be sufficient to entice new mothers of babies 2-12 months of age to come to participate in the 90-minute interview. In the 2014 and 2016 PRAMS cognitive interviewing studies, we found that new mothers had difficulties keeping appointments due to problems with childhood illnesses, baby sitters for older children, and transportation. In total, for this project, the maximum respondent burden will be 68 hours. A burden table for this project is shown below:

Form Name	Number of Participants	Number of Responses/ Participant	Average hours per response	Response Burden (in hours)
Screener (recruited from newspaper/flyer)	96	1	5/60	8
Questionnaire	40	1	85/60	57
Respondent Data Collection Sheet	40	1	5/60	3
Total	68

Attachments (6)

cc:

V. Buie

J. Zirger

DHHS RCO

1 Willson, S. (2017). Cognitive Interview Evaluation of Survey Items to Measure Substance Use and Impaired Driving. National Center for Health Statistics. Hyattsville, MD. https://wwwn.cdc.gov/QBank/Report.aspx?1186. Accessed 4/9/2018.

6 | Page