Request - Questionnaire Design and Evaluation Researchcognitive functioning and other questions for (NHIS)

DEPARTMENT OF HEALTH & HUMAN SERVICES Public Health Service

Centers for Disease Control and Prevention

National Center for Health Statistics

3311 Toledo Road

Hyattsville, Maryland 20782

February 29, 2016

Margo Schwab, Ph.D.

Office of Management and Budget

725 17th Street, N.W.

Washington, DC 20503

Dear Dr. Schwab:

The staff of the NCHS Center for Questionnaire Design and Evaluation Research (CQDER) (OMB No. 0920-0222, exp. 07/31/2018) plans to conduct an evaluation of questions on cognitive functioning and other questions for the National Health Interview Survey (NHIS OMB #0920-0214, expires 1/31/19).

We propose to start recruiting for volunteer participants as soon as we receive clearance and to start testing as soon as possible after that.

Background Information about Cognitive Testing of Questionnaires

The methodological design of this proposed study is consistent with the design of typical cognitive testing research. As you know, the purpose of cognitive testing is to obtain information about the processes people use to answer survey questions as well as to identify any potential problems in the questions. The analysis will be qualitative.

Proposed project: Testing of questions on cognitive functioning and other questions for the National Health Interview Survey (NHIS)

The primary purpose of this study is to test a follow up question to the NHIS question on cognitive function. One of the six questions to identify disability that is being used with increasing frequency in Department of Health and Human Services (HHS) health surveys is “Because of a physical, mental, or emotional condition, do you have serious difficulty concentrating, remembering, or making decisions?” This question is used frequently by the Disability and Health Branch (DHB) to identify those with a disability related to cognition. While this question identifies difficulty with cognitive function, it does not provide information as to the underlying cause of the disability (i.e., intellectual disability, mental illness, etc.). Therefore, it is believed that a follow-up question(s) is needed to identify the condition underlying the cognitive limitation among those respondents who report “yes” to the above question either as self or proxy report. This data will enhance public health surveillance and programmatic work by identifying the most frequently reported underlying causes of the cognitive disability and (with a large enough sample size) assessing the health risks and behaviors associated with those specific conditions. The cognitive functioning and other questions we are evaluating are included as Attachment 1. The testing procedure conforms to the cognitive interviewing techniques that have been described in CQDER’s generic OMB clearance package (No. 0920-0222, exp. 07/31/2018).

The secondary purpose of this study is to test additional NHIS questions in the domains of general health and well-being, sleep, and preventive care. The questions on mental health and stress were used on the Canadian Community Health Survey, 2015 (record number 3226, see full listing of survey numbers for Statistic’s Canada’s active surveys at http://www23.statcan.gc.ca/imdb-bmdi/pub/index-eng.htm). The questions on sleep and preventive care are currently used on the NHIS; the purpose of testing at this time is to refine the response categories.

We propose to recruit 40 English speaking adults (aged 18 and over) who meet at least one of the following criteria: 1) have difficulty with concentrating or remembering or 2) live in a household with another adult (aged 18 and over) who has difficulty concentrating or remembering. Sample selection will also ensure demographic variety, particularly in terms of gender, education, and race.

The newspaper advertisements/flyers used to recruit respondents are shown in Attachment 2. The 5 minute screener used to determine eligibility of individuals responding to the newspaper advertisements/flyers is shown in Attachment 3. Note that wording of the template has been approved and is contained within our umbrella package. Only project specific information has been added to the document. It is anticipated that as many as 72 individuals may need to be screened in order to recruit 40 participants.

Interviews averaging 60 minutes (including the completion of a Respondent Data Collection Sheet) will be conducted by CQDER staff members with English speaking respondents. All respondents will answer questions related to their own health and functioning. In order to test question placement and phrasing, half of the respondents will receive COG_2 and half of the respondents will receive COG_2a. Additionally, respondents recruited on the basis of living with another adult (aged 18 and over) who has difficulty concentrating or remembering, will serve as proxy-respondents when answering the single follow-up question on cognitive functioning, answering on behalf of the person in their household.

Interviews will be conducted in the Questionnaire Design and Evaluation Research Laboratory as well as at off-site locations. All interviews conducted in the Questionnaire Design and Evaluation Research Laboratory will be video and audio recorded to allow researchers to review the behaviors and body language of the respondents. Interviews conducted off-site will only be audio recorded. These recordings will allow researchers to ensure the quality of their interview notes.

In the case of respondents who are accompanied to the interview by a relative or caretaker, the companion will be asked to wait in the waiting room while the respondent completes the interview.

After respondents have been briefed on the purpose of the study and the procedures that CQDER routinely takes to protect human subjects, respondents will be asked to read and sign an Informed Consent document (Attachment 4). Only project specific information has been added to the document. Respondents will also be asked to fill in their demographic characteristics on the Respondent Data Collection Sheet. This document is contained in our umbrella package. The burden for completion of this form is captured in the interview.

The interviewer will then ask the respondent to confirm that he/she understands the information in the Informed Consent, and then state that we would like to record the interview. The recorder will be turned on once it is clear that the procedures are understood and agreed upon.

For respondents who indicate in the screener that they have cognitive difficulties, the interviewer will explicitly document that the respondent understands the consent by asking the following questions:

“I am going to ask you questions about difficulties you may have. Is that ok with you?”

“There is a video camera/audio recorder that is going to record our conversation. Is it ok with you for us to make a video/audio recording of our conversation?”

Taking into account their interactions up to that point, if the interviewer feels the respondent is cognitively capable of participating in the interview and if the respondent answers “yes” to both questions, the interview will proceed. If at any time, the interviewer determines that the respondent is not able to answer or understand the procedure, the interview will be halted.

The interviewer will then orient the respondent to the cognitive interview with the following introduction:

[Fill staff name] may have told you that we will be working on some questions that will eventually be added to national surveys. Before that happens, we like to test them out on a variety of different people. The questions we are testing today are about [your/your and your adult household member’s] physical and mental health, and healthcare. We are interested in your answers, but also in how you go about making them. I may also ask you questions about the questions—whether they make sense, what you think about when you hear certain words, and so on.

I will read each question to you, and I’d like you to answer as best you can. Please try to tell me what you are thinking as you figure out how to answer. Also, please tell me if:

there are words you don’t understand,

the question doesn’t make sense to you,

you could interpret it more than one way,

it seems out of order,

or if the answer you are looking for is not provided.

The more you can tell us, the more useful it will be to us as we try to develop better questions. Okay? Do you have any questions before we start? If yes, answer questions. If not, let’s get started.

After the interview is over, respondents will be given the thank-you letter (document contained in umbrella package) signed by Charles J. Rothwell, Director of NCHS, a copy of the informed consent, and $40. After the cognitive interview is over, respondents will be asked to read the Special Consent for Expanded Use of Video and Audio Recordings (Attachment 5). There will be no coercion and the respondents will be told that they can call and reverse the decision at any time if they change their minds. For respondents who indicated that they have cognitive difficulties, the interviewer will ensure that they understand the special consent by asking, “Is it alright for us to show the video of our interview to other people?” If respondents do sign the special consent form they will be given a copy of that as well.

Extreme care will be taken with all recordings and paperwork from the interviews conducted off-site. Recordings and identifying paperwork will be stored in a secured travel case until returned to NCHS, at which point they will be transferred to the usual secured locked storage cabinets.

We propose giving participants $40 incentives, which is our standard incentive. In total, for this project, the maximum respondent burden will be 46 hours. A burden table for this project is shown below:

Form Name	Number of Participants	Number of Responses/ Participant	Average hours per response	Response Burden (in hours)
Screener	72	1	5/60	6
Questionnaire	40	1	60/60	40

Attachments (5)

cc:

V. Buie

T. Richardson

DHHS RCO

3 | Page