Cognitive and Field Testing Results

The purpose of this study is to conduct cognitive interviews to test questions about heart disease and chronic disease management for BRFSS. These cognitive interviews were conducted to learn about how adults interpret, understand and respond to a set of questions and response options proposed for inclusion in the BRFSS survey used by CDC to collect data about the health of adults throughout the United States.

Cognitive testing is a technique used during survey design to gain a sharper understanding of underlying comprehension or retrieval problems among the respondent population. The results of cognitive interviews may lead to revisions of survey questions and changes in question order so as to enhance response rates and data quality. In a cognitive interview, an interviewer administers draft survey questions to a respondent while collecting additional verbal information about the survey responses with the goal of evaluating the quality of the responses or to determine if the wording is generating the intended information.^¹

This topline summary report presents the initial findings from nine cognitive interviews. All interviews were conducted by a trained interviewer using a CDC-approved, semi-structured interview guide. They averaged 40 minutes in length and were audio-recorded. Six of the nine interviews were conducted in person. In person interviewees were required to review and sign an informed consent statement. Phone interviewees were required to verbally consent to the interview after being read an informed consent statement.

Each participant was invited to respond to all the items in the guide, in the order the items appeared. Throughout the interview, ICF’s trained qualitative research specialist noted respondent comprehension issues, think-aloud processes, and additional feedback. This report summarizes feedback for each item.

Recruitment

ICF recruited adults with a history of heart attack, stroke, and/or hypertension to participate in cognitive interviews. In addition to posting several ads on Craigslist, ICF posted advertisements at an area hospital and community health center, and reached out to the VT American Heart Association Marketing Coordinator and Patient Coordinator within adult Neurology at UVM Medical Center. While we were able to successfully recruit individuals with hypertension and a history of heart attack for in-person interviews, interviews with three out of four stroke survivors were conducted over the phone. All respondents received an incentive of $50 for participating in the interview.

Respondents

Cognitive testing was conducted with 5 females and 4 males. Two participants had a history of both heart attack and hypertension, 4 participants had a history of stroke, and three participants had a history of hypertension only. Of the selected participants, 5 were between the ages of 18-49 years old, three were between the ages of 50-64 years old, and 1 was 65 or older. Seven participants self-identified as white, 1 self-identified as Black or African American, and 1 identified as American Indian or Alaska Native. As shown in Tables 1 and 2 below, participants represented a range of incomes and education levels.

Table 1: Participant Income

Income	Number of Respondents	Percentage
Under $25,000	1	11%
$25,000 – $34,999	3	33%
$35,000 - $49,999	1	11%
$50,000 - $74,999		0%
$75,000 or above	4	44%
Total:	9	100%

Table 2: Participant Education

Education Attained	Number of Respondents	Percentage
Some High school or less		0%
High school graduate (or equivalent)		0%
Some college or technical school (1-4 years, no degree)	2	22%
College graduate (e.g., BA, BS, etc.)	5	56%
Graduate degree (e.g., MA, MS, PhD, etc.)	2	22%
Medical degree (e.g., RN, NP, MD, DDS)		0%
Total:	9	100%

Overall Findings

All questions were asked of all nine participants. Comprehension was high for most questions and response options. Respondents found most questions to be clearly worded and easy to follow.

The most significant feedback from participants is summarized below:

Questions about taking medications “as directed.”

Two respondents noted that most people will want to report that they always take their medications as directed, even if this is not always the case. These respondents also mentioned that the timeframe of the question is “a little broad,” and that this would allow people to generalize and select the response perceived to be most favorable (“always” or “most of the time”). Two respondents felt that the response options were subjective (e.g., what qualifies as “most of the time”). One respondent noted that if this question is about taking medications for a specific condition, this condition should be included in the question wording.

Recommendations:

Change the wording to refer to the specific condition being treated.
Consider adding a more specific timeframe (e.g., over the past two weeks) to reduce the likelihood that respondents will generalize to the most favorable answer.

Questions about “outpatient rehab.”

When asked the definition of “outpatient rehab,” two respondents mentioned “drug or alcohol treatment facility” or a “place to get sober,” and two respondents mentioned that they did not know what outpatient rehab is.

Recommendation:

Given the low comprehension of this question we recommend adding a short description of outpatient rehab.

Phrasing of conditions under Chronic Disease and Self-Management

This set of questions includes a space for filling in the condition of most concern:

Example: Q2. Has your (NAME OF CONDITION OF MOST CONCERN) gotten better, worse, or stayed the same in the last 2 weeks?

These questions are clear when inserting “hypertension,” but is less clear when directly inserting heart attack or stroke, which are medical events.

Recommendations:

For heart attack and stroke, questions should be clarified to specify whether the respondent should refer to underlying conditions precipitating the medical event, or the consequences of the medical event.

Findings by Question

4.0 When was the last time you had your blood pressure checked by a doctor, nurse, or other health professional?

Read only if necessary: By “other health professional” we mean a nurse practitioner, a physician assistant, or some other licensed health professional.

Read only if necessary:

Never
Within the past year (anytime less than 1 year ago)
Within the past 2 years (1 year but less than 2 years ago)
Within the past 3 years (2 years but less than 3 years ago)
Within the past 4 years (3 years but less than 4 years ago)
Within the past 5 years (4 years but less than 5 years ago)
6 or more years ago
Don’t know / Not sure
Refused

No respondents had any difficulty answering this question.
One respondent noted that if a person had not had their blood pressure checked recently, it could be difficult for them to remember when this was.

Clarity/Comprehension: High

Recommendations: No changes

4.2 Are you currently taking [prescription] medicine for your high blood pressure?

Yes
No
Don’t know/Not sure
Refused

This question was asked two ways: Half of the respondents were asked whether they had taken prescription medicine, and half were asked whether they had taken medicine (the word “prescription” was not used). No respondents had any issues answering the question with either wording. All respondents had a similar understanding of the word prescription. Respondents who were not originally asked about “prescription” medicine indicated that adding the word “prescription” would not have changed their answer. Two respondents noted that they could not think of a non-prescription medicine for managing high blood pressure.

Clarity/Comprehension: High

Recommendations: No changes to the wording of this question. The use of the word prescription did not impact respondent comprehension and interpretation of the question.

4.2.3 If “Yes”, ask: “How often do you take your medications as directed? Would you say …

Interviewer note: ‘As directed’ would include advice from a doctor, nurse practitioner, physician assistant, or some other licensed health professional. Would you say…

Always
Most of the time
Sometimes
Rarely
Never
Don’t know/Not sure
Refused

All respondents had a similar understanding of what “as directed” means, noting that this is what is on the prescription bottle;
One respondent noted that at times what is on the bottle may not match exactly what the doctor says, but noted that this would not change her answer;
Two respondents noted that most people will want to report that they always take their medications as directed, even if this is not always the case. One respondent also mentioned that the timeframe of the question is “a little broad,” and that this would allow people to generalize and select the response perceived to be most favorable (“always” or “most of the time”).
Two respondents felt that this question was slightly “subjective,” as respondents may have different interpretations of what qualifies as “most of the time” and “sometimes”
One respondent noted that if this question is specific to medications for managing high blood pressure, this should be included in the question wording.

Clarity/Comprehension: Medium

Recommendations:

Change the wording to specifically refer to medications for managing high blood pressure.
Consider adding a more specific timeframe (e.g., over the past two weeks) to reduce the likelihood that respondents will generalize to the most favorable answer.

4.2.4 If “No”, ask: “Is that because it was prescribed and you never started taking it, or you have stopped taking it?”

Was prescribed but never started
Was prescribed but stopped taking it
Never was prescribed

7 Don’t know / Not sure

9 Refused

Most respondents found this question to be clearly worded and straightforward.
One respondent noted that this question seemed “confrontational,” and that it didn’t reflect many of the barriers that patients face in taking their medications.
One respondent noted that the question does not consider alternative ways of managing blood pressure other than prescription medicine.

Clarity/Comprehension: High

Recommendations: Consider adding a response option for managing hypertension through diet and exercise, as appropriate.

5.1 About how long has it been since you last had your blood cholesterol checked?

[Blood cholesterol is a fatty substance found in the blood.]

Read only if necessary:

1 Never

2 Within the past year (anytime less than 1 year ago)

3 Within the past 2 years (1 year but less than 2 years ago)

4 Within the past 3 years (2 years but less than 3 years ago)

5 Within the past 4 years (3 years but less than 4 years ago)

6 Within the past 5 years (4 years but less than 5 years ago)

8 6 or more years ago

7 Don’t know / Not sure

9 Refused

All respondents were able to easily answer this question with certainty.
One respondent noted that there are two types of cholesterol, and she wasn’t sure which type this question was referring to (although a more specific question would not have changed her answer).
One respondent noted that most people refer to just “cholesterol.” This respondent noted that the unfamiliar term “blood cholesterol” caused him to hesitate and think about his answer.

Clarity/Comprehension: High

Recommendations: Consider referring just to “cholesterol” rather than “blood cholesterol,” as appropriate.

Have you ever been prescribed medicine for your blood cholesterol?

Yes
No
Don’t know/Not sure
Refused

All respondents reported a high level of comprehension/clarity for this question. No issues were encountered or feedback was provided.

Clarity/Comprehension: High

Recommendations: No changes

5.2.2 Are you currently taking [prescription] medicine for your blood cholesterol?

Yes
No
Don’t know/Not sure
Refused

As with question 4.2, this question was asked two ways: Half of the respondents were asked whether they had taken prescription medicine, and half were asked whether they had taken medicine (the word “prescription” was not used). As with question 4.2, no respondents had any issues answering the question with either wording. All respondents had a similar understanding of the word prescription. Respondents who were not originally asked about “prescription” medicine indicated that adding the word “prescription” would not have changed their answer.

Clarity/Comprehension: High

Recommendations: No changes

If “Yes” to 5.2.2

5.2.3 How often do you take your medications as directed? Would you say..

Interviewer note: ‘As directed’ would include advice from a doctor, nurse practitioner, physician assistant, or some other licensed health professional.

Always
Most of the time
Sometimes
Rarely
Never
Don’t know/Not sure
Refused

Please see feedback listed under Question 4.2.3.

Clarity/Comprehension: Medium

Recommendations:

Change the wording to specifically refer to medications for managing high blood pressure.
Consider adding a more specific timeframe (e.g., over the past two weeks) to reduce the likelihood that respondents will generalize to the most favorable answer.

If “No”, ask:

5.2.4 “Is that because it was prescribed and you never started taking it, or you have stopped taking it?”

Was prescribed but never started
Was prescribed but stopped taking it

7 Don’t know / Not sure

9 Refused

Please see feedback listed under Question 4.2.4.

Clarity/Comprehension: High

Recommendations: Consider adding a response option for managing hypertension through diet and exercise, as appropriate.

Aspirin for CVD prevention

How often do you take an aspirin to prevent or control heart disease, heart attacks or stroke? Would you say:

1 Daily

2 Some days

3 Used to take but, had to stop due to side effects, or

4 No, do not take

7 Don’t know / Not sure

Refused

All respondents were able to answer this question without difficulty. One respondent noted that some people may not understand what is meant by aspirin, although he personally found this question easy to understand and answer as worded. Two respondents felt that the question should specify “baby aspirin.”

Clarity/Comprehension: High

Recommendations: Specify low-dose adult aspirin or baby aspirin, as appropriate.

Home/Self Measured Blood Pressure

Ask only among those with hypertension:

Has your healthcare provider recommended you check your blood pressure out of the office or at home?

1 Yes

2 No

7 Don’t know/not sure

All respondents reported a high level of clarity/comprehension for this question.

Clarity/Comprehension: High

Recommendations: No changes

Do you regularly check your blood pressure outside of your healthcare provider’s office or at home?

If “Yes”, ask: “Do you take it mostly at home or on a machine at a pharmacy, grocery or similar location?”

If “No”, SKIP question 3.

Read only if necessary:

1 Mostly at home [Go to question 3]

2 Mostly on a machine at a pharmacy, grocery or similar location [Go to question 3]

3 Do not check it [SKIP question 3]

7 Don’t know / Not sure [SKIP question 3]

9 Refused [SKIP question 3]

All respondents reported a high level of clarity/comprehension for this question.

Clarity/Comprehension: High

Recommendations: No changes

How do you share your blood pressure numbers that you collected with your healthcare provider? Is it…

1 Mostly by telephone

2 Mostly by other methods (email, internet portal, fax)

3 Mostly in person

4 Do not share information

7 Don’t know / Not sure

9 Refused

All respondents reported a high level of clarity/comprehension for this question.

Clarity/Comprehension: High

Recommendations: No changes

Rehabilitation

Ask only among participations with a prior history of a heart attack (1-3)

About how many sessions of outpatient rehab did you attend?

1 Less than 12 [Go to question 3]

2 12 to 24 [Go to question 3]

3 25 or more

4. Completed the recommended sessions / Finished

7 Don’t know / Not sure [Go to question 3]

9 Refused [Go to question 3]

When asked the definition of “outpatient rehab,” two respondents mentioned either a “drug or alcohol treatment facility” or a “place to get sober.”
One respondent mentioned that they have “no idea what outpatient rehab is.”
One respondent mentioned that this question should be specific about the type of rehab being referenced.

Clarity/Comprehension: Low

Recommendations: Given the low comprehension of this question even within a small sample size, we recommend adding a short description of outpatient rehab for this question.

What was the main reason you stopped going or did not go to outpatient rehab?

Read only if necessary:

1 Not recommended by my doctor or other healthcare provider

2 Didn’t know about it

3 Hard to travel to the rehab facility

4 Work conflict

5 Unable to afford (because no health insurance coverage or high copayments)

6 Did not think that it would help me

8 Completed the recommended sessions / Finished

7 Don’t know / Not sure

9 Refused

The only issue with this question is related to the definition of “outpatient rehab,” as described above. No other issues were encountered with this question.

Clarity/Comprehension: High

Recommendations: No changes.

Ask only among participations with a prior history of a stroke (4-6)

Following your stroke, did you go to any kind of rehab to strengthen muscles and regain skills in walking, swallowing, or speaking?

1 Yes [Go to Question 5]

2 No [Go to Question 6]

7 Don’t know / Not sure [End]

9 Refused [End]

All respondents reported high clarity/comprehension for this question. One respondent noted that listing examples of types of rehab might clarify this question (e.g., Occupational Therapy, Speech Therapy, and Physical Therapy).

Clarity/Comprehension: High

Recommendations: Add a READ ONLY IF NECESSARY interviewer note to provide examples of types of rehab.

Where did you go to rehab after your stroke? Was it in a:

1 Facility you were staying at (inpatient rehab or a nursing home),

2 A facility you had to travel to or in your home, or in

3 Both types

7 Don’t know / Not sure

9 Refused

Four respondents found the response options for this question to be confusing. Specifically, response option 2 lists two locations, leading to confusion as to what the third option (both types) was referring to: Option 1 and 2, or both locations within Option 2. Two respondents questioned whether rehab support provided during their hospital stay immediately following the stroke should count toward response #1.

Clarity/Comprehension: Low

Recommendations: Consider separate response options for “facility you traveled to” and “in your home.” Reword to allow respondents to select all that apply (rather than referring to “both”). Include “hospital” as an example in response 1, IF this answer should apply. If the hospital does not apply, rephrase the wording of the question to specify that this question refers to follow-up rehab AFTER being discharged from the hospital.

What was the main reason you didn’t go to rehab?

Read only if necessary:

1 Not recommended by doctor or other healthcare provider [Only among those with No to Q1]

2 Didn’t know about it [Only among those with No to Q1]

3 Hard to travel to the rehab facility (includes being too far away from a facility)

4 Work conflict

5 Unable to afford (because no health insurance coverage or high copayments)

6 Did not think that it would help

7 Don’t know / Not sure

9 Refused

All respondents reported high clarity/comprehension for this question.

Clarity/Comprehension: High

Recommendations: No changes

Chronic Disease and Self-Management Questions

Q1 Earlier you told me that you have (FILL FROM ABOVE history of heart attack, history of stroke, hypertension). Which of these conditions concerns you most?

Heart attack
Stroke
Hypertension

All respondents reported high clarity/comprehension for this question.

Clarity/Comprehension: High

Recommendations: No changes

Q2. Has your (NAME OF CONDITION OF MOST CONCERN) gotten better, worse, or stayed the same in the last 2 weeks?

A. Better (GO TO Q3-A)

B. Worse (GO TO Q3-B)

C. Stayed the same (GO TO Q3-C)

D. Not sure / don’t know (GO TO Q3-B)

E. Refused (GO TO Q3-B)

F. Never had condition (GO TO Q4)

Most respondents reported high clarity/comprehension for this question. All respondents paused to reflect upon this question prior to answering. All respondents were able to articulate the factors they considered when selecting their response (e.g., running times, blood pressure numbers). One respondent recommended that the survey might ask the respondent how they determined whether their condition had gotten better, worse, or stayed the same. One respondent noted that the time frame for answering this question is very short, and that if the interviewer had asked for the last “two months,” his answer would have changed. The only respondent who was unsure of how to answer this question asked whether her response should refer to the underlying conditions precipitating her stroke, or the consequences of her stroke.

Clarity/Comprehension: High

Recommendations: For medical events such as heart attack or stroke, specify whether this question is referring to conditions precipitating the event, or the outcomes of the event.

If Q2 Response was A: Q3A. Did you do anything that helped make it better?

A. Yes

B. No

C. Not sure / don’t know

D. Refused

If Q2 Response was B, D, or E: Q3B. Did you do anything to try to make it better?

A. Yes

B. No

C. Not sure / don’t know

D. Refused

If Q2 Response was C: Q3C. Did you do anything to keep it from getting worse?

A. Yes

B. No

C. Not sure / don’t know

D. Refused

All respondents reported high clarity/comprehension for the above questions. However, after reading the third response option, two of the respondents noted that the wording of the question (Did you do anything to keep it from getting worse?) is clearer than the previous two questions. Since none of the respondents expressed any initial trouble understanding or answering the first two questions, we do not recommend any changes to any of these questions.

Clarity/Comprehension: High

Recommendations: No changes

Q4. Do you know of any courses, classes, or specialized training in your community to help people manage (NAME OF CONDITION OF MOST CONCERN )?

A. Yes

B. No

C. Don’t Know

D. Refused

Most respondents reported high clarity/comprehension for this question. Only one respondent who had some difficulty processing information (due to stroke) had some difficulty understanding what courses and classes were referring to.

Clarity/Comprehension: High

Recommendations: No changes

Q5. Have you ever participated in a course or class, or received special training, on how you can manage your (NAME OF CONDITION OF MOST CONCERN)?

A. Yes

B. No

C. Don’t Know

D. Refused

Clarity/Comprehension: High

Recommendations: No changes

WRAP UP (5 minutes) – 3 questions

1. How comfortable were you sharing this information about yourself?

All respondents stated that they were very comfortable sharing this information about themselves. One respondent noted that the interviewer should stress again throughout the interview that all of this information should be kept confidential.

2. Would you be comfortable sharing this information about someone in your household?

All respondents stated that they would be very comfortable sharing this information about someone in their household.

3. Is there anything else about the survey questions and responses that you would like share with me?

One respondent noted that having the “government stamp” on this survey might be something that people will not like, and that it may cause them to not be truthful.

One respondent recommend that some terms used in these questions (rehab, support groups, blood cholesterol) should be simplified. Recommendations for simplifying question wording are captured above.

Sun Exposure and Food Stamps Questions

Top Line Summary Report

Background/Purpose

The purpose of this study is to conduct cognitive interviews to test questions about sun exposure and food stamps for BRFSS. These cognitive interviews were conducted to learn about how adults interpret, understand, and respond to a set of questions and response options proposed for inclusion in the BRFSS survey. This survey is used by CDC to collect data about the health of adults throughout the United States.

Cognitive testing is a technique used during survey design to gain a sharper understanding of underlying comprehension or retrieval problems among the respondent population. The results of cognitive interviews may lead to revisions of survey questions and changes in question order to enhance response rates and data quality. In a cognitive interview, an interviewer administers draft survey questions to a respondent while collecting additional verbal information about the survey responses. The goal of this process is to evaluate the quality of the responses or to determine if the wording of the questions generates the intended information.^²

This topline summary report presents the initial findings from nine in-person cognitive interviews. All interviews were conducted by trained interviewers using a CDC-approved, semi-structured interview guide. All cognitive interviews were approximately 30 minutes in length and were audio-recorded. All interviewees were required to agree to an informed consent statement.

Each participant was invited to respond to all the items in the guide, in the order the items appeared. Throughout the interview, ICF’s trained qualitative research specialists noted respondent comprehension issues, think-aloud processes, and additional feedback. This report summarizes feedback for each item.

Recruitment

Initial recruitment was done via an advertisement posted to Craigslist linked to an eligibility screener survey. If eligible, respondents were asked to provide their name, address, and contact information so that an ICF recruiter could contact them to schedule an interview. Additional respondents were recruited via the personal networks of ICF staff members. ICF staff members were not eligible to participate.

Respondents

Cognitive testing was conducted with five females and four males. Of the selected females, two were between the ages of 18 and 40 years old, two were between the ages of 41 and 64 years old, and one was over the age of 65. Two of the females self-identified as Black or African American, two as white, and one as Asian. Additionally, one female self-identified as Hispanic. Of the selected males, two were between the ages of 18 and 40 years old, one was between the ages of 41 and 64 years old, and one was over the age of 65. Two of the males self-identified as Black or African American, while the other two self-identified as white. Additionally, one male self-identified as Hispanic. Finally, three participants reported that they or someone in their household had received food stamps in the past year, while two other participants reported either that they or someone in their household had received food stamps at some point in the past.

Overall Findings

All questions were asked of all nine participants.

In general, respondents did not report any concerns or discomfort sharing information about themselves related to the topics in the two modules for cognitive testing: (1) Sun Exposure, and (2) Food Stamps. Only two people said that they would feel uncomfortable answering questions about someone else in their household over the phone. For each question set in each module:

Module 1 questions were commonly thought to be straight-forward and easy to follow and respond to.
- As with previous testing, respondents’ ethnicity/race may have influenced their perceived risk of damage from exposure to the sun or use of indoor tanning devices. Given this in the descriptions of responses to the questions for this module, respondents’ race has been noted when relevant.

The Module 2 question was slightly more confusing for interview participants. The changes recommended for this question are intended to help both participants/households who have received food stamps and participants/households who have not received food stamps.

Below we report key themes across respondents for each question tested. In addition, if deemed relevant to help the CDC consider question revision, we have included information on where only one or a few respondents offered responses different than what was commonly reported by others.

Module 1 – Cognitive Testing of Sun Exposure Questions

This module began with a brief description of “sunburn”:

By "sunburn" we mean even a small part of your skin hurts or you experienced a change in skin color such as redness for 12 hours or more. Also include burns from tanning beds and other indoor tanning devices.

Regarding the introduction:

When asked to repeat the description all respondents included both pain and redness as symptoms of sunburn. Additionally, all participants included artificial tanning as a potential sources of sunburn. Only about half of respondents repeated that redness should last for 12 hours or more. However, when asked to comment on the 12 hour time frame, most participants thought it was a reasonable time frame to include in the description.

When asked what came to mind when hearing the description, participants reporting thinking about their own experiences with sunburn such as when it happens to them (“when I lay in the sun too long”) or how their sunburn experience is different than the description (one white respondent stated his skins gets red but not particularly painful). Three African American participants and one Asian participant noted that they don’t “tan” or experience the redness associated with sunburn.

Despite any differences between respondent experience and the description, all respondents reported that the description made sense. Respondents did not report anything unclear or confusing about the wording of the description.

Clarity/Comprehension: High

Recommendation: Keep the wording of the description as is.

1. During the past 12 months, how many times have you had a sunburn?

DO NOT READ RESPONSES

None

1-365 times

Don’t know / Not sure

Refused

Respondent answers to the question ranged from zero to “5 or 6 times” (with the African American and Asian respondents emphasizing that they have never had a sunburn).

Respondents were commonly able to repeat this question in their own words.

Most respondents remembered the question asked about the “past 12 months”, though a few stated “past year” instead.

All respondents noted the question asked about “how many times”.

All respondents noted the question asked about sunburn, though participants did not always repeat the exact words “have you had a sunburn.” Instead, some participants stated similar phrases such as “did you get a sunburn” or “were you sunburned”.

Eight participants considered “the past 12 months” to be the prior 12 months starting from the day of the interview. However, one participant initially thought the “past 12 months” might be the past year of 2016.

Of the five participants who have had, or could get, a sunburn, all but one felt confident they could remember how many times they had a sunburn in the past 12 months. Of these, one participant was not sunburned due to being very vigilant about using sun protection, while the other participants remembered the 1 or 2 times they had a sunburn. One participant estimated he had been sunburned 5 or 6 times and was not sure of the exact number. Another participant who did remember the number of times sunburned thought it might be difficult for future interviewees to remember this. The participant suggested the question might be easier to answer if ranges were provided to select from, such as, 0-5 times and 6-10 times, etc, instead of the question being open ended.

None of the participants thought the question was confusing. Only the participant who did not remember the exact number of sunburns thought the question was somewhat difficult to answer.

Clarity/Comprehension: High

Recommendation: While clarity was high during testing, it is still possible that different respondents might interpret “past 12 months” in different ways. We recommend further defining “past 12 months”, such as “past 12 months, that is, since last (name of current month, e.g. March)”.

The revised item might read as follows:

During the past 12 months, that is, since last (name of current month, e.g. March), how many times have you had a sunburn?

Additionally, as one participant suggests, CDC might consider developing response option ranges instead of leaving the question as open ended.

2. When you go outside on a warm sunny day for more than one hour, how often do you try to protect yourself from the sun (for example, using sunscreen, wearing a wide-brimmed hat, or wearing a long-sleeved shirt)?

Always or Most of the time

Sometimes

Rarely or Never

Don’t go out in the sun

Don’t know / Not sure

Refused

INTERVIEWER: Listen and record respondent answer and verbal think aloud stream.

The majority of participants answered “Sometimes” with one participant indicating “Always or Most of the time”, one participant indicating, “Rarely or never”, and one indicating “Don’t go out in the sun”.

When asked to repeat the question, all participants included the concept of “protecting yourself from the sun when you go outside” in their response, as well as one or more form of protection (i.e. sunscreen, wearing a hat, or wearing a long-sleeved shirt). Only a few participants included in their answers either it being “a warm sunny day” or being outside “for more than one hour.”

All participants were comfortable estimating how often they try to protect themselves from the sun, and felt confident the answer choice they selected reflected their behavior. Additionally, the answer choices listed appeared to be distinct, as participants did not note any confusion between them nor express any difficulty deciding between two options.

Participants were also asked to consider whether or not their answer would change if the question did not specifically ask about “a warm sunny day”. While a majority of participant would have given the same answer, those who would change their answer noted that they associated both sun and warmth with needing sun protection. Participants said they were less likely to use sun protection on days that were sunny and cold, such as in the fall and winter.

Participants were further asked to consider whether or not their answer would change if the question did not specifically ask about being outside for “more than one hour”. Eight participants said their answer would not change if those words were not in the question. These participants indicated that the weather and their activities influenced whether or not they used sun protection. These participants did not consider length of time outside to be a meaningful influencer of their use of sun protection. The remaining participant stated he did think about how long he would be outside when deciding whether or not to use sun protection.

When ask to explain what “Sometimes” means to them, some participants stated they thought it meant using sun protection about half the time they went out in the sun. Some participants acknowledged that they might not use sun protection due to forgetting it or because they decided it was not necessary for what they were doing. Some of these participants stressed that using sun protection was activity based. They might use it at the beach “Always or most of the time”, but overall they used it “sometimes”. Additionally, one participant selected “Always or Most of the time” because he wears a hat when going outside to protect his face from the sun “most of the time”. This participant also quite often uses sunscreen. Furthermore, one participant stated that “Rarely or never” really meant never as he doesn’t need sun protection so doesn’t use it. The remaining participant selected “Don’t go out in the sun” because he simply dislikes being in the sun and stays in the shade, the house, and/or waits for the sun to go down to go outside.

None of the participants suggested that any additional answer choices should be added. However, one participant suggested that Always and Most of the time should be two separate response options, and another participant suggested that Rarely and Never should be two separate response options.

Respondents did not believe the question was unclear or difficult to answer.

Clarity/Comprehension: High

Recommendation: Keep the wording of the description as is.

Module 2 – Food Stamps Question

1. In the past 12 months, have you or anyone in your household received food stamps, also called SNAP (the Supplemental Nutrition Assistance Program), or food benefits on EBT (Electronic Benefit Transfer)?

Interviewer note:

Food Stamps or SNAP (Supplemental Nutrition Assistance Program) is a government program that provides plastic cards that can be used to buy food. In the past, SNAP was called the Food Stamp Program and gave people benefits in paper coupons or food stamps.

Yes

Three respondents reported that they had received food stamps in the past 12 months. No respondents noted members of their household receiving food stamps in the past 12 months. However, two participants stated they had previously lived in households that received food stamps.
When asked to repeat the question, all participants included the 12 month time frame and the detail of personally receiving food stamps. Most participants also included household receipt of food stamps. Some participants remembered and included the terms SNAP, food benefits, and/or EBT.
During follow up probing, all participants noted being familiar with the term food stamps, though some participants were not as familiar with the terms SNAP, food benefits, and/or EBT (including two participants who had received food stamps). However, all participants were able to understand from the question that food stamps, SNAP, and food benefits were names for the same program. Additionally, all participants were familiar enough with EBT to understand from the context of the question that EBT was a mechanism through which people received their food stamps.
Participants generally agreed that it was appropriate to use the word “receive” in the question. Participants could not think of a better word for the question.
Participants who had received food stamps stated they had no difficulty remembering when they had received them. These participants did not think anyone who received food stamps would be likely to forget this. The remaining participants also thought it was unlikely that individuals who had received food stamps would not be able to remember whether or not this had happened in the past year.
Participants did have varying definitions for the word “household”. All participants thought “household” meant people living in the same house. Most participants thought that the people living together had to be family to make a household, but some participants thought it might mean friends, boyfriend/girlfriend, or roommates as long as they were living together and sharing responsibilities in the house.
A majority of participants, including some who had received food stamps and some who had not, thought it was possible for a future interviewee to not know if members of their household (whether family, friends, or roommates) had received food stamps in the past 12 months. However, all participants felt that if a future interviewee did know, he or she would not forget whether or not it had happened in the past 12 months.
Additionally, four participants expressed that possible future interviewees might not be comfortable talking about receiving food stamps as part of a phone survey. Participants stated it is a sensitive topic, and some people are embarrassed about being on food stamps. Unfortunately, participants could not devise a better way to ask the question. Participants in this effort were not uncomfortable with the question, and thought the question should be included in future survey efforts.
In order to make the question clearer, a few participants suggested adding a definition of household. In order to make the question easier to answer, one participant suggested splitting it into two questions, one about the interviewee and one about the household.

Clarity/Comprehension: Medium

Recommendation: As currently structured, the question requires a respondent to simultaneously consider whether or not he or she received food stamps and whether or not someone else in their household received food stamps. We think making this item into two questions would make it easier to answer for participants. The first question would ask only about the interviewee. The second question would ask about the household, would provide a definition for “member of a household”, and would include options for “don’t know/not sure” and “there are no other members in my household.” Additionally, in both questions we recommend further defining “past 12 months”, such as “past 12 months, that is, since last (name of current month, e.g. March)”.

The revised items might read as follows:

In the past 12 months, that is, since last (name of current month, e.g. March), have you received food stamps, also called SNAP (the Supplemental Nutrition Assistance Program), or food benefits on EBT (Electronic Benefit Transfer)?

Interviewer note:

Yes

In the past 12 months, that is, since last (name of current month, e.g. March), has anyone in your household received food stamps, also called SNAP (the Supplemental Nutrition Assistance Program), or food benefits on EBT (Electronic Benefit Transfer)?

Interviewer note:

The Internal Revenue Service (IRS) defines a member of household as a person who is related to you or lives with you for the entire year as a member of your household.

Yes

Don’t know/Not Sure

There are no other members in my household

WRAP UP

Shape1

1. How comfortable were you sharing this information about yourself?

All respondents were comfortable sharing this information about themselves.

Shape2

2. Would you be comfortable sharing this information about someone in your household?

Seven respondents would be comfortable answering these questions for someone in their household, while the other two respondents said they would not want to do that.

Shape3

3. Is there anything else about the survey questions and responses that you would like share with me?

Participants did not have anything else to share that was relevant.

Hepatitis Questions

Top Line Summary Report

Background/Purpose

The purpose of this study is to conduct cognitive interviews to test questions about hepatitis for BRFSS. These cognitive interviews were conducted to learn about how adults interpret, understand and respond to a set of questions and response options proposed for inclusion in the BRFSS survey used by CDC to collect data about the health of adults throughout the United States.

This topline summary report presents the initial findings from nine cognitive interviews. All interviews were conducted by a trained interviewer using a CDC-approved, semi-structured interview guide. They averaged 30 minutes in length and were audio-recorded. In person interviewees were required to review and sign an informed consent statement. Phone interviewees were required to verbally consent to the interview after being read an informed consent statement.

Recruitment

Adults who have been diagnosed with hepatitis B or C and adults who have been treated for hepatitis C were recruited to participate in cognitive interviews. As the incidence of hepatitis in the general population of Vermont (site of in-person interviews) is relatively low, and treatment for hepatitis can be a potentially sensitive subject matter, a variety of recruitment methods were utilized, including Craigslist advertisements, flyers posted at a dialysis clinics, and outreach to an online hepatitis support group. ICF staff members were not eligible to participate, but were asked to inform friends and family who met these criteria about the possibility of participating in the study, resulting in one participant recruited through personal networks. Potential respondents who are unable to participate in an in-person interview in Burlington, VT were able to participate in a telephone interview. All respondents received an incentive of $50 for participating in the in-person or telephone interviews.

Respondents

Cognitive testing was conducted with nine individuals, including five females and four males. Four participants had been diagnosed with hepatitis B, and 6 participants had been diagnosed with hepatitis C. Of the selected females, two women were between the ages of 18-40 years old, one was 41-64 years old, and two were 65 or older. One female self-identified as white non-Hispanic, one identified as white Hispanic, two self-identified as Black or African American, and one identified as American Indian or Alaska Native. Two men were between the ages of 18 and 40 years old, and one was 41-64 years old, and the other two were aged 65 or older. One self-identified as white non-Hispanic, one self-identified as Hispanic, one self-identified as Asian, and one self-identified Native Hawaiian or Pacific Islander. As shown in Tables 1 and 2 below, participants represented a wide range of incomes and education levels.

Table 1: Participant Income

Income	Number of Respondents	Percentage
Under $25,000	3	33%
$25,000 – $34,999
$35,000 - $49,999	1	11%
$50,000 - $74,999	4	44%
$75,000 or above	1	11%
Skipped
Total:	9	100%

Table 2: Participant Education

Education Attained	Number of Respondents	Percentage
Some High school or less
High school graduate (or equivalent)	2	22%
Some college or technical school (1-4 years, no degree)	2	22%
College graduate (e.g., BA, BS, etc.)	4	44%
Graduate degree (e.g., MA, MS, PhD, etc.)	1	11%
Medical degree (e.g., RN, NP, MD, DDS)
Skipped
I do not wish to answer
	9	100%

Overall Findings

All questions were asked of all nine participants. Comprehension was very high for all the questions and all the different response options. Respondents found the questions to be clearly worded and easy to follow.

The main challenges respondents faced when answering these questions include:

Respondent lack of clarity around diagnosis. Three respondents had questions about their own diagnosis, which created some difficulty with answering questions. Two of these respondents tested positive for hepatitis antibodies, but did not have the hepatitis virus. These respondents were not sure how to answer questions 1, 3, and 6.

Recommendation: We recommend that help language be included in the instrument to clarify whether individuals testing positive for hepatitis antibodies (not the virus) should answer yes to this question. We also recommend including a response option for “not sure whether hepatitis A, B, or C”

Respondent recall issues around vaccines. Four out of the nine participants had difficulty recalling whether they had received the hepatitis B or A vaccine. While respondents gave an initial “yes” or “no” response to this question, when probed respondents indicated that they were not sure, or couldn’t remember.

Recommendation: We recommend providing guidance to help respondents identify events that would likely trigger vaccination (born after the year when the vaccination became available; received prior to starting college; provided by employer due to occupational exposure, etc.)

Findings by Question

MODULE 1 – Cognitive Testing of Emerging Core-Hepatitis

Has a doctor, nurse, or other health professional ever told you that you had hepatitis C?

READ IF RESPONDENT ASKS A QUESTION: Hepatitis C is an infection of the liver from the hepatitis C virus.

Yes
No
Don’t know / Not sure
Refused

Many participants asked the interviewer to repeat whether they had said hepatitis “B” or “C” as it can be difficult to differentiate these letters over the phone, even with the intentional emphasis by the interviewer.
Some participants had been told that they had hepatitis C, but questions around their diagnosis made them unsure whether it was accurate to answer “yes” to this question. For example, one participant stated that they were told they had hepatitis C “antibodies,” but follow up lab work was “inconclusive.” Another participant stated that at the time of their diagnosis, there was no differentiation between hepatitis A, B, and C. Although they believed they had hepatitis C, they could not be certain.
All participants found the wording of this question to be clear.

Clarity/Comprehension: High

Recommendations:

In the intro, have the interviewer state “C as in Charlie.”
Two of the 9 participants (one with hepatitis C, one with hepatitis B [see Question 3]) cited instances where testing indicated that presence of hepatitis antibodies, but not the presence of the hepatitis virus. As the presence of antibodies may indicate that the individual may have contracted hepatitis at one point, participants were not sure whether they should answer “yes” to this question. Both participants raised this question without probing from the interviewer. As such, we recommend that help language be included in the instrument to clarify whether individuals testing positive for hepatitis antibodies (not the virus) should answer yes to this question.
Include a response option for “not sure whether hepatitis A, B, or C”

Were you ever prescribed medicine to treat hepatitis C?

READ IF RESPONDENT IS NOT SURE: Many different types of medicine have been used to treat hepatitis C including injections and oral medicine.

Yes
No
Don’t know / Not sure
Refused

There were no issues encountered with this question. Participants found the wording to be clear and were very sure of their responses.

Clarity/Comprehension: High

Recommendation: Keep the question worded as is.

Thank you for answering those questions. Now I would like to ask you some questions about hepatitis B.

Has a doctor, nurse, or other health professional ever told you that you had hepatitis B?

READ IF RESPONDENT ASKS A QUESTION: Hepatitis B is an infection of the liver from the hepatitis B virus.

Yes
No
Don’t know / Not sure
Refused

Many participants asked the interviewer to repeat whether they had said hepatitis “B” or “C” as it can be difficult to differentiate these letters over the phone, even with the intentional emphasis by the interviewer.
One participant stated that at the time of their diagnosis, there was no differentiation between hepatitis A, B, and C. Although they believed they had hepatitis C, they could not be certain.
One participant stated that tests indicated the presence of hepatitis B antibodies. They expressed uncertainty as to whether they should answer yes to this question (as they may have been exposed to hepatitis B in the past) or no (because of the presence of antibodies indicated that they are immune).
There were no issues with question wording or participant comprehension.

Clarity/Comprehension: High

Recommendation:

In the intro, have the interviewer state “B as in boy.”

Two of the 9 participants (one with hepatitis C [see Question 1], one with hepatitis B) cited instances where testing indicated that presence of hepatitis antibodies, but not the presence of the hepatitis virus. As the presence of antibodies may indicate that the individual may have contracted hepatitis at one point, participants were not sure whether they should answer “yes” to this question. Both participants raised this question without probing from the interviewer. As such, we recommend that help language be included in the instrument to clarify whether individuals testing positive for hepatitis antibodies (not the virus) should answer yes to this question.
Include a response option for “not sure whether hepatitis A, B, or C”

Are you currently taking medicine to treat hepatitis B?

READ IF RESPONDENT IS NOT SURE: Many different types of medicine can be used to treat hepatitis B including injections and oral medicine.

Yes
No
Don’t know / Not sure
Refused

There were no issues encountered with this question. Participants found the wording to be clear, and were very sure of their responses.

Clarity/Comprehension: High

Recommendation: Keep the question worded as is.

MODULE 2 – Cognitive Testing of Hepatitis Treatment and Vaccination

Were you treated for hepatitis C in 2015 or after?

READ IF RESPONDENT IS NOT SURE: Most hepatitis C treatments offered in 2015 or after were oral medicines or pills, including Harvoni, Viekira, Zepatier, Epclusa, and others.

Yes
No
Don’t know / Not sure
Refused

There were no issues encountered with this question. Participants found the wording to be clear and were very sure of their responses. When asked to repeat the question in their own words, all participants correctly referenced the “2015 or after” timeline.

Clarity/Comprehension: High

Recommendation: Keep the question worded as is.

Were you treated for hepatitis C prior to 2015?

READ IF RESPONDENT IS NOT SURE: Most hepatitis C treatments offered prior to 2015 were injections or shots and pills given weekly or more often over many months.

Yes
No
Don’t know / Not sure
Refused

There were no issues encountered with this question. Participants found the wording to be clear, and were very sure of their responses. When asked to repeat the question in their own words, all participants correctly referenced the “prior to 2015” timeline.

Clarity/Comprehension: High

Recommendation: Keep the question worded as is.

Do you still have hepatitis C?

READ IF RESPONDENT IS NOT SURE: You may still have hepatitis C and feel healthy. Your blood must be tested again to tell if you still have hepatitis C.

Yes
No
Don’t know / Not sure
Refused

One participant who had questions about their original diagnosis was also not sure whether they still have hepatitis C, as antibodies are present and follow up testing was “inconclusive.”
All participants who received treatment after being diagnosed with hepatitis C were very sure of their response, as they received regular follow-up blood tests and discussed the results with their physicians.
There were no issues with question wording or participant comprehension.

Clarity/Comprehension: High

Recommendation: Keep the question worded as is. Clarification on questions related to diagnosis and the existence of antibodies should resolve the questions raised by this participant.

Now I would like to ask you some questions about hepatitis vaccines. Vaccines are for hepatitis B and hepatitis A.

Have you ever received the hepatitis B vaccine? The hepatitis B vaccine is completed after the third shot is given.

INTERVIEWER NOTE: Response is “Yes” only if respondent has received the entire series of three shots.

Yes
No
Don’t know / Not sure
Refused

Four out of the 9 respondents were not sure of their answers to this question:
- One respondent answered “pretty sure,” but with some hesitation. This respondent had campaigned for access to this vaccination due to work hazards. Although the details surrounding the vaccination were very clear to her, she stated that she “kind of” remembered receiving the vaccination, but that it shows up on her vaccination card.
- Three respondents said “no,” but after probing seemed less sure of their answers, indicating that they may have received it as a child or as an adult. While their initial answer was “no,” they were not sure of their answer.
Another respondent said “yes” before the interviewer could finish the question. When the second half of the question was stated, the respondent then clarified that they had received 2 out of the 3 shots.
Respondents noted that the information about the number of shots given to complete the vaccine was informative and helpful.

Clarity/Comprehension: High

Recommendation:

Since many respondents answered this question before it could be finished, we recommend reordering this question as follows: The hepatitis B vaccine is completed after the third shot is given. Have you completed the hepatitis B vaccine?
4 out of the 9 respondents had difficulty remembering whether they had received this vaccine. These respondents answered “yes” or “no” rather than “don’t know” to this question, but then indicated that they were unsure of their answer after probing. We recommend providing guidance to help respondents identify events that would likely trigger vaccination (born after year when vaccination became available; received prior to starting college; provided by employer due to occupational exposure, etc.)

Have you ever received the hepatitis A vaccine? The hepatitis A vaccine is completed after the second shot is given.

INTERVIEWER NOTE: Response is “Yes” only if respondent has received two shots.

Yes
No
Don’t know / Not sure
Refused

Respondents experienced the same issues with recall for this question as with question 8.

Clarity/Comprehension: High

Recommendation:

Since many respondents answered this question before it could be finished, we recommend reordering this question as follows: The hepatitis A vaccine is completed after the second shot is given. Have you completed the hepatitis A vaccine?

We recommend providing guidance to help respondents identify events that would likely trigger vaccination (born after year when vaccination became available; received prior to starting college; provided by employer due to occupational exposure, etc.)

WRAP UP

1. How comfortable were you sharing this information about yourself?

All respondents felt comfortable sharing this information about themselves. Several mentioned that they felt reassured by promises of confidentiality.

2. Would you be comfortable sharing this information about someone in your household?

All but one respondent indicated that they felt comfortable sharing this information about people in their household.

3. Is there anything else about the survey questions and responses that you would like share with me?

The only additional feedback came from respondents who indicated that they wanted to see more questions to be able to share more information about their experience (including treatment).

Chronic Fatigue Syndrome (ME) Questions

CFS Cognitive Testing Round 1

Main Findings for Prevalence Questions: 1 (Lifetime) and 2 (Current)

Q1 Have you ever been told by a doctor or other healthcare professional that you had CFS or ME?

1. Were participants able to respond accurately to the question?

All participants responded “Yes” to this question, and based on their description of the progression and current status of their CFS/ME, responded accurately.

Participants all described a long process of seeing various doctors, having tests, and ruling out alternative diagnoses in order to get a diagnosis of CFS/ME. Due to the fact that there are few doctors who specialize in CFS/ME, there is a general lack of knowledge about the disease. They typically had difficulty in getting a diagnosis; it can be a very long process – a year or more.

When asked to say what the phrase “… ever been told by a doctor or other health professional …” means to them, all participants used the words “diagnosis” or “diagnosed” and/or “confirmed.” This indicates that they focused on the formal outcome of the protracted process.

2. How did participants react to the question and the terms used in it?

Although all participants responded “Yes” to this question without hesitation, there was some negative feedback about the term “chronic fatigue syndrome.”

Multiple participants described how the terms “chronic fatigue,” “chronic fatigue syndrome,” and “CFS” are not considered to accurately describe the disease, since fatigue is not considered to be the most significant issue.
Multiple participants stated a perception that although the terms “chronic fatigue,” “chronic fatigue syndrome,” and “CFS” are commonly used, they diminish the disease or cause others to underestimate its severity.
Several participants said they prefer to use “Myalgic encephalomyelitis” (ME) rather than CFS (or forms of it). They stated that ME indicates that the disease is neurological – meaning muscle and brain dysfunction, and that affects cognitive functions.
Several participants said that which term they use depends on who they are talking to and the context. For those who prefer ME, they said that since that term is less known and harder for people to understand, they often use CFS for simplicity.

Q2: Do you still have CFS or ME?

Were participants able to respond accurately to this question?

Participants all said Yes to this question, and based on their description of the progression and current status of their CFS/ME, responded accurately.

How did participants react to being asked this question (do they still have CFS/ME)?

Several participants had neutral reactions to this question or seemed to understand why it was asked.

One participant said this was a reasonable question because many people will say that they have recovered; CFS is a disease that waxes and wanes, so it would be possible to catch someone at a “good” point before they relapse.
Several said that some who suffer from the disease recover somewhat, but do not get back to full ability (100%).
One described having been diagnosed with CFS once. Then after some years passed with a significant level of recovery, she had a second onset that was worse than the first time.

Several of the participants in this round reacted negatively to being asked if they still have CFS/ME.

One said “It’s a terrible question -- I find it offensive.” She explained her reaction: if this was a question on a survey, she would think that “the person [asking the question] wouldn’t know what they were talking about.” And: “If there’s a cure, I want to know about it.” She said it is a lifelong condition, something that doesn’t go away.
Another commented that this question was like asking: “Do you still have brown eyes?”

One participant was confused about the question. She seemed to think this was the same question she was asked earlier. The “had” in Q1 vs. the “still have” in Q2 was not obvious to her – perhaps due to all the intervening prompts in the cog interview setting (vs. how this would be delivered under real field conditions).

Recommendations for Questions 1 and 2

The consistent interpretation of the question as asking about diagnosis leads to the question: What is the exact measurement goal for this question? Does CDC want to measure the prevalence of those who have received a formal diagnosis of CFS/ME (lifetime and current)? What about those who think they have CFS, and/or are in the process of getting a diagnosis, but have not yet received a formal diagnosis? Would they qualify as “Yes” response to “ever been told…”?

We did not have any participants who fell into that category – they all had a diagnosis. We suggest monitoring during Round 2 testing to see if there are additional negative reactions to Q2.

It is important to retain both “CFS” and “ME” (vs. shortening to one or the other), since there are individual differences in acceptance of these terms. Having both terms seemed to alert participants that there is an understanding that these might be interchangeable (even though participants differed in their view of whether CFS and ME are the same or different diseases).

Caveat: Maybe we just did not have representation (within our 9) of those who have recovered enough to be considered completely better. However, some of our participants were very involved in support groups and are likely knowledgeable enough about the disease to know of instances of full recovery. We also had one participant who had had a period of remission, so we did hear that perspective.

Main Findings for Disability Questions 3 (Homebound) and 4 (Employment)

We tested these questions with the frequency scale, which was of most interest to CDC for the first round of testing. Due to the comments most of the participants made about their choices, we also offered the alternative version of these questions and probed about how the concepts of frequency and difficulty meshed with their way of thinking about limitations due to their CFS/ME.

Q3: How often does your CFS make it difficult for you to go outside the home alone or visit a doctor’s office? (All the time, Most of the time, Some of the time, Rarely, Never)

1. Were participants able to select a difficulty response option for this question?

Most participants were able to select a scale point from those provided. One or two said they would choose a point between two of the options provided because their symptoms vary (e.g., said they would choose something between All of the time and Some of the time).

2. How did participants react to the question and frequency scale?

One said that “never” would not be a viable option for people with CFS.

One pointed out that this question was double-barreled – there could be differences for going outside alone vs. going to a doctor’s office.
While some participants felt more comfortable answering in terms of frequency, others preferred the difficulty version.

For some whose symptoms vary a great deal, their response to a frequency scale would depend on when they are asked the question. One called this the “time element” – when you are asked the question will have a big impact on how you answer.

When asked if they had a preference, or which version was easier for them to answer, participants were split. A few preferred answering with the frequency scale but others felt that the difficulty scale meshed better with how they think about their CFS.
Several said a “Yes/No” version of the question would be easy to answer. One explicitly said that for those CFS sufferers who have trouble making decisions or handling multiple options, it would be easier to answer “Yes” or “No.” However, these participants also acknowledged that Yes/No response yield less information than a response distribution.
When asked what period of time they were responding about, participants’ responses varied, indicating that they used their own reference period or thought about a recent stable period for their health status, whether it was a few weeks or a longer span of years (e.g., the period of time since a relapse in 2007).

Q4: How often does your CFS make it difficult for you to work at a job or business? (All the time, Most of the time, Some of the time, Rarely, Never)

1. Were participants able to respond accurately to this question?

Almost all of the participants in this round said they are on disability (not able to work) or have not worked for some time. They tended to choose the most extreme point of the scale (frequency or difficulty) because they are unable to work.

2. How did participants react to the question?

One said that this question was a Yes/No question – that you are either able to work or not.

The participants who said they cannot work chose the most extreme response. This suggests that for survey respondents who are not working because they are unable to work, there may be some issues with responding to Question 4: both the difficulty scale and the frequency scale might seem illogical.

Recommendations for Questions 3 and 4

Retain the text we tried out in this round (“because of your CFS/ME…”) to ensure that respondents will base their judgment on that condition, rather than something else (since some have one or more other long-term conditions like fibromyalgia, or even short-term illnesses that they may also consider in their response unless this explicit text is provided).

To make the question more standard across survey respondents, use a reference period for reporting disability-- a reference period that would be of most interest to CDC’s CFS Research Program.

When talking about their disease progression, many participants described up and down cycles or slow steady decline over time. Several also talked about not knowing when they would have good days or bad days – these are unpredictable due to the nature of CFS. These types of fluctuations vary from individual to individual, so without specifying a reference period, survey respondents will just base their judgment on a meaningful timeframe of their own. Note that the interval should not be too long; participants also noted that memory issues can be a symptom for CFS sufferers. For example, 30 days may be long enough for to achieve some accuracy in reporting past behavior, while not taxing individuals who may have CFS-related memory issues.

Based on the issues with the scales that we tried out and CDC’s stated measurement concepts (the CDC’s Chronic Viral Disease proposal stated “homebound disability” and “employment disability”), we suggest pursuing development of a question using the following approach, and then testing it in Round 2.

Over the past 30 days, have you been able to:

1 Perform daily self-care activities? (Yes/No)

2 Provide care for others? (Yes/No)

3 Work at a job part time? (Yes/No)

4 Work at a job full time? (Yes/No)

Would a Yes/No approach provide CDC with adequate information? If so, we would tweak the wording to base reporting of these behavior-related items to CFS/ME.

Behavioral Risk Factor Surveillance System (BRFSS)

Chronic Fatigue Syndrome

Report on Round 2 Cognitive Interviewing

January 2014

Prepared for:

Centers for Disease Control and Prevention

Atlanta, Georgia

Prepared by:

WESTAT

Rockville, Maryland

Table of Contents

Section Page

1 BRFSS Background 1

2 Study Overview 2

3 Methodology 2

3.1 Protocol Development 3

4 Participant Recruitment 4

5 Expert Review 5

6 Findings and Recommendations 6

6.1 General Reactions to Questions 7

6.2 Question-by Question Findings and Recommendations 7

7 Overall Recommendations 22

List of Appendices

Appendix

A Participant Screener 24

B Cognitive Interview Protocol 29

1. BRFSS Background

The Centers for Disease Control and Prevention (CDC) conducts the Behavioral Risk Factor Surveillance System (BRFSS) survey every year in all 50 states, the District of Columbia, Guam, and the Virgin Islands. This large Random Digit Dialing (RDD) survey is used to collect information from a random sample of people over the age of 18. The topic areas covered by the survey include preventive health behaviors, behaviors that may put people at risk for chronic diseases or injuries, and health care access. The survey results are used to design a variety of health care programs and to revise health care policies.

In preparation for each BRFSS cycle, the CDC evaluates the need to modify the survey content. Various divisions and external stakeholders identify draft questions that might be added to the core as well as emerging topics to be considered for new rotating core or optional modules. Westat conducted a first round of in-person testing of the module during July 22 – 31, 2013. Westat drafted preliminary results and recommendations from Round 1 and discussed these with CDC on the telephone in early August. The second round of testing was conducted over the telephone from August 26 – September 4, 2013. Westat discussed these preliminary results with CDC in person on September 6.

Westat used a different methodology in Round 2 than in the first round. We conducted interviews by telephone, so the primary focus of the testing was respondent behavior related to telephone administration. This round also tested three different versions of the Homebound Disability questions. Additionally, the Employment Disability question was broken out of the Homebound question and became a separate, individual question that was tested on six participants.

This report summarizes the findings from Round 2. In the following pages, we describe the methods that we used to develop the interview protocols, recruit participants, and conduct the interviews (Methodology Section). In the Findings and Recommendations Section, we present the findings from this round of interviews and, where appropriate, offer recommendations for changes to the survey questions. Appendixes include the Round 2 materials: the general screener (Appendix A) used to identify eligible participants for the module and the cognitive interview protocol used during the sessions (Appendix B).

2. Study Overview

Westat conducted all Round 2 interviews by telephone. At the start of each interview, the interviewer administered the informed consent by reading scripted text which described the purpose of the project, the sponsor, the expected burden and risk to the respondent, and the procedures for the interview.

All interviews were audio recorded. The consent text described the purpose for the recording and asked permission to record. Once the participant consented to the interview and recording, the interviewer administered the interview protocol, including retrospective probes. At the conclusion of the session, the interviewer asked if the participant had any questions about what had been discussed. Then, the interviewer confirmed the participant’s address for mailing the $30 incentive check and thanked the respondent for participating.

After each interview, the survey methodologist reviewed notes and the recording from the interview to extract information about question administration, participant quotes, and question-related findings. We compiled these individual summaries to produce a comprehensive summary of results from the second round of testing. The analysis was conducted on the comprehensive summary that represents the entire content of all the interviews. The analysis used many of the basic principles and methods of qualitative research. The ultimate analysis criteria were the extent to which the respondent correctly understood question intent and the extent to which the response options accurately and precisely represented the respondent’s experience and situation.

3. Methodology

Two senior methodologists with expertise in cognitive testing techniques administered the protocol over the telephone to a total of nine participants. The questions were delivered in a fashion that mimics the actual conditions of a telephone survey. All probing was done retrospectively, that is, after the respondent had answered all of the survey questions. Nine health questions from the BRFSS core (see page 33 for the questions) were included for the sake of context and as a warm-up for the respondent. The interviewers only probed to determine comprehension and response accuracy for the CFS/ME and the Homebound and Employment Disability questions; the nine health questions from the BRFSS core were administered without probing. In the sections that following the Protocol Development section we describe the question pretesting methods in greater detail.

3.1 Protocol Development

Based on the findings from Round 1, we developed the protocol for Round 2. Questions 1 and 2 (“Have you ever been told by a doctor or other health professional that you had Chronic Fatigue Syndrome (CFS) or Myalgic Encephalomyelitis (ME)?” and “Do you still have CFS/ME?”) tested well in Round 1. In Round 2, we used the same types of basic probes for Questions 1 and 2 that were used in Round 1. We included scripted probes to capture the respondent’s experience of CFS because we felt the narrative gave context and meaning to the respondent’s answers. We also used the approach of asking the respondent what the question meant and what their answers meant. In addition to the scripted probes, we used emergent probing as needed to assess comprehension and response accuracy. This approach has significant theoretical and empirical underpinning in the cognitive interviewing literature as being nonbiasing and nondirective. We feel that we get the most accurate findings using this methodology.

Over the course of the testing with 9 participants, the Homebound Disability Questions were revised and tested in 3 different versions. After we learned of an issue with a question, we revised that question before proceeding with additional testing. The versions were developed sequentially based on the findings from the cognitive testing. As part of the question revision process, the Employment Disability part of the Homebound Disability Question was broken out into a separate question, with the aim of improving the accuracy of reporting. This is discussed in greater detail in Section 6.2. Also see Appendix B for the versions of questions used in Round 2 testing.

The task of the Round 2 testing was to test the Lifetime Prevalence (i.e., “Have you ever been told by a doctor or other health professional that you had ….” and Current Prevalence (“Do you still have Chronic Fatigue Syndrome or …”) and a Homebound/Employment Disability question. To provide context for the prevalence and disability questions, we included nine health questions from the BRFSS core (see Appendix B). It is considered good cognitive interviewing practice to give the respondent some warm-up questions before delivering the questions that are of analytic interest. This gives the respondent some training in the question asking/answering process and focuses the respondent on health-related questions. The nine warm-up questions from the BRFSS core were not probed on and in this sense, were not part of the cognitive testing. Only the CFS/ME Prevalence questions and Homebound and Employment Disability questions were probed. All analysis and findings refer to the CFS/ME Prevalence questions and Homebound and Employment Disability questions.

4. Participant Recruitment

For Round 2 testing, we used the same recruiting sources as for Round 1 – we posted on support group Internet sites and networked with members of the CFS community. Because this round was on the telephone, we were able to interview people from approximately five states with a significant geographic spread.

Nine eligible CFS/ME individuals were scheduled for a telephone interview. Table 1 shows selected demographic characteristics of the Round 2 participants and the version of the Homebound and Employment Disability question they received.

Table 1. Demographic Characteristics of Round 2 Participants

Gender	Age	Ethnicity	Education
Initial Set of Homebound & Employment Disability Questions
F	30-39	Asian	College
F	70	White	Some College
F	57	White	Post-graduate
First Revision to the Homebound & Employment Disability Questions
F	56	White	Post-graduate
F	67	White	Post-graduate
F	60 - 69	White	Post-graduate
Second Revision to the Homebound Disability Questions
F	51	White	Post-graduate
F	60 - 69	White	Some college
M	64	White	College

5. Expert Review

Before developing the protocol, Westat conducted an expert review on the set of questions received from CDC. The first two questions (i.e., “Have you ever been told by a doctor or other health professional that you had Chronic Fatigue Syndrome (CFS) or Myalgic Encephalomyelitis (ME)?” and “Do you still have CFS/ME?”) were deemed appropriate for telephone administration, but the Homebound and Employment Disability Questions were viewed as problematic. In particular, the stem and item design was verbally difficult to deliver over the telephone. The second issue was that the Yes/No response options were not particularly informative about the respondent’s condition. The original Homebound and Employment Disability questions in their revised format are displayed below:

Original Format of the Homebound and Employment Disability Question

Do you have difficulty in doing any of following activities:

Going outside the home alone to shop or visit a doctor's office?

(1) Yes

(2) No

(7) Don’t Know/Not Sure

(9) Refused

Working at a job or business?

(1) Yes

(2) No

(7) Don’t Know/Not Sure

(9) Refused

Revised Format of the Homebound and Employment Disability Question Used for First 3 Respondents

3. Thinking about your CFS, during the past 6 months, have you been able to leave your home to…

Visit a doctor’s office Yes/No

Go to some other medical appointment Yes/No

Go shopping Yes/No

Socialize with friends or family Yes/No

Work even a few hours at a job or business Yes/No

6. Findings and Recommendations from the Cognitive Interviews

The discussion in this section relates the findings from the Round 2 cognitive testing. First, we discuss the respondents’ general reactions to the questions and the fact that CFS/ME questions are being tested. Then we present the analysis of the performance of each question individually. To support our findings about the questions, we include direct quotations and paraphrasing of the respondents comments. Some of these will be more relevant and valuable to CDC than others. We have included quotations that demonstrate the broad spectrum of comments that the questions generated. We have done this to generate a complete and thorough record of the cognitive interviews.

6.1 General Reactions to Questions

Respondents expressed high levels of interest in participating in the interviews. They felt that their illness was very misunderstood and they wanted to make some contribution to clearing up the misunderstandings. A few of the respondents were extremely ill and expected a relapse as a consequence of the exertion of participating in the interview. They nevertheless wanted to feel that they had made some contribution to the cause of CFS. The participants were encouraged by the interest CDC was taking in their illness.

All of the participants were able to answer all of the questions. For all but one of the participants, there were no delivery issues detected with any of the versions tested. One participant declined to answer the homebound disability questions when asked to choose a response from a scale. On the other hand, the question flaws were revealed in the debriefing sessions. When the respondents were asked to explain their answers, the lack of correspondence between their answers and their actual situations and conditions became clear. For this reason, there were multiple attempts made to improve the Homebound and Employment Disability questions.

6.2 Question-by-Question Findings and Recommendations

In this section, we present the question that was tested. Then we provide a general comment on how the question functioned and show the distribution of the respondents’ answers in a table. After that, we provide quotations and comments that the respondents made to demonstrate our findings using the respondents’ own words.

CFS Prevalence Module

Question 1

Have you ever been told by a doctor or other health professional that you had Chronic Fatigue Syndrome (CFS) or Myalgic Encephalomyelitis (ME)?

Yes

Do not read:

7 Don’t know/Not sure

9 Refused

Findings

Interviewers noted no question delivery issues (e.g., participant requesting a repeat of the question or requesting clarification of the response options) for this question for phone administration. Participant responses are shown below. All of the participants provided a codeable response.

Number of Responses

Response

Code

Response Option

Yes

Based on their explanations during the retrospective debriefing, all nine respondents provided evidence that they reported accurately. Specifically, all nine respondents provided a narrative of receiving their diagnoses of CFS or ME. All of the narratives stressed that the diagnosis of CFS is exclusionary. The respondents described different onsets, but all have extreme fatigue, post-exertional malaise, sleep disorders, pain, cognitive issues, and no stamina. Paraphrases and quotes from these narratives are provided below. It is notable that the respondents used a variety of terms to refer to their illness – these terms are also listed below. Further, respondents provided various explanations for their choice of term to describe their illness.

Respondents’ narratives of receiving their diagnoses.

One respondent said that she was diagnosed in 2004. She had debilitating fatigue "such that [she] had to stop working… [she] tried to work, but got sicker and rundown." She had constant colds, viruses, sore throat. She gave up all exercise, despite previously being highly athletic. "[I] got sicker and more exhausted." Her symptoms include sleep disorders (abnormal sleep patterns), low blood pressure ("a hallmark of CFS"). All of her blood work fit the pattern for CFS and her main symptom is complete exhaustion.

Another respondent said "It was diagnosed in 1984" but she probably had it longer than that. "It is very debilitating. You do not get quality sleep. You are very tired all the time. Muscles ache. Your ability to think clearly is impacted. You are in pain all the time." It interferes with your quality of life. "I wake up and feel as if I never went to sleep in the first place." "It affects the whole body." This respondent added that she was completely bedridden a few days per week. To make the diagnosis of CFS, doctors do blood tests, follow-up, etc.

"Yes, I've been diagnosed" in 1990 or so. "I'm not typical but I have that as well as ME." "There is no specific marker to show it reliably"; other symptoms include irritable bowel syndrome and sleep disorders.

Another respondent has had CFS for over 30 years. She received the diagnosis after a process of elimination. She thinks her CFS came on gradually. She first became ill in 1982. Looking back, she had a number of viral infections. She had a number of cases of severe influenza and a very bad case of bronchitis. She has no idea how it started, but it just got worse and worse. Today, she is homebound, but not bedbound. CFS “is a fatigue that just never goes away”; her sleep is disturbed.

This respondent’s illness started 20 years ago when he was under a lot of stress from a number of situations. He started to have sleep problems. Then, about 2 years later, he went to bed one night and the next morning, he could not get out of bed. He was completely exhausted, in a lot of pain, and had a great deal of mental confusion. He could not navigate in his own neighborhood. Less than a week later, he went to his doctor and the doctor thought he had CFS. The respondent received a referral to a physician who was trying to help CFS patients. This doctor did the exclusionary tests, but still was not able to help much. Over the course of about 5 years, he recovered enough to be able to function in a relatively quiet environment for about 4 or 5 hours. He could never return to work. His health has not improved much in the last 10-12 years. He can do a few hours of low level activity. He is light and chemical sensitive. He has pain in his body, but he does not have fibromyalgia. He also has many chronic conditions. He has trouble sleeping and takes medication for his sleep problems.

One respondent reported that she had a severe flu, and she seemed to recover. But then her health turned and she grew increasingly ill. She stopped working in 1994. First she saw a neurologist for seizures and cognitive issues. Then in 1995 she went to Johns Hopkins Hospital and was diagnosed with CFS. She feels that her CFS is severe to very severe. She experiences severe post-exertional malaise. She can sustain activity for about an hour per day. She cannot stand up and move around for more than four hours a day. She experiences significant cognitive problems. She experiences a dehydration that cannot be replenished by fluid or salt, so she is treated with saline IVs.

The nine respondents use somewhat different terms to describe their illness. These terms include CFS, ME, CFIDS, and Chronic Fatigue. The comments below give examples of the way the respondents use these terms and what these terms mean to the respondents.

Two respondents said that they used the term CFS.

One respondent said that CFS was the diagnosis she was originally given and that is the term she uses. She added that ME entered the discussion about 10 years ago. In Europe and Canada, CFS is called ME. On the other hand, another respondent said that no one in the US has been given the diagnosis of ME.

The other respondent reported that she uses the term CFS. "If you use some other term, no one knows what you are talking about. I don't like the term. It is like a cancer patient being called CFS just because they are fatigued. ME is more precise. It emphasizes the cognitive and other symptoms other than the fatigue.”

One respondent used CF and said that if that generated confusion then she said CFS.

One respondent said that she primarily used CFS but also uses ME

“ … use Chronic Fatigue and Chronic Fatigue Syndrome. ME is less known.”

One respondent said that she used CFIDS, but also used ME and CFS to describe her condition.

This respondent reported using the term CFIDS, or CFS, but also uses ME.

Two respondents said that they used ME to describe their condition.

“ME is scientific, has merit, and is recognized by WHO. ME is used worldwide.”

The other respondent use Myalgic Encephalomyelitis because it sounds more medical -- "it sounds like something." The respondent said that there are problems associated with the name CFS. CFS has been trivialized.

Two patients said that they used Fibromyalgia to describe their condition. Fibromyalgia was a more recognized term. There has been much more on TV about Fibromyalgia. One of these respondents added that when she was diagnosed, the doctor said that Fibromyalgia, CFS, ME were all the same disease.

One of these respondents said that the term she uses to talk about CFS is "fibromyalgia." Fibromyalgia is the term you hear on TV. "Fibromyalgia covers it." The respondent said that ME is "some sort of unknown problem with muscles or soft tissue. It does not show up on an X-Ray or MRI." This respondent only uses the term fibromyalgia. She has seen the term ME in an article, but knows no one who uses it.

One respondent who uses the term "fibromyalgia" Said she has very sensitive points on her body so that is the reason she says that she has fibromyalgia. She received the diagnosis of CFS and Fibromyalgia at the same time with information that they were the same thing.

Question 2

Do you still have Chronic Fatigue Syndrome (CFS) or Myalgic Encephalomyelitis (ME)?

Yes

Do not read:

7 Don’t know/Not sure

9 Refused

Findings

Number of Responses

Response

Code

Response Option

Yes

Based on their explanations during the retrospective debriefing, all nine respondents provided evidence that they reported accurately – all still experience CFS symptoms and still consider themselves to have CFS. On the other hand, one respondent’s symptoms were relatively mild while another respondent’s symptoms were extremely severe. All nine respondents felt that asking whether they still had CFS was a reasonable question. Some individuals manage to recover: “Some cases resolve.” No one in our sample had resolved completely, thus the uniformity of response. The following sample of comments and quotations provides a sense of the continuation of the respondent’s CFS symptoms.

“I still have my symptoms. I'm still disabled… can't think like a normal person, do normal activities. I have to rest, pace myself, can’t go back to school, can't do a job. It’s extremely difficult to do cognitive activities."

"It has never gone away. It is always here."

"It hasn't gone into remission or disappeared."

"It's a lifelong thing."

"It means that I've gotten back to about 25% of my functionality and I have plateaued at that point."

"It hasn’t gone away in almost 20 years."

“I heard of a few who have some improvements - they are at a higher level of functioning -- even working part time. Very few fully recover; some get worse."

Two respondents made comments on the underlying diagnostic criteria. The comments are included for the sake of completeness, but may not be of primary interest to CDC. One respondent thought that the question should ask what diagnostic criteria had been used to make the determination of CFS. This respondent thought that it would be more meaningful to ask about the variety and intensity level of the CFS. Another respondent thought that it would be more meaningful to ask about the severity of the CFS. Both of these comments are shown below.

This respondent said that there were many different definitions of CFS -- if you are going to ask people about CFS, you have to specify which definition you are using. This respondent was so sick that she qualifies for all of the CFS definitions, including the one for ME. This respondent thought the question was too broad -- it does not distinguish among the different varieties and intensity levels.

The other respondent said that CFS has a variable course. The milder cases resolve. Some CFS cases are "normal with some limitations." This respondent added that it was not particularly meaningful to ask whether people “still” have it. It is more meaningful to ask about severity.

First Revised Design of the Homebound and Employment Disability Question Used for First 3 Respondents

The question below represents the first revised question used to ask the Homebound and Employment Disability Question for the first 3 respondents in Round 2 testing.

Question 3

Thinking about your CFS, during the past 6 months, have you been able to leave your home to …

1. Visit a doctor’s office Yes/No

2. Go to some other medical appointment Yes/No

Go shopping Yes/No

Socialize with friends or family Yes/No

Work even a few hours at a job or business Yes/No

Do not read:

7. Don’t know/Not sure

9. Refused

Findings

Respondent Number	Responses
1	Yes, Yes, Yes, Yes, No
2	Yes, Yes, Yes, Yes, No
3	Yes, No, No, No, No

Even though the three respondents were able to answer this question and there were no question comprehension issues, Westat nonetheless felt that this question did not perform well. The main issue was that this question failed to capture whether individuals with CFS could do these things (visit a doctor’s office, go to some other type of medical appointment, go shopping, socialize with friends or family, work even a few hours at a job) on their own or whether they needed assistance. The “Yes” responses do not represent ability to perform these tasks independently. The “Yes” responses represent what CFS individuals are able to do with assistance. In some cases, the assistance was of such magnitude that one truly marveled at the “Yes” response. The comments and quotations below demonstrate the degree to which the “Yes” responses represent assistance.

This respondent reported that she can do the things named in the question only with help. Her spouse helps or she would not be able to do any of the things listed. She worries about what would happen if the current level of help were not available. "I would be homebound without help."

Another respondent said that she can only leave home once a year and that is to visit the doctor to get an order for a home-health aid renewed. Leaving home is a very great effort where she has to lie on a mattress in a van with earplugs and blinders. The respondent cannot deal with sensory input. This annual visit to the doctor always triggers a severe relapse. This respondent has gone years completely unable to leave home even to see a doctor. This respondent’s illness has been progressive with continuous deterioration over time.

The third respondent who received this question said that she was normally at home for weeks at a time. To leave home, she has to be in a wheelchair and a parent accompanies her.

Recommendation

To address the issues presented by respondents reporting that they could do various activities, when in actuality, they could accomplish these activities only with assistance, Westat rewrote the Homebound and Employment Disability Question and split it into 3 parts: (1) activities that can be accomplished alone; (2) activities that can be accomplished with assistance, and (3) hours worked on average per week. The hours worked per week became Question 4 which is discussed separately from Question 3a and 3b.

The questions below represent the second revised format used to ask the Homebound and Employment Disability Question for the second set of three respondents in Round 2 testing.

Questions 3a and 3b

3a.Thinking about your CFS, during the past 6 months, have you been able to leave your home alone to …

1. Visit a doctor’s office Yes/No

2. Go to some other medical appointment Yes/No

3. Attend events or social activities Yes/No

Do not read:

7. Don’t know/Not sure

9. Refused

FOR ANY “NO” RESPONSES TO Q3A, PLEASE DELIVER 3B.

3b.Thinking about your CFS, during the past 6 months, have you been able to leave your home with assistance from someone else to ….

1. Visit a doctor’s office Yes/No

2. Go to some other medical appointment Yes/No

3. Attend events or social activities Yes/No

Do not read:

7. Don’t know/Not sure

9. Refused

Responses to Question 3a

Respondent Number	Responses
4	Yes, No, No
5	No, No, No
6	Yes, No, No

Responses to Question 3b

Respondent Number	Responses
4	Yes, Yes, No
5	Yes, No, No
6	Yes, Yes, Yes

Findings

The revised Question 3a and 3b functioned well as far as comprehension. All three respondents understood the question’s intent and were able to produce a codeable response. When asked to paraphrase the question, the respondents said that the question meant:

“Whether I was able to drive myself to any kind of medical appointment."
"The question is asking whether or not I am able to get myself to a grocery store or the doctor's office without assistance.”

The respondents offered the following comments about Question 3a and 3b and the following explanations of their answers.

One respondent commented that the question was not specific enough. "I take care of basic medical needs. Socializing is important, but not something you can do very frequently."

Another respondent emphasized that she cannot do anything that requires driving. The questions made her think that it was asking about whether she could drive herself to do these tasks. Driving is not possible for this respondent. This respondent answered “No” to things that she can do because she can walk short distances. In a number of cases, this respondent should have answered “Yes” because her doctor’s office is very close and she can walk. The respondent nevertheless answered negatively because the question made her think of driving. The driving a car assumption went through both Question 3a and Question 3b.

Another respondent said “Yes” to some of the items in Question 3a, but when she explained her answers, it was obvious that a more accurate answer would have been “No.” This respondent chose the "Yes" answer based on a number of assumptions that would be difficult to make happen all on the same day. For example, it would have to be an absolute superlative day for her; the distance to the doctor’s would have to be very short. Even on her best day, she cannot process the amount of information that it takes to make a long drive. Additionally, she would have "to have the stamina to sit in a doctor's office and wait to be seen and still have the energy to get home." Further, this respondent reported that she can go grocery shopping -- but that means that she can only buy 4 or 5 items because of the over stimulation of the grocery store. A grocery store has many colors and noises that are overwhelming. We interpreted this respondent’s “Yes” to actually mean that she can go to the grocery store “Sometimes” or “Rarely.”

The revised Question 3a and 3b performed so poorly that Westat could see no value in testing it further or putting more work into it. The testing has demonstrated that CFS/ME is a highly variable illness with great variation day to day, week to week, and month to month. Any question that fails to capture the variation will be subject to very high levels of measurement error. Based on this understanding, Westat revised Question 3a and 3b to the question shown below. The question shown below retained the distinction of what a CFS/ME individual can do alone or with assistance, but it adds a frequency scale in an attempt to capture some of the variability in what CFS individuals are able to do. Questions 4a, 4b, and 4c are follow-up questions in the event that a respondent answers “no” to any of Questions 3a, 3b, and 3c.

Third Revised Homebound Disability Questions 3a, 3b, 3c, 4a, 4b, and 4c

3. Thinking about your CFS, during the past 6 months, how often have you been able to leave your home alone to…

3a Visit a doctor’s office

All of the time

Most of the time
Some of the time

None of the time

3b Go shopping

All of the time

Most of the time

Some of the time

None of the time

3c Attend events or social activities

All of the time

Most of the time
Some of the time

None of the time

Responses to Question 3

Respondent Number	Responses to Qx 3a	Responses to Qx 3b	Responses to Qx 3c
7	No	Declined	Declined
8	All of the time	All of the time	All of the time
9	Most of the time	Most of the time	Some of the time

FOR ANY “NONE OF THE TIME” RESPONSES TO Q3, ADMINISTER CORRESPONDING PART OF Q4:

4. Thinking about your CFS, during the past 6 months, how often have you been able to leave your home with assistance from someone else to…

4a Visit a doctor’s office

All of the time
Most of the time

Some of the time
None of the time

4b Go shopping

All of the time

Most of the time
Some of the time
None of the time

4c Attend events or social activities

All of the time

Most of the time
Some of the time

None of the time

Responses to Question 4

Respondent Number	Responses to Qx 4a	Responses to Qx 4b	Responses to Qx 4c
7	Declined	Declined	Declined
8	Skipped out	Skipped out	Skipped out
9	Skipped out	Skipped out	Skipped out

Findings for Questions 3a, 3b, 3c, 4a, 4b, and 4c

The revised Question 3a, 3b, 3c, 4a, 4b, and 4c functioned well as far as comprehension. All three respondents understood the question’s intent; two were able to produce a codeable response and the third declined to answer, saying that the scale was not appropriate for her. When asked to paraphrase the question, the respondents said that the question was asking whether they could get out to do the various tasks alone without assistance. On the other hand, the improvement that we expected to see by the inclusion of a frequency scale was not borne out. The respondents expected the response options to capture what they wanted to tell us about their illness. The frequency scale did not capture and express the way the respondents experience their CFS. For this reason, we think the frequency scale should be omitted.

One respondent commented that the extreme variability of the illness makes it very difficult to determine whether the person is on an upward or downward trend. It is important to gauge the direction of the illness. Further, this respondent said that asking about the last 30 days was too short of a reference period. A CFS individual can be doing better for the last couple of months, but the overall trend could be in the opposite direction.

It is worth noting that the respondent who had the easiest time answering Question 3a, 3b, and 3c said that her condition had been relatively stable over the last 6 months, so answering was not a problem.

None of the 3 respondents who received Question 3a, 3b, and 3c proceeded to Question 4a, 4b, and 4c. For the two who skipped out of Question 4a, 4b, and 4c did so correctly because they had answered “All, Most, or Some of the Time.” The respondent who declined felt that the response options failed to capture her experience of CFS and thus declined to answer.

Recommendation

The Homebound Disability question tested on the final three respondents did well for comprehension, but not so well for the response options. The frequency scale did not reflect what the respondents wanted to tell us about their illness and for this reason, we suggest changing the frequency scale to a simple “Yes/No” response option. See the question suggested below.

3. Thinking about your CFS, during the past 6 months, how often have you been able to leave your home alone to…

3a Visit a doctor’s office Yes/No

3b Go shopping Yes/No

3c Attend events or social activities Yes/No

FOR ANY “NONE OF THE TIME” RESPONSES TO Q3, ADMINISTER CORRESPONDING PART OF Q4:

4. Thinking about your CFS, during the past 6 months, how often have you been able to leave your home with assistance from someone else to…

4a Visit a doctor’s office Yes/No

4b Go shopping Yes/No

4c Attend events or social activities Yes/No

Revised Employment Disability Question Broken out as a Separate Question

As part of the revision of the Homebound Disability Question, the part that asked about employment was broken out into a separate question – see question 5 below. This question was asked of a total of six respondents.

5. Thinking about your CFS, during the past 6 months, how many hours a week on average have you been able to work at a job or business for pay?

1 0 or no hours -- cannot work at all because of my CFS

2 1 - 10 hours a week

3 11- 20 hours a week

4 21- 30 hours a week

5 31 - 40 hours a week

Do not read:

7 Don’t know / Not sure

9 Refused

All six respondents reported zero hours of work per week. No one in our sample of CFS/ME individuals worked. All were either retired or on disability. Everyone understood the question and answered correctly given the question intent. Given that no one in the sample worked at all, we are not able to provide detailed information on the functioning of this question. We expect that if this question were fielded on a population that worked at least some hours per week, it would function adequately since it is similar to the other work for pay questions that are commonly fielded.

7. Overall Recommendations

Throughout Round 1 and Round 2, Questions 1 and 2 (“Have you ever been told by a doctor or other health professional that you had Chronic Fatigue Syndrome (CFS) or Myalgic Encephalomyelitis (ME)?” and “Do you still have Chronic Fatigue Syndrome (CFS) or Myalgic Encephalomyelitis (ME)?”) functioned well and demonstrated robust properties across both in-person and telephone testing. We recommend their use in any survey CDC fields them in.

Based on our discussions with the CDC staff working on developing the CFS optional module, CDC would like to obtain the following measures for homebound disability:

A prevalence estimate for ability to leave home alone (for each of 3 activities) within the prior six-month period, and for those it applies to, a follow-up
A prevalence estimate for ability to leave home with assistance (for each of the same three activities and the same time period)

If asked as Yes/No questions, these would be similar to many other BRFSS items, which measure prevalence rather than a frequency (e.g., how often able to leave home alone, how many times able to leave home alone). However, we caution that the Yes/No versions of these Homebound Disability questions have not been adequately tested on actual test participants, so we do not have empirical evidence to support a strong recommendation to proceed with fielding these versions. We recommend further testing of the Yes/No versions to collect empirical evidence.

We also believe that the Employment Disability question will work in the field since our test respondents understood it and voiced no objections.

Appendix A

Participant Screener

INTRO

Hello, my name is ______. I am calling on behalf of Westat, a research company in Rockville, Maryland. Westat is conducting a research project for the Centers for Disease Control and Prevention. Are you interested in helping us test several questions for an upcoming national survey? We want to ask people with your health condition if the questions are easy to understand and answer, or if we need to make changes to the questions before the survey actually goes out.

Would you be interested in helping with this project?

No – GO TO THANK AND TERMINATE (“Thank you anyway for your time today, and have a great day.”)

Yes – Great, thank you. Now I just need to make sure that you meet the requirements of the study, so I need to ask you some questions.

Q1. Our first round of interviews will be taking place at Westat’s headquarters, located off of Route 28 -- on Research Boulevard in Rockville, Maryland. Are you within driving or commuting distance of our location?

PROVIDE MORE INFO IF NEEDED: 1600 RESEARCH; NEAR THE CORNER OF RESEARCH AND GUDE DRIVE; 63 OR 54 BUS STOPS IN FRONT OF 1600; ETC.

Yes [GO TO Q2]
No, not within driving distance of Westat  We will be conducting Phase 2 of this study by telephone in several weeks. Would you be available for a telephone interview during [FILLWITH EXPECTED TIMEFRAME FOR PHONE INTERVIEWS]?
1. Yes  Okay, I just have a few questions to see if you are eligible for the study. [GO TO Q2]
2. No  THANK AND TERMINATE (Thank you for taking the time to talk with me today. You are not eligible for this study. Have a great (day/evening)).

Q2. ASK ONLY IF NOT OBVIOUS: Are you male or female?

Male
Female

INTERVIEWER: RECRUIT A MIX OF BOTH SEXES.

Q3. Which of the following age categories are you in?

Under 18 – TERMINATE AND THANK (“Based on the requirements of this study, it looks like we can’t include you at this time.”)
18 – 29
30 – 39
40 – 49
50 – 59
60 – 69
70 or over

INTERVIEWER: WE WANT TO RECRUIT A MIX OF AGES.

Q4. Are you of Hispanic or Latino Origin?

a. Yes

b. No

INTERVIEWER: RECRUIT A MIX.

Q5. Are you…READ OPTIONS ALOUD

White
Black or African American
Asian/Pacific Islander
American Indian or Alaskan Native
Other? ______________________

INTERVIEWER: RECRUIT A MIX OF RACES.

Q6. What is the highest level of education you have completed?

11^th grade or less
12 years of school, no diploma
GED or high school graduate (diploma)
Some college or technical school
College or technical school graduate
Post-graduate

INTERVIEWER: RECRUIT A MIX.

Q7. Have you ever been diagnosed with Chronic Fatigue Syndrome (CFS) or Myalgic Encephalomyelitis (ME)?

Yes
No  THANK AND TERMINATE (Thank you for taking the time to talk with me today. You are not eligible for this study. Have a great (day/evening)).

INTERVIEWER: ONLY THOSE WHO HAVE HAD CFS AT SOME POINT ARE ELIGIBLE.

Q8. Do you currently have Chronic Fatigue Syndrome (CFS) or Myalgic Encephalomyelitis (ME)?

THANK AND TERMINATE

Thank you for taking the time to talk with me today. You are not eligible for this study. Have a great (day/evening).

END

[IF ELIGIBLE] You may be are eligible to help with this research project. We will call you back to schedule a specific time. You will receive [$50 cash upon completion of the in-person interview / $30 check upon completion of the telephone interview]. Are you interested in participating?

YES — Thank you so much for your willingness to help us. [IF PERSON MEETS TARGET CRITERIA, SCHEDULE FOR A TIME SLOT AS SOON AS POSSIBLE.]

[VERIFY THAT PHONE NUMBER IS BEST ONE TO REACH R, AND GET A BACKUP NUMBER IN CASE OF NEED TO REMIND RESPONDENT OF APPOINTMENT TIME. ALSO GET EMAIL ADDRESS.]

No — Thank you for taking the time to talk with me today and have a great (day/evening).

Appendix B

Cognitive Interview Protocol

Date/Time:_______________ ID:______ Interviewer:_______________

Cognitive Testing of Questions for CDC/BRFSS

Thank you for taking the time to help us out today. First, I’ll give you a little background about what we’ll be doing today.

Westat is working on this project for the Centers for Disease Control and Prevention (CDC). Every year, the Behavioral Risk Factor Surveillance System (BRFSS) survey is conducted in all 50 states, the District of Columbia, Guam, and the Virgin Islands. This large survey is used to collect information from a random sample of people 18 years old or older. The topics covered by the survey include preventive health behaviors, behaviors that may put people at risk for chronic diseases or injuries, and access to health care. The information from the survey is used to design health care programs and policies.

We here at Westat are helping CDC test questions that may be added to the survey. It's important that we learn how people interpret and respond to the new questions so that we can improve them before they are used in the real study.

We want to make sure that people taking part in the survey find the questions easy to understand and respond to. Today, I will ask you to answer questions, and at the end of a set of related questions I will stop and ask you to share your reactions about them. Hearing your reactions will help us learn how to improve the questions.

CONSENT

READ CONSENT FORM.

With your permission, I would like to record the interview so that I have an accurate account of what you say in case I miss anything while taking notes. [START RECORDER AND GET ORAL PERMISSION TO RECORD.]

CLOSING

We will mail a check to [GET ADDRESS]:

Name: _____________________

Address: ___________________________

___________________________

Do you have any questions before we get started?

END: That’s the last question. We appreciate your help today. Thank you for your time.

Consent Form

Purpose: We are inviting you to participate in a research activity. The research involves testing some questions about Chronic Fatigue Syndrome that are being developed for a large national health survey. The feedback will help us to develop recommendations for improving the survey.

Sponsorship: Westat is conducting this project on behalf of, and funded by, the Centers for Disease Control and Prevention (CDC).

What is involved: We will be asking you to answer some survey items, and then answering some questions about those items. Your participation in this research project is voluntary, and you have the right to stop at any time or to refuse to answer any question. The session will take approximately 30-60 minutes.

Confidentiality: We would like to record the interview. Sometimes it is helpful to review a portion of a recording as we make recommendations for improving the survey. If the recording is reviewed later, it will only be by a few Westat staff and possibly the client. During this session, we may have a note-taker in the next room or an observer representing the CDC.

You will never be identified by name. The things you say may be put in a written summary of this discussion, but there will be no way to identify who said what, and your name will not be used anywhere.

Risks: There are no known risks for participation in this research activity. You may skip any question(s) that you do not want to answer, or that you find uncomfortable to talk about, both in the questionnaire and in the discussion afterwards. The information you provide will be treated as confidential. The recordings will be destroyed within 6 weeks of the end of the study.

Benefits: There are no direct benefits to you for participating in this study. However, you will be helping with an important research project.

Questions: If you have questions about the project, you may call the Task Manager, Jennifer Crafts, at 800-937-8281, Ext. 4881. If you have questions about your rights and welfare as a research participant, please call the Westat Human Subjects Protection office at 1-888-920-7631. Please leave a message with your full name, the name of the research study that you are calling about, and a phone number beginning with the area code. Someone will return your call as soon as possible.

You will be sent a $30 check for completing the session.

[ASK FOR CONSENT TO PARTICIPATE AND TO BE RECORDED.]

PROTOCOL: Chronic Fatigue Syndrome Questions

Note to interviewers: Administer the Section 6 and Section 7 questions first for context only; no probing. Conduct retrospective probing for new questions about CFS.

Current BRFSS questions from Module 6 & 7 of Core

6.1 Blood cholesterol is a fatty substance found in the blood. Have you EVER had your blood cholesterol checked?

1 Yes

2 No [Go to next section]

Do not read:

7 Don’t know / Not sure [Go to next section]

9 Refused [Go to next section]

6.2 About how long has it been since you last had your blood cholesterol checked?

Read only if necessary:

1 Within the past year (anytime less than 12 months ago)

2 Within the past 2 years (1 year but less than 2 years ago)

3 Within the past 5 years (2 years but less than 5 years ago)

4 5 or more years ago

7 Don’t know / Not sure

9 Refused

6.3 Have you EVER been told by a doctor, nurse or other health professional that your blood cholesterol is high?

1 Yes

2 No

7 Don’t know / Not sure

9 Refused

Now I would like to ask you some questions about general health conditions. Has a doctor, nurse, or other health professional EVER told you that you had any of the following? For each, tell me “Yes,” “No,” or you’re “Not sure.”

7.1 (Ever told) you that you had a heart attack also called a myocardial infarction?

1 Yes

2 No

7 Don’t know / Not sure

9 Refused

7.2 (Ever told) you had angina or coronary heart disease?

1 Yes

2 No

7 Don’t know / Not sure

9 Refused

7.3 (Ever told) you had a stroke?

1 Yes

2 No

7 Don’t know / Not sure

9 Refused

7.4 (Ever told) you had asthma?

1 Yes

2 No [Go to Q7.6]

7 Don’t know / Not sure [Go to Q7.6]

9 Refused [Go to Q7.6]

7.5 Do you still have asthma?

1 Yes

2 No

7 Don’t know / Not sure

9 Refused

7.6 (Ever told) you had skin cancer?

1 Yes

2 No

7 Don’t know / Not sure

9 Refused

7.7 (Ever told) you had any other types of cancer?

1 Yes

2 No

7 Don’t know / Not sure

9 Refused

7.8 (Ever told) you have Chronic Obstructive Pulmonary Disease or COPD, emphysema or chronic bronchitis?

1 Yes

2 No

7 Don’t know / Not sure

9 Refused

7.9 (Ever told) you have some form of arthritis, rheumatoid arthritis, gout, lupus, or fibromyalgia?

1 Yes

2 No

7 Don’t know / Not sure

9 Refused

INTERVIEWER NOTE: Arthritis diagnoses include: rheumatism, polymyalgia rheumatic; osteoarthritis (not osteoporosis); tendonitis, bursitis, bunion, tennis elbow; carpal tunnel syndrome, tarsal tunnel syndrome; joint infection, Reiter’s syndrome; ankylosing spondylitis; spondylosis; rotator cuff syndrome

connective tissue disease, scleroderma, polymyositis, Raynaud’s syndrome; vasculitis (giant cell arteritis, Henoch-Schonlein purpura, Wegener’s granulomatosis; polyarteritis nodosa).

CFS Prevalence Module

Have you ever been told by a doctor or other health professional that you had Chronic Fatigue Syndrome (CFS) or Myalgic Encephalomyelitis (ME)?

YES
NO [END – WAS NOT ELIGIBLE FOR THIS TEST]

Do not read:

Don’t Know/Not Sure

Refused

Do you still have Chronic Fatigue Syndrome (CFS) or Myalgic Encephalomyelitis (ME)?

(1) YES [GO TO 3 and 4]

(2) NO [FOR TESTING PURPOSES, CONTINUE ON TO 3 and 4; ASK P TO RESPOND ABOUT THE TIME S/HE DID HAVE CFS SO WE CAN GET FEEDBACK ABOUT THEM; NOTE Ps WERE SCREENED TO STILL HAVE CFS]

(7) Don’t Know/Not Sure

(9) Refused

IF YES FOR Q2, ADMINSTER Q3

3. Thinking about your CFS, during the past 6 months, have you been able to leave your home to…

Visit a doctor’s office Yes/No
Go to some other medical appointment Yes/No
Go shopping Yes/No
Socialize with friends or family Yes/No
Work even a few hours at a job or business Yes/No

RETROSPECTIVE PROBING FOR STORY ON Q1:

For this question: “Have you ever been told by a doctor or other health professional that you had Chronic Fatigue Syndrome (CFS) or Myalgic Encephalomyelitis (ME)? you answered “~~”. Can you say more about that?

IF NEEDED:

Can you tell me about your CFS?

How severe would you say your CFS is?

How long have you had CFS?

How does someone know whether they have CFS?

What does it mean to “have ever been told by a doctor or other health professional”?

Can you say in your own words what “Chronic Fatigue Syndrome” (CFS) is?

Is CFS a term that you use or do you use some other term?

Can you say in your own words what Myalgic Encephalomyelitis (ME) is?

Is ME a term that you use or do you use some other term?

RETROSPECTIVE PROBING FOR STORY ON Q2:

For the question: “Do you still have Chronic Fatigue Syndrome (CFS) or Myalgic Encephalomyelitis (ME)?” you answered “~~”. Can you say more about that?

What does it mean to “still have CFS”?

How does someone know whether they “still have CFS or ME”?

IF NEEDED:

Can you talk about how CFS plays out?

[IS IT SOMETHING THAT GOES AWAY OR DO YOU HAVE IT FOREVER?]

The last question about CFS/ME was:

Thinking about your CFS, during the past 6 months, have you been able to leave your home to…

Visit a doctor’s office Yes/No
Go to some other medical appointment Yes/No
Go shopping Yes/No
Socialize with friends or family Yes/No
Work even a few hours at a job or business Yes/No

Can you say in your own words what this question is asking for?

FOR EACH PART 1 - 5:

For [~~] you answered “[Yes/No].” Can you say more about that?

ASSESS HOW THE REFERENCE PERIOD WORKS:

What period of time were you thinking about when you answered…?

How easy or difficult was it to decide on your answers for this period of time [during the past 6 months]?

Why?
Can you say more about that?

What does “have you been able to leave your home” mean to you?

IF NEEDED:

Can you talk about what it takes for you to “be able to leave your home…?

Revised Homebound and Employment Disability Protocol – Used with the Second Set of 3 Respondents

3. Thinking about your CFS, during the past 6 months, have you been able to leave your home alone to…

Visit a doctor’s office Yes/No/NA --no doctor visit in past 6 mths
Go shopping Yes/No
Attend events or social activities Yes/No

FOR ANY NO RESPONSES TO Q3:

4. Thinking about your CFS, during the past 6 months, have you been able to leave your home with assistance from someone else to…

1 Visit a doctor’s office Yes/No

2 Go shopping Yes/No

3 Attend events or social activities Yes/No

5. Thinking about your CFS, during the past 6 months, how many hours a week on average have you been able to work at a job or business for pay?

1 0 or no hours -- cannot work at all because of my CFS

2 1 - 10 hours a week

3 11- 20 hours a week

4 21- 30 hours a week

5 31 - 40 hours a week

Do not read:

7 Don’t know / Not sure

9 Refused

The next question about CFS/ME was:

3. Thinking about your CFS, during the past 6 months, have you been able to leave your home alone to…

1 Visit a doctor’s office Yes/No

2 Go shopping Yes/No

3 Attend events or social activities Yes/No

Can you say in your own words what this question is asking for?

FOR EACH PART 1 - 3:

For [~~] you answered “[Yes/No].” Can you say more about that?

ASSESS HOW THE REFERENCE PERIOD WORKS:

What period of time were you thinking about when you answered…?

How easy or difficult was it to decide on your answers for this period of time [during the past 6 months]?

Why?
Can you say more about that?

What does “have you been able to leave your home alone” mean to you?

IF NEEDED:

Can you talk about what it takes for you to “be able to leave your home alone…?

4. Thinking about your CFS, during the past 6 months, have you been able to leave your home with assistance from someone else to…

1 Visit a doctor’s office Yes/No

2 Go shopping Yes/No

3 Attend events or social activities Yes/No

Can you say in your own words what this question is asking for?

IF NEEDED:

How is it the same or different from the last question we just talked about?

FOR EACH PART THAT WAS ASKED:

For [~~] you answered “[Yes/No].” Can you say more about that?

Can you talk about what “able to leave your home with assistance from someone else” means to you…?

Second Revised Formats for the Homebound Disability Questions

3. Thinking about your CFS, during the past 6 months, how often have you been able to leave your home alone to…

3a Visit a doctor’s office

All of the time
Most of the time
Some of the time
None of the time

NA --no doctor visit in past 6 mths

3b Go shopping

All of the time
Most of the time
Some of the time
None of the time

3c Attend events or social activities

All of the time
Most of the time
Some of the time
None of the time

FOR ANY “NONE OF THE TIME” RESPONSES TO Q3, ADMINISTER CORRESPONDING PART OF Q4:

4. Thinking about your CFS, during the past 6 months, how often have you been able to leave your home with assistance from someone else to…

4a Visit a doctor’s office

All of the time
Most of the time
Some of the time
None of the time

NA --no doctor visit in past 6 mths

4b Go shopping

All of the time
Most of the time
Some of the time
None of the time

4c Attend events or social activities

All of the time
Most of the time
Some of the time
None of the time

5. Thinking about your CFS, during the past 6 months, how many hours a week on average have you been able to work at a job or business for pay?

1 0 or no hours -- cannot work at all because of my CFS

2 1 - 10 hours a week

3 11- 20 hours a week

4 21- 30 hours a week

5 31 - 40 hours a week

Do not read:

7 Don’t know / Not sure

9 Refused

The next question about CFS was:

3. Thinking about your CFS, during the past 6 months, how often have you been able to leave your home alone to…

1 Visit a doctor’s office

2 Go shopping

3 Attend events or social activities

Can you say in your own words what this question is asking for?

FOR EACH PART 1 - 3:

For [~~] you answered “[ ].” Can you say more about that?

ASSESS HOW THE REFERENCE PERIOD WORKS:

What period of time were you thinking about when you answered…?

How easy or difficult was it to decide on your answers for this period of time [during the past 6 months]?

Why?
Can you say more about that?

What does “have you been able to leave your home alone” mean to you?

IF NEEDED:

Can you talk about what it takes for you to “be able to leave your home…?

Frequency Scale:

All of the time
Most of the time
Some of the time
None of the time

Can you please talk about how well the scale worked for you? All of the time, Most of the time, Some of the time, None of the time – did these work for you? Did they allow you to say what you want to tell us about your CFS/ME?

4. Thinking about your CFS, during the past 6 months, how often have you been able to leave your home with assistance from someone else to…

1 Visit a doctor’s office

2 Go shopping

3 Attend events or social activities

Can you say in your own words what this question is asking for?

IF NEEDED:

How is it the same or different from the last question we just talked about?

FOR EACH PART THAT WAS ASKED:

For [~~] you answered “[ ].” Can you say more about that?

Frequency Scale:

All of the time
Most of the time
Some of the time
None of the time

Adverse Childhood Experiences (ACE) Questions

Behavioral Risk Factor Surveillance System (BRFSS):

Round 1 and 2 Results of Cognitive Testing of Proposed Items for 2008 Survey

FINAL REPORT

Submitted to:

Ruth Jiles, M.S., MPH., Ph.D.

Behavioral Surveillance Branch

Division of Adult and Community Health

National Center for Chronic Disease Prevention and Health Promotion

Centers for Disease Control and Prevention (CDC)

Submitted by:

RTI International

3040 Cornwallis Road

P.O. Box 12194

Research Triangle Park, NC 27709-2194

RTI Project Numbers 08654.028.001 & .002

September 29, 2006

Behavioral Risk Factor Surveillance System (BRFSS):

Results of Round 1 Results of Cognitive Testing of Proposed Items for 2008 Survey

FINAL REPORT

September 29, 2006

Submitted to:

Ruth Jiles, M.S., MPH., Ph.D.

Behavioral Surveillance Branch

Division of Adult and Community Health

National Center for Chronic Disease Prevention and Health Promotion

Centers for Disease Control and Prevention (CDC)

PREPARED BY:

Michael Schwerin, Ph.D.

Doug Currivan, Ph.D.

Emily McFarlane, M.A.

Submitted by:

Michael Schwerin, Ph.D.

RTI International*

Survey Research Division

Research Triangle Park, NC 27709

RTI Project Numbers: 0208654.028.000.001 & .002

*RTI International is a trade name of Research Triangle Institute.

Contents

Section Page

1. Introduction 1

1.1 Background 1

1.2 Methodology 1

1.3 Instrument/Questions 3

1.4 Participants 5

1.5 Procedures 6

2. Results 9

2.1 General Themes—Round 1 9

2.1.3 Adverse Childhood Experiences 12

2.2 General Themes—Round 2 14

2.2.3 Adverse Childhood Experiences 19

2.3 Question-by-Question Comments—Round 2 22

2.4 Recommendations 49

2.5 Limitations 51

References 53

Appendix

A Question Appraisal System (QAS-99) Review of 2007 BRFSS Questions A-1

B Participant Informed Consent Form B-1

C BRFSS Cognitive Interviewing Protocol—Round 1 C-1

D Question-by-Question Results for Round 1 Cognitive Testing D-1

E BRFSS Cognitive Interviewing Protocol—Round 2 E-1

List of Tables

Table Page

Table 1. Summary of Round 1 Recommendations for Revision to Proposed New BRFSS Items 3

Table 2. Cognitive Interview Participant Characteristics—Rounds 1 and 2 6

Table 3. Summary of Final Recommendations for Revision to Proposed New BRFSS Items 49

1. Introduction

1.1 Background

The Behavioral Risk Factor Surveillance System (BRFSS), which was established in 1984 and began national data collection in 1994, was developed to collect prevalence data on risk behaviors and preventive health practices that affect health status (Centers for Disease Control and Prevention [CDC], 2005). BRFSS survey data are collected annually, and results are used for health promotion and disease prevention program planning, development, and evaluation. The BRFSS survey consists of a core interview (i.e., a fixed core, a rotating core, and emerging core questions that reflect newer issues), optional modules, and state-added questions. RTI International conducts a variety of activities for the BRFSS, including survey item appraisal and cognitive testing.

Cognitive testing for this project occurs in two rounds: (1) in-person cognitive interviews and (2) cognitive testing over the telephone. For Round 1 cognitive testing, CDC asked RTI to test new items proposed for six BRFSS modules: Adverse Childhood Experiences. Across these modules, RTI tested a total of 37 questions, 35 of which were new. Eight people participated in Round 1 testing. For Round 2 testing, RTI tested CDC revisions to the previously described modules/items. A total of 18 participants participated in Round 2 testing.

This report summarizes the results of Round 1 and Round 2 cognitive testing conducted using the additional questions proposed for the 2008 BRFSS. This section of the report is organized into the following sections: Methodology, Participants, and Procedure. Section 2, Results, is devoted primarily to general themes (Rounds 1 and 2) and question-by-question comments for Round 2 testing, followed by recommendations. Appendixes include the question appraisal conducted prior to cognitive testing, the cognitive interviewing protocols (Rounds 1 and 2), Round 1 question-by-question cognitive testing results, and the participant informed consent form.

1.2 Methodology

Prior to conducting cognitive interviews, the BRFSS modules were evaluated using RTI's Question Appraisal System (QAS). The QAS is a structured, standardized instrument review methodology that assists a survey design expert in evaluating questions relative to the tasks they require of participants, specifically with regard to how participants understand and respond to survey questions. The QAS also allows the reviewer to evaluate the structure and effectiveness of the questionnaire form itself. In part, the QAS is a coding system (i.e., an item taxonomy) that describes the cognitive demands of the questionnaire and documents the question features that are likely to lead to response error. These potential errors include errors related to comprehension, task definition, information retrieval, judgment, and response generation. This forms appraisal can be used as a starting point for identifying particular instructions, questions, or response categories that may be problematic and could compromise the quality of the data in surveys. The review of the questionnaire helps to identify issues to target for cognitive interviewing (see Appendix A for BRFSS 2008 QAS findings).

Cognitive interviewing techniques have come into common use as a way to assess a range of problems with survey questions, including those related to comprehension, retrieval, decision, and response processes used by participants in answering questions. The most frequently used model for cognitive interviewing has four primary features: (1) question comprehension, (2) retrieval of relevant information from memory, (3) decision processes, and (4) response processes (Tourangeau, 1984). Cognitive interviewing is used to explore a person’s decision-making processes in each of these areas. Willis (1999) summarizes how cognitive interviewing can be used to explore these four aspects of responding to questions:

1. Question comprehension—studies both question intent (what does the participant believe the question to be asking?) and the specific meanings of terms in the question.

2. Retrieval from memory—examines ability to recall information and the strategies used to retrieve that information (i.e., estimation strategies or counting of individual events).

3. Decision processes—examines the participant’s motivation to thoughtfully provide an accurate response, as well as issues related to desirable responding (or social desirability—the desire to respond in such a way as to make oneself look better, either through intentional deception or unconscious self-deception).

4. Response processes—evaluates the ability of the participant to match his or her estimation (e.g., perception, behavior) with the response options available.

Cognitive interviewing uses two approaches to probe question understanding and responding: think-aloud techniques and verbal probing. Briefly, the think-aloud technique asks that the participant provide a continuous verbal narrative while reading survey questions, recalling the information requested, considering how to respond, and matching his or her response to the response options listed on the survey form. Verbal probing techniques can be either concurrent probing, in which the probes are administered as the participant completes a survey item, or retrospective probing, in which the participant completes the entire survey and is then presented with specific probes about the question, retrieval, decision processes, and response processes. Each approach has advantages and disadvantages. For Round 1, concurrent verbal probing was used for testing the BRFSS items, primarily to elicit immediate feedback upon asking the question and to use in-depth probes to further explore the participant’s response. For Round 2, retrospective verbal probing was used for BRFSS items that were tested previously. This was done to create a testing environment that more closely resembled an actual telephone survey.

1.3 Instrument/Questions

During the first round of cognitive interviews, RTI recommended several revisions to the proposed modules to the 2007 BRFSS (see Table 1). These recommendations were forwarded by CDC’s behavioral Surveillance Branch (BSB) staff to each respective CDC branch that was responsible for reviewing the content of the recommendation and providing a revised survey item. CDC provided revised items to RTI for Round 2 testing.

Table 1. Summary of Round 1 Recommendations for Revision to Proposed New BRFSS Items

Module	Item Number	Proposed Recommendation for Revision
All	All new items	All new items should be clearly and consistently formatted so that interviewers understand which question text and response options should/should not be read.













Adverse Childhood Experiences	New 1	Format this item into three separate items, one for alcoholic, one for problem drinker, and one for illicit drug user, in this order. Consider providing expanded descriptions for the questions on “problem drinker” and “illicit drug user.” For example, problem drinker can be specified as “someone whose drinking was excessive and caused financial or emotional problems in your household?” Illicit drug user could be specified as “someone who uses illegal drugs or abuses legal drugs, such as using prescription drugs without a valid prescription.”
Adverse Childhood Experiences	New 3	Revise the question text to more precisely specify what kinds of imprisonment should be counted. For example, if short stays in jail (1 or 2 days) before trial or sentencing are not to be counted, then the question should specify this.
Adverse Childhood Experiences	New 5	Format this question as three separate items, one for depression, one for mental illness, and one for attempting suicide. Also, refine the terminology used to describe depression and mental illness, for greater precision.
Adverse Childhood Experiences	New 7	Consider expanding this question to indicate the nature of these behaviors. For example, an instruction could inform participants to not include incidents where a parent may have grabbed or pushed a child for safety reasons. Consider adding the term “hit” to the list of behaviors.
Adverse Childhood Experiences	all	Standardize the text in all items to refer to “anyone living in your household.”
Adverse Childhood Experiences	all	Consider gender matching interviewers and participants when asking this series of questions, if feasible.

Revised BRFSS questions for Round 2 testing appear in section 2 of this report. Following each question are follow-up questions or probes that resulted from a review of the questions using the Question Appraisal System (QAS-99; see Appendix A for BRFSS QAS findings) and Round 1 cognitive testing results that are summarized in Table 1. Participant numbers with relevant demographics in parentheses and comments elicited in the cognitive testing appear throughout the instrument in bold blue text, with an abbreviation of relevant subject demographics in parentheses. The abbreviations are as follows: age (numerically noted in years), education (High school diploma = HS, Associate’s Degree = AD, College Degree = CD, Graduate Degree = GD, race (African American = AA, Asian = A, Hispanic = H, White = W), and gender (female = F, male = M), received flu shot in past year (yes =shot, no otherwise), had illness with fever (yes = ill, no otherwise), and works in a health-care facility (yes = HCW, no otherwise). Participant numbers indicate which participant provided each response (to questions and prompts). When a numbered response does not appear after a follow-up prompt, it indicates that the question was skipped by the interviewer because it did not apply or was not asked.

1.4 Participants

Participants were recruited in the Raleigh/Durham, North Carolina, area through known contacts, a local newspaper advertisement, and flyers describing the requirements for participation and instructing interested persons to contact RTI for more information. All participants were screened for eligibility, which required meeting one of the following three criteria: having a flu shot in the past 12 months, having an illness with fever in the past 30 days, or working in a health-care facility such as a medical clinic, hospital, or nursing home. We also screened participants on demographic characteristics, gender, race, and education to draw an appropriate group of participants that reflected the diversity of the BRFSS survey sample.

For Round 1 testing, a total of eight participants were recruited for cognitive testing. All eight participants completed the cognitive testing. All eight participants had a flu shot in the past 12 months, seven had an illness with fever in the past 30 days, and three currently worked in a health-care facility. For Round 2 testing, a total of 18 participants were recruited for cognitive testing. Nine participants who were health-care workers received Module 1 (SHS I) and Module 3 (the Influenza High-Risk/Health-Care Workers). The remaining nine participants received Module 2 (SHS II), Module 4 (Flu Shot History), Module 5 (Flu Morbidity & Cost Estimate), and Module 6 (Adverse Childhood Experiences). Six of these participants had a flu shot in the past year, and six had been sick with the flu or flu-like symptoms in the past 6 months.

All participants answered the key screening questions (e.g., “During the past 12 months, have you had a flu shot?”) as part of the cognitive testing to verify our initial telephone screening. Each participant received a $40 cash incentive immediately following the cognitive interview. Table 2 presents the breakdown of participant characteristics in the cognitive interviews for both rounds.

Table 2. Cognitive Interview Participant Characteristics—Rounds 1 and 2

Participant Number	Age	Education	Race	Gender	Shot	Ill	Health-Care Worker
ROUND 1—IN-PERSON COGNITIVE INTERVIEWING
1	26	College Degree	White	F	Y	N	Registered Nurse
2	40	College Degree	White	F	Y	Y	Certified Nursing Assistant
3	44	Associate’s Deg	White	F	Y	Y	No
4	41	College Degree	African American	F	Y	Y	Home-Care Nurse
5	25	College Degree	African American	F	Y	Y	No
6	60	Grad Degree	White	M	Y	Y	No
7	25	College Degree	White	M	Y	Y	No
8	42	HS Diploma	White	M	Y	Y	No
ROUND 2—TELEPHONE-ADMINISTERED COGNITIVE INTERVIEWING
Health-Care Workers
3	27	Grad Degree	White	F			Immunologist
5	44	College Degree	White	F			Medical Technician
7	31	College Degree	White	M			Nurse
10	36	HS Diploma	African American	M			Lab Technician
11	37	College Degree	White	F			Physician Assistant
12	32	College Degree	African American	F			Paraoptometric Technician
14	24	College Degree	Asian	M			EMT
17	31	College Degree	African American	F			Clinical Trials Coordinator
18	34	Grad Degree	White	M			Pharmacist
Non-Health-Care Workers
1	53	College Degree	White	M	Y	N
2	57	HS Diploma	White	F	Y	N
4	21	HS Diploma	African American	F	N	Y
6	30	College Degree	African American	M	Y	Y
8	4	Associate Deg	African American	F	Y	Y
9	39	College Degree	African American	F	Y	Y
13	37	HS Diploma	AA/Hispanic	F	Y	Y
15	61	Grad Degree	White	M	Y	N
16	62	HS Diploma	White	F	Y	Y

1.5 Procedures

Adults 18 years of age or older responding to recruiting advertisements called the telephone number for RTI, heard a description of the study, and were asked several brief screening questions. If the callers met the screening criteria and were interested in participating, RTI staff scheduled a cognitive testing session.

Cognitive testing was conducted in Research Triangle Park, North Carolina, at the RTI Cognitive Interviewing Laboratory, a private room with one-way mirrors and audio and visual monitoring, enabling unobtrusive observation of interviews in the adjacent conference room. Upon arrival, each participant was greeted and escorted to the Cognitive Interviewing Laboratory conference room, where he or she met with the survey methodologist who conducted the cognitive interviews. The survey methodologist read aloud the participant informed consent form (Appendix B), provided a copy to the participant, and asked for a signature and permission for observation.

For Round 1 testing, cognitive interviews using the concurrent verbal-probing technique were conducted to investigate potential sources of response error on a draft version of BRFSS items. The question appraisal results provided some structured probes and spontaneous probes for the cognitive interviewing protocol (see Appendix C for Round 1 BRFSS Cognitive Interviewing protocol, which includes draft survey questions and cognitive interviewing probes). The Round 2 cognitive interviews used the retrospective verbal-probing technique. Here we administered the revised questions provided for Round 2. After all items in a section were administered, the survey methodologist administered probes to explore potential sources of error. The cognitive interviewing protocol was developed using results from the Round 1 question-by-question comments (see Appendix D). The cognitive interviewing protocol for Round 2 appears in Appendix E.

Typical spontaneous probes included comprehension or interpretation probes (e.g., In your own words, what does this term mean to you?), recall probes (e.g., How did you come to figure out your response?), and other general probes (e.g., How did you arrive at that answer? Was it easy or hard to recall that answer? Do you think people would answer that question honestly? and How difficult was it to match your answer to the answers that were available?). To minimize participant burden, the survey administration and cognitive interview were designed to last no more than an hour.

Two interviewers with training and experience in cognitive interviewing methods conducted the interviews. Because of the sensitive nature of the items on adverse childhood experiences, RTI’s Institutional Review Board (IRB) recommended gender matching of interviewers and participants. For Round 1, Interviewer 1 interviewed the five female participants and Interviewer 2 interviewed the three male participants. Interviews were conducted over 3 days, August 10, 11, and 14, 2006. For Round 2, Interview 1 interviewed 10 participants, and Interviewer 2 interviewed the remaining 8 participants. These interviews were conducted over a 3-day period: from September 13 to 15, 2006. Specifically, each participant was asked about the following topics:

1. Issues or problems in understanding individual questions, including sentence structure, individual word comprehension, answer choices, and context relative to question order.

2. Socially desirable responding from a personal perspective and how he or she believed that other people would perceive questions.

3. Instructions or transitional statements used in the questionnaire.

4. Spontaneous probes for issues that arose during the interview process.

For each participant, interviewers took written notes on a paper copy of the cognitive interviewing protocol during the interview and then entered comments in an electronic version of the protocol. From the aggregated notes, the cognitive testing team composed a final cognitive testing report that presented a summary of general issues, detailed findings for each item tested, and recommendations for item revision.

2. Results

The overall results for the first round of testing are discussed further in Section 2.1 for each of the modules proposed for testing. Question-by-question results for Round 1 cognitive testing appear in Appendix D. Overall results from the second round of testing appear in Section 2.2. The questions are discussed in the order they appear in the instrument; it may be helpful to review a copy of the instrument while reading the discussion. It may also be helpful to review both the summary of results and the specific answers given by participants, provided in Section 2.2. Section 2.3 includes the question-by-question answers given by participants as well as their response to the scripted and spontaneous probes administered during cognitive interviewing. Section 2.4 provides a summary of recommendations for question revision. Section 2.5 briefly describes some of the limitations encountered in the two rounds of cognitive testing.

2.1 General Themes—Round 1

Cognitive testing was successfully completed with a total of eight adults who met the screening criteria to respond to the proposed BRFSS survey items. Cognitive testing results indicate that proposed items generally appear to be functioning as intended, with a few minor problems, which are noted below. A review of responses throughout the testing indicates that participants frequently made use of heuristics and other means to help them remember factual information, such as the date of their flu shot. When probed about their decision process in answering attitude or opinion questions, such as their views on smoking policies, participants gave careful thought and consideration to their responses.

2.1.1 Secondhand Smoke (SHS)

2.1.2 Influenza

2.1.3 Adverse Childhood Experiences

Because of the sensitive nature of the eight questions in Module 6, RTI’s IRB recommended two protocols for testing these items. The first four items were Q1-growing up with alcohol abuse, Q2-parental separation/divorce, Q3-incarcerated household member, and Q4-emotional abuse. For these items, interviewers instructed participants that they could provide yes or no answers, but that they should not provide details of any of their relevant experiences. Interviewers followed a different protocol with the additional four items: (5) mental illness in the home, (6) witnessing domestic violence, (7) physical abuse, and (8) sexual abuse. For these items, interviewers instructed participants not to answer each question and to only answer the follow-up probes for each item. As a result, specific answers were not given for items Q5 through Q8 in this module.

On Q1, the participants varied in their interpretation of the terms “problem drinker” and “alcoholic.” For some participants, these terms were synonymous, whereas others made a distinction between the two. The latter group distinguished a category of people who drink “too much” or “excessively” as problem drinkers who are not necessarily alcoholics.

Participants indicated a few problems in understanding or answering Q2. One (#3 [44, W, F, Shot, Ill]) indicated that she knew her parents were apart during war time, but this did not count as “separated.” Two participants further indicated that “separated” did not necessarily mean legal separation.

An important issue arose on Item Q3 in this module. Participants varied on how they interpreted the phrase “go to prison.” Some participants indicated that any arrest resulting in incarceration would count as “go to prison,” whereas others did not count short stays (e.g., one or two days) in jail. Six of the eight participants indicated that a stay of greater than one or two days would be necessary to answer yes to this item. At the same time, two participants (#3 [44, W, F, Shot, Ill] and #4 [41, W, F, AA, Shot, Ill, HC]) provided ambiguous responses as to what kinds of situations would count as “go to prison.”

Q4 did not present any comprehension problems. All participants provided examples of “put you down” that appeared to be consistent with the intent of the question. Two participants indicated some possible ambiguity in whether certain statements by parents would count in this question. For example, participant #5 (25, AA, F, Shot, Ill) indicated that parents might swear in certain situations but that a distinction could be made between swearing in this context and swearing “at you.” Participant #7 (25, W, M, Shot, Ill) suggested that the issue was related to anger and that a distinction could be made between appropriate discipline and put-down statements.

The interpretation of the terms “mentally depressed” and “mentally ill” in Q5 varied greatly among participants. Some interpreted these terms as essentially the same, whereas others made clear distinctions between them. The most common distinction was that “mentally ill” included other mental conditions, such as psychotic disorders, bipolar, manic, and schizophrenia. One participant (#6 [60, W, M, Shot, Ill]) responded that the terminology needed to be more inclusive of different conditions and precise in labeling these.

Q6 presented no significant problems for participants. Overall, they could identify only a few scenarios that might cause some slight confusion in answering the question. None of the participants felt that this item was confusing.

Participants identified more scenarios that might cause ambiguity in answering Q7, compared with Q6. Four of the participants suggested that situations where a parent might grab a child for safety or protection could create confusion in responding to this item. The primary issue identified by these participants was the intent of the parent’s actions, which might not be understood by the child at the time; similar behaviors with different intent could cause ambiguity in responding to Q7.

Cognitive interview participants did not report any problems in interpreting the intent of Item Q8, although three participants suggested that survey participants might be unwilling to answer this question honestly. To assess how people might react to being asked all eight items in Module 6 as part of a telephone interview, interviewers asked two further probes of all participants. Participants varied in their assessment of how they would react to such questions in a telephone survey. Two indicated that they would have some concerns about answering these questions over the telephone or would prefer to answer the questions in person. Two others indicated that it would be preferable or easier to answer these items over the telephone. Others indicated that a telephone interview would be acceptable but also indicated concerns about the sensitive nature of the questions. Participants were similarly diverse in their responses as to whether they had any preference for the gender of the interviewer asking them these eight items. Three participants indicated that interviewer gender would not matter, two participants (one female and one male) indicated that they would prefer a female interviewer, and one male responded that he would be less comfortable with a female interviewer.

2.2 General Themes—Round 2

Round 2 cognitive testing was successfully completed with a total of 18 adults, of whom 9 were health-care workers and 9 were not but met the screening criteria for having a flu shot or recent flu-like illness. The cognitive testing results indicated that the revised items were significant improvements over the original versions tested in Round 1, with a few minor problems, noted in this section. As in Round 1, the responses indicated that participants frequently made use of heuristics and other means to help them remember factual information, such as the date of their flu shot or having a recent illness with fever. When probed about their decision process in answering attitude or opinion questions, such as their views on smoking policies, participants generally gave careful thought and consideration to their responses.

2.2.1 Secondhand Smoke (SHS)

2.2.2 Influenza

2.2.3 Module 6—Adverse Childhood Experiences

Module 6, Adverse Childhood Experiences, was administered only to the nine non-health-care-worker participants. Because of the sensitive nature of the questions in Module 6, RTI’s IRB mandated that the same protocol used in Round 1 be followed in Round 2. For the first five questions, interviewers instructed participants that they could provide yes or no answers, but that they should not provide details of any of their relevant experiences. For the final four items in this module, interviewers instructed participants not to answer each question and to answer only the follow-up probes for each item. As a result, specific answers were not given for items Q6 through Q9 in this module. In addition, to facilitate recall for retrospective probing, interviewers read the first five items followed by the appropriate probes, then completed the last four items and probes.

Only one of the nine participants who completed this module said yes to any of the first three items. For Items 4 and 5, yes was more common. Three participants indicated that their parents had been separated or divorced, and five indicated that a parent or other adult had sworn at them, insulted them, or put them down. Overall, participants seemed to have more varied understandings of the first five items than the final four.

Participants varied in their interpretation of a few key terms in the first five questions. In Item 1, participants all agree that the terms “problem drinker” and “alcoholic” were synonymous or very close in meaning. Their interpretations of what constitutes either a problem drinker or an alcoholic did vary widely, though, from “someone who feels they have to have at least one or two drinks each day” (#2 [57, CD, W, F, Shot]) to “someone who gets drunk a lot” (#4 [21, HS, AA, F, Shot]). These results indicate that some definition in the questions of what constitutes a “problem drinker” and an “alcoholic” might provide greater uniformity of understanding among participants.

An important change to Item 2 between rounds was the addition of the term “street drugs.” Participants had fairly consistent notions of what kinds of substances counted as street drugs; they named similar drugs when probed. The only difference in interpretation among participants was whether prescriptions drugs, when used illegally, would count as street drugs. Of the nine participants, five indicated that they would count prescription drugs used illegally as street drugs, and four indicated that they would not. One of the five who indicated that prescription drugs would count stated that she did not initially think about prescription drugs when answering the question [#9 (39, CD, AA, F, Shot)]. The intention of what substances should be included in this question appears to need clarification.

As in the first round, participants had somewhat varied interpretations of the phrase “went to prison” in Item 3. Most participants indicated that they were thinking of situations where a person is convicted of a crime and spends more than 1 or 2 nights in a prison facility. At the same time, three indicated that any type of facility would count, and one specifically indicated that she would count a situation “even if jailed over night” (#2 [57, CD, W, F, Shot]). The addition of the term “convicted of a crime” may have clarified the question for some participants, but others still appeared to include short stays in jail, even an overnight stay prior to a conviction. Clarifying whether short presentencing stays in a jail count in this question would improve comprehension. This point is especially important given that participants expressed similar understandings of the phrase “convicted of a crime” but provided somewhat varied interpretations for what counted as “went to prison.”

Q4 also produced some variation in how participants understood the term “separated.” Seven of nine participants indicated that separation includes both legal and other separations where spouses are not living together. The other two indicated that they were only counting legal separations. All participants indicated that separation due to a spouse serving in the military would not count as “separated.” The intent of this question would be clearer if some indication was provided on whether “separated” involves only legal separations or other separations.

Item 5 presented almost no variation in how participants understood key terms. Participants gave similar examples for what counted as “put you down.” Overall, responses indicated that the intention of the question to measure potential verbal or emotional abuse was understood clearly by all participants.

Among the final four items, only question 6 produced differences in interpretation, and these differences were fairly subtle. Four participants indicated that the terms “depressed” and “mentally ill” were virtually synonymous, whereas the other five made distinctions between the two terms. The most common distinction was that participants understood mentally ill to encompass psychological conditions other than depression. With respect to the terms “suicidal,” participants provided a range of interpretations from “someone who’s not on medication and has a history of depression” (#1 [53, CD, W, M, Shot]) to “someone who is thinking about or has tried to take their own life” (#4 [21, HS, AA, F, Shot]).

Items 7, 8, and 9 presented little variation in participants’ understanding, and seemed relatively straightforward. To assess how people might react to being asked the items in Module 6 as part of a telephone interview, especially the last four items, interviewers again asked two further probes of all participants:

Would they be comfortable answering these questions in a telephone survey in their home?

Would the gender of the interviewer make any difference in how comfortable they would be answering these questions?

Among the participants who completed this module, two indicated that they would be comfortable responding to these items as part of a telephone survey, and one indicated that he would be comfortable answering only the first five questions in a telephone survey. The other participants indicated that they would not be comfortable responding to these items in a telephone survey. The most common objection was providing sensitive information over the telephone when unable to precisely confirm the caller’s identity. With respect to the second question, only two participants indicated that gender would matter. Both of these participants were female and indicated that they would be more comfortable providing responses to these items if the interviewer were female. Overall, the concern participants indicated about answering these kinds of questions in a telephone interview suggests that special procedures may be needed to implement these questions as part of BRFSS.

2.3 Question-by-Question Comments—Round 2

For each question, participant numbers are to the right of the responses. Responses to probes or potentially significant sources of response error are noted in bold blue text, with an abbreviation of relevant subject information in parentheses. The abbreviations are as follows: age (numerically noted in years), race (African American = AA, Hispanic = H, White = W), and gender (female = F, male = M), received flu shot in past year (yes = shot, no otherwise), had illness with fever in past month (yes = ill, no otherwise), and works in health-care facility (yes = HC, no otherwise). Participant numbers indicate which participant provided each response to questions and prompts. When a numbered response does not appear after a follow-up prompt, it indicates that the question was skipped by the interviewer because it did not apply or was not asked.

Core Employment Question

S4. Are you currently:

Please read:

1. Employed for wages #3 (27, GD, W, F, HCW), #5 (44, CD, W, F, HCW), #7 (31, CD, W, M, HCW), #10 (36, HS, AA, M, HCW), #12 (32, CD, AA, F, HCW), #17 (33, CD, AA, F, HCW), #18 (34, GD, W, M, HCW)

2. Self-employed

3. Out of work for more than 1 year

4. Out of work for less than 1 year

5. A Homemaker

6. A Student #11 (37, CD, W, F, HCW)

7. Retired, or

8. Unable to work

Do not read:

9. Refused

Interviewer note: If response to Core S4 = 1 (Employed) or 2 (Self-employed), continue with Q1. Otherwise, go to Q2.

1. On how many of the past 7 DAYS, that is since [FILL DATE __________], did someone smoke in your indoor workplace while you were there?

Do not read:

1 – 7 days (enter days ________)

55 Did not work in the past 7 days

66. I do not work indoors most of the time

77. Don’t Know/Not sure

88. NONE #3 (27, GD, W, F, HCW), #5 (44, CD, W, F, HCW), #7 (31, CD, W, M, HCW), #10 (36, HS, AA, M, HCW), #11 (37, CD, W, F, HCW), #12 (32, CD, AA, F, HCW), #14 (24, CD, A, M, HCW), #17 (33, CD, AA, F, HCW), #18 (34, GD, W, M, HCW)

99. Refused

2. On how many of the past 7 days, that is since [FILL DATE ____________], have you been inside your home with someone who was smoking?

Do not read:

1 – 7 days (enter days ________)

#7 (31, CD, W, M, HCW) – 2 days

#14 (24, CD, A, M, HCW) – 1 day

8. I was not at home in the past 7 days

55. I was not home in the past 7 days.

77. Don’t Know/Not sure

88. NONE #3 (27, GD, W, F, HCW), #5 (44, CD, W, F, HCW), #10 (36, HS, AA, M, HCW), #11 (37, CD, W, F, HCW), #12 (32, CD, AA, F, HCW), #17 (33, CD, AA, F, HCW), #18 (34, GD, W, M, HCW)

99. Refused

3. Which statement best describes the rules about smoking inside your home? Do not include decks, garages, or porches.

Please read:

1. Smoking is not allowed anywhere inside your home #3 (27, GD, W, F, HCW), #5 (44, CD, W, F, HCW), #11 (37, CD, W, F, HCW), #12 (32, CD, AA, F, HCW), #17 (33, CD, AA, F, HCW), #18 (34, GD, W, M, HCW)

2. Smoking is allowed in some places or at some times #7 (31, CD, W, M, HCW), #14 (24, CD, A, M, HCW)

3. Smoking is allowed anywhere inside your home #10 (36, HS, AA, M, HCW)

Do not read:

7. Don’t know / Not sure

9. Refused

OPTION 1 (half of interviews)

4. In public places and indoor workplaces, do you THINK that smoking should be allowed in all areas, allowed in some areas, or not allowed at all?

Do not read:

1. Allowed in all areas [Go to Module 2, Question 1]

2. Allowed in some areas [Go to Module 2, Question 1] #3 (27, GD, W, F, HCW), #10 (36, HS, AA, M, HCW), #12 (32, CD, AA, F, HCW)

3. Not allowed at all [Go to Module 2, Question 1] #7 (31, CD, W, M, HCW)

OPTION 2 (half of interviews)

4. Do you THINK that smoking should be allowed in bars?

Do not read:

1. Yes #17 (33, CD, AA, F, HCW), #14 (24, CD, A, M, HCW)

2. No #5 (44, CD, W, F, HCW), #11 (37, CD, W, F, HCW), #18 (34, GD, W, M, HCW)

7. Don’t know / Not sure

9. Refused

5. Do you THINK that smoking should be allowed in restaurants?

Do not read:

1. Yes #14 (24, CD, A, M, HCW)

2. No #5 (44, CD, W, F, HCW), #11 (37, CD, W, F, HCW), #17 (33, CD, AA, F, HCW), #18 (34, GD, W, M, HCW)

7. Don’t know / Not sure

9. Refused

6. Do you THINK that smoking should be allowed in indoor workplaces?

Do not read:

1. Yes

2. No #5 (44, CD, W, F, HCW), #11 (37, CD, W, F, HCW), #14 (24, CD, A, M, HCW), #17 (33, CD, AA, F, HCW), #18 (34, GD, W, M, HCW)

7. Don’t know / Not sure

9. Refused

#14 (24, CD, A, M, HCW) – Participant indicated that his “yes” answers to items 4 and 5 meant that designated smoking areas should be created to allow smoking.

Module 1 Probes

Q1. f Q1 = 1-7, 55, or 88:

In the first question of this section, what did you think of as your indoor workplace? Were you thinking of just your work area, the entire building you work in, or some other area?

#3 (27, GD, W, F, HCW) – The entire building, it ‘s a small building and smoking is not allowed inside.

#5 (44, CD, W, F, HCW) – The entire hospital, including around the outside.

#7 (31, CD, W, M, HCW) – Anywhere inside the hospital where I work.

#10 (36, HS, AA, M, HCW) – Any place inside the building

#11 (37, CD, W, F, HCW) – Urgent care offices where I work; it’s one big building, but I was thinking of my particular practice.

#12 (32, CD, AA, F, HCW) – My immediate workplace, the inside of the building.

#14 (24, CD, A, M, HCW) – I work on a cubicle in a ground floor office, so I was thinking of my floor.

#17 (33, CD, AA, F, HCW) – The place inside the building where I work. The entire building.

#18 (34, GD, W, M, HCW) – My specific department, which is in a larger building (hospital). There is no smoking allowed inside. The designated areas are just outside the door, so there is a chance of smoke coming in if someone opens the door.

The first question asked about someone smoking in your workplace “while you were there?” What did this phrase mean to you? Did you think you we were asking about actually seeing someone smoking while you were at work, or somehow knowing that someone was smoking during the time you were at work?

#3 (27, GD, W, F, HCW) – In my presence, so I could see or smell it.

#5 (44, CD, W, F, HCW) – Legally or not legally when I was in my immediate area.

#7 (31, CD, W, M, HCW) – I took it to mean while I was on the clock. I was thinking of seeing someone smoke. WHEN PROBED: smelling the smoke would also count.

#10 (36, HS, AA, M, HCW) – While I was at work, was someone smoking inside. I was thinking just now that someone was smoking.

#11 (37, CD, W, F, HCW) – Just that I would have knowledge, be able to smell it.

#12 (32, CD, AA, F, HCW) – A co-worker or patient during my shift inside my building.

#14 (24, CD, A, M, HCW) – Aware of the presence of smoking.

#17 (33, CD, AA, F, HCW) – They hours I was physically at work. Smelling smoke or seeing it. Don’t have to see it to know.

#18 (34, GD, W, M, HCW) – Seeing someone smoke.

Q2. If smoker, from screening items:

When you were answering the second question about whether you been inside your home with someone who was smoking, were you thinking of yourself and others, or just others?

#10 (36, HS, AA, M, HCW) – Just others

#14 (24, CD, A, M, HCW) – Just other people.

Q3. (Other than yourself,) is there anyone else in your home who smokes regularly?

#5 (44, CD, W, F, HCW) – No one smokes.

#7 (31, CD, W, M, HCW), #10 (36, HS, AA, M, HCW) – No.

#12 (32, CD, AA, F, HCW) – No

#14 (24, CD, A, M, HCW) – No, I live by myself.

#18 (34, GD, W, M, HCW) – No.

If Q3 = 1 or 2:

What, if anything, do you do to enforce the rules about smoking in your home?

#3 (27, GD, W, F, HCW) – Nothing, it’s not really an issue.

#5 (44, CD, W, F, HCW) –If anyone asks to smoke, I would say no.

#7 (31, CD, W, M, HCW) – I ask my family not to smoke in the general area where I am at.

#11 (37, CD, W, F, HCW) – I have never had to do anything. No one has tried to smoke.

#12 (32, CD, AA, F, HCW) – My husband has a sign on the coffee table saying you can’t smoke. My younger brother used to smoke, but we made him go outside.

#14 (24, CD, A, M, HCW) – I might allow 1 or 2 if the windows are open, but that’s it.

#17 (33, CD, AA, F, HCW) – No one smokes. I tell visitors not to smoke.

#18 (34, GD, W, M, HCW) – If someone were to light a cigarette, I’d tell them to go outside. But the people I know don’t smoke.

If Q3 = 1 or 2:

Do you ever break your rules about smoking for certain guests or certain situations?

#3 (27, GD, W, F, HCW) – No.

#5 (44, CD, W, F, HCW) – No.

#7 (31, CD, W, M, HCW) – Yes, for my parents. They can smoke anywhere.

#11 (37, CD, W, F, HCW), #17 (33, CD, AA, F, HCW) – No.

#12 (32, CD, AA, F, HCW) – Yes, when my father visits he can smoke in the house.

#14 (24, CD, A, M, HCW) – Yes, the rules loosen up if there’s some drinking going on.
#18 (34, GD, W, M, HCW) – No.

Q4. If option 1 administered:

In answering the question about whether you think smoking should be allowed in public places and indoor workplaces, what did the phrase “public places” mean to you?

#3 (27, GD, W, F, HCW) – Indoor or outdoor public areas, parks, restaurants, bars, movie theaters.

#7 (31, CD, W, M, HCW) – I was thinking of malls, general shopping areas that kind of situation.

#10 (36, HS, AA, M, HCW) – Any place that is not your home or a friend’s home such as a store, restaurant, any place of business.

#12 (32, CD, AA, F, HCW) – Bars, movie theaters, restaurants, work environments – any common areas. They can have designated smoking areas.

Also, on that same question, were you thinking in terms of creating laws or policies to prohibit smoking in certain places?

#3 (27, GD, W, F, HCW) – I was thinking about laws to prohibit smoking, but I believe in having smoking areas.

#7 (31, CD, W, M, HCW) – I was thinking about whether I think they should be allowed to smoke, not about laws

#10 (36, HS, AA, M, HCW) – Creating laws or policies

#12 (32, CD, AA, F, HCW) – Yes, having a policy on where you can smoke.

Q4/Q5/Q6. If option 2 administered:

In answering the three last questions on whether you think smoking should be allowed in certain places, were you thinking in terms of creating laws or policies to prohibit smoking in these places?

#5 (44, CD, W, F, HCW) – I was thinking about allowing smoking, that it’s up to each establishment whether they want to be “smoke free.”

#11 (37, CD, W, F, HCW) – No particular laws, just my opinions

#14 (24, CD, A, M, HCW) – I was thinking about individual places setting policies on whether or not to have a smoking area.

#17 (33, CD, AA, F, HCW) – Just personal preference because I wouldn’t go to a smoky bar. But if you know a place is smoky and you don’t like it you don’t have to go there. I wasn’t thinking laws just my personal opinion.

#18 (34, GD, W, M, HCW) – I was just thinking would the establishments choose to allow it.

MODULE 2: Secondhand Smoke II (including core screening items)

Core Tobacco Items

S1. Have you smoked at least 100 cigarettes in your entire life?

Do not read:

(5 packs 1. Yes #2 (57, CD, W, F, Shot, Ill), #6 (30, CD, AA, M, Shot,

= 100 Ill), #15 (61, GD, W, M, Shot), #16 (62, HS, W, F, Shot, Ill)

Cigarettes) 2. No [Go to S4] #1 (53, CD, W, M, Shot), #4

(21, HS, AA, F, Shot, Ill), #8 (45, AD, AA, F, Shot, Ill), #9 (39, CD, AA, F, Shot), #13 (37, HS, AA/H, F, Ill)

7. Don’t know / Not sure [Go to S4]

9. Refused [Go to S4]

S2. Do you now smoke cigarettes every day, some days, or not at all?

Do not read:

1. Every day

2. Some days

3. Not at all [Go to S4] #2 (57, CD, W, F, Shot, Ill), #6 (30, CD, AA, M, Shot, Ill), #15 (61, GD, W, M, Shot), #16 (62, HS, W, F, Shot, Ill)

9. Refused [Go to S4]

S3. During the past 12 months, have you stopped smoking for one day or longer because you were trying to quit smoking?

Do not read:

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

Core Employment Question

S4. Are you currently:

Please read:

1. Employed for wages #4 (21, HS, AA, F, Shot, Ill), #6 (30, CD, AA, M, Shot, Ill), #8 (45, AD, AA, F, Shot, Ill), #9 (39, CD, AA, F, Shot)

2. Self-employed #13 (37, HS, AA/H, F, Ill)

3. Out of work for more than 1 year

4. Out of work for less than 1 year

5. A Homemaker

6. A Student

7. Retired, or #1 (53, CD, W, M, Shot), #15 (61, GD, W, M, Shot), #16 (62, HS, W, F, Shot, Ill)

8. Unable to work #2 (57, CD, W, F, Shot, Ill)

Do not read:

9. Refused

#2 (57, CD, W, F, Shot, Ill) – PARTICIPANT ORIGINALLY SELECTED RETIRED. SHE THEN ASKED WHAT UNABLE TO WORK MEANT. I ASKED HER WHAT SHE THOUGHT IT MEANT AND SHE SAID, ”Disabled or on Social Security benefits.” SHE THEN DECIDED TO CHANGE HER ANSWER TO “Unable to work.”

Interviewer note: If response to Core S4 = 1 (Employed) or 2 (Self-employed), continue with Q1. Otherwise, go to Q4.

1. On how many of the past 7 DAYS, that is since [DATEFILL], did anyone smoke in your indoor workplace while you were there?

Do not read:

1 – 7 days (enter days ________) [Go to Q2]

55. I did not work in the past 7 days [Go to Q2]

66. I do not work indoors most of the time [Go to Q4]

77. Don’t know / Not sure [Go to Q2]

88. NONE [Go to Q2] #6 (30, CD, AA, M, Shot, Ill)

99. Refused [Go to Q2]

#4 (21, HS, AA, F, Shot, Ill), #8 (45, AD, AA, F, Shot, Ill), #9 (39, CD, AA, F, Shot), #13 (37, HS, AA/H, F, Ill) – PARTICIPANT ANSWERED 0, WHICH IS NOT AN OPTION.

2. Which of the following best describes the official policy on smoking in your indoor workplace?

Please read:

1. I do not work indoors most of the time [Go to Q4]

2. Smoking is not allowed in any indoor areas in my workplace [Go to Q3] #4 (21, HS, AA, F, Shot, Ill), #6 (30, CD, AA, M, Shot, Ill), #8 (45, AD, AA, F, Shot, Ill), #9 (39, CD, AA, F, Shot), #13 (37, HS, AA/H, F, Ill)

3. Smoking is allowed in some indoor areas in my workplace [Go to Q3]

4. Smoking is allowed in all indoor areas in my workplace [Go to Q3]

Do not read:

5. There is no official policy [Go to Q4]

7. Don’t know / Not sure [Go to Q3]

9. Refused [Go to Q3]

3. Do smokers follow the policy on smoking in your indoor workplace?

Read if necessary: Include employees, customers, and visitors.

Please read:

1. Smokers always follow the policy on smoking #4 (21, HS, AA, F, Shot, Ill), #6 (30, CD, AA, M, Shot, Ill), #8 (45, AD, AA, F, Shot, Ill), #9 (39, CD, AA, F, Shot), #13 (37, HS, AA/H, F, Ill)

2. Smokers usually follow the policy on smoking

Smokers sometimes follow the policy on smoking

Smokers rarely follow the policy on smoking

Smokers never follow the policy on smoking

There are no smokers in my indoor workplace

Do not read:

7. Don’t know / Not sure

9. Refused

#4 (21, HS, AA, F, Shot, Ill), #9 (39, CD, AA, F, Shot) – PARTICIPANT ANSWERED BEFORE I READ THE RESPONSE OPTIONS.

4. On how many of the past 7 days, that is since [DATE FILL], have you been inside the vehicle you drive or ride in the most with someone who was smoking?

Do not read:

1 – 7 days (enter days ________)

55. I was not in a vehicle in the past 7 days

77. Don’t know / Not sure

88. NONE #1 (53, CD, W, M, Shot), #6 (30, CD, AA, M, Shot, Ill), #15 (61, GD, W, M, Shot)

99. Refused

#2 (57, CD, W, F, Shot, Ill), #4 (21, HS, AA, F, Shot, Ill), #8 (45, AD, AA, F, Shot, Ill), #9 (39, CD, AA, F, Shot), #13 (37, HS, AA/H, F, Ill), #16 (62, HS, W, F, Shot, Ill) – PARTICIPANT SAID 0 DAYS, WHICH IS NOT AN OPTION.

5. Which statement best describes the rules about smoking inside the vehicle you drive or ride in the most?

Please read:

1. Smoking is not allowed inside the vehicle at any time #1 (53, CD, W, M, Shot), #2 (57, CD, W, F, Shot, Ill), #4 (21, HS, AA, F, Shot, Ill), #6 (30, CD, AA, M, Shot, Ill), #8 (45, AD, AA, F, Shot, Ill), #9 (39, CD, AA, F, Shot), #13 (37, HS, AA/H, F, Ill), #15 (61, GD, W, M, Shot), #16 (62, HS, W, F, Shot, Ill)

2. Smoking is sometimes allowed inside the vehicle

3. Smoking is always allowed inside the vehicle

Do not read:

7. Don’t know/Not sure

8. I don’t drive or ride in a vehicle frequently enough to provide an answer

9. Refused

#15 (61, GD, W, M, Shot) – Participant indicated that if they were with a client, as part of their business, they would allow them to smoke.

MODULE 2 PROBES

Q1. If Q1 = 1-7, 55, or 88:

What did you think of as your indoor workplace? Were you thinking of just your work area, the entire building you work in, or some other area?

#4 (21, HS, AA, F, Shot, Ill) – The office I work in and the break room and the elevators; basically the areas where I work.

#6 (30, CD, AA, M, Shot, Ill) – The inside area of the entire building.

#8 (45, AD, AA, F, Shot, Ill) – The entire building that I work in

#9 (39, CD, AA, F, Shot) – Inside the office, the area in my building that I work. Actually the whole building.

#13 (37, HS, AA/H, F, Ill) – I do home healthcare, so I was thinking of the client’s home. The other person who takes care of my patient doesn’t smoke either.

In this question, what do you think we meant by someone smoking “while you were there?” Did you think you we were asking about actually seeing someone smoking while you were at work, or somehow knowing that someone was smoking during the time you were at work?

#4 (21, HS, AA, F, Shot, Ill) – While I was in the office or elevator. I was thinking about actually seeing them smoke.

#6 (30, CD, AA, M, Shot, Ill) – While I was there, if I could see smoking or smell the scent.

#8 (45, AD, AA, F, Shot, Ill) – I just know you’re not allowed to. I wasn’t thinking of whether I saw them or not. If you smoke it would set off an alarm, so people don’t do it.

#9 (39, CD, AA, F, Shot) – While I was there was anyone smoking. Just know that they were smoking, I wouldn’t have to see them.

#13 (37, HS, AA/H, F, Ill) – While I’m at work including family, coworkers or friends, anyone but me. I thought it meant knowing they were smoking.

Q3. How did you come up with your answer on whether smokers follow the policy on smoking in your indoor workplace? (Probe: Were you thinking mostly about the people you work with who are smokers you work with, or were you thinking more about how often you seen smoking, or both?)

#4 (21, HS, AA, F, Shot, Ill) – I have just never seen anyone smoke inside.

#6 (30, CD, AA, M, Shot, Ill) – How often I’d seen smoking inside the building.

#8 (45, AD, AA, F, Shot, Ill) – I know that they can’t because it would set off an alarm. Plus there are designated smoking areas outside.

#9 (39, CD, AA, F, Shot) – As long as I have been there I have not know anyone to smoke.

#13 (37, HS, AA/H, F, Ill) – Depending on the workplace. It has to be followed because it would be reported. Plus I would have smelled the smoke.

If Q4 =1-7, 77, or 88:

For the term vehicle, were you just thinking about cars, trucks, and SUVs? Would a motorcycle or bike count, to you? What about public transportation such as buses or trains?

#1 (53, CD, W, M, Shot) – All of those count, including buses and trains. A bike would not count.

#2 (57, CD, W, F, Shot, Ill) – Cars, trucks, SUV’s vans. I would not count a motorcycle. Public transportation would count.

#4 (21, HS, AA, F, Shot, Ill) – Just cars, not trucks, SUVs or anything else.

#6 (30, CD, AA, M, Shot, Ill) – I was just thinking of your personal car. But a bus or train would count.

#8 (45, AD, AA, F, Shot, Ill) – I was thinking of just my car. WHEN PROBED SHE INDICATED THAT SHE WOULD CONSIDER THINGS LIKE PUBLIC TRANSPORTATION.

#9 (39, CD, AA, F, Shot) – I was thinking of my car or a truck or SUV. Public transportation would count.

#13 (37, HS, AA/H, F, Ill), #16 (62, HS, W, F, Shot, Ill) – Cars, trucks, and SUVs would count. Motorcycles and public transportation would not count.

#15 (61, GD, W, M, Shot) – Personal vehicles. [PARTICIPANT ALSO NOTED THAT TOWNS OR STATES RESTRICT WHETHER OR NOT YOU CAN SMOKE.]

module 3: Influenza – Flu, High-risk/Health Care Workers

Q12.8 Has a doctor, nurse, or other health professional ever said that you have any of the following health problems or conditions?

[READ IF NECESSARY]: By “other health professional” we mean a nurse practitioner or a physician’s assistant.

Read all items listed below before waiting for an answer:

Lung problems or conditions, other than asthma;

Kidney problems or conditions;

Sickle cell anemia or other anemia;

-or-

A weakened immune system caused by a chronic illness or by medicines taken for a chronic illness?

Do not read:

1. Yes [Go to Q12.9]

2. No [Go to Q12.10] #3 (27, GD, W, F, HCW), #5 (44, CD, W, F, HCW), #7 (31, CD, W, M, HCW), #10 (36, HS, AA, M, HCW), #11 (37, CD, W, F, HCW), #12 (32, CD, AA, F, HCW), #14 (24, CD, A, M, HCW), #17 (33, CD, AA, F, HCW), #18 (34, GD, W, M, HCW)

7. Don’t know / Not sure [Go to Q12.10]

9. Refused [Go to Q12.10]

[READ IF THE RESPONDENT HESITATES OR SAYS NO, DK OR NOT SURE]: Illnesses such as cancer or HIV/AIDS can cause a person to have a weakened immune system. Medicines such as steroids or transplant medications can cause a person to have a weakened immune system. Do you need me to repeat this question?

Q12.9 Do you still have (this/any of these) problem(s) or conditions?

Do not read:

1. Yes

2. No

7. Don’t know / Not sure

9. Refused

Q12.10. Do you currently work in a hospital, medical clinic, doctor’s office, nursing home or some other health care facility?

[READ]: This includes part-time and volunteer work in a health care facility as well as professional nursing care provided in the home.

[READ IF NECESSARY]: Dental offices are included as health care facilities.

Do not read:

1. Yes [Go to Q12.11] #3 (27, GD, W, F, HCW), #5 (44, CD, W, F, HCW), #7 (31, CD, W, M, HCW), #10 (36, HS, AA, M, HCW), #11 (37, CD, W, F, HCW), #12 (32, CD, AA, F, HCW), #14 (24, CD, A, M, HCW), #17 (33, CD, AA, F, HCW), #18 (34, GD, W, M, HCW)

2. No [Go to Q13.1]

7. Don’t know / Not sure [Go to Q13.1]

9. Refused [Go to Q13.1]

#11 (37, CD, W, F, HCW) – PARTICIPANT WAS UNSURE IF IT COUNTED BECAUSE SHE IS NOT PAID AND IT IS PART OF HER STUDENT ROTATION. EVEN AFTER READING THE PART ABOUT VOLUNTEER WORK SHE WAS STILL UNCERTAIN, BUT DECIDED YES.

Q12.11. Do you have direct face-to-face or hands-on contact with patients as a part of your routine work?

[READ IF NECESSARY]: By direct face-to-face or hands on contact we mean closer patient contact than usually would occur between a receptionist and a patient.

Do not read:

1. Yes #3 (27, GD, W, F, HCW)#7 (31, CD, W, M, HCW), #11 (37, CD, W, F, HCW), #12 (32, CD, AA, F, HCW), #14 (24, CD, A, M, HCW), #17 (33, CD, AA, F, HCW)

2. No #3 (27, GD, W, F, HCW), #5 (44, CD, W, F, HCW), #10 (36, HS, AA, M, HCW), #18 (34, GD, W, M, HCW)

7. Don’t know / Not sure (Probe by repeating question)

9. Refused

Module 3 Probes

Q12.8. In the first question in this section, you said that a doctor, nurse or other health professional told you that you had one of those health conditions. Who specifically told you that? [If probe was not read]

When I read “other health professional,” can you tell me who you thought of [if anyone]?

#3 (27, GD, W, F, HCW) – I didn’t pay attention to that, but I would include PAs.

#5 (44, CD, W, F, HCW) – No, I was just thinking of doctors.

#7 (31, CD, W, M, HCW), #11 (37, CD, W, F, HCW) – Nurse Practitioner, Physician Assistant

#10 (36, HS, AA, M, HCW) – I didn’t think about it; I mainly thought about a nurse

#12 (32, CD, AA, F, HCW) – NA’s, radiologists, auxiliary personnel, anyone who might administer tests.

#14 (24, CD, A, M, HCW) – I was assuming you were only talking about doctors, since they would be the one to diagnose you. I guess psychiatrists and physical therapists would count.

#17 (33, CD, AA, F, HCW) – Nurse practitioner, Physician Assistant, any type of professional medical staff

#18 (34, GD, W, M, HCW) – Pharmacists, respiratory therapists, physical therapists.

Were there are conditions or problems that you were uncertain whether you had? If yes, why?

#3 (27, GD, W, F, HCW) – No.

#5 (44, CD, W, F, HCW) – No.

#7 (31, CD, W, M, HCW), #11 (37, CD, W, F, HCW), #17 (33, CD, AA, F, HCW) – No.

#10 (36, HS, AA, M, HCW) – PARTICIPANT MENTIONED THAT HE HAD SICKLE CELL TRAIT WHEN I READ THE QUESTION. I PROBED ON WHETHER HE WOULD ANSWER YES OR NO. HE SAID HE WOULD ANSWER NO BECAUSE IT WAS DIFFERENT. I ASKED IF IT WAS CONFUSING AND HE SAID NOT FOR HIM BUT THAT SOME PEOPLE WITH SICKLE CELL TRAIT MIGHT NOT KNOW THE DIFFERENCE.

#12 (32, CD, AA, F, HCW) – No.

#18 (34, GD, W, M, HCW) – No.

Q12.10 I asked you if you currently worked in a hospital, medical clinic or other health care facility. In your opinion, is this question only asking about trained health care professionals, such as nurses and doctors, or anyone who works in a health care facility, including receptionists?

#3 (27, GD, W, F, HCW) – Anyone who works there.

#5 (44, CD, W, F, HCW) – All encompassing to anyone who works there, all different levels of employees.

#7 (31, CD, W, M, HCW) – I considered it trained healthcare professionals

#10 (36, HS, AA, M, HCW) – Anyone who works there in a healthcare facility

#11 (37, CD, W, F, HCW) – Anyone who works in a healthcare facility regardless of title.

#12 (32, CD, AA, F, HCW) – Only trained health care professionals.

#14 (24, CD, A, M, HCW) – I was using an inclusive definition of anybody who worked there.

#17 (33, CD, AA, F, HCW) – Well, the question is very general. So I was thinking of anyone who worked there.

#18 (34, GD, W, M, HCW) – Everybody – administrative people, secretaries, and others.

Q12.11 For the last question, what is direct face-to-face contact and hands-on contact? Can you give me examples of what this would be?

#3 (27, GD, W, F, HCW) – Contact with an actual person – talking to them, examining them, or doing treatments like drawing blood.

#5 (44, CD, W, F, HCW) – Whether it’s part of your routine work, like doctors, nurses, or PTs, who have direct contact as part of their responsibilities. Talking to patient counts, but only if routine.

#7 (31, CD, W, M, HCW) – WHEN PROBED ABOUT FACE-TO-FACE CONTACT: It is patient care; transporting patients; anything that puts you with 3 feet of a patient; WHEN PROBED: receptionists would not count. I THEN PROBED ABOUT HANDS-ON CONTACT: Hands-on is different. It’s doing stuff with the patient. I take back what I said earlier – receptionists do have face-to-face contact. They could have it greeting or talking to a patient. Hands-on would be starting IVs, changing linens. I ASKED IF DIRECT PATIENT CARE WOULD SOUND BETTER THAN FTF OR HANDS-ON CONTACT AND HE SAID THAT IT WOULD. When we come up with the disaster plans for our hospital in case of a Smallpox outbreak or something of that nature, receptionists are most at risk because they come in contact with every single patient who walks through the door. Our protocol is that they are quarantined first. Them and triage nurses because they are the most likely to be exposed. It is the same for influenza, they come in contact with the most patients. Our receptionists are not behind glass or anything so they are not protected.

#10 (36, HS, AA, M, HCW) – A receptionist is face-to-face or a financial advisor. A nurse, phlebotomist, doctors, any sort of physical contact would be hands-on.

#11 (37, CD, W, F, HCW) – In my case, FTF would be taking patient history. It could also be the receptionist that check them in or the nurses that triage them in. Hands-on would be a physical exam, taking blood pressure, temperature. This is less applicable to reception and front desk staff.

#12 (32, CD, AA, F, HCW) – Being in an exam room, administering a test. Talking to a patient or asking questions could be face-to-face, but not hands-on.

#14 (24, CD, A, M, HCW) – Face-to-face would include talking to patients, this could be administrative. Hands-on involves diagnosing, evaluating, etc. I think hands-on encompasses face-to-face.

#17 (33, CD, AA, F, HCW) – FTF would be direct patient care, one-on-one physical therapy, gate training. Hands-on is pretty much the same thing: demonstrating, showing, walking with them. I guess FTF could be talking to them.

#18 (34, GD, W, M, HCW) – I was thinking on a daily basis. Face-to-face is different than hands-on. Face-to-face would just be talking to them, not touching in any way. Hands-on would involve touching. I have face-to-face contact with patients every once in a while, but not on a regular basis.

MODULE 4: Flu Shot History

Q9.1 A flu shot is an influenza vaccine injected into your arm. During the past 12 months, have you had a flu shot?

Do not read:

1. Yes [Go to Q9.2] #1 (53, CD, W, M, Shot), #2 (57, CD, W, F, Shot, Ill), #6 (30, CD, AA, M, Shot, Ill), #8 (45, AD, AA, F, Shot, Ill), #9 (39, CD, AA, F, Shot), #13 (37, HS, AA/H, F, Ill), #15 (61, GD, W, M, Shot), #16 (62, HS, W, F, Shot, Ill)

2. No [Go to Q9.3] #4 (21, HS, AA, F, Shot, Ill)

7. Don’t know/Not sure [Go to Q9.3]

9. Refused [Go to Q9.3]

Q9.2 During what month and year did you receive your most recent flu shot?

Do not read:

_ _/ _ _ _ _ Month / Year

7 7/ 7 7 7 7 Don’t know/Not Sure

9 9/ 9 9 9 9 Refused

#1 (53, CD, W, M, Shot) – December, 2005.

#2 (57, CD, W, F, Shot, Ill) – November, 2005.

#6 (30, CD, AA, M, Shot, Ill) – January, 2006. Participant wasn’t sure at first whether it was late December 2005 or January 2006.

#8 (45, AD, AA, F, Shot, Ill) – March, 2006. PARTICIPANT HESITATED AND THE SAID THAT SHE KNEW IT WAS DURING THE FIRST QUARTER OF THE NEW YEAR AND THEN GUESSED MARCH.

#9 (39, CD, AA, F, Shot) – September, 2005. It was the beginning of the flu season either August or September.

#13 (37, HS, AA/H, F, Ill) – August, 2006

#15 (61, GD, W, M, Shot) – October, 2005.

#16 (62, HS, W, F, Shot, Ill) – January, 2006

Q9.3 During the past 12 months, have you had a flu vaccine that was sprayed in your nose? The flu vaccine sprayed in the nose is also called FluMist™.

Do not read:

1. Yes [Go to Q9.4]

2. No [Go to Module 3, question 1] #1 (53, CD, W, M, Shot), #2 (57, CD, W, F, Shot, Ill), #4 (21, HS, AA, F, Shot, Ill), #6 (30, CD, AA, M, Shot, Ill), #8 (45, AD, AA, F, Shot, Ill), #9 (39, CD, AA, F, Shot), #13 (37, HS, AA/H, F, Ill), #15 (61, GD, W, M, Shot), #16 (62, HS, W, F, Shot, Ill)

7. Don’t know/Not sure [Go to Module 3, question 1]

9. Refused [Go to Module 3, question 1]

Q9.4 During what month and year did you receive your most recent flu vaccine that was sprayed in your nose?

Do not read:

_ _/ _ _ _ _ Month / Year

7 7/ 7 7 7 7 Don’t know/Not Sure

9 9/ 9 9 9 9 Refused

MODULE 4 PROBES

If Q9.1 or Q9.3 = Yes:

How certain are you of when you had your last flu shot?

#1 (53, CD, W, M, Shot) – 100% certain.

#2 (57, CD, W, F, Shot, Ill) – Very certain.

#4 (21, HS, AA, F, Shot, Ill) - Very certain. I know that I have never had a flu shot in my life.

#6 (30, CD, AA, M, Shot, Ill) – 75% certain, almost positive.

#8 (45, AD, AA, F, Shot, Ill) – Uncertain. I know it was at the beginning of the year and I think it was March.

#9 (39, CD, AA, F, Shot) – Almost certain but not positive

#13 (37, HS, AA/H, F, Ill) – Positive. It was the last time I went to the doctor.

#15 (61, GD, W, M, Shot) – Very certain, because I get it the same time every year.

#16 (62, HS, W, F, Shot, Ill) – Pretty certain.

How did you come up with your answer? How did you remember that was when you had your last flu shot?

#1 (53, CD, W, M, Shot) – I’ve been getting a flu shot at the same time for the past 10-15 years.

#2 (57, CD, W, F, Shot, Ill) – A friend of mine moved into her home at the end of October last year and I remember that I had the flu shot a week or two later.

#6 (30, CD, AA, M, Shot, Ill) – I was trying to decide how far after Christmas it was.

#8 (45, AD, AA, F, Shot, Ill) – Because of my job, I get flu shots for free and it is around the same time each year.

#9 (39, CD, AA, F, Shot) – I know it was right before the flu season started

#13 (37, HS, AA/H, F, Ill) – It was the sickest I had ever been.

#15 (61, GD, W, M, Shot) – I get it as part of a scheduled appointment with my doctor.

#16 (62, HS, W, F, Shot, Ill) – I talked to my husband about it and we both agreed it was January. We got shots at the same time.

MODULE 5: Flu Morbidity & Cost Estimate

Next, we would like to ask you some questions about influenza, or the flu.

Q1. Last month, did you have any illnesses with a fever?

Do not read:

1. Yes [Go to Q1a] #2 (57, CD, W, F, Shot, Ill), #4 (21, HS, AA, F, Shot, Ill), #6 (30, CD, AA, M, Shot, Ill), #8 (45, AD, AA, F, Shot, Ill), #13 (37, HS, AA/H, F, Ill), #16 (62, HS, W, F, Shot, Ill)

2. No [Go to Module 4, question 1] #1 (53, CD, W, M, Shot), #9 (39, CD, AA, F, Shot), #15 (61, GD, W, M, Shot)

7. Don’t know/Not sure [Go to Q1a]

9. Refused [Go to Q1a]

Q1A. Were any of these illnesses accompanied by a cough and/or sore throat?

Do not read:

1. Yes #4 (21, HS, AA, F, Shot, Ill), #6 (30, CD, AA, M, Shot, Ill), #8 (45, AD, AA, F, Shot, Ill), #13 (37, HS, AA/H, F, Ill), #16 (62, HS, W, F, Shot, Ill)

2. No #2 (57, CD, W, F, Shot, Ill)

7. Don’t know/Not sure

9. Refused

Q2. Did you get care from a doctor, nurse, or other health professional for this illness?

Do not read:

1. Yes [Go to Q2A1] #4 (21, HS, AA, F, Shot, Ill), #6 (30, CD, AA, M, Shot, Ill), #8 (45, AD, AA, F, Shot, Ill), #13 (37, HS, AA/H, F, Ill), #16 (62, HS, W, F, Shot, Ill)

2. No [Go to Q2A2] #2 (57, CD, W, F, Shot, Ill)

7. Don’t know/Not sure [Go to Q2A2]

9. Refused [Go to Q2A2]

Q2A1. Why did you go to the doctor, nurse, or other health professional for this illness?

Do not read answer choices below. Select categories that best match response. Multiple categories may be selected.

1. Want to know the diagnosis [Go to Q2B] #4 (21, HS, AA, F, Shot, Ill), #8 (45, AD, AA, F, Shot, Ill)

2. Don’t want to make other people sick [Go to Q2B]

3. Too sick to care for myself or others [Go to Q2B]

4. Need a sick note for work [Go to Q2B]

5. Want to shorten illness [Go to Q2B]

6. Want to get back to work [Go to Q2B]

7. Worried that my other chronic conditions will get worse [Go to Q2B]

8. No improvement after several days [Go to Q2B]

9. Need to get a prescription [Go to Q2B] #6 (30, CD, AA, M, Shot, Ill), #13 (37, HS, AA/H, F, Ill), #16 (62, HS, W, F, Shot, Ill)

10. Other reason not listed above [Go to Q2B] #4 (21, HS, AA, F, Shot, Ill) – I was having trouble breathing/pain. #13 (37, HS, AA/H, F, Ill) – Pain, OTCs not working. #16 (62, HS, W, F, Shot, Ill) – felt reassured talking to a doctor about it.

Q2A2. At what point would you go to the doctor, nurse, or other health professional for this illness?

Do not read answer choices below. Select categories that best match response. Multiple categories may be selected.

1. Want to know the diagnosis [Go to Q3]

2. Don’t want to make other people sick [Go to Q3]

3. Too sick to care for myself or others [Go to Q3]

4. Need a sick note for work [Go to Q3]

5. Want to shorten illness [Go to Q3]

6. Want to get back to work [Go to Q3]

7. Worried that my other chronic conditions will get worse [Go to Q3]

8. No improvement after several days [Go to Q3]

9. Need to get a prescription [Go to Q3]

10. Other reason not listed above [Go to Q3] #2 (57, CD, W, F, Shot, Ill) – She said if the illness had gone into her chest or otherwise gotten worse.

Q2B. Did the doctor, nurse, or other health professional tell you this illness was the flu?

Do not read:

1. Yes [Go to Q2C] #6 (30, CD, AA, M, Shot, Ill), #8 (45, AD, AA, F, Shot, Ill), #13 (37, HS, AA/H, F, Ill), #16 (62, HS, W, F, Shot, Ill)

2. No [Go to Q3] #4 (21, HS, AA, F, Shot, Ill)

7. Don’t know/Not sure [Go to Q2C]

9. Refused [Go to Q2C]

#6 (30, CD, AA, M, Shot, Ill) – Doctor indicated it was “bacterial,” and that he had flu-like symptoms.

#8 (45, AD, AA, F, Shot, Ill) – They said it was hard to detect but that it could be the flu. I PROBED AS TO WHETHER THIS WAS A YES OR NO AND SHE SAID YES. PARTICIPANT SAID THAT WE SHOULD HAVE A “They said it could be the flu.”

Q2C. Did you have a flu test that was positive for this illness?

Do not read:

1. Yes #6 (30, CD, AA, M, Shot, Ill), #8 (45, AD, AA, F, Shot, Ill)

2. No #13 (37, HS, AA/H, F, Ill), #16 (62, HS, W, F, Shot, Ill)

7. Don’t know/Not sure

9. Refused

Q2D. Did you receive Tamiflu, or an inhaled medicine called Relenza to treat this illness?

Do not read:

1. Yes #8 (45, AD, AA, F, Shot, Ill), #16 (62, HS, W, F, Shot, Ill) – Tamiflu

2. No #6 (30, CD, AA, M, Shot, Ill), #13 (37, HS, AA/H, F, Ill)

7. Don’t know/Not sure

9. Refused

Q2E. Were you hospitalized for this illness?

Do not read:

1. Yes

2. No #6 (30, CD, AA, M, Shot, Ill), #8 (45, AD, AA, F, Shot, Ill), #13 (37, HS, AA/H, F, Ill), #16 (62, HS, W, F, Shot, Ill)

7. Don’t know/Not sure

9. Refused

Q3. How many days of work did you miss because of this illness?

_____ Days missed (enter number; lowest allowed = 0.50)

66 Not employed

77 Don't know/Not Sure

88 None

99 Refused

#4 (21, HS, AA, F, Shot, Ill) – 2 days

#6 (30, CD, AA, M, Shot, Ill) – 4 days

#8 (45, AD, AA, F, Shot, Ill) – About 3 days

#13 (37, HS, AA/H, F, Ill) – 4 days

Q4. How many other members of your household had an illness similar to yours during the past month?

_____ persons [Go to Q4A]

77 Don't know/Not Sure [Go to Module 3, question 1]

88 None [Go to Module 3, question 1]

99 Refused [Go to Module 3, question 1]

#2 (57, CD, W, F, Shot, Ill) – No one else. There are no other members in my household.

#4 (21, HS, AA, F, Shot, Ill), #13 (37, HS, AA/H, F, Ill), #16 (62, HS, W, F, Shot, Ill) – 1 person

#6 (30, CD, AA, M, Shot, Ill) – One person. [Participant hesitated, but then answered that girlfriend had similar symptoms.]

#8 (45, AD, AA, F, Shot, Ill) – 0

Q4A. What are their ages? (The number of persons should add to the total in 4.)

#4 (21, HS, AA, F, Shot, Ill) – 20

#6 (30, CD, AA, M, Shot, Ill) – 28

#13 (37, HS, AA/H, F, Ill) – 34

#16 (62, HS, W, F, Shot, Ill) - 61

MODULE 5 PROBES

Q1.-Q1a. The first few questions in this section asked you about any illnesses with a fever that you had. Were you thinking about the flu only or other illnesses as well?

#1 (53, CD, W, M, Shot) – I was thinking only about the flu.

#2 (57, CD, W, F, Shot, Ill) – I was thinking about any illness.

#4 (21, HS, AA, F, Shot, Ill), #16 (62, HS, W, F, Shot, Ill) – Other illnesses as well

#6 (30, CD, AA, M, Shot, Ill) – No, I was also thinking about colds, or strep throat.

#8 (45, AD, AA, F, Shot, Ill) – I was thinking about sinus infections or other flu-like illnesses only.

#13 (37, HS, AA/H, F, Ill) – Just the flu.

#15 (61, GD, W, M, Shot) – I was thinking of this globally, not necessarily the flu, just anything if the symptoms are flu-like.

Q2. If Q2 = Yes:

You said that you got care from a doctor, nurse or other health professional. Who specifically did you see?

#4 (21, HS, AA, F, Shot, Ill) – A doctor in the ER.

#6 (30, CD, AA, M, Shot, Ill) – My doctor.

#8 (45, AD, AA, F, Shot, Ill) – An urgent care doctor.

#16 (62, HS, W, F, Shot, Ill) – My internist.

The second question asked if you got care from a doctor, nurse or other health professional. Who do you consider a health professional to be? Can you give me some examples of who would count and who does not count?

#2 (57, CD, W, F, Shot, Ill) – Nurse practitioner or physician assistant

#4 (21, HS, AA, F, Shot, Ill) – Doctor, nurse’s aid, someone in a clinic (with a degree), a pharmacist.

#6 (30, CD, AA, M, Shot, Ill) – I was only thinking about doctors and nurses; I kind of ignored the other part.

#8 (45, AD, AA, F, Shot, Ill) – I was just thinking of a nurse or a doctor

#13 (37, HS, AA/H, F, Ill) – No one else.

#16 (62, HS, W, F, Shot, Ill) – Dentist, chiropractor, physician assistant

Q2d. If Q2d = Yes:

Did you actually receive Tamiflu or Relenza in the doctor’s office or did you get a prescription for Tamiflu or Relenza that you filled from a pharmacy?

#8 (45, AD, AA, F, Shot, Ill), #16 (62, HS, W, F, Shot, Ill) – Prescription I filled from the pharmacy

Q2E. Can you tell me what it means to be hospitalized? Is that different from just going to the hospital? How?

#4 (21, HS, AA, F, Shot, Ill) – Treated in a hospital where you have to stay overnight.

#6 (30, CD, AA, M, Shot, Ill) – Being hospitalized means spending at least one night there, not just going to the hospital.

#8 (45, AD, AA, F, Shot, Ill) – To stay in the hospital overnight.

#13 (37, HS, AA/H, F, Ill) – Were you admitted and stayed at least one night.

#16 (62, HS, W, F, Shot, Ill) – Admitted as in-patient or through the ER, probably a minimum stay of 24 hours.

Q3. If Q3 < 66:

You said that you missed X number of days of work. How did you come up with this number? Were you uncertain at all about what counted as missed work?

#4 (21, HS, AA, F, Shot, Ill) – I’m not uncertain. I think it was 2-3 days. I would count missing work for a doctor’s appointment

#6 (30, CD, AA, M, Shot, Ill) – I just rounded up, wasn’t sure if it was 3 or 4.

#8 (45, AD, AA, F, Shot, Ill) – I work from a remote location sometimes and on the days that I normally go in, I had to work from home.

#13 (37, HS, AA/H, F, Ill) – I know that my husband was not working for four days and he was home the whole time that I was sick.

Q4. When we asked about other members of your household having an illness, what did you consider to be an illness “similar to yours?”

#2 (57, CD, W, F, Shot, Ill) – Having similar symptoms

#4 (21, HS, AA, F, Shot, Ill) – Having similar symptoms also with sinus.

#6 (30, CD, AA, M, Shot, Ill) – Having a cold or strep throat, or any symptoms related to a cold.

#8 (45, AD, AA, F, Shot, Ill) – The same symptoms: sore throat/fever.

#13 (37, HS, AA/H, F, Ill) – Having flu symptoms.

#16 (62, HS, W, F, Shot, Ill) – Having the same symptoms – Achy, fever, tired, no appetite.

MODULE 6: Adverse Childhood Experiences

I’d like to ask you some questions about events that happened during your childhood. This information will allow us to better understand problems that may occur early in life, and may help others in the future. This is a sensitive topic. Some people may feel uncomfortable with these questions. Remember that your phone number has been chosen randomly and your answers are strictly confidential. At the end of this section, if you would like, I will give you phone numbers for organizations that can provide information and referral for these issues. Please keep in mind that you can ask me to skip any question that you do not want to answer.

Looking back at your childhood, before age 18:

1. Did you live with anyone who was a problem drinker or alcoholic?

Do not read:

1. Yes #16 (62, HS, W, F, Shot, Ill)

2. No #1 (53, CD, W, M, Shot), #2 (57, CD, W, F, Shot, Ill), #4 (21, HS, AA, F, Shot, Ill), #6 (30, CD, AA, M, Shot, Ill), #8 (45, AD, AA, F, Shot, Ill), #9 (39, CD, AA, F, Shot), #13 (37, HS, AA/H, F, Ill), #15 (61, GD, W, M, Shot)

7. Don’t know/Not sure

9. Refused

2. Did you live with anyone who used street drugs?

Do not read:

1. Yes

7. Don’t know/Not sure

9. Refused

3. Did you live with anyone who went to prison as a result of being convicted of a crime?

Do not read:

1. Yes

7. Don’t know/Not sure

9. Refused

4. While you were growing up were your parents ever separated or divorced?

Do not read:

1. Yes #4 (21, HS, AA, F, Shot, Ill), #15 (61, GD, W, M, Shot), #16 (62, HS, W, F, Shot, Ill)

2. No #1 (53, CD, W, M, Shot), #2 (57, CD, W, F, Shot, Ill), #6 (30, CD, AA, M, Shot, Ill), #8 (45, AD, AA, F, Shot, Ill), #9 (39, CD, AA, F, Shot), #13 (37, HS, AA/H, F, Ill)

7. Don’t know/Not sure

9. Refused

5. Did a parent or adult in your home ever swear at you, insult you, or put you down?

Do not read:

1. Yes #2 (57, CD, W, F, Shot, Ill), #6 (30, CD, AA, M, Shot, Ill), #8 (45, AD, AA, F, Shot, Ill), #15 (61, GD, W, M, Shot), #16 (62, HS, W, F, Shot, Ill)

2. No #1 (53, CD, W, M, Shot), #4 (21, HS, AA, F, Shot, Ill), #9 (39, CD, AA, F, Shot), #13 (37, HS, AA/H, F, Ill)

7. Don’t know/Not sure

9. Refused

MODULE 6 PROBES – PART 1

Q1. In your own words, what do you think a “problem drinker” is? Is this the same to you as an alcoholic, or are these different?

#1 (53, CD, W, M, Shot) – They’re basically the same, a problem drinker usually becomes an alcoholic.

#2 (57, CD, W, F, Shot, Ill) – Someone who feels they have to have at least 1-2 drinks per day. Problem drinkers and alcoholics are similar. Alcoholics drink to excess, have poor reasoning skills. Problem drinkers just need to drink, don’t recognize themselves as alcoholics

#4 (21, HS, AA, F, Shot, Ill) – Someone who gets drunk a lot. It’s the same thing as an alcoholic.

#6 (30, CD, AA, M, Shot, Ill) – Drinks every day, or gets drunk. Problem drinker is the same as alcoholic to me.

#8 (45, AD, AA, F, Shot, Ill) – Someone who is like an alcoholic; cannot control their actions from drinking

#9 (39, CD, AA, F, Shot) – An alcoholic. Someone who can’t control their drinking habits

#13 (37, HS, AA/H, F, Ill) – Same as an alcoholic.

#15 (61, GD, W, M, Shot) – Someone who drinks to the point where they lack control, or it impacts their cognitive thinking or ability. Alcoholic and problem drinker are the same.

#16 (62, HS, W, F, Shot, Ill) – Same as an alcoholic. Someone who has an addiction over their ability to stop drinking

Q2. In the second question, what kinds of drugs or other substances were you thinking of as “street drugs?”

#1 (53, CD, W, M, Shot) – Marijuana, coke, speed, any you can buy on the street and are illegal to purchase. Prescription drugs, if use illegally, would count.

#2 (57, CD, W, F, Shot, Ill) – I was thinking of marijuana, cocaine, or heroine. I PROBED ON WHETHER ABUSING PRESCRIPTION DRUGS WOULD COUNT AND SHE SAID YES.

#4 (21, HS, AA, F, Shot, Ill) – Cocaine, marijuana, heroin. I would not count prescription drugs.

#6 (30, CD, AA, M, Shot, Ill) – Pot/marijuana, coke, valium, not prescription drugs.

#8 (45, AD, AA, F, Shot, Ill) – Marijuana, cocaine, pills. Prescription medications would not count.

#9 (39, CD, AA, F, Shot) – Marijuana, cocaine, heroine. I wasn’t thinking about prescription drugs, but I guess if someone bought it on the street maybe it could count

#13 (37, HS, AA/H, F, Ill) – marijuana, (non-prescription) pills, cocaine. Abusing prescription drugs would be something else.

#15 (61, GD, W, M, Shot) – Pills, narcotics, opium, pot, prescription drugs used illegally.

#16 (62, HS, W, F, Shot, Ill) – Illicit drugs, non-prescription drugs. Prescription drugs used incorrectly or abused. Anything illegal or used incorrectly.

Q3. What did you think the phrase “went to prison” meant? (Probes: Were you thinking about any kind of jail or prison? Any other kinds of correctional facilities that you considered when thinking about that question? Were you thinking about just adults who went to prison or people less than 18 as well?)

#1 (53, CD, W, M, Shot) – State or federal prison, not a county jail, not for less than one year.

#2 (57, CD, W, F, Shot, Ill) – Even if jailed over night; spent the night behind bars for doing something illegal or inappropriate.

#4 (21, HS, AA, F, Shot, Ill) – Someone that was tried and convicted of a crime and spent time in a jail. WHEN PROBED ON WHETHER A JAIL OR A PRISON WOULD COUNT, PARTCIPANT SAID THAT JUST A PRISON WOULD COUNT.

#6 (30, CD, AA, M, Shot, Ill) – Any type of facility. I was also thinking about someone under 18.

#8 (45, AD, AA, F, Shot, Ill) - That they were in jail for a period of time. Jail/prison/any facility would count.

#9 (39, CD, AA, F, Shot) – Going to jail, locked up, serving time.

#13 (37, HS, AA/H, F, Ill) – Being incarcerated X amount of years; being behind bars. WHEN PROBED SHE SAID THAT JUST BEING IN PRISON WOULD COUNT.

#15 (61, GD, W, M, Shot) – Any type of incarceration. Age had nothing to do with it.

#16 (62, HS, W, F, Shot, Ill) – Someone convicted of a crime. Any correctional facility

What did the phrase “convicted of a crime” mean to you?

#1 (53, CD, W, M, Shot) – A misdemeanor or felony.

#2 (57, CD, W, F, Shot, Ill) – Found guilty of breaking the law.

#4 (21, HS, AA, F, Shot, Ill) – Found guilty by a jury or judge.

#6 (30, CD, AA, M, Shot, Ill) – Just any kind of crime.

#8 (45, AD, AA, F, Shot, Ill) – Convicted for doing something illegal.

#9 (39, CD, AA, F, Shot) – You’ve done something wrong and a judge found you guilty.

#13 (37, HS, AA/H, F, Ill) – Being charged with something that is illegal.

#15 (61, GD, W, M, Shot) – Held accountable, found guilty by jury or judge.

#16 (62, HS, W, F, Shot, Ill) – Someone who broke the law.

Q4. Were you uncertain at all about whether your parents had ever been separated? Can you tell me about what the term “separated” means to you? Probe for whether they were thinking about legally separated or just separated.

#1 (53, CD, W, M, Shot) – Separated means living apart for some time, not necessarily a legal separation.

#2 (57, CD, W, F, Shot, Ill) – I was just thinking of legally separated. My father moved out for a week or two, but I didn’t count that. Just legally separated.

#4 (21, HS, AA, F, Shot, Ill) – No, they were separated first and then divorced. Separated means just living apart.

#6 (30, CD, AA, M, Shot, Ill) – Legal separation, military wouldn’t count.

#8 (45, AD, AA, F, Shot, Ill) – It means not living in the same household or not wanting to live under the same roof. Military separation would not count.

#9 (39, CD, AA, F, Shot) – No. Separated means not living together. Military separation does not count.

#13 (37, HS, AA/H, F, Ill) – No. Separated means not living in the same home, legal separation or other. My dad was in the military and I think that’s different.

#15 (61, GD, W, M, Shot) – Doesn’t have to be legal, just not living together. A military separation would not count.

#16 (62, HS, W, F, Shot, Ill) – My parents were first separated and then divorced. Separated means no longer living in the same household while married. Military separation would not count. While technically separated, I would not count it.

Q5. In general, what kinds of things do you think count as things that “put you down” – can you give any examples?

#1 (53, CD, W, M, Shot) – If they were to belittle you in front of relatives or friends.

#2 (57, CD, W, F, Shot, Ill) – being demeaning.

#4 (21, HS, AA, F, Shot, Ill) – Saying something to hurt someone’s feelings

#6 (30, CD, AA, M, Shot, Ill) – No, not really.

#13 (37, HS, AA/H, F, Ill) – Saying someone is stupid, that they never do anything right, look at what you did this time, etc.

#15 (61, GD, W, M, Shot) – Saying your stupid, can’t do anything right, you’re a failure.

#16 (62, HS, W, F, Shot, Ill) – Anything humiliating, criticism, mean-spirited, non-constructive.

Now I'm going to ask you about a few more questions about some other childhood experiences that some people have had. We understand why some people might not be comfortable answering these questions, so we are going to conduct this next session a little bit differently. We do not want you to actually answer the questions, we just want to get your opinion on the question itself and whether it makes sense to you and is easy to understand.

For example in the last question, we asked you if an adult or parent in your home ever swore at you. In these next questions, we do NOT want to know your actual “yes/no” answer. Instead we want to know your answers to some follow-up questions, such as "What does the phrase “swear at you” mean?" or "What were you thinking about when you heard the phrase “put you down".

Let's practice with this question “How many rooms do you have in your home? Now, tell me how you came up with your answers? For example, did you count bathrooms porches?”

6. Did you live with anyone who was depressed, mentally ill, or suicidal?

Do not read:

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

Q6. What do the terms ”depressed” or “mentally ill” mean to you? Do these mean the same thing to you, or are they different somehow?

#1 (53, CD, W, M, Shot) – Depressed may be considered suicidal. The person probably needs medication. Mentally ill and depressed are the same.

#2 (57, CD, W, F, Shot, Ill) – Mental illness means inappropriate feelings about one’s self, misperceptions about reality, excessive inappropriate behavior. Depressed means sad or withdrawn.

#4 (21, HS, AA, F, Shot, Ill) – Depressed means taking medication to treat depression. Mentally ill is not a disease, but psychosis. Mentally ill can mean a lot of things, but if you are depressed then you are mentally ill.

#6 (30, CD, AA, M, Shot, Ill) – Having a problem and needing to use medicines, being in a hospital or psychiatric ward, constantly going to a psychiatrist. Mental illness could be different from depressed, like a serial killer.

#8 (45, AD, AA, F, Shot, Ill) – Depressed means moping around, tired, don’t want to do anything. Mentally ill means diagnosed with an illness, unable to comprehend like a normal person

#9 (39, CD, AA, F, Shot) – A chemical imbalance in the brain causing them to be withdrawn, act differently from normal. Depressed and mentally ill mean the same thing.

#13 (37, HS, AA/H, F, Ill) – Mentally ill means you can’t function doing normal daily, day-to-day things; doing things out of order; urinating on themselves; not bathing. Depression means sadness, want to be alone, not talking to anyone.

#15 (61, GD, W, M, Shot) – Having psychotic problems. Depression is a form of mental illness, which can be mild to severe.

#16 (62, HS, W, F, Shot, Ill) – Mentally ill and depressed meant the same thing. Someone who is not responding appropriately to the occasion.

What does the term “suicidal” mean to you? (Probe: In what situations would you describe someone as “suicidal?”)

#1 (53, CD, W, M, Shot) – Someone who’s not on medication and has a history of depression.

#2 (57, CD, W, F, Shot, Ill) – Someone who has ideation or any issue with suicide. Even slight, occasional thoughts about suicide would count.

#4 (21, HS, AA, F, Shot, Ill) – Someone who is thinking about or has tried to take their own life.

#6 (30, CD, AA, M, Shot, Ill) – Thinking about or planning to take their life.

#8 (45, AD, AA, F, Shot, Ill) – someone who wants to take their own life. Is thinking about it or planning it.

#9 (39, CD, AA, F, Shot) – someone who feels they are capable of killing themselves

#13 (37, HS, AA/H, F, Ill) – Having thoughts of killing yourself.

#15 (61, GD, W, M, Shot) – Using a means to end one’s life – pills, weapons, jumping.

#16 (62, HS, W, F, Shot, Ill) – Someone who is a danger to themselves; no motivation to be alive; at risk.

7. Did your parents or adults in your home ever hit, punch or beat each other up?

Do not read:

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

Q7. Can you think about any incidences a person might observe that would be difficult to decide on whether they count in this question?

#1 (53, CD, W, M, Shot) – If there was an argument, and one person loses his temper and uses any kind of physical force. It might matter if there was a history of verbal abuse in the past.

#2 (57, CD, W, F, Shot, Ill), #4 (21, HS, AA, F, Shot, Ill), #8 (45, AD, AA, F, Shot, Ill), #9 (39, CD, AA, F, Shot), #13 (37, HS, AA/H, F, Ill), #16 (62, HS, W, F, Shot, Ill) – No, it was very clear.

#6 (30, CD, AA, M, Shot, Ill) – No, can’t think of any. Maybe if parents pushed each other.

#15 (61, GD, W, M, Shot) – Physical abuse or words, which can do just as much damage.

8. Not including spanking, did a parent or adult in your home ever hit, beat, kick, or physically hurt you in any way?

Do not read:

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

Q8. Can you think about any incidences a person might observe that would be difficult to decide on whether they count in this question?

#1 (53, CD, W, M, Shot) – Maybe if the parent was really frustrated and didn’t know how to handle it.

#2 (57, CD, W, F, Shot, Ill), #4 (21, HS, AA, F, Shot, Ill), #8 (45, AD, AA, F, Shot, Ill), #9 (39, CD, AA, F, Shot), #16 (62, HS, W, F, Shot, Ill) – No, it was very clear.

#6 (30, CD, AA, M, Shot, Ill) – No, this was straightforward to me.

#13 (37, HS, AA/H, F, Ill) – I think some parents spank their child in a hard way. Certain types of spanking are too harsh.

#15 (61, GD, W, M, Shot) – Physical is physical.

9. Did an adult or anyone at least 5 years older than you ever touch you sexually or try to make you touch them sexually or force you to have sex?

Do not read:

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

9. For all of the questions in this section, were you still thinking only of experiences that happened in childhood, that is, before age 18 or did you think of any experiences that happened when you were older?

#1 (53, CD, W, M, Shot) – I was just thinking before 18.

#2 (57, CD, W, F, Shot, Ill), #8 (45, AD, AA, F, Shot, Ill) – I was thinking about both before and after 18.

#4 (21, HS, AA, F, Shot, Ill) – I was thinking about older as well. I was just answering this question this way. The others were under 18.

#6 (30, CD, AA, M, Shot, Ill) – Under 18.

#9 (39, CD, AA, F, Shot), #16 (62, HS, W, F, Shot, Ill) – Just before 18.

#13 (37, HS, AA/H, F, Ill) – Any experiences from any time.

#15 (61, GD, W, M, Shot) – Only under 18.

I just have two final questions for you. For the last several questions about different childhood experiences:

Would you be comfortable answering these kinds of questions with an interviewer who called you on your home phone? (Probe: Why or why not?)

#1 (53, CD, W, M, Shot) – I would not feel comfortable, not knowing the person and who you’re giving the information to.

#2 (57, CD, W, F, Shot, Ill), #16 (62, HS, W, F, Shot, Ill) – Yes, I would be.

#4 (21, HS, AA, F, Shot, Ill) – No because you can’t see that person. They might not be who they say they are.

#6 (30, CD, AA, M, Shot, Ill) – The beginning set would be ok, but I would be uncomfortable with the second half.

#8 (45, AD, AA, F, Shot, Ill) – No because it would be a complete stranger. I wouldn’t do it with a complete stranger over the phone.

#9 (39, CD, AA, F, Shot) – I don’t think so; it’s too personal

#13 (37, HS, AA/H, F, Ill) – Yes, it’s not too sensitive.

#15 (61, GD, W, M, Shot) – I would be very hesitant, not knowing who was asking. There’s no way to know for sure.

Would you have any concerns about answering these questions based on whether the interviewer asking the questions was male or female?

#1 (53, CD, W, M, Shot) – This would make no difference at all.

#2 (57, CD, W, F, Shot, Ill), #4 (21, HS, AA, F, Shot, Ill), #8 (45, AD, AA, F, Shot, Ill) – No, it wouldn’t matter.

#6 (30, CD, AA, M, Shot, Ill) – No.

#9 (39, CD, AA, F, Shot) – No, not really depending on the situation or surroundings. I might have problems if they came to my home.

#13 (37, HS, AA/H, F, Ill) – Yes, I would prefer a female. If a male asked me this I might think that he was some sort of pervert. It would make me feel uncomfortable.

#15 (61, GD, W, M, Shot) – I don’t think it would matter. I wouldn’t give it to either gender. I’d be more comfortable if it was in person, or maybe if it was sent out as a mail questionnaire.

#16 (62, HS, W, F, Shot, Ill) – I would be more comfortable with someone from the same sex.

2.4 Recommendations

The revisions made to the questions following Round 1 generally improved respondent understanding of these proposed new survey items. Round 2 testing did reveal some question issues that still require revisions to improve understanding and reliability for these items. Table 3 provides a summary of our specific recommendations in the order in which questions were presented in the cognitive testing protocol. For most concerns noted from cognitive testing, our recommendations include specific rewording of items. For the issues involving clarification of question objectives, specific CDC branches will need to provide input.

Table 3. Summary of Final Recommendations for Revision to Proposed New BRFSS Items

Module	Item	Proposed Recommendation for Revision
















Adverse Childhood Experiences	1	Consider providing a definition of problem drinker or alcoholic that references the frequency of drinking alcoholic drinks
Adverse Childhood Experiences	2	Add a specific instruction as to whether illegally used prescription drugs should be counted.
Adverse Childhood Experiences	3	Consider changing question to read, “Did you live with anyone who was convicted of a crime and sentenced to serve time in a prison, jail, or other corrections facility.”
Adverse Childhood Experiences	4	Clarify the question text to indicate whether only legal separations should be counted. If other separations (i.e., not legal) should be counted, then consider adding a probe to indicate that other non-legal separations should be counted.
Adverse Childhood Experiences	4	Include a volunteered option for participants whose parents were never married.
Childhood Experiences	All	Consider repeating the phrase, “Looking back at your childhood, before age 18” one or two times through the section.
Adverse Childhood Experiences	All	In order to address the sensitive nature of these questions, consider gender matching for female participants only wherever feasible.
Adverse Childhood Experiences	All	To further address the sensitive nature of these questions, consider splitting the items randomly across participants so that (1) each respondent only answers five of the nine items and (2) each respondent only answers two of the items 6 through 9. If sample sizes allow, this procedure will minimize the overall burden on individual participants to provide answers to sensitive items.

2.5 Limitations

As in any cognitive-testing study, the objective of this method of question refinement is to identify potential sources of response error. Results from this study are not intended to generalize to a larger population but rather reflect how typical participants understand and respond to proposed survey items. Further, although this group of participants varied in age, gender, and ethnicity, it is difficult to generalize the current cognitive interviewing with so few participants to the target population for the survey. We believe that the first round of cognitive testing identified the major problems with the current draft survey items. The second round of interviews was necessary to confirm that the suggested revisions to those items were effective. One limitation during Round 1 was that all health-care workers recruited were females. Also, because of the time of year (August 2006), none of the seven participants who had recent illnesses had influenza. These limitations were addressed in Round 2, in which approximately half of the health-care workers were male, and we specifically recruited participants who had the flu or flu-like symptoms. Finally, we did not require participants to provide answers to the last four items in the Adverse Childhood Experiences section. Their reactions to these items may have differed somewhat had we asked them to provide answers.

References

Centers for Disease Control and Prevention. (2005). Behavioral Risk Factors Surveillance System operational and user’s guide, version 3.0. Atlanta, GA: National Center for Chronic Disease Prevention and Health Promotion, CDC.

Tourangeau, R. (1984). Cognitive sciences and survey methods. In T. Jabine, M. Straf, J. Tanur, et al. (Eds.), Cognitive aspects of survey methodology: Building a bridge between disciplines (pp. 73–100). Washington, DC: National Academy Press.

Willis, G. B. (1999). Cognitive interviewing: A “how to” guide. Research Triangle Park, NC: Research Triangle Institute.

Willis, G. B., & Lessler, J. T. (1999). Question Appraisal System: QAS-99. Rockville, MD: Research Triangle Institute.

Appendix A

Question Appraisal System (QAS-99) Review of 2008 BRFSS Questions

QAS of 2008 BRFSS Questions for Cognitive Testing

field code during an interview.

Recommendation: Make response question and response options match to assist interviewers.

---------------------------------------------------------------------------------------------

Q2B. Did the doctor or health professional tell you this illness was the flu?

- yes -- go to Q2C

- no -- skip to Q3

- don’t know/refuse -- skip to Q2C

Problem: INCONSISTENT WITH OTHER QUESTIONS. Similar questions in the other flu module used “doctor, nurse, or health professional.”

Recommendation: Include nurse so that it is consistent with other questions in the instrument. Also too make consistent with other changes, consider “for being ill” instead of “for this illness.”

---------------------------------------------------------------------------------------------

Q2C. Did you have a flu test that was positive for this illness?

- yes

- no

- don’t know/refuse

Problem: WORDING. The phrase “for this illness” is unnecessary.

Problem: DOUBLE-BARRELED. If they say “no” you don’t know if it is because the flu test was negative or because they did not have a flu test.

Problem: KNOWLEDGE MIGHT NOT EXIST. Participant might not know if they were given a flu test or not.

Recommendation: Consider deleting the phrase “for this illness.” If it is important to know the distinction between a negative test and not having had a test, include an additional question that asks if they received a test for the flu shot.

---------------------------------------------------------------------------------------------

Q2D. Did you receive Tamiflu, also known as oseltamivir, or an inhaled medicine called Relenza or zanamivir to treat this illness?

- yes

- no

- don’t know/refuse

Problem: INCONSISTENCY WITH OTHER QUESTIONS. This question says “to treat this illness” yet the next question refers to “the flu.”

Recommendation: Make questions consistent. Either keep referring to it as “the illness” or switch over to “the flu” once it has been diagnosed.

---------------------------------------------------------------------------------------------

Q2E. Were you hospitalized for the flu?

- yes

- no

- don’t know/refuse

Problem: TECHNICAL TERMS. Participants might not know what “hospitalized” means, and whether or not that is different from going to the hospital.

Recommendation: Make the question clearer for participants. Consider, “Were you hospitalized for 24 hours or longer for the flu?”

---------------------------------------------------------------------------------------------

Q3. Did you miss work for this illness?

- yes -- go to Q3A

- no -- skip to Q4

- no, not employed -- skip to Q4

- don’t know/refuse -- skip to Q4

Problem: VAGUE. “Miss work” is vague. Does that mean an entire day or does it include an hour during lunch used to go to the doctor?

Recommendation: Consider providing a time frame. For example, “Did you miss work for one day or longer because of the flu?” Or if you are also interested in whether they took off work to see a doctor, consider revising the time from to “one hour or longer.”

---------------------------------------------------------------------------------------------

Q3A. Approximately how many days of work did you miss for this illness?

_____ [enter number] days

Problem: VAGUE. Can participants give partial days answers?

Recommendation: Specify range for response, and consider partial day answers.

---------------------------------------------------------------------------------------------

Q4. Was anyone in your household sick with an illness similar to yours during the past 30 days?

yes -- go to Q4A

no – skip to next section

- don’t know/refuse – skip to next section

Problem: NONE.

---------------------------------------------------------------------------------------------

Q4A. How many individuals?

- ____ [enter number]

- don’t know/refuse

Problem: NONE.

---------------------------------------------------------------------------------------------

Q4B. What are their ages: ____, ____, ____, ____, ____, ____, ____,

[the number of persons should add to the total in 4A.]

Problem: KNOWLEDGE. Participants may not know all the ages of the household members, particularly in a roommate situation.

Recommendation: Consider adding an interview note to gather approximate ages if necessary.

Appendix B

Participant Informed Consent Form

Participant Informed Consent

The purpose of this study is to test some questions that will be used in a national study measuring the prevalence of behaviors that place people at risk for illness and injury. Each state completes 125 to 635 interviews each month for a total of more than 150,000 interviews a year. The results of the national study will be used in planning, initiating, supporting, and evaluating health promotion and disease prevention programs. We want to add some new questions to the interview, but first we want to learn whether the new questions can be understood. So right now we’re just testing how well the new questions work.

RTI, in collaboration with the Centers for Disease Control and Prevention, is conducting a study to test survey questions on various topics related to health behaviors and some of your life experiences. You are one of a number of participants who will test these questions. You were chosen to participate because you responded to a newspaper advertisement or were otherwise recruited and met the screening criteria for participation.

The interview will be conducted in person or over the telephone and will take approximately 60 minutes. I will ask questions about your health, exposure to second hand smoke, and potentially adverse childhood experiences and then ask for additional information about how you formed your answers and what you were thinking. Once you’ve completed the interview, you will receive $40 in appreciation for your time.

It is important to understand that taking part in this interview is entirely voluntary. There are no benefits to you for participating. The only perceived risk is that some of the questions may be uncomfortable to some people. You can skip any question that you do not want to answer. There is no penalty for skipping any question or any part of the interview.

In order to make best use of our findings, we ask you to allow us to audio-tape your interview. The tape will only be heard by authorized people who are working on this project. The only purpose of tape-recording is to allow us to review the interview in order to hear how well the survey questions work. We will destroy the tape within two weeks of this interview. If at any time you would rather that the interview not be taped, just tell the interviewer and the interview will no longer be taped. In addition, our client may want to observe the interview in order to assist in identifying any problems with the questionnaire as I administer it to you. Federal law requires us to keep your answers confidential and to use your answers only for research purposes, according to the Confidential Information Protection and Statistical Efficiency Act of 2002. We will not share your answers with anyone outside this project. There are, however, some exceptions to our promise of confidentiality. If you tell us that you are in immediate danger or that you intend to harm yourself or someone else, we may need to inform the appropriate authorities according to state and local law.

If you have any questions about the study, you may call Michael Schwerin at 919-316-3878. If you have any questions about your rights as a research participant in this study, you may contact RTI's Office of Research Protection at 1-866-214-2043.

I agree to participate in this interview by signing below:

You have my consent to record the interview by audiotape:……………………□ Yes……..□ No

You have my consent for the interview to be observed by the project client:….□ Yes….□ No….□ NA

Participant Name:______________________________ Date: ______________

Interviewer Name:_____________________________ Date: ______________

Participant Informed Consent

Telephone Interview

The purpose of this study is to test some questions that will be used in a national study measuring the prevalence of behaviors that place people at risk for illness and injury. Each state completes 125 to 635 interviews each month for a total of more than 350,000 interviews a year. The results of the national study will be used in planning, initiating, supporting, and evaluating health promotion and disease prevention programs. We want to add some new questions to the interview, but first we want to learn whether the new questions can be understood. So right now we’re just testing how well the new questions work.

The interview will be conducted over the telephone and will take approximately 30 minutes. I will ask questions about your health, exposure to second hand smoke, and potentially adverse childhood experiences and then ask for additional information about how you formed your answers and what you were thinking. Once you’ve completed the interview, you will receive $40 in appreciation for your time.

It is important to understand that taking part in this interview is entirely voluntary. There are no benefits to you for participating. The only perceived risk is that some of the questions may be uncomfortable to some people. Please let me know if a question makes you feel uncomfortable. You can skip any question that you do not want to answer. There is no penalty for skipping any question or any part of the interview.

Our client may be listening to the interview via a conference line, in order to assist us in identifying any problems with the questionnaire as I administer it to you. Federal law requires us to keep your answers confidential and to use your answers only for research purposes, according to the Confidential Information Protection and Statistical Efficiency Act of 2002. We will not share your answers with anyone outside this project. There are, however, some exceptions to our promise of confidentiality. If you tell us that you are in immediate danger or that you intend to harm yourself or someone else, we may need to inform the appropriate authorities according to state and local law.

Do I have your consent to be interviewed over the telephone? .………………..□ Yes……..□ No

Do I have your consent for the interview to be observed by the project client:…□ Yes……..□ No

INTERVIEWER: By signing below you are indicating that the respondent has given responses to the two consent questions above.

Interviewer Name:______________________________________ Date: ______________

Appendix C

Cognitive Interview Protocol – Round 1

2008 BRFSS Questions for Cognitive Testing

6 Topics/Modules

Round 1 Interviews

__________________________________________________

Interviewer: _______________________

Date: _______________________

Time: _______________________

Respondent First Name: _______________________

__________________________________________________

Thank you for participating in this project. In this interview, we’re less interested in your actual answers to the questions then we are in what you think about the questions and how you understand them. We want to make sure the questions make sense and that you are familiar with all the words we use. So, after some of the questions, we may ask you additional follow-up questions, such as “tell me in your own words what this question is asking" or "how hard was it to come up with your answer?" This will help us determine if we are asking the right questions.

Even though some of our questions may make it sound like we are giving you a test, there are no right or wrong answers to these questions. Our main goal is to make sure that the questions are understandable. You can help us by pointing out any questions you find confusing or unclear.

Do you have any questions before we get started?

[ANSWER QUESTIONS AS NEEDED.]

MODULE 6: Adverse Childhood Experiences

These next set of questions are about experiences some people have when they are children. Please note that we’re just looking for yes or no answers, we’re not asking you to give us details of any of experiences that you may have had.

Looking back at your childhood, (that is) before age 18:

Q1. Growing up with alcohol abuse: Did you live with anyone who was a problem drinker, alcoholic, or who used illicit drugs?

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

In your own words, what do you think a “problem drinker” is? Is this the same to you as an alcoholic, or different?

What kinds of things do you think counts as “illicit drugs?”

Q2. Parental separation/divorce: Were your parents ever separated or divorced?

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

Q3. Incarcerated household member: Did anyone in your household go to prison?

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

Q4. Did a parent or adult in your home ever swear at you, insult you, or put you down?

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

What kinds of things do you think count as “put you down” – can you give any examples?

Now I'm going to ask you a few more questions about childhood experiences that some people have had. We understand why some people might not be comfortable answering these questions so we are going to conduct this next session a little bit differently. We do not want you to actually answer the questions, we just want to get your opinion on the questions itself and whether it makes sense to you and is easy to understand.

For example in the above question, we asked you if an adult or parent in your home ever swore at you. In this case we do NOT want to know your actual answer . Instead we want to know your answers to some follow-up questions, such as "What does the phrase “swear at you” mean?" or "Were you thinking about when you heard the phrase “put you down".

Let's practice with this question “Do you own a car?” Now, what does “car” mean to you in this question?

Q5. Was anyone in your home mentally depressed or mentally ill or tried to attempt suicide?

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

In this question, what did “mentally depressed” or “mentally ill” mean to you?

Q6. Did parents or adults in your home ever push, grab, shove, slap, or throw something at each other?

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

Did you have any experiences as child that you weren’t sure whether they counted for this question?

Q7. Physical Abuse: Not including spanking, did a parent or adult in your home ever push, grab, shove, slap or throw something at you?

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

Did you have any experiences as child that you weren’t sure whether they counted for this question?

Q8. Did an adult or anyone at least 5 years older than you ever touch you sexually or try to make you touch them sexually or force you to have sex?

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

How difficult was it for you to follow this question and provide an answer?

Did you have any experiences as child that you weren’t sure whether they counted for this question?

Appendix D

Question-by-Question Results for Round 1

Question-by-Question Comments from Round 1 Cognitive Testing

MODULE 6: Adverse Childhood Experiences

The next few questions are about experiences some people have when they were children. Please note that we’re just looking for “yes" or “no” answers to these questions, we’re not asking you to give us details of any of experiences that you may have had.

Looking back at your childhood, that is, before age 18:

Q1. Did you live with anyone who was a problem drinker, alcoholic, or who used illicit drugs?

Do not read:

1. Yes #8 (42, W, M, Shot, Ill)

2. No #1 (26, W, F, Shot, HC), #2 (40, W, F, Shot, Ill, HC), #3 (44, W, F, Shot, Ill), #4 (41, W, F, AA, Shot, Ill, HC), #5 (25, AA, F, Shot, Ill), #6 (60, W, M, Shot, Ill), #7 (25, W, M, Shot, Ill)

7. Don’t know/Not sure

9. Refused

In your own words, what do you think a “problem drinker” is? Is this the same to you as an alcoholic, or are these different?

#1 (26, W, F, Shot, HC) - Someone who gets out of control with their drinking. I think it’s pretty similar to an alcoholic, but an alcoholic is almost constantly intoxicated

#2 (40, W, F, Shot, Ill, HC), #3 (44, W, F, Shot, Ill), #4 (41, W, F, AA, Shot, Ill, HC), #5 (25, AA, F, Shot, Ill) – Alcoholic. Same thing.

#6 (60, W, M, Shot, Ill) – Same thing.

#7 (25, W, M, Shot, Ill) – A problem drinker is a lesser version than an alcoholic, someone who drinks too much.

#8 (42, W, M, Shot, Ill) – Someone who drinks every day, exceeds intoxication limits, not just a glass of wine with dinner or a beer at a cookout.

What kinds of things do you think count as “illicit drugs?”

#1 (26, W, F, Shot, HC) – Marijuana, drugs, opiates. I PROBED PARTICIPANT ON WHETHER PRESCRIPTION DRUGS TAKEN WITHOUT A PRESCRIPTION WOULD COUNT. I would count it now that you said it, but it didn’t come to mind initially

#2 (40, W, F, Shot, Ill, HC) – IV drug use, cocaine, not something like marijuana

#3 (44, W, F, Shot, Ill) – Illegal drugs: pot, LSD, crack, heroin, WHEN PROBED SHE SAID PRESCRIPTION DRUGS WOULD COUNT

#4 (41, W, F, AA, Shot, Ill, HC) – Illegal drugs, cocaine, marijuana, prescription drugs without a prescription

#5 (25, AA, F, Shot, Ill) – Any drug that’s abused or illegal for you

#6 (60, W, M, Shot, Ill) – Illegal drugs – marijuana, cocaine, hashish, that sort of thing.

#7 (25, W, M, Shot, Ill) – Drugs you’re not supposed to take, including illegal drugs and prescription drugs.

#8 (42, W, M, Shot, Ill) – Illegal drugs, or prescription drugs without a prescription.

Q2. Were your parents ever separated or divorced?

Do not read:

1. Yes #8 (42, W, M, Shot, Ill)

2. No #1 (26, W, F, Shot, HC), #2 (40, W, F, Shot, Ill, HC), #3 (44, W, F, Shot, Ill), #4 (41, W, F, AA, Shot, Ill, HC), #5 (25, AA, F, Shot, Ill), #6 (60, W, M, Shot, Ill), #7 (25, W, M, Shot, Ill)

5. Parents not ever married

7. Don’t know/Not sure

9. Refused

Were you uncertain at all about whether your parents had ever separated?

#1 (26, W, F, Shot, HC), #2 (40, W, F, Shot, Ill, HC), #4 (41, W, F, AA, Shot, Ill, HC) - No

#3 (44, W, F, Shot, Ill) – No, my parents were apart while my dad was at war, but never separated

#6 (60, W, M, Shot, Ill) – No

#7 (25, W, M, Shot, Ill) – No

#8 (42, W, M, Shot, Ill) – No, I definitely knew they had been.

If yes: Can you tell me more about what “separated” means to you?

#4 (41, W, F, AA, Shot, Ill, HC) – Means lived apart. PARTICIPANT DID NOT THINK IT NECESSARILY MEANT LEGALLY SEPARATED

#5 (25, AA, F, Shot, Ill) – Either ever being apart or legally apart

Q3. Did anyone in your household go to prison?

Do not read:

1. Yes

2. No #1 (26, W, F, Shot, HC), #2 (40, W, F, Shot, Ill, HC), #3 (44, W, F, Shot, Ill), #4 (41, W, F, AA, Shot, Ill, HC), #5 (25, AA, F, Shot, Ill), #6 (60, W, M, Shot, Ill), #7 (25, W, M, Shot, Ill), #8 (42, W, M, Shot, Ill)

7. Don’t know/Not sure

9. Refused

What does the phrase “go to prison” mean to you? (Probes: Were you thinking about any kind of jail or prison? Any other kinds of correctional facilities?)

#1 (26, W, F, Shot, HC) - For being arrested for DUI or murder or stealing. Jail counts. PARTICIPANT WAS ABLE TO REMEMBER THAT THE QUESTIONS WERE ASKING ABOUT BEFORE SHE WAS 18

#2 (40, W, F, Shot, Ill, HC) – Incarcerated, jail would count but only for an extended period like more than 30 days; 2-3 months or more

#3 (44, W, F, Shot, Ill) – Lock-up; jail; sentenced not just arrested and waiting for bond

#4 (41, W, F, AA, Shot, Ill, HC) – Well, I wasn’t thinking of visiting. Jail would count.

#5 (25, AA, F, Shot, Ill) – Incarcerated. Prison means long-term, felony. Not jail.

#6 (60, W, M, Shot, Ill) – Convicted of a crime, sentenced, given a term, and incarcerated. There are different kinds of prisons, from the one like Martha Stewart was in to maximum security. Jail is a short-term facility, and not the same, so I wouldn’t count that.

#7 (25, W, M, Shot, Ill) – Spend a lengthy time in prison. A night in jail doesn’t count. Also doesn’t count if it’s a rehabilitation facility such as drug treatment, only if convicted of a crime.

#8 (42, W, M, Shot, Ill) – Incarcerated, one or two nights in jail wouldn’t count.

Q4. Did a parent or adult in your home ever swear at you, insult you, or put you down?

Do not read:

1. Yes #6 (60, W, M, Shot, Ill), #8 (42, W, M, Shot, Ill)

2. No #1 (26, W, F, Shot, HC), #2 (40, W, F, Shot, Ill, HC), #3 (44, W, F, Shot, Ill), #4 (41, W, F, AA, Shot, Ill, HC), #5 (25, AA, F, Shot, Ill), #7 (25, W, M, Shot, Ill),

7. Don’t know/Not sure

9. Refused

In general, what kinds of things do you think count as things that “put you down” – can you give any examples?

#1 (26, W, F, Shot, HC) – Saying you weren’t smart enough or did poorly

#2 (40, W, F, Shot, Ill, HC) – Insulting me, ridiculing me because of a disability, that sort of thing

#3 (44, W, F, Shot, Ill) – Telling you you’re stupid or fat or lazy.

#4 (41, W, F, AA, Shot, Ill, HC) – Calling you stupid, crazy, saying you can’t do anything right

#5 (25, AA, F, Shot, Ill) – Abusive emotionally to self esteem and confidence. When I got in trouble they might have said, “damn it” but it was never swearing at me.

#6 (60, W, M, Shot, Ill) – I think the issue is anger – is it just discipline or a put-down.

#7 (25, W, M, Shot, Ill) – Maybe passive-aggressive behavior, making discouraging remarks.

#8 (42, W, M, Shot, Ill) – Comparing to another child in some way.

Now I'm going to ask you to review a few more questions about some other childhood experiences that some people have had. We understand why some people might not be comfortable answering these questions, so we are going to conduct this next session a little bit differently. We do not want you to actually answer the questions, we just want to get your opinion on the question itself and whether it makes sense to you and is easy to understand.

Q5. Was anyone in your home mentally depressed or mentally ill or tried to attempt suicide?

Do not read:

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

What do “mentally depressed” or “mentally ill” mean to you? Do these mean the same thing to you, or are they different somehow?

#1 (26, W, F, Shot, HC) – Mentally depressed means unhappy and can’t do daily functions. Mentally ill includes the same plus psychotic disorders

#2 (40, W, F, Shot, Ill, HC) – It means bipolar or manic. I was only thinking about serious depression. PARTICIPANT DID NOT GIVE A SEPARATE DEFINITION FOR MENTALLY ILL

#3 (44, W, F, Shot, Ill) – Unable to function, very down, not just blues. Mild or serious depression.

#4 (41, W, F, AA, Shot, Ill, HC) – Depression, past-partum depression, people who try to hurt themselves; mild or serious depression.

#5 (25, AA, F, Shot, Ill) – Bouts with depression. Both serious or mild

#6 (60, W, M, Shot, Ill) – Depression is the wrong word. You have bipolar, depression, schizophrenia, all are different conditions. This needs to be more inclusive and use the right terms. Mentally ill is the all-encompassing term.

#7 (25, W, M, Shot, Ill) – Having a problem with depression, being lethargic, unwilling to do things each day. Depression is depressed, but mentally ill could be schizophrenia, other conditions.

#8 (42, W, M, Shot, Ill) – These are the same to me, someone on medication for the problem, or just never happy.

Q6. Did parents or adults in your home ever push, grab, shove, slap, or throw something at each other?

Do not read:

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

Can you think about any incidences a person might observe that would be difficult to decide on whether they count in this question? For you, would any other terms clarify the kinds of things asked about in this question?

#1 (26, W, F, Shot, HC) – Only if they were doing it for fun, you know being silly.

#2 (40, W, F, Shot, Ill, HC), #3 (44, W, F, Shot, Ill) – No, it’s clear.

#4 (41, W, F, AA, Shot, Ill, HC) - If they are wrestling or playing around, a child might misinterpret that.

#5 (25, AA, F, Shot, Ill) – Young kids could interpret some things their parents did in the wrong way

#6 (60, W, M, Shot, Ill) – No, this is ok.

#7 (25, W, M, Shot, Ill) – Someone could make physical contact out of frustration, but not anger, which might be inappropriate, but not abuse.

#8 (42, W, M, Shot, Ill) – The only thing I can think of is that this might depend on whether you were asking someone younger.

Q7. Not including spanking, did a parent or adult in your home ever push, grab, shove, slap or throw something at you?

Do not read:

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

Can you think about any experiences a person might have that would be difficult to decide on whether they count in this question? For you, would any other terms clarify the kinds of things asked about in this question?

#1 (26, W, F, Shot, HC) – Maybe if a small child was about to run out in front of a car and the parent grabbed them to be safe.

#2 (40, W, F, Shot, Ill, HC), #3 (44, W, F, Shot, Ill) – No, it’s pretty straight forward.

#4 (41, W, F, AA, Shot, Ill, HC) – Playing around. AGREED THAT IF A SMALL CHILD RAN OUT IN FRONT OF A CAR AND PARENT GRABBED THEM THAT IT WOULDN’T CARE.

#5 (25, AA, F, Shot, Ill) – If a parent grabbed a child for protection, it might be hard to tell from the child’s perspective that they were doing it to save them

#6 (60, W, M, Shot, Ill) – Something is missing here – “hit” should be included. Also, using a tool for discipline – Belt, yardstick, or whatever.

#7 (25, W, M, Shot, Ill) – If a parent grabbed a child from running out into the street.

#8 (42, W, M, Shot, Ill) – Same as Q7.

Q8. Did an adult or anyone at least 5 years older than you ever touch you sexually or try to make you touch them sexually or force you to have sex?

Do not read:

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

If you were asked to answer this question, how difficult do you think it would it be to follow it and provide an answer? (Probe, if needed: Would you say not at all difficult, somewhat difficult, or very difficult?)

#1 (26, W, F, Shot, HC), #3 (44, W, F, Shot, Ill), #5 (25, AA, F, Shot, Ill) – Not difficult.

#2 (40, W, F, Shot, Ill, HC) – Difficult because it is an intimate question, something that you do not want to answer.

#4 (41, W, F, AA, Shot, Ill, HC) – Clear to follow, it is not difficult for me, but it might be too personal for some others.

#6 (60, W, M, Shot, Ill) – The only thing is whether this is engaging in sex or sexual abuse. Engaging in sex can be forced or seduced.

#7 (25, W, M, Shot, Ill) - Seems straightforward, sort of three things in one, but not confusing.

#8 (42, W, M, Shot, Ill) – I don’t think it would be difficult, it’s cut and dry, clear to me. Someone might not want to answer it. The question is will it be answered honestly?

For this question, were you thinking only of experiences that happened in childhood, that is, before age 18?

#1 (26, W, F, Shot, HC), #3 (44, W, F, Shot, Ill) – Yes.

#4 (41, W, F, AA, Shot, Ill, HC) – Anytime.

#5 (25, AA, F, Shot, Ill) – Yes, but it might help to use a reminder

#6 (60, W, M, Shot, Ill) – Yes, I remembered.

#7 (25, W, M, Shot, Ill) – Yes.

#8 (42, W, M, Shot, Ill) – Yes.

How difficult would it be to answer these type of questions in a telephone survey?

#1 (26, W, F, Shot, HC) – If some of these things had happened to someone it might be uncomfortable, but generally they are okay.

#2 (40, W, F, Shot, Ill, HC) – Not on a telephone interview. The other ones are okay, but not the last one. Well, the question about going to prison might be too personal also.

#3 (44, W, F, Shot, Ill) – It would be okay

#4 (41, W, F, AA, Shot, Ill, HC) – I probably wouldn’t answer on the telephone. Well, it would be okay except for the last one.

#5 (25, AA, F, Shot, Ill) – Easier over the phone than face to face, especially if you were a victim

#6 (60, W, M, Shot, Ill) – It would be ok, but people might want to add more information about their experiences.

#7 (25, W, M, Shot, Ill) – Might be easier if the questions are yes/no.

#8 (42, W, M, Shot, Ill) – I’d be more comfortable in person, you may not know who really is calling on the phone.

Would it matter if the interviewer was male or female?

#2 (40, W, F, Shot, Ill, HC), #5 (25, AA, F, Shot, Ill) – Female only

#3 (44, W, F, Shot, Ill) – Wouldn’t matter

#4 (41, W, F, AA, Shot, Ill, HC) – Gender wouldn’t matter for most of the questions

#6 (60, W, M, Shot, Ill) – I prefer a female interviewer; I just like the sound of their voice.

#7 (25, W, M, Shot, Ill) – I might be more concerned if the interviewer was female, depending on experiences.

#8 (42, W, M, Shot, Ill) – No difference to me.

Appendix E

BRFSS Cognitive Interview Protocol – Round 2

2008 BRFSS Questions for Cognitive Testing

Round 2 Interviews

Version 1 (HCW)

__________________________________________________

Interviewer: _______________________

Date: _______________________

Time: _______________________

Respondent First Name: _______________________

__________________________________________________

Thank you for participating in this project. In this interview, I’m going to ask you some questions about how you understood the survey question and how you came up with your answer. We’re less interested in your actual answers to the questions then we are in what you think about the questions and how you understand them. We want to make sure the questions make sense and that you are familiar with all the words we use. So, after some of the questions, we may ask you additional follow-up questions, such as “tell me in your own words what this question is asking" or "how hard was it to come up with your answer?" This will help us determine if we are asking the right questions.

Because these questions are designed for a study that is conducted by telephone, we will do the interview together over the phone.

Do you have any questions before we get started?

[ANSWER QUESTIONS AS NEEDED.]

Ok, I’m going to go to the next room to start the interview by telephone.

2008 BRFSS Questions for Cognitive Testing

Round 2 Interviews

Version 2 (non-HCW)

__________________________________________________

Interviewer: _______________________

Date: _______________________

Time: _______________________

Respondent First Name: _______________________

__________________________________________________

Because these questions are designed for a study that is conducted by telephone, we will do the interview together over the phone.

Do you have any questions before we get started?

[ANSWER QUESTIONS AS NEEDED.]

Ok, I’m going to go to the next room to start the interview by telephone.

MODULE 4: Adverse Childhood Experiences

Looking back at your childhood, before age 18:

1. Did you live with anyone who was a problem drinker or alcoholic?

Do not read:

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

2. Did you live with anyone who used street drugs?

Do not read:

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

3. Did you live with anyone who went to prison as a result of being convicted of a crime?

Do not read:

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

4. While you were growing up were your parents ever separated or divorced?

Do not read:

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

5. Did a parent or adult in your home ever swear at you, insult you, or put you down?

Do not read:

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

MODULE 4 PROBES – Part 1

Q1. In your own words, what do you think a “problem drinker” is? Is this the same to you as an alcoholic, or are these different?

Q2. In the second question, what kinds of drugs or other substances were you thinking of as “street drugs?”

What did the phrase “convicted of a crime” mean to you?

Q5. In general, what kinds of things do you think count as things that “put you down” – can you give any examples?

MODULE 4 – Part 2

Let's practice with this question “How many rooms do you have in your home? Now, tell me how you came up with your answers? For example, did you count bathrooms porches?”

6. Did you live with anyone who was depressed, mentally ill, or suicidal?

Do not read:

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

Q6. What do the terms ”depressed” or “mentally ill” mean to you? Do these mean the same thing to you, or are they different somehow?

What does the term “suicidal” mean to you? (Probe: In what situations would you describe someone as “suicidal?”)

7. Did your parents or adults in your home ever hit, punch or beat each other up?

Do not read:

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

Q7. Can you think about any incidences a person might observe that would be difficult to decide on whether they count in this question?

8. Not including spanking, did a parent or adult in your home ever hit, beat, kick, or physically hurt you in any way?

Do not read:

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

Q8. Can you think about any incidences a person might observe that would be difficult to decide on whether they count in this question?

9. Did an adult or anyone at least 5 years older than you ever touch you sexually or try to make you touch them sexually or force you to have sex?

Do not read:

1. Yes

2. No

7. Don’t know/Not sure

9. Refused

I just have two final questions for you. For the last several questions about different childhood experiences:

Would you be comfortable answering these kinds of questions with an interviewer who called you on your home phone? (Probe: Why or why not?)

Would you have any concerns about answering these questions based on whether the interviewer asking the questions was male or female?

2018 Field Test

Questionnaire By Version

Version 1 – Edits noted in red text

Version 2 – Edits noted in orange text

Version 3 – Edits noted in blue text

Version 4 – Edits noted in purple text

06-07-2018

TABLE OF CONTENTS

OMB Header and Introductory Text		………………………………………………………………………………..	2
Landline Introduction		………………………………………………………………………………..	3
Cell Phone Introduction		………………………………………………………………………………..	4
Core Section 1:	Health Status	………………………………………………………………………………..	5
Core Section 2:	Healthy Days	………………………………………………………………………………..	6
Rotating Module:	Healthcare Access	………………………………………………………………………………..	7
Core Section 6	Chronic Health Conditions	………………………………………………………………………………..	8
Rotating Core Section:	Arthritis	………………………………………………………………………………..	9
Core Section 8:	Demographics	………………………………………………………………………………..	10
Module:	Aspirin for CVD Prevention	………………………………………………………………………………..	11
Module:	Home/Self-measured Blood Pressure	………………………………………………………………………………..	12
Core Section 9:	Tobacco Use	………………………………………………………………………………..	13
Core Section 10:	Alcohol Consumption	………………………………………………………………………………..	14
Core Section 11:	Immunization	………………………………………………………………………………..	15
Module:	ME/CFS	………………………………………………………………………………..	16
Module:	Hepatitis Treatment and Vaccination	………………………………………………………………………………..	17
Module:	Family Planning	………………………………………………………………………………..	18
Module:	Caregiving	………………………………………………………………………………..	19
Closing Statement		………………………………………………………………………………..	20

OMB Landline Header and Introductory Text

Read if necessary	Read	Interviewer instructions
Read if necessary	Read	(not read)
Public reporting burden of this collection of information is estimated to average 27 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to CDC/ATSDR Reports Clearance Officer; 1600 Clifton Road NE, MS D-74, Atlanta, Georgia 30333; ATTN: PRA (0920-1061).	HELLO, I am calling for the Kansas Department of Health & Environment. My name is (name). We are gathering information about the health of Kansas residents. This project is conducted by the health department with assistance from the Centers for Disease Control and Prevention. Your telephone number has been chosen randomly, and I would like to ask some questions about health and health practices.	Form Approved OMB No. 0920-1061 Exp. Date 3/31/2018 Interviewers do not need to read any part of the burden estimate nor provide the OMB number unless asked by the respondent for specific information. If a respondent asks for the length of time of the interview provide the most accurate information based on the version of the questionnaire that will be administered to that respondent. If the interviewer is not sure, provide the average time as indicated in the burden statement. If data collectors have questions concerning the BRFSS OMB process, please contact Carol Pierannunzi at [email protected].

Landline Introduction

Question Number	Question text	Responses (DO NOT READ UNLESS OTHERWISE NOTED)	SKIP INFO/ CATI Note	Interviewer Note(s)
LL01.	Is this [PHONE NUMBER]?	1 Yes	Go to LL02
		2 No	TERMINATE
LL02.	Is this a private residence?	1 Yes	Go to LL04	Read if necessary:
		2 No	Go to LL03	By private residence we mean someplace like a house or apartment.
				Do not read:
				Private residence includes any home where the respondent spends at least 30 days including vacation homes, RVs or other locations in which the respondent lives for portions of the year.
		3 No, this is a business	TERMINATE	Read:
				Thank you very much but we are only interviewing persons on residential phones at this time.
LL03.	Do you live in college housing?	1 Yes	Go to LL04	Read if necessary:
				By college housing we mean dormitory, graduate student or visiting faculty housing, or other housing arrangement provided by a college or university.
		2 No	TERMINATE	Read:
				Thank you very much, but we are only interviewing persons who live in private residences or college housing at this time.
LL04.	Do you currently live in Kansas?	1 Yes	Go to LL05
		2 No	TERMINATE	Read:
				Thank you very much but we are only interviewing persons who live in Kansas at this time.
LL05.	Is this a cell phone?	1 Yes, it is a cell phone	TERMINATE	Read:
				Thank you very much but we are only interviewing by landline telephones in private residences or college housing at this time.
		2 Not a cell phone	Go to LL06	Read if necessary:
				By cell phone we mean a telephone that is mobile and usable outside your neighborhood.
				Do not read:
				Telephone service over the internet counts as landline service (includes Vonage, Magic Jack and other home-based phone services).

LL06.	Are you 18 years of age or older?	~~1 Yes, male respondent~~		~~Do not read:~~
		~~2 Yes, female respondent~~ 1 Yes		~~Sex will be asked again in demographics section.~~
		~~3 No~~ 2 No	TERMINATE	Read:
				Thank you very much but we are only interviewing persons aged 18 or older at this time.
LLSex (C08Q01)	Are you male or female?	1 Male
		2 Female
		7 DON’T KNOW/NOT SURE
		9 REFUSED

Transitions to Core Section 1

Question Number	READ	SKIP INFO/ CATI Note	Interviewer Note(s)
	I will not ask for your last name, address, or other information that can identify you. You do not have to answer any question you do not want to, and you can end the interview at any time. Any personal information that you provide will not be used to identify you. If you have any questions about the survey, please call 1-866-445-1429.	Go to C01.01	Introductory text may be reread when selected respondent is reached.

OMB Cell Phone Header and Introductory Text

Read if necessary			Read		Interviewer instructions
					(not read)
Public reporting burden of this collection of information is estimated to average 27 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to CDC/ATSDR Reports Clearance Officer; 1600 Clifton Road NE, MS D-74, Atlanta, Georgia 30333; ATTN: PRA (0920-1061).			HELLO, I am calling for the Kansas Department of Health & Environment. My name is (name). We are gathering information about the health of Kansas residents. This project is conducted by the health department with assistance from the Centers for Disease Control and Prevention. Your telephone number has been chosen randomly, and I would like to ask some questions about health and health practices.		Form Approved OMB No. 0920-1061 Exp. Date 3/31/2018 Interviewers do not need to read any part of the burden estimate nor provide the OMB number unless asked by the respondent for specific information. If a respondent asks for the length of time of the interview provide the most accurate information based on the version of the questionnaire that will be administered to that respondent. If the interviewer is not sure, provide the average time as indicated in the burden statement. If data collectors have questions concerning the BRFSS OMB process, please contact Carol Pierannunzi at [email protected].
Cell Phone Introduction (split introduction around the vehicle and safety questions – no changes to the questions themselves in version 3)

Question Number	Question text	Responses (DO NOT READ UNLESS OTHERWISE NOTED)		SKIP INFO/ CATI Note		Interviewer Note(s)
KCP01.	Are you driving a vehicle or operating machinery ~~at this time~~ right now? (The Kansas IRB requires Addition of question.)	1 Yes		Set appointment if possible. TERMINATE.		Read:
						Thank you very much. We will call you back at a more convenient time.
		2 No		Go to CP01
CP01.	Is this a safe time to talk with you?	1 Yes		Go to CP02
		2 No		Set appointment if possible. TERMINATE		Read:
						Thank you very much. We will call you back at a more convenient time.

Cell Phone Introductory Text continued

Read if necessary	Read	Interviewer instructions
Read if necessary	Read	(not read)
	~~HELLO, I am calling for the~~ ~~Kansas Department of Health & Environment. My name is (name). We are gathering information about the health of~~ ~~Kansas~~ ~~residents.~~ This project is conducted by the health department with assistance from the Centers for Disease Control and Prevention. Your telephone number has been chosen randomly, and I would like to ask some questions about health and health practices.

Cell Phone Introduction continued

Question Number	Question text	Responses (DO NOT READ UNLESS OTHERWISE NOTED)	SKIP INFO/ CATI Note	Interviewer Note(s)
CP02.	Is this [PHONE NUMBER]?	1 Yes	Go to LL02
		2 No	TERMINATE
CP03.	Is this a cell phone?	1 Yes, it is a cell phone	TERMINATE	Read:
				Thank you very much but we are only interviewing by landline telephones in private residences or college housing at this time.
		2 Not a cell phone	Go to LL06	Read if necessary:
				By cell phone we mean a telephone that is mobile and usable outside your neighborhood.
				Do not read:
				Telephone service over the internet counts as landline service (includes Vonage, Magic Jack and other home-based phone services).
CP04. (CPADULT) CP04B (CPADLTB)	Are you 18 years of age or older?	~~1 Yes, male respondent~~		~~Do not read:~~
		~~2 Yes, female respondent~~ 1 Yes		~~Sex will be asked again in demographics section.~~
		~~3 No~~ 2 No	TERMINATE	Read:
				Thank you very much but we are only interviewing persons aged 18 or older at this time.
CP05.	Do you live in a private residence?	1 Yes	Go to CP07	Read if necessary:
		2 No	Go to CP06	By private residence we mean someplace like a house or apartment.
				Do not read:
				Private residence includes any home where the respondent spends at least 30 days including vacation homes, RVs or other locations in which the respondent lives for portions of the year.

CP06.	Do you live in college housing?		1 Yes	Go to CP07	Read if necessary:
			1 Yes	Go to CP07	By college housing we mean dormitory, graduate student or visiting faculty housing, or other housing arrangement provided by a college or university.
			2 No	TERMINATE	Read:
			2 No	TERMINATE	Thank you very much, but we are only interviewing persons who live in private residences or college housing at this time.
CP07.	Do you currently live in Kansas?		1 Yes	Go to CP09
CP07.	Do you currently live in Kansas?		2 No	Go to CP08
CP08.	In what state do you currently live?		1 Alabama
			2 Alaska
			4 Arizona
			5 Arkansas
			6 California
			8 Colorado
			9 Connecticut
			10 Delaware
			11 District of Columbia
			12 Florida
			13 Georgia
			15 Hawaii
			16 Idaho
			17 Illinois
			18 Indiana
			19 Iowa
			20 Kansas
			21 Kentucky
			22 Louisiana
			23 Maine
			24 Maryland
			25 Massachusetts
			26 Michigan
			27 Minnesota
			28 Mississippi
			29 Missouri
			30 Montana
			31 Nebraska
			32 Nevada
			33 New Hampshire
			34 New Jersey
			35 New Mexico
			36 New York
			37 North Carolina
			38 North Dakota
			39 Ohio
			40 Oklahoma
			41 Oregon
			42 Pennsylvania
			44 Rhode Island
			45 South Carolina
			46 South Dakota
			47 Tennessee
			48 Texas
			49 Utah
			50 Vermont
			51 Virginia
			53 Washington
			54 West Virginia
			55 Wisconsin
			56 Wyoming
			66 Guam
			72 Puerto Rico
			78 Virgin Islands
			99 Refused
CPSex (C08Q01)		Are you male or female?	1 Male
			2 Female
			7 DON’T KNOW/NOT SURE
			9 REFUSED

Transitions to Core Section 1

Question Number	READ	SKIP INFO/ CATI Note	Interviewer Note(s)
	I will not ask for your last name, address, or other information that can identify you. You do not have to answer any question you do not want to, and you can end the interview at any time. Any personal information that you provide will not be used to identify you. If you have any questions about the survey, please call 1-866-445-1429.	Go to C01.01	Introductory text may be reread when selected respondent is reached.

Core Section 1: Health Status

Question Number	Question text	Responses (DO NOT READ UNLESS OTHERWISE NOTED)	SKIP INFO/ CATI Note	Interviewer Note(s)
C01.01	Would you say that in general your health is—	Read:
		1 Excellent
		2 Very Good
		3 Good
		4 Fair
		5 Poor
		Do not read:
		7 Don’t know/Not sure
		9 Refused

Core Section 2: Healthy Days

Question Number	Question text	Responses (DO NOT READ UNLESS OTHERWISE NOTED)	SKIP INFO/ CATI Note	Interviewer Note(s)
C02.01	Now thinking about your physical health, which includes physical illness and injury, for how many days during the past 30 days was your physical health not good?	_ _ Number of days (01-30)
		88 None
		77 Don’t know/not sure
		99 Refused
C02.02	Now thinking about your mental health, which includes stress, depression, and problems with emotions, for how many days during the past 30 days was your mental health not good?	_ _ Number of days (01-30)
		88 None
		77 Don’t know/not sure
		99 Refused
C02.03	During the past 30 days, for about how many days did poor physical or mental health keep you from doing your usual activities, such as self-care, work, or recreation?	_ _ Number of days (01-30)	Skip if C02.01, PHYSHLTH, is 88 and C02.02, MENTHLTH, is 88
		88 None
		77 Don’t know/not sure
		99 Refused

Rotating Module: Healthcare Access

Question Number	Question text	Responses (DO NOT READ UNLESS OTHERWISE NOTED)	SKIP INFO/ CATI Note	Interviewer Note(s)
HC.01 HC.1B	What is the primary source of your health care coverage? ~~Is it…~~	Read if necessary: 1 A plan purchased through an employer or union Notes: includes plans purchased through another person's employer
		2 A plan that you or another family member buys on your own
		3 Medicare
		4 Medicaid or other state program
		5 TRICARE (formerly CHAMPUS), VA, or Military
		6 Alaska Native, Indian Health Service, Tribal Health Services
		7 Some other source
		8 None (no coverage) Do not read:
		77 Don’t know/Not Sure
		99 Refused

Core Section 6: Chronic Health Conditions

Question Number	Question text	Responses (DO NOT READ UNLESS OTHERWISE NOTED)	SKIP INFO/ CATI Note	Interviewer Note(s)
	Has a doctor, nurse, or other health professional ever told you that you had any of the following? For each, tell me Yes, No, Or You’re Not Sure.
C06.01	~~(Ever told)~~ Ever told you that you had a heart attack also called a myocardial infarction?	1 Yes
		2 No
		7 Don’t know / Not sure
		9 Refused
C06.02	~~(Ever told) you had~~ (Ever told you that you had) angina or coronary heart disease?	1 Yes
		2 No
		7 Don’t know / Not sure
		9 Refused
C06.03	~~(Ever told) you had~~ (Ever told you that you had) a stroke?	1 Yes
		2 No
		7 Don’t know / Not sure
		9 Refused
C06.04	~~(Ever told) you had~~ (Ever told you that you had) asthma?	1 Yes
		2 No	Go to C06.06
		7 Don’t know / Not sure
		9 Refused
C06.05	Do you still have asthma?	1 Yes
		2 No
		7 Don’t know / Not sure
		9 Refused
C06.06	~~(Ever told) you had~~ (Ever told you that you had) skin cancer?	1 Yes
		2 No
		7 Don’t know / Not sure
		9 Refused
C06.07	~~(Ever told) you had~~ (Ever told you that you had) any other types of cancer?	1 Yes
		2 No
		7 Don’t know / Not sure
		9 Refused
C06.08	~~(Ever told) you have~~ (Ever told you that you had) chronic obstructive pulmonary disease, C.O.P.D., emphysema or chronic bronchitis?	1 Yes
		2 No
		7 Don’t know / Not sure
		9 Refused

C06.10	~~(Ever told) you have~~ (Ever told you that you had) a depressive disorder (including depression, major depression, dysthymia, or minor depression)?	1 Yes
		2 No
		7 Don’t know / Not sure
		9 Refused
C06.11	Not including kidney stones, bladder infection or incontinence, were you ever told you have kidney disease?	1 Yes		Read if necessary:
		2 No		Incontinence is not being able to control urine flow.
		7 Don’t know / Not sure
		9 Refused
C06.12	~~(Ever told) you have~~ (Ever told you that you had) diabetes?	1 Yes		If yes and respondent is female, ask:
		2 Yes, but female told only during pregnancy	Go to next section.	Was this only when you were pregnant?
		3 No		If respondent says pre-diabetes or borderline diabetes, use response code 4.
		4 No, pre-diabetes or borderline diabetes
		7 Don’t know / Not sure
		9 Refused
C06.13	How old were you when you were told you have diabetes?	_ _ Code age in years
		[97 = 97 and older]
		98 Don‘t know / Not sure
		99 Refused

Rotating Core Section: Arthritis

Question Number	Question text	Responses (DO NOT READ UNLESS OTHERWISE NOTED)	SKIP INFO/ CATI Note	Interviewer Note(s)
ARTH.01	~~Ever told you have~~ (Ever told you have) some form of arthritis, rheumatoid arthritis, gout, lupus, or fibromyalgia?	1 Yes		Do not read:
		2 No		Arthritis diagnoses include: rheumatism, polymyalgia rheumatic, osteoarthritis (not osteoporosis), tendonitis, bursitis, bunion, tennis elbow, carpal tunnel syndrome, tarsal tunnel syndrome, joint infection, Reiter’s syndrome, ankylosing spondylitis; spondylosis, rotator cuff syndrome, connective tissue disease, scleroderma, polymyositis, Raynaud’s syndrome, vasculitis, giant cell arteritis, Henoch-Schonlein purpura, Wegener’s granulomatosis, polyarteritis nodosa)
		7 Don’t know / Not sure
		9 Refused
ARTH.02	Has a doctor or other health professional ever suggested physical activity or exercise to help your arthritis or joint symptoms?	1 Yes		If the respondent is unclear about whether this means increase or decrease in physical activity, this means increase.
		2 No	If Arth01 = 2, 7 or 9, skip to next section
		7 Don’t know / Not sure
		9 Refused
ARTH.03	Have you ever taken an educational course or class to teach you how to manage problems related to your arthritis or joint symptoms?	1 Yes
		2 No
		7 Don’t know / Not sure
		9 Refused
ARTH.04	Are you now limited in any way in any of your usual activities because of arthritis or joint symptoms?	1 Yes		If a respondent question arises about medication, then the interviewer should reply:
		2 No
		7 Don’t know / Not sure
		9 Refused		"Please answer the question based on how you are when you are taking any of the medications or treatments you might use."

ARTH.05	In the next question, we are referring to work for pay. Do arthritis or joint symptoms now affect whether you work, the type of work you do or the amount of work you do?	1 Yes	If respondent gives an answer to each issue (whether works, type of work, or amount of work), then if any issue is "yes" mark the overall response as "yes."
		2 No
		7 Don’t know / Not sure
		9 Refused
			If a question arises about medications or treatment, then the interviewer should say:
			"Please answer the question based on your current experience, regardless of whether you are taking any medication or treatment."
ARTH.06	Please think about the past 30 days, keeping in mind all of your joint pain or aching and whether or not you have taken medication. During the past 30 days, how bad was your joint pain on average on a scale of 0 to 10 where 0 is no pain and 10 is pain or aching as bad as it can be?	__ __ Enter number
		[00-10]
		77 Don’t know/ Not sure
		99 Refused

Core Section 8: Demographics & Food Stamp

Question Number	Question text	Responses (DO NOT READ UNLESS OTHERWISE NOTED)	SKIP INFO/ CATI Note	Interviewer Note(s)
~~C08.01~~ ~~(Format 1)~~ ~~C08.1B~~ LLSex or CPSex	~~What is your sex?~~ ~~Are you male or female?~~	~~Do not read:~~		Moved to Introduction, will not be asked in Demographics.
		~~1 Male~~
		~~2 Female~~ ~~Do not read:~~
		~~7 Don’t know / Not sure~~
		~~9 Refused~~
C08.02	What is your age?	_ _ Code age in years
		07 Don’t know / Not sure
		09 Refused

C08.03	Are you Hispanic, Latino/a, or Spanish origin?	If “yes”, read: Are you…		One or more categories may be selected.
		1 Mexican, Mexican American, Chicano/a
		2 Puerto Rican
		3 Cuban
		4 Another Hispanic, Latino/a, or Spanish origin
		Do not read:
		5 No
		7 Don’t know / Not sure
		9 Refused
C08.04	Which one or more of the following would you say is your race?	Please read:	If more than one response to C08.04; continue. Otherwise, go to C08.06.	If 40 (Asian) or 50 (Pacific Islander) is selected read and code subcategories underneath major heading.
		10 White
		20 Black or African American
		30 American Indian or Alaska Native
		40 Asian
		41 Asian Indian		One or more categories may be selected.
		42 Chinese
		43 Filipino
		44 Japanese
		45 Korean
		46 Vietnamese
		47 Other Asian
		50 Pacific Islander
		51 Native Hawaiian
		52 Guamanian or Chamorro
		53 Samoan
		54 Other Pacific Islander
		Do not read:
		60 Other
		88 No additional choices
		77 Don’t know / Not sure
		99 Refused

C08.05	Which one of these groups would you say best represents your race?	Please read:	If 40 (Asian) or 50 (Pacific Islander) is selected read and code subcategories underneath major heading.
		10 White
		20 Black or African American
		30 American Indian or Alaska Native
		40 Asian
		41 Asian Indian	If respondent has selected multiple races in previous and refuses to select a single race, code refused
		42 Chinese
		43 Filipino
		44 Japanese
		45 Korean
		46 Vietnamese
		47 Other Asian
		50 Pacific Islander
		51 Native Hawaiian
		52 Guamanian or Chamorro
		53 Samoan
		54 Other Pacific Islander
		Do not read:
		60 Other
		77 Don’t know / Not sure
		99 Refused
C08.06	Are you…	Please read:
		1 Married
		2 Divorced
		3 Widowed
		4 Separated
		5 Never married
		Or
		6 A member of an unmarried couple
		Do not read:
		9 Refused

C08.07	What is the highest grade or year of school you completed?	Read if necessary:
		1 Never attended school or only attended kindergarten
		2 Grades 1 through 8 (Elementary)
		3 Grades 9 through 11 (Some high school)
		4 Grade 12 or GED (High school graduate)
		5 College 1 year to 3 years (Some college or technical school)
		6 College 4 years or more (College graduate)
		Do not read:
		9 Refused
C08.08	Do you own or rent your home?	1 Own	Other arrangement may include group home, staying with friends or family without paying rent.
		2 Rent
		3 Other arrangement	Home is defined as the place where you live most of the time/the majority of the year.

		7 Don’t know / Not sure	Read if necessary:
		9 Refused	We ask this question in order to compare health indicators among people with different housing situations.
C08.14	Have you ever served on active duty in the United States Armed Forces, either in the regular military or in a National Guard or military reserve unit?	1 Yes	Read if necessary:
		2 No	Active duty does not include training for the Reserves or National Guard, but DOES include activation, for example, for the Persian Gulf War.
		7 Don’t know / Not sure
		9 Refused

C08.15	Are you currently…?	Read:	If more than one, say:
		1 Employed for wages	“Select the category which best describes you”.
		2 Self-employed
		3 Out of work for 1 year or more
		4 Out of work for less than 1 year
		5 A Homemaker
		6 A Student
		7 Retired
		Or
		8 Unable to work
		Do not read:
		9 Refused
FOODSTAMP.01	In the past 12 months, have you received food stamps, also called SNAP, the Supplemental Nutrition Assistance Program on an EBT card?	1 Yes	Food Stamps or SNAP (Supplemental Nutrition Assistance Program) is a government program that provides plastic cards, also known as EBT (Electronic Benefit Transfer) cards, that can be used to buy food. In the past, SNAP was called the Food Stamp Program and gave people benefits in paper coupons or food stamps.
		2 No
		7 Don’t Know/Not Sure
		9 Refused
C08.16	How many children less than 18 years of age live in your household?	_ _ Number of children
		88 None
		99 Refused

C08.17	Is your annual household income from all sources—	Read if necessary:		If respondent refuses at ANY income level, code ‘99’ (Refused) Field Test Note: Ci3 Programming correction to C08Q17i. The word “Interviewer” has been removed.
		04 Less than $25,000	If no, ask 05;
		($20,000 to less than $25,000)	if yes, ask 03
		03 Less than $20,000	If no, code 04;
		($15,000 to less than $20,000)	if yes, ask 02
		02 Less than $15,000	If no, code 03;
		($10,000 to less than $15,000)	if yes, ask 01
		01 Less than $10,000	If no, code 02
		05 Less than $35,000	If no, ask 06
		($25,000 to less than $35,000)
		06 Less than $50,000	If no, ask 07
		($35,000 to less than $50,000)
		07 Less than $75,000	If no, code 08
		($50,000 to less than $75,000)
		08 $75,000 or more
		Do not read:
		77 Don’t know / Not sure
		99 Refused
C08.18	About how much do you weigh without shoes?	_ _ _ _ Weight		If respondent answers in metrics, put 9 in first column. Round fractions up
		(pounds or kilograms)
		7777 Don’t know / Not sure
		9999 Refused
C08.19	About how tall are you without shoes?	_ _ / _ _ Height		If respondent answers in metrics, put 9 in first column. Round fractions down
		(ft/inches or meters/centimeters)
		77/ 77 Don’t know / Not sure
		99/ 99 Refused
C08.20	To your knowledge, are you now pregnant?	1 Yes	Skip if C08.01, SEX, is coded 1; or C08.02, AGE, is greater than 49.
		2 No
		7 Don’t know / Not sure
		9 Refused

Module: Aspirin for CVD Prevention

Question Number	Question text	Responses (DO NOT READ UNLESS OTHERWISE NOTED)	SKIP INFO/ CATI Note	Interviewer Note(s)
ASP.01	How often do you take an aspirin to prevent or control heart disease, heart attacks or stroke? Would you say you take it….	Read:
		1 Daily,
		2 Some days,
		3 You used to take it but had to stop due to side effects,
		4 You used to take it but stopped because you didn’t think it was useful
		or
		4 You never took it.
		Do not read:
		7 Don’t know / Not sure
		9 Refused

Module: Home/Self-measured Blood Pressure

Question Number	Question text	Responses (DO NOT READ UNLESS OTHERWISE NOTED)	SKIP INFO/ CATI Note	Interviewer Note(s)
HSBP.01 HSBP.1B	Has your ~~healthcare provider~~ doctor, nurse, or other ~~health~~ healthcare professional recommended you check your blood pressure ~~out~~ outside of the office or at home?	1 Yes		By ~~healthcare provider~~ other ~~health~~ healthcare professional we mean nurse practitioner, a physician assistant, or some other licensed health professional.
		2 No
		7 Don’t know / Not sure
		9 Refused
HSBP.02	Do you regularly check your blood pressure outside of your healthcare ~~provider’s~~ professional’s office or at home?	1 Yes
		2 No	Go to next section
		7 Don’t know / Not sure
		9 Refused

HSBP.03 HSBP.3B	Do you take it mostly at home or on a machine at a pharmacy, grocery or similar location?	1 ~~Mostly~~ At home
		2 ~~Mostly~~ On a machine at a pharmacy, grocery or similar location
		3 Do not check it
		7 Don’t know / Not sure
		9 Refused
HSBP.04 HSBP.4B	How do you share your blood pressure numbers that you collected with your healthcare ~~provider~~ professional? Is it… mostly by telephone, other methods such as emails, internet portal or fax, or in person?	Read:
		1 ~~Mostly by~~ Telephone
		2 ~~Mostly by~~ Other methods such as email, internet portal, or fax, or
		3 ~~Mostly~~ In person
		Do not read:
		4 Do not share information
		7 Don’t know / Not sure
		9 Refused

Core Section 9: Tobacco Use

Question Number	Question text	Responses (DO NOT READ UNLESS OTHERWISE NOTED)	SKIP INFO/ CATI Note	Interviewer Note(s)
C09.01	Have you smoked at least 100 cigarettes in your entire life?	1 Yes		Do not include electronic cigarettes (e-cigarettes, njoy, bluetip), herbal cigarettes, cigars, cigarillos, little cigars, pipes, bidis, kreteks, water pipes (hookahs) or marijuana.
		2 No	Go to C09.05
		7 Don’t know/Not Sure
		9 Refused		5 packs = 100 cigarettes
C09.02	Do you now smoke cigarettes every day, some days, or not at all?	1 Every day
		2 Some days
		3 Not at all	Go to C09.04
		7 Don’t know / Not sure	Go to C09.05
		9 Refused
C09.03	During the past 12 months, have you stopped smoking for one day or longer because you were trying to quit smoking?	1 Yes	Go to C09.05
		2 No
		7 Don’t know / Not sure
		9 Refused

C09.04	How long has it been since you last smoked a cigarette, even one or two puffs?	Read if necessary:
		01 Within the past month
		(less than 1 month ago)
		02 Within the past 3 months
		(1 month but less than 3 months ago)
		03 Within the past 6 months
		(3 months but less than 6 months ago)
		04 Within the past year
		(6 months but less than 1 year ago)
		05 Within the past 5 years
		(1 year but less than 5 years ago)
		06 Within the past 10 years
		(5 years but less than 10 years ago)
		07 10 years or more
		08 Never smoked regularly
		Do not read:
		77 Don’t know / Not sure
		99 Refused
C09.05	Do you currently use chewing tobacco, snuff, or snus every day, some days, or not at all?	1 Every day	Read if necessary:
		2 Some days	Snus (Swedish for snuff) is a moist smokeless tobacco, usually sold in small pouches that are placed under the lip against the gum.
		3 Not at all
		7 Don’t know / Not sure
		9 Refused

Core Section 10: Alcohol Consumption

Question Number	Question text	Responses (DO NOT READ UNLESS OTHERWISE NOTED)	SKIP INFO/ CATI Note	Interviewer Note(s)
C10.01	During the past 30 days, how many days per week or per month did you have at least one drink of any alcoholic beverage such as beer, wine, a malt beverage or liquor?	1 _ _ Days per week		~~INTERVIEWER NOTE:~~ ~~A 40-ounce beer would count as 3 drinks, or a cocktail drink with 2 shots would count as 2 drinks.~~ INTERVIEWER NOTE: One drink is equivalent to a 12-ounce beer, a 5-ounce glass of wine, or a drink with one shot of liquor.
		2 _ _ Days in past 30 days
		888 No drinks in past 30 days	Go to next section
		777 Don’t know / Not sure
		999 Refused
C10.02	One drink is equivalent to a 12-ounce beer, a 5-ounce glass of wine, or a drink with one shot of liquor. During the past 30 days, on the days when you drank, about how many drinks did you drink on the average?	_ _ Number of drinks		Read if necessary:
		88 None		A 40-ounce beer would count as 3 drinks, or a cocktail drink with 2 shots would count as 2 drinks.
		77 Don’t know / Not sure
		99 Refused
C10.03	Considering all types of alcoholic beverages, how many times during the past 30 days did you have X [CATI X = 5 for men, X = 4 for women] or more drinks on an occasion?	_ _ Number of times	CATI X = 5 for men, X = 4 for women
		77 Don’t know / Not sure
		99 Refused
C10.04	During the past 30 days, what is the largest number of drinks you had on any occasion?	_ _ Number of drinks
		77 Don’t know / Not sure
		99 Refused

Core Section 11: Immunization

Question Number	Question text	Responses (DO NOT READ UNLESS OTHERWISE NOTED)	SKIP INFO/ CATI Note	Interviewer Note(s)
C11.01 C11.1B	During the past 12 months, have you had either ~~a flu shot or a flu vaccine that was sprayed in your nose?~~ a flu vaccine that was sprayed in your nose or a flu shot injected into your arm?	1 Yes		A new flu shot came out in 2011 that injects vaccine into the skin with a very small needle. It is called Fluzone Intradermal vaccine. This is also considered a flu shot.
		2 No
		7 Don’t know / Not sure
		9 Refused
C11.02 C11.2B	During what month and year did you receive your most recent ~~flu shot injected into your arm or flu vaccine that was sprayed in your nose?~~ flu vaccine that was sprayed in your nose or flu shot injected into your arm?	_ _ / _ _ _ _ Month/ Year
		777777 Don’t know/ Not sure
		999999 Refused
C11.03	At what kind of place did you get your last flu shot or vaccine?	Read if necessary:		Read if necessary:
		01 A doctor’s office or health maintenance organization (HMO)		How would you describe the place where you went to get your most recent flu vaccine?
		02 A health department
		03 Another type of clinic or health center (a community health center)
		04 A senior, recreation, or community center
		05 A store (supermarket, drug store)
		06 A hospital (inpatient)
		07 An emergency room
		08 Workplace
		09 Some other kind of place
		11 A school
		Do not read:
		10 Received vaccination in Canada/Mexico
		77 Don’t know / Not sure
		99 Refused

C11.04 C11.4C	~~A vaccine to prevent the human papilloma virus or HPV infection is available and is called the cervical cancer or genital warts vaccine, HPV shot,~~ ~~[Fill: if female “GARDASIL or CERVARIX”, if male “GARDASIL”]~~. Have you ever had the human papilloma virus vaccination or HPV vaccination?	1 Yes	If age >= 49, go to next section (not working, Carol P. asked to let it be)	A vaccine to prevent the human papilloma virus or HPV infection is available and is called the cervical cancer or genital warts vaccine, HPV shot, [Fill: if female “GARDASIL or CERVARIX”, if male “GARDASIL”]. (Human Papilloma Virus (Human Pap•uh•loh•muh Virus), Gardasil (Gar•duh• seel), Cervarix (Serv a rix))

		2 No	Go to C11.06
		7 Don’t know / Not sure
		9 Refused
C11.05	How many HPV shots did you receive?	_ _ Number of shots (1-2)
		3 All shots
		77 Don’t know / Not sure
		99 Refused
C11.06	Have you ever had the shingles or zoster vaccine?	1 Yes	If age ≤ 49, go to next section	Shingles is an illness that results in a rash or blisters on the skin, and is usually painful. There are two vaccines now available for shingles: Zostavax, which requires 1 shot and Shingrix which requires 2 shots.
		2 No
		7 Don’t know / Not sure
		9 Refused

Module: ME/CFS

Question Number	Question text	Responses (DO NOT READ UNLESS OTHERWISE NOTED)	SKIP INFO/ CATI Note	Interviewer Note(s)
ME/CFS.01 ME/CFS.1B	Have you ever been told by a doctor or other health professional that you had Chronic Fatigue Syndrome (CFS) or ~~Myalgic Encephalomyelitis (ME)~~ ME (Myalgic Encephalomyelitis)?	1 Yes		My-al-gic En-ceph-a-lo-my-eli-tis
		2 No	Go to next section
		7 Don’t know / Not sure
		9 Refused
ME/CFS.02 ME/CFS.2B	Do you still have Chronic Fatigue Syndrome (CFS) or ~~Myalgic Encephalomyelitis (ME)~~ ME (Myalgic Encephalomyelitis)?	1 Yes		My-al-gic En-ceph-a-lo-my-eli-tis
		2 No
		7 Don’t know/ Not sure
		9 Refused
ME/CFS.03	Thinking about your CFS or ME, during the past 6 months, how many hours a week on average have you been able to work at a job or business for pay?	Read if necessary:
		1 0 or no hours -- cannot work at all because of CFS or ME
		2 1 - 10 hours a week
		3 11- 20 hours a week
		4 21- 30 hours a week
		5 31 - 40 hours a week
		Do not read:
		7 Don’t know/ Not sure
		9 Refused

Module: Hepatitis

Question Number	Question text	Responses (DO NOT READ UNLESS OTHERWISE NOTED)	SKIP INFO/ CATI Note	Interviewer Note(s)
HTV.01	Have you ever been told by a doctor or other health professional that you had Hepatitis C?	1 Yes		Hepatitis C is an infection of the liver from the Hepatitis C virus
		2 No	Go to HTV.05
		7 Don’t know / Not sure
		9 Refused

HTV.02	Were you treated for Hepatitis C in 2015 or after?	1 Yes		Most hepatitis C treatments offered in 2015 or after were oral medicines or pills. Including Harvoni, Viekira, Zepatier, Epculsa and others.
		2 No
		7 Don’t know/ Not sure
		9 Refused
HTV.03	Were you treated for Hepatitis C prior to 2015?	1 Yes		Most hepatitis C treatments offered prior to 2015 were shots and pills given weekly or more often over many months.
		2 No
		7 Don’t know/ Not sure
		9 Refused
HTV.04	Do you still have Hepatitis C?	1 Yes		You may still have Hepatitis C and feel healthy. Your blood must be tested again to tell if you still have Hepatitis C.
		2 No
		7 Don’t know/ Not sure
		9 Refused
HTV.05 HTV.5B	The next question is about Hepatitis B. Has a doctor, nurse, or other health professional ever told you that you had hepatitis B?	1 Yes		Hepatitis B is an infection of the liver from the hepatitis B virus.
		2 No	Go to HTV.07
		7 Don’t know/ Not sure
		9 Refused
HTV.06	Are you currently taking medicine to treat hepatitis B?	1 Yes
		2 No
		7 Don’t know/ Not sure
		9 Refused
~~HTV.07~~**	~~Have you ever received the Hepatitis B vaccine?~~	~~1 Yes~~		~~The Hepatitis B vaccine is completed after a third shot is given.~~ ~~Only code “yes” if respondent indicates all shots have been completed.~~
		~~2 No~~
		~~7 Don’t know/ Not sure~~
		~~9 Refused~~
~~HTV.08~~ ~~HTV.8B~~**	~~The next question is about Hepatitis A.~~ ~~Have you ever received the Hepatitis A vaccine?~~	~~1 Yes~~
		~~2 No~~
		~~7 Don’t know/ Not sure~~
		~~9 Refused~~

**NOTE: Per Carol P, CDC program asked to remove the last two questions in the Hepatitis Module.

Module: Family Planning

Question Number	Question text	Responses (DO NOT READ UNLESS OTHERWISE NOTED)	SKIP INFO/ CATI Note	Interviewer Note(s)
FP.01	The last time you had sex with a man, did you or your partner do anything to keep you from getting pregnant?	1 Yes	Ask module only if female age 18-49.
		2 No	Go to FP.04
		3 No partner/ not sexually active	Go to next section
		4 Same sex partner
		7 Don’t know / Not sure
		9 Refused
FP.02	The last time you had sex with a man, what did you or your partner do to keep you from getting pregnant?	Read if necessary:	Go to FP.03	If respondent reports using more than one method, please code the method that occurs first on the list.
		01 Female sterilization (ex. Tubal ligation, Essure, Adiana) 02 Male sterilization (vasectomy)
		03 Contraceptive implant (ex. Nexplanon, Jadelle, Sino Implant, Implanon)		If respondent reports using “condoms,” probe to determine if “female condoms” or “male condoms.”
		04 IUD, Levonorgestrel (LNG) or other hormonal (ex. Mirena, Skyla, Liletta, Kylena)		If respondent reports using an “I.U.D.” probe to determine if “levonorgestrel I.U.D.” or “copper-bearing I.U.D.”
		05 IUD, Copper-bearing (ex. ParaGard)		If respondent reports “other method,” ask respondent to “please specific” and ensure that their response does not fit into another category. If response does fit into another category, please mark appropriately.
		06 IUD, type unknown
		07 Shots (ex. Depo-Provera or DMPA)
		08 Birth control pills, any kind
		09 Contraceptive patch (ex. Ortho Evra, Xulane)
		10 Contraceptive ring (ex. NuvaRing)
		11 Male condoms	Go to next module
		12 Diaphragm, cervical cap, sponge
		13 Female condoms
		14 Not having sex at certain times (rhythm or natural family planning)
		15 Withdrawal (or pulling out)
		16 Foam, jelly, film, or cream
		17 Emergency contraception (morning after pill)
		18 Other method
		Do not read:
		77 Don’t know/ Not sure
		99 Refused
FP.04	Some reasons for not doing anything to keep you from getting pregnant the last time you had sex might include wanting a pregnancy, not being able to pay for birth control, or not thinking that you can get pregnant.	Read if necessary:		If respondent reports “other reason,” ask respondent to “please specify” and ensure that their response does not fit into another category. If response does fit into another category, please mark appropriately.
		01 You didn’t think you were going to have sex/no regular partner
		02 You just didn’t think about it
		03 Don’t care if you get pregnant
		04 You want a pregnancy
		05 You or your partner don’t want to use birth control
		06 You or your partner don’t like birth control/side effects
		07 You couldn’t pay for birth control
		08 You had a problem getting birth control when you needed it
		09 Religious reasons
		10 Lapse in use of a method
		11 Don’t think you or your partner can get pregnant (infertile or too old)
		12 You had tubes tied (sterilization)
		13 You had a hysterectomy
		14 Your partner had a vasectomy (sterilization
		15 You are currently breast-feeding
		16 You just had a baby/postpartum 17 You are pregnant now
		18 Same sex partner
		19 Other reasons
		Do not read:
		77 Don’t know/Not sure
		99 Refused

Module: Caregiving

Question Number	Question text	Responses (DO NOT READ UNLESS OTHERWISE NOTED)	SKIP INFO/ CATI Note	Interviewer Note(s)
CG.01 CG.1B	~~People may provide regular care or assistance to a friend or family member who has a health problem or disability.~~	1 Yes		People may provide regular care or assistance to a friend or family member who has a health problem or disability. If caregiving recipient has died in the past 30 days, say: “I’m so sorry to hear of your loss.” and code 8.
		2 No	Go to CG.09
		7 Don’t know / Not sure
		9 Refused
	During the past 30 days, did you provide regular care or assistance to a friend or family member who has a health problem or disability?	8 Caregiving recipient died in past 30 days	Go to next section
CG.02	What is his or her relationship to you?	01 Mother		For example is he or she your mother or daughter or father or son?
		02 Father
		03 Mother-in-law
		04 Father-in-law		If more than one person, say:
		05 Child		“Please refer to the person to whom you are giving the most care.”
		06 Husband
		07 Wife
		08 Same-sex partner
		09 Brother or brother-in-law
		10 Sister or sister-in-law
		11 Grandmother
		12 Grandfather
		13 Grandchild
		14 Other relative
		15 Non-relative/ Family friend
		16 Unmarried partner
		77 Don’t know/Not sure
		99 Refused

CG.03 CG.3B	For how long have you provided care for that person? ~~Would you say…~~	~~Read:~~ Read if necessary:
		1 Less than 30 days
		2 1 month to less than 6 months
		3 6 months to less than 2 years
		4 2 years to less than 5 years
		5 5 years or more
		Do not read:
		7 Don’t Know/ Not Sure
		9 Refused
CG.04 CG.4B	In an average week, how many hours do you provide care or assistance? ~~Would you say…~~	~~Read:~~ Read if necessary:
		1 Up to 8 hours per week
		2 9 to 19 hours per week
		3 20 to 39 hours per week
		4 40 hours or more
		Do not read:
		7 Don’t know/Not sure
		9 Refused
CG.05	What is the main health problem, long-term illness, or disability that the person you care for has?	1 Arthritis/ Rheumatism		If respondent provides more than one say:
		2 Asthma
		3 Cancer		“Please tell me which one of these conditions would you say is the major problem?”
		4 Chronic respiratory conditions such as Emphysema or COPD
		5 Alzheimer’s disease, Dementia and other Cognitive Impairment Disorders ~~such as Alzheimer’s disease~~
		6 Developmental Disabilities such as Autism, Down’s Syndrome, and Spina Bifida
		7 Diabetes
		8 Heart Disease, Hypertension or stroke
		9 Human Immunodeficiency Virus Infection (HIV)
		10 Mental Illnesses, such as Anxiety, Depression, or Schizophrenia
		11 Other organ failure or diseases such as kidney or liver problems
		12 Substance Abuse or Addiction Disorders
		13 Injuries, including broken bones
		14 Old age/ infirmity/frailty
		15 Other
		Do not read:
		77 Don’t know/Not sure
		99 Refused
CG.06 CG.6B	Does the person you care for also have Alzheimer’s disease, dementia or other cognitive impairment disorder?	1 Yes	If CG.05 = 5 (Dementia and other Cognitive Impairment Disorders such as Alzheimer’s disease), go to CG.07 Otherwise, continue.
		2 No
		7 Don’t know/ Not sure
		9 Refused
CG.07	In the past 30 days, did you provide care for this person by managing personal care such as giving medications, feeding, dressing, or bathing?	1 Yes
		2 No
		7 Don’t know/ Not sure
		9 Refused
CG.08	(In the past 30 days, did you provide care for this person by…) managing household tasks such as cleaning, managing money, or preparing meals?	1 Yes	If CG.01 = 1 or 8, go to next module
		2 No
		7 Don’t know/ Not sure
		9 Refused
CG.09	In the next 2 years, do you expect to provide care or assistance to a friend or family member who has a health problem or disability?	1 Yes
		2 No
		7 Don’t know/ Not sure
		9 Refused

Closing

Read if necessary	Read	Interviewer instructions
Read if necessary	Read	(not read)
	That was my last question. Everyone’s answers will be combined to help us provide information about the health practices of people the state. Thank you very much for your time and cooperation.

Field Test Modifications and Comments by Version

Question Number			Question text	Responses		SKIP INFO/ CATI Note		Interviewer	Ci3 Variable Name
								Note(s)
Changes are in red text!
Version 1 -- Noon Edits -- 6-5-2018
C08.01			~~What is your sex?~~						C08Q1B
			Are you male or female?
HSBP.01			Has your ~~healthcare~~ ~~provider~~ doctor, nurse, or other health professional recommended you check your blood pressure outside of the office or at home?					By ~~healthcare provider~~ other health professional we mean nurse practitioner, a physician assistant, or some other licensed health professional.	HSBPQ1B
HSBP.03			Do you take it mostly at home or on a machine at a pharmacy, grocery or similar location?	1 ~~Mostly~~ At home					HSBPQ3B
				2 ~~Mostly~~ On a machine at a pharmacy, grocery or similar location
				3 Do not check it
				7 Don’t know / Not sure
				9 Refused
HSBP.04			How do you share your blood pressure numbers that you collected with your healthcare provider? Is it mostly by telephone, other methods such as email, internet portal or fax, or in person?	~~Read:~~					HSBPQ4B
				1 ~~Mostly~~ by Telephone
				2 ~~Mostly by~~ Other methods such as email, internet portal, or fax, or
				3 ~~Mostly~~ In person
				~~Do not read:~~
				4 Do not share information
				7 Don’t know / Not sure
				9 Refused
ME/CFS.01			Have you ever been told by a doctor or other health professional that you had Chronic Fatigue Syndrome (CFS) or ~~Myalgic Encephalomyelitis (ME)~~ ME (Myalgic Encephalomyelitis)?						CFSQ1B
ME/CFS.02			Do you still have Chronic Fatigue Syndrome (CFS) or ~~Myalgic Encephalomyelitis (ME)~~ ME (Myalgic Encephalomyelitis)?						CFSQ2B
HTV.05			The next question is about Hepatitis B. Has a doctor, nurse, or other health professional ever told you that you had hepatitis B?						HTVQ5B	Introduction sentence added to emphasis change to Hep B.
HTV.08			The next question is about Hepatitis A. Have you ever received the Hepatitis A vaccine?						HTVQ8B	Introduction sentence added to emphasis change to Hep A.
CG.06			Does the person you care for also have Alzheimer’s disease, dementia or other cognitive impairment disorder?	1 Yes		If CG.05 = 5 (Dementia and other Cognitive Impairment Disorders such as Alzheimer’s disease), go to CG.07			CGQ6B	Skip added to CGQ05 in Ci3 programming.
				2 No
				7 Don’t know/ Not sure
				9 Refused
Question Number		Question text		Responses			SKIP INFO/ CATI Note	Interviewer	Ci3 Variable Name	NOTE
								Note(s)
Changes are in red text!
Version 2 -- 5pm Edits -- 6-5-2018
CPIntroQ		HELLO, I am calling for the Kansas Department of Health & Environment. My name is (name). We are gathering information about the health of Kansas residents. This project is conducted by the health department with assistance from the Centers for Disease Control and Prevention. Your telephone number has been chosen randomly, and I would like to ask some questions about health and health practices.							CPIntroQ	Striked out text was moved to after safety questions: Vehicle & CPSafe, see CPINTCON. No variable name change.
CPINTCON		This project is conducted by the health department with assistance from the Centers for Disease Control and Prevention. Your telephone number has been chosen randomly, and I would like to ask some questions about health and health practices.							CPINTCON	New variable.
HSBP.1B		Has your doctor, nurse, or other ~~health~~ healthcare professional recommended you check your blood pressure outside of the office or at home?						By other ~~health~~ healthcare professional we mean nurse practitioner, a physician assistant, or some other licensed health professional.	HSBPQ1B	No variable name change.
HSBP.02		Do you regularly check your blood pressure outside of your healthcare ~~provider’s~~ professional’s office or at home?							HSBPQ02	No variable name change.




HSBP.4B		How do you share your blood pressure numbers that you collected with your healthcare ~~provider~~ professional? Is it mostly by telephone, other methods such as email, internet portal or fax, or in person?							HSBPQ4B	No variable name change.





C11.4C		Have you ever had the human papilloma virus vaccination or HPV vaccination?					The age skip was not working, so all adults asked the question.		C11Q4C	The variables skipped "B", went directly to "C".
Question Number	Question text				Responses		SKIP INFO/ CATI Note	Interviewer	Ci3 Variable Name	NOTE
								Note(s)
Changes are in red text!
Version 3 -- 9:30am Edits -- 6-6-2018
LL06.	Are you 18 years of age or older?				~~1 Yes, male respondent~~			Do not read: Sex will be asked again in demographics section.	~~LLAdult~~ LLAdltB
					~~2 Yes, female respondent~~
					~~3 No~~
					1 Yes
					2 No
LLSex	Are you male or female?				1 Male				LLSex	New Variable. Moved to out of demographics to introduction.
					2 Female
					7 Don't know/Not sure
					9 Refused
CP04.	Are you 18 years of age or older?				~~1 Yes, male respondent~~			Do not read: Sex will be asked again in demographics section.	~~CPAdult~~ CPAdltB
					~~2 Yes, female respondent~~
					~~3 No~~
					1 Yes
					2 No
CPSex	Are you male or female?				1 Male				CPSex	New Variable. Moved to out of demographics to introduction.
					2 Female
					7 Don't know/Not sure
					9 Refused
ARTH.01	(Ever) told you have) some form of arthritis, rheumatoid arthritis, gout, lupus, or fibromyalgia?								ARTHQ01	No variable change.
C08.01	~~Are you male or female?~~								C08Q1B	Deleted question in demographics.
C08.17	~~INTERVIEWER:~~ Annual household income is								C08Q17i	No variable change.

C10.01	During the past 30 days, how many days per week or per month did you have at least one drink of any alcoholic beverage such as beer, wine, a malt beverage or liquor?							INTERVIEWER NOTE: A 40-ounce beer would count as 3 drinks, or a cocktail drink with 2 shots would count as 2 drinks.	C10Q01	No variables change. Just added interviewer note.
C11.01	During the past 12 months, have you had either ~~a flu shot or a flu vaccine that was sprayed in your nose?~~ a flu vaccine that was sprayed in your nose or a flu shot injected into your arm?								C11Q1B
C11.02	During what month and year did you receive your most recent ~~flu shot injected into your arm or flu vaccine that was sprayed in your nose?~~ flu vaccine that was sprayed in your nose or flu shot injected into your arm?								C11Q2B
C11.4C	A vaccine to prevent the human papilloma virus or HPV infection is available and is called the cervical cancer or genital warts vaccine, HPV shot, [Fill: if female “GARDASIL or CERVARIX”, if male “GARDASIL”]. Have you ever had the human papilloma virus vaccination or HPV vaccination?							A vaccine to prevent the human papilloma virus or HPV infection is available and is called the cervical cancer or genital warts vaccine, HPV shot, [Fill: if female “GARDASIL or CERVARIX”, if male “GARDASIL”]. (Human Papilloma Virus (Human Pap•uh•loh•muh Virus), Gardasil (Gar•duh• seel), Cervarix (Serv a rix))	C11Q4C	No variable change.
CG.01	~~People may provide regular care or assistance to a friend or family member who has a health problem or disability.~~ During the past 30 days, did you provide regular care or assistance to a friend or family member who has a health problem or disability?							People may provide regular care or assistance to a friend or family member who has a health problem or disability. If caregiving recipient has died in the past 30 days, say: “I’m so sorry to hear of your loss.” and code 8.	CG.1B
Question Number	Question text				Responses		SKIP INFO/ CATI Note	Interviewer	Ci3 Variable Name		NOTE
								Note(s)
Changes are in red text!
Version 4 -- Noon Edits -- 6-6-2018
C10.01	During the past 30 days, how many days per week or per month did you have at least one drink of any alcoholic beverage such as beer, wine, a malt beverage or liquor?							INTERVIEWER NOTE: A 40-ounce beer would count as 3 drinks, or a cocktail drink with 2 shots would count as 2 drinks. INTERVIEWER NOTE: One drink is equivalent to a 12-ounce beer, a 5-ounce glass of wine, or a drink with one shot of liquor.	C10Q01		No variable change. Just corrected interviewer note.
HTV.07	~~Have you ever received the Hepatitis B vaccine?~~								HTVQ07		Question removed per CDC program. Will not be part of the module.
HTV.8B	The next question is about Hepatitis A. Have you ever received the Hepatitis A vaccine?								HTVQ8B		Question removed per CDC program. Will not be part of the module.

1 Beatty PC (2003). “Answerable Questions: Advances in the Methodology for Identifying and Resolving Questionnaire Problems in Survey Research.” Doctoral Dissertation, University of Michigan in Dissertation Abstracts International 64(09): 3504A.

2 Beatty PC (2003). “Answerable Questions: Advances in the Methodology for Identifying and Resolving Questionnaire Problems in Survey Research.” Doctoral Dissertation, University of Michigan in Dissertation Abstracts International 64(09): 3504A.

3 Beatty PC (2003). “Answerable Questions: Advances in the Methodology for Identifying and Resolving Questionnaire Problems in Survey Research.” Doctoral Dissertation, University of Michigan in Dissertation Abstracts International 64(09): 3504A.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Modified	0000-00-00
File Created	0000-00-00