3 English Cognitive Interview Script

ATTACHMENT B-1: INTERVIEW PROTOCOL, ENGLISH

OMB No. 0952-0589

Exp. 04/30/2014

STATEMENT OF PRIVACY: Collection of this information is authorized by The Public Health Service Act, Sections 411 (42 USC 285 a) and 412 (42 USC 285a-1.a and 285a1.3). The purpose of this data collection is to evaluate whether the survey questions are easy to understand. The results of the data collection will be used to improve the survey instrument. Rights of study participants are protected by The Privacy Act of 1974. Participation is voluntary, and there are no penalties for not participating or withdrawing from the study at any time. Refusal to participate will not affect your benefits in any way. The information collected in this study will be kept private under the Privacy Act and will only be seen by people authorized to work on this project. The report summarizing the findings will not contain any names or identifying information. Identifying information will be destroyed when the project ends.

NOTIFICATION TO RESPONDENT OF ESTIMATED BURDEN: Public reporting burden for this collection of information is estimated to average 90 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to: NIH, Project Clearance Branch, 6705 Rockledge Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-0589). Do not return the completed form to this address.

Cognitive Interview Protocol – FDA Cycle

Thank you for agreeing to come in today. Let me tell you a little about Westat and what we’re going to do today.

Westat is a research company that conducts research on many different topics, under contract with many different organizations. This particular project is for the National Institutes of Health, specifically, the National Cancer Institute. Westat has been contracted to conduct a survey about how adults in the United States get information about health, and other related topics such as their attitudes and behaviors regarding health. For this survey we will be sending questionnaires out in the mail to thousands of people nationwide.

At this stage we are still developing the questionnaire and trying to figure out how to make it as easy as possible for people to understand and fill out correctly. So that is what you’ll be helping us with – I’ll have you look at and fill out the questionnaire to see what your reaction is to it, and what difficulties you have with it.

Before we get started, I want to let you know that everything we cover in the interview will be kept private - only the few people actually working on the project will have access to the information you share with us. If there are any questions you don’t want to answer that is perfectly fine - We’re really more interested in how you interpret the questions.

If it’s OK with you, I would like to tape our conversation, just so that I can review it later when we’re analyzing all the interview data. (And finally, some of our colleagues from the National Cancer Institute are here today – behind the mirror - to observe and see for themselves how the questions work.)

CONSENT FORM: Here is a form I must ask you to look over and sign – it basically covers the points I’ve just gone over with you and indicates you have agreed to take part (ONE COPY FOR RESPONDENT, ONE COPY FOR US)

This survey will be mailed to a random sample of home addresses across the country. The only information we know when we mail the survey is the address, nothing else. We don’t know who lives at the address. Before we actually start talking about the questionnaire we’re developing, I’d like to get an understanding of how the mail is usually handled at your house. For example...

• Who gets the mail, how does it get distributed, and so on?

• IF NEEDED: Who would probably look at this (POINT TO MAILING PACKAGE) first, in your household? (It won’t be addressed to any specific person, it will say something like “The Rockville Resident at….” and then the street address) (What would likely happen to it?)

So let’s move on. I’d like you to fill it out the best you can, but let me say a few things first….

• I’d like you to read aloud as you go along – just read the question and whatever your answer is. I’m going to be filling in your answers on my copy.

• As much as possible, just read and respond to this the way you think you normally would if you were at home doing this on your own. Again, some people might read practically every word they see, while others might be inclined to skip stuff here and there – just do whatever feels right to you.

• Please share whatever reactions come to mind while you’re filling it out. We’re especially interested in knowing about things that confuse you, such as when you’re not quite sure what a question is asking you – just let me know when that happens.

• Finally, we’ll pause now and then to discuss something about a question – I might ask you to tell me why you answered the way you did, or ask you to tell me in your own words what you think the question is asking.

ANY QUESTIONS?

HAND RESPONDENT THE MAILING PACKAGE. PACKAGE INCLUDES:

• Cover letter

• A questionnaire

SURVEY PACKAGE:

NOTE ANY REACTIONS THAT R HAS: [So what are you thinking at this point?]

OBSERVATION: DID R READ THE COVER LETTER?

• R read letter completely and very carefully

• R read it completely but very quickly/not so carefully

• R read some of the letter

• R skipped most of the letter

• R basically ignored the letter

INTERVIEWERS, PLEASE NOTE:

• Fill out your copy of the questionnaire as the respondent says his/her answers – it will help with the probing.

• When respondents miss a skip instruction let them proceed and on their own unless they appear confused by a question not applying to them. When this happens, point out the skip instruction and show them where to go next. If appropriate, probe on what happened at the skip instruction initially.

• Observe carefully and probe as appropriate for potential misinterpretations, inability to recall, failures to answer, missing response choices, or anything else that my result in response error. Clues to look for include taking a long time to answer, having to read a question more than once, changing one’s answer, and body language (e.g., I noticed it took you longer to answer that question than most of the other questions here – what were you thinking about there? OR I noticed you got a funny look on your face when you read that question – can you tell me what that was about?)

• In general, do your probing once the respondent has completed the section. Focus the respondent on a question by saying something like “Let’s talk about question B6.” The main exception is when a respondent is showing clear difficulty/confusion with a question – in that case, probe immediately.

QUESTIONNAIRE SECTION A (LOOKING FOR HEALTH INFORMATION)

A17: Probe about any responses that are marked (e.g. Tell me more about that.) Is there anything else that should be in this list?

A19: How did you come up with your answer?

A20: Probe about any responses that are marked (e.g. Tell me more about that.) Is there anything else that should be in this list?

QUESTIONNAIRE SECTION B (USING THE INTERNET TO FIND INFORMATION)

B3: How often do you use the Internet?

QUESTIONNAIRE SECTION C (TOBACCO PRODUCTS)

C17: How did you come up with your answer? Can you give me an example of what you bought?

C18: Did you look at the images at the top of the page? Did the images at the top of the page help you answer these questions?

C26: Probe on two of the items in the list (e.g. how did you come up with your answer?) May also want to ask, what do you know about nicotine?

C27: Probe on two different items in the list where the respondent answers differently. Try to understand the differences between the categories.

QUESTIONNAIRE SECTION D (EXPOSURE TO MESSAGES ABOUT TOBACCO)

D1, D3, D7: If the respondent indicates that they’ve heard any of these messages, ask “What have you seen, heard or read about [MESSAGE]?” Probe on two of the items in the list (e.g. How did you come up with your answer?). Try to understand the difference between the responses. Also ask, can you tell the difference between the different types of newspapers in this question? What about the different Internet sites? Can you provide some examples of each?

QUESTIONNAIRE SECTION E (CONTENTS OF TOBACCO PRODUCTS):

E2: Tell me about the last time you saw this list.

E3: What kind of list do you think this question is asking about?

E8: What kind of list is this question asking about?

E9: In your own words, why would you be interested in seeing such a list?

E10: In your own words, why would you not be interested in seeing such a list?

E14: If yes, tell me more about your answer. If no, what is this question asking about? What does it mean for someone to review tobacco products?

E15: What is this question asking about?

E16: What does it mean to test tobacco products for harmful chemicals?

E17: Pick one or two yes responses and ask the respondent to tell you about the answer.

E18: What does it mean to be qualified to review tobacco products? Probe on two items in the list where the respondent answers differently.

E19: What does it mean to be unbiased? Probe on two items in the list where the respondent answers differently. Is qualified different from unbiased? Are there other words that might be easier to understand in these last two questions?

E20: How did you decide whether something was [high, medium, low] priority? Probe on two items from the list where the respondent answers differently.

E33-E34: How did you come up with your answer? Have you ever heard of this claim? What does it mean for it to “make a claim”? Where would you expect to see this claim?

QUESTIONNAIRE SECTION F (BELIEFS ABOUT CIGARETTE CLAIMS)

F1-F3: How did you come up with your answer? Try to understand what it is about the package that the respondent is basing their answer on. Do these packs (and brand names) look believable (like a real cigarette pack)? Why?

F4: Tell me about your answer. Have you ever heard of low nicotine cigarettes? What have you heard?

F5-F7: How did you come up with your answer? Try to understand what it is about the package that the respondent is basing their answer on. Do these packs (and brand names) look believable (like a real cigarette pack)? Why?

F8: Tell me about your answer. Have you ever heard of a cigarette being TSNA free? What have you heard?

F10: How did you come up with your answer?

F11: How did you come up with your answer?

F12: What does TSNA free mean?

F13: How did you come up with your answer?

F14: How did you come up with your answer? Did having the packs to look at help you to think about your responses (compare questions with packs F1-F8 to those without F9F14—which were easier to answer and why?)

QUESTIONNAIRE SECTION G (DIETARY SUPPLEMENTS)

G1: What is a dietary supplement? Have you ever heard the claim that a dietary supplement product "may produce anticarcinogenic effects in the body?"  Probe on two items in the list to understand how the respondent is interpreting them.

G2: Tell me about your answer.

G3: Tell me about your answer.

G4: Probe on two items in the list where the respondent answers differently (e.g. how did you come up with your answer?)

G6-G8: If yes, tell me about the last time you gave a dietary supplement to a child in your family.

G9: Tell me about your answer.

G10: Tell me more about these symptoms.

G11: How confident are you that this supplement was related to your problem? Why do think the supplement was related?

G12: Did you tell anyone else? Who was it?

QUESTIONNAIRE SECTION H (FOOD AND DRUG ADMINISTRATION)

H1: What do you know about the FDA? In your own words, what is the FDA’s main purpose?

H2: Tell me about the last time you heard that the FDA has approved a drug.

H3: Has a drug you have been using ever been recalled? IF YES, what did you do? IF NO, Tell me about the last time you heard about a drug being recalled.

H4: Tell me about the last time that you paid attention to an FDA investigation.

H5: What type of information is this question asking about?

H6: Has a type of food you bought ever been recalled? IF YES, what did you do? IF NO, Tell me about the last time you heard about food being recalled.

H7: How did you come up with your answer?

H8: Probe on two items in the list where the respondent answers differently (e.g. how did you come up with your answer?)

H9: Probe on two items in the list where the respondent answers differently (e.g. how did you come up with your answer?)

H10: Probe on two items in the list where the respondent answers differently (e.g. how did you come up with your answer?)

SECTION I (YOU AND YOUR HOUSEHOLD)

I19-I20: Try to understand what type of tobacco use this captures. Tell me about the people who use tobacco in your household. If no one uses tobacco, ask does anyone ever use tobacco in your house? What types of tobacco products are you thinking about when you answer these questions?

FINAL SUMMARY PROBE

So now that you’ve looked at and answered all these questions, let me ask you something. When we conduct surveys we worry about people starting to fill out the questionnaire, but then getting bored or perhaps even irritated by the questionnaire, and thus not finishing it and sending it back to us.

• If you had been doing this at home on your own, how do you think you would have handled this questionnaire once you began answering it?

• Were there any questions that stick out in your mind in this questionnaire, for any reason? You can take a minute and glance through the questionnaire if you want to. IF NEEDED - For example,

  • anything that seemed kind of irrelevant or unimportant?

  • Anything that seemed particularly important for National Cancer Institute or policy makers to know?

  • Anything that seemed a bit too personal or sensitive to ask about?

  • Anything that seemed difficult to answer?

  • Anything else about the questionnaire that you think we should think about?

INTERVIEWERS: If there is any time left, check with observers to see if they have any additional questions for this respondent. If not, thank the respondent and dismiss (give them their incentive and have them sign the receipt).

10