Delaware Cognitive Interview Protocol

Delaware Client Integrity Message Testing Protocol

Participant ID: ______ Interviewer initials: _____ SNAP? __Current __Past __Never

Interview Date: ___ / ___ /2023 Start Time: ____________ End Time: ____________

Read or paraphrase the following intro text:

Hello. My name is ________________, and I work for RTI International. We are working with the U.S. Census Bureau to conduct these interviews. Thank you for agreeing to participate in our study.

Today we are talking to people about the information they receive from the SNAP food assistance program. The information should be clear and easy to understand, so people who are eligible receive their benefits know about rules and requirements to continue participating in SNAP. We have found that the best way to see whether information is clear is to meet with people in person and get their feedback as they read letters and things like that. You will be helping us evaluate some of the information that Delaware provides to people who participate in SNAP.

To ensure we capture notes of your feedback during today’s interview we will call my colleague so that they can take notes.

[INTERVIEWER: CALL NOTETAKER]

Before we start, there is a form I would like you to read and sign. [HAND PARTICIPANT CONSENT FORM]

This is a consent form. It explains the purpose of today’s session and your rights as a participant. It also informs you that we would like to audio record this session. We do this to get an accurate record of your feedback for our notes. Our session today is completely confidential. If you currently receive or are applying for benefits, this interview will not have any impact on your benefits. Your participation in this study is completely voluntary. You may decline to answer any particular question and you may stop the interview at any time. We do not share information with anyone. Only the people on the project have access to the recording and we do not use any names in our reports. Take a look and let me know if you have any questions. Please sign it when you are finished, and then I will also sign it.

[SIGN CONSENT FORM AFTER PARTICIPANT SIGNS]

Think Aloud: As you read the materials today, I am interested in the thoughts and reactions that go through your mind as you process the messages. So I would like you to tell me everything that you are thinking as we go through these materials.

Practice: Let’s practice before we start. [SHOW PARTICIPANT PRACTICE PARAGRAPH]

Please read this short paragraph and say what comes into your mind as you’re reading it.

Our solar system was formed about 4.5 billion years ago. It consists of our sun and eight main planets: Mercury, Venus, Earth, Mars, Jupiter, Saturn, Uranus and Neptune. Mercury, Venus, Earth and Mars are mostly made of rock and metal and are called “terrestrial planets.” Jupiter, Saturn, Uranus and Neptune are called “gas giants” because they are mostly composed of gas.

Practice probes: In your own words, can you tell me what this paragraph is saying?

Did you find any part of it difficult to understand?

Thanks; that was great. So as we go through the SNAP materials, I will ask you questions like these. I really want to hear your opinions and reactions, so don’t hesitate to speak up whenever something is unclear, or not easy to understand. We are going to look at some materials and I’m going to ask you questions about each one. Do you have any questions before we begin?

[INTERVIEWER TURN ON RECORDER. CHECK BATTERY LEVEL. NOTETAKER START RECORDING IN CENSUS VDI.]

General questions

• The SNAP program provides information about various rules, instructions, and things like that to people receiving or applying for food assistance. Do you recall receiving any of these kinds of information?

  • (If yes) What kinds of rules or instructions do you recall receiving?

• In your opinion, would you say you have a good understanding of the SNAP rules? Why do you say that?

• Do you recall getting any of these kinds of information that was not clear?

  • (If yes) Please tell me about that. What wasn’t clear about it?

  • Were you able to figure out what that information really meant? If so, how did you figure it out?

• If you had a question about your SNAP benefits, where would you go for more information?

Topic 1 - Card management

Exhibit 1: Program Information and Rights and Responsibilities (Page 18/19)

[HAVE PARTICIPANT REVIEW SPECIFIC SECTIONS OF THIS DOCUMENT BELOW FOR THIS TOPIC.]

This is part of a packet of information provided by the DE SNAP program. Please take a few minutes to read this and then we will talk about it. If necessary: Please remember to think aloud as you read.

Now let’s look at this section.

• In your own words, can you tell me what this is saying?

• In your opinion, is the information in this section useful or not useful? Why do you say that?

• Is anything here confusing or unclear? Why do you say that?

• Is there any other information you would want to see that was not here?

• Please tell me in your own words what a PIN is.

Exhibit 2: EBT Brochure

[HAVE PARTICIPANT REVIEW BROCHURE IN ITS ENTIRETY. THEN POINT PARTICIPANT TO RELEVANT SUBTOPIC EXCERPTS BELOW AND PROBE.]

Here is a brochure with some information that someone would get when they are approved for SNAP. Please take a few minutes to read this page and then we will talk about it. If necessary: Please remember to think aloud as you read.

• In your own words, what is the purpose of this document?

• Is there any information here that you think is especially important?

  • If yes: What information do you think is important? Why do you say that?

• Did you find any part of it difficult to understand?

  • If yes: Which parts were difficult?

• Did you read anything on this page that you didn’t know before?

• Was there anything that you expected to see in this page that was not here?

I now have some questions about specific sections. [POINT PARTICIPANT TO SELECTED SECTIONS.]

• How would you describe this section in your own words?

Now how about this section?

• What does this section mean to you?

• This section says to keep your card in a safe place. What does that mean to you?

  • (If needed) What are some ways someone might keep their card safe?

• Is anything in this section confusing or unclear?

Thank you for sharing your thoughts on that. Now let’s take a look at another section of this brochure. Some of the questions that I ask you may sound repetitive, so thank you for your patience as we talk about these materials.

Topic 2 - Eligible/ineligible purchases

Exhibit 2: EBT Brochure (Continued)

[PARTICIPANT REVIEWED BROCHURE IN ITS ENTIRETY UNDER TOPIC ONE. FOR TOPIC 2, POINT PARTICIPANT TO RELEVANT SUBTOPIC EXCERPTS BELOW AND PROBE.]

Please take a moment to re-read this section. If necessary: Please remember to think aloud as you read.

• In your own words, can you tell me what this section means to you?

• Did you find any part of it difficult to understand?

  • If yes: Which parts were difficult?

• What does “unprepared food” mean to you?

• What does “non-food items” mean to you?

Thank you for sharing your thoughts on that. Now let’s take a look at another brochure from the DE SNAP program.

Exhibit 3: DSS Brochure

[POINT PARTICIPANT TO RELEVANT SUBTOPIC EXCERPTS IN BROCHURE BELOW AND PROBE.]

Now please take a look at this information in this brochure. If necessary: Please remember to think aloud as you read.

• What does this information mean to you?

• Did you find any part of it difficult to understand?

  • If yes: Which parts were difficult?

• How would you describe the kinds of food that you are allowed to buy with SNAP?

• How would you describe the kinds of foods and other items that you are NOT allowed to buy with SNAP?

• Did you see anything in this section that you did not know before?

• Is there any other information you would want to see on a page like this?

Topic 3 – Trafficking

Exhibit 3: DSS Brochure (Continued)

[POINT PARTICIPANT TO RELEVANT SUBTOPIC EXCERPTS IN BROCHURE BELOW AND PROBE.]

Now let’s look at the statement at the bottom of that brochure. Please remember to think aloud as you read.

• Please tell me in your own words what this section means to you.

• In your opinion, is anything about this page confusing or unclear?

  • If yes: Which parts were not clear?

• What does it mean to buy, sell, or trade an EBT card?

• Did you see anything in this section that you did not know before?

• Is there any other information you would want to see in a statement like this?

Exhibit 1: Program Information and Rights and Responsibilities (Page 18)

[HAVE PARTICIPANT REVIEW SPECIFIC SECTIONS OF THIS DOCUMENT BELOW FOR THIS TOPIC.]

Now please take a few minutes to read this section which is part of the application for benefits in DE.

• Please tell me in your own words what this section is about.

• Did you learn anything in this section that you did not know before? What was that?

• Is there any other information you would want to see in a page like this?

• In your opinion, is anything about this page confusing or unclear?

  • If yes: Which parts were not clear?

    • Do you have any suggestions for making this page clearer or easier to read?

• This section talks about trafficking.

  • What does this mean to you?

  • [IF NEEDED] Can you think of any examples of what might be considered trafficking?

• [IF NEEDED; POINT TO FIRST BULLET] How about this first bullet, what does this mean to you?

  • In your opinion, what is the difference between the first bullet and the second bullet?

• [IF NEEDED; POINT TO FIFTH BULLET] And how about this bullet, what does that mean to you?

Exhibit 4. Application (Page 12)

[HAVE PARTICIPANT REVIEW SPECIFIC SECTIONS OF PAGE 12 BELOW FOR THIS TOPIC.]

Now please take a look at this information. If necessary: Please remember to think aloud as you read.

• Please tell me in your own words what this section is about.

• Is there any information that you think is especially important?

  • If yes: What information do you think is important? Why do you say that?

• Did you find any part of this difficult to understand?

  • If yes: Which parts were difficult?

    • Do you have any suggestions for making this page clearer or easier to read?

Topic 4 - Fraud reporting

Exhibit 5. Fraud Reporting Webpage

[HAVE PARTICIPANT REVIEW WEBSITE EXCERPT BELOW IN ITS ENTIRETY -.]

Next, we will review some information from the DE DHSS website. Please remember to think aloud as you read this information.

• In your opinion, what is the purpose of this web page?

• In your opinion, is this information clear or not clear? Why do you say that?

• Did you learn anything in this section that you did not know before? If yes, what information here is new for you?

• [IF NEEDED, POINT TO FIRST SECTION “Overview”] Please tell me in your own words what this first section means to you.

  • [IF NEEDED, POINT TO FIRST SENTENCE “Audit and Recovery…”] What does this first sentence mean to you?

    • [IF NEEDED] What does “acts of fraud or error that cause an overpayment of benefits” mean to you?

  • [IF NEEDED, POINT TO SECOND SENTENCE “ARMS is also responsible…”] What does this next sentence mean to you?

• [IF NEEDED, POINT TO SECTION UNDER “Guidelines…”] How about this next section, what does that mean to you?

• If you were thinking about reporting some kind of fraud, would you have any concerns about providing your name or other personal information?

  • [NOTE WHETHER P ATTENDS TO SENTENCE “(Persons reporting welfare fraud can choose to remain anonymous)”]

  • [IF NEEDED, POINT TO “(Persons reporting welfare fraud…”] Do you recall whether you saw this sentence when you reading this page?

    • What does that mean to you?

  • If you wanted to report fraud, do recall seeing any information on this page about how you could do that?

Exhibit 6. Fraud Reporting Flyer

[HAVE PARTICIPANT REVIEW IN ITS ENTIRETY AND PROBE.]

Next, we will take a look at another flyer. Please remember to say anything that comes to mind as you review.

• In your opinion, what is the purpose of this document?

• Do you have any questions after reviewing this sheet?

• In your opinion, is anything about this page confusing or unclear?

  • If yes: Which parts were not clear?

• Is welfare fraud a term you have heard before?

  • What comes to mind when you hear welfare fraud?

• [POINT TO *TIPS] What does *TIPS mean to you?

• Thinking about the information we reviewed before about trafficking, in your opinion is it clear whether there is a connection between trafficking and the information on this page? Why do you say that?

  • In your opinion, do trafficking and welfare fraud mean the same thing, or are they different? Why do you say that?

• Do you have any comments on this flyer that you want to share?

Those are all the documents I have to review with you today. Thank you for sharing your thoughts with me about them.

Debriefing Questions (time permitting):

Now I just have a few more questions before we are finished.

Thinking about all the materials we have reviewed today, is there anything you would like to add that we didn’t get a chance to talk about yet?

Government agencies collect information when people fill out forms for programs. One example is the information given when someone fills out an application for food assistance benefits. Government agencies also collect data using surveys, like the census that is taken every ten years about the population.

How would you feel if the information that agencies get from the application forms, such as the one for food assistance benefits, was shared with other government agencies instead of those other agencies asking you directly?

[IF NEEDED:] For example if you completed the SNAP application, it asks a question about your date of birth. How would you feel if about your date of birth that you gave to SNAP was used by another agency that was conducting a survey, instead of the other agency asking you for your date of birth?

[IF NEEDED:] So the answer that you had given on your SNAP application would be used by the other agency. In your opinion, would that be okay or would that not be okay?

[IF NEEDED:] Tell me more about why you answered that way?

Do you have any other last thoughts you’d like to share?

Thank you for your help today!

[ASK PARTICIPANT TO FILL OUT VOUCHER FORM. GIVE PARTICIPANT INCENTIVE]

9