Alabama Client Integrity Cognitive Testing Protocol

Alabama Client Integrity Message Testing Protocol

Participant ID: ______ Interviewer initials: _____ SNAP? __Current __Past __Never

Interview Date: ___ / ___ /2023 Start Time: ____________ End Time: ____________

Read or paraphrase the following intro text:

Hello. My name is ________________, and I work for RTI International. We are working with the U.S. Census Bureau to conduct these interviews. Thank you for agreeing to participate in our study.

Today we are talking to people about the information they receive from the SNAP food assistance program. The information should be clear and easy to understand, so people who are eligible receive their benefits know about rules and requirements to continue participating in SNAP. We have found that the best way to see whether information is clear is to meet with people in person and get their feedback as they read letters and things like that. You will be helping us evaluate some of the information that the state of Alabama provides to people who participate in SNAP.

To ensure we capture notes of your feedback during today’s interview we will be connecting to a virtual meeting so that my colleague [NAME] can take notes.

[INTERVIEWER: START TEAMS MEETING. ONCE CONNECTED, INTRODUCE NOTETAKER TO PARTICIPANT]

Before we start, there is a form I would like you to read and sign. [HAND PARTICIPANT CONSENT FORM]

This is a consent form. It explains the purpose of today’s session and your rights as a participant. It also informs you that we would like to audio record this session. We do this to get an accurate record of your feedback for our notes. Our session today is completely confidential. If you currently receive or are applying for benefits, this interview will not have any impact on your benefits. Your participation in this study is completely voluntary. You may decline to answer any particular question and you may stop the interview at any time. We do not share information with anyone. Only the people on the project have access to the recording and we do not use any names in our reports. Take a look and let me know if you have any questions. Please sign it when you are finished, and then I will also sign it.

[SIGN CONSENT FORM AFTER PARTICIPANT SIGNS]

Think Aloud: As you read the materials today, I am interested in the thoughts and reactions that go through your mind as you process the messages. So I would like you to tell me everything that you are thinking and feeling as we go through these materials.

Practice: Let’s practice before we start. [SHOW PARTICIPANT PRACTICE PARAGRAPH]

Please read this short paragraph and say what comes into your mind as you’re reading it.

Practice probes: In your own words, can you tell me what this paragraph is saying?

Did you find any part of it difficult to understand?

Thanks; that was great. So as we go through the SNAP materials, I will ask you questions like these. I really want to hear your opinions and reactions, so don’t hesitate to speak up whenever something is unclear, or not easy to understand. We are going to look at some materials and I’m going to ask you to questions about each one. Do you have any questions before we begin?

[INTERVIEWER TURN ON RECORDER. CHECK BATTERY LEVEL. NOTETAKER START RECORDING IN CENSUS VDI.]

General questions

• The SNAP program provides information about various rules, instructions, and things like that to people receiving or applying for food assistance. Do you recall receiving any of these kinds of information?

  • (If yes) What kinds of rules or instructions do you recall receiving?

• In your opinion, would you say you have a good understanding of the SNAP rules? Why do you say that?

• Do you recall getting any of these kinds of information that was not clear?

  • (If yes) Please tell me about that. What wasn’t clear about it?

  • Were you able to figure out what that information really meant? If so, how did you figure it out?

• If you had a question about your SNAP benefits, where would you go for more information?

Topic 1 - Card management

Exhibit 1: AL SNAP Flyer

[SHOW PARTICIPANT ENTIRE INFORMATIONAL FLYER. START WITH GENERAL PROBES AND THEN PROBE ON SELECTED SECTIONS]

Here is a flyer that someone can get from a SNAP office. [POINT TO “EBT CARD” SECTION] Please take a few minutes to read this section and then we will talk about it. If necessary: Please remember to think aloud as you read.

• In your own words, can you tell me what this section is saying?

• Is there any information in this section that you think is especially important?

  • If yes: What information do you think is important? Why do you say that?

• Did you find any part of it difficult to understand?

  • If yes: Which parts were difficult?

• This section talks about an ‘authorized representative.’ What does that mean to you?

• Can you tell me in your own words what a PIN is?

• Are there any situations where someone might want to give their PIN to others?

  • (if yes) What are some examples of those kinds of situations?

• What would you do if someone knew your PIN and you didn’t necessarily want them to know it?

• [POINT TO LAST BLOCK] What does this part mean to you?

Thank you for sharing your thoughts on that. Now let’s take a look at another section of this flyer. Some of the questions that I ask you may sound repetitive. Thank you for your patience as we talk about these materials.

Topic 2 - Eligible/ineligible purchases

Exhibit 1: AL SNAP Flyer

[AFTER REVIEWING EXCERPTS FOR TOPIC ONE, POINT PARTICIPANT TO RELEVANT SUBTOPIC EXCERPTS BELOW FOR TOPIC 2 AND PROBE.]

Please take a few minutes to read this section and then we will talk about it. If necessary: Please remember to think aloud as you read.

• In your own words, can you tell me what this is saying?

• Is there any information in this section that you think is especially important?

  • If yes: What information do you think is important? Why do you say that?

• Did you find any part of it difficult to understand?

  • If yes: Which parts were difficult?

• How would you describe the kinds of food that you are allowed to buy with SNAP?

• How would you describe the kinds of foods and other items that you are NOT allowed to buy with SNAP?

  • (if needed) What does “prepared or hot foods” mean to you?

• Did you find any part of it difficult to understand?

  • If yes: Which parts were difficult?

• Is there any other information you would want to see in a flyer like this?

Thank you. Now let’s take a look at another section.

Topic 3 – Trafficking

Exhibit 1 : AL SNAP Flyer

[AFTER REVIEWING EXCERPTS FOR TOPIC TWO, POINT PARTICIPANT TO RELEVANT SUBTOPIC EXCERPTS BELOW FOR TOPIC 3 AND PROBE.]

Please take a few minutes to read this section and then we will talk about it. If necessary: Please remember to think aloud as you read.

• In your own words, can you tell me what this section is saying?

• Is there any information in this section that you think is especially important?

  • If yes: What information do you think is important? Why do you say that?

• In your opinion, is anything about this page confusing or unclear?

  • If yes: Which parts were not clear?

• How would you describe what trafficking means in your own words?

  • Can you name any examples of trafficking?

• [POINT TO LAST BULLET] Please tell me in your own words what this last part means to you.

• Is there any other information you would want to see in a flyer like this?

Now please look at this section:

• Did you find any part of this section difficult to understand?

  • If yes: Which parts were difficult?

• What does “Disqualification Periods” mean to you?

Thank you for sharing your thoughts about those sections. Now let’s take a look at a different document.

Exhibit 2: AL Application

[SHOW PARTICIPANT PAGE AND POINT TO SPECIFIC EXCERPTS/PAGES BELOW.]

Next we’ll look at part of the benefits application for the state of Alabama. [POINT TO “PENALTY WARNINGS” SECTION.]

Please take a few minutes to read this section and then we will talk about it. If necessary: Please remember to think aloud as you read.

• In your opinion, what is the purpose of this page?

  • (if needed) This section asks for a signature. Why do you think that is?

• In your opinion, is anything about this page confusing or unclear?

  • If yes: Which parts were not clear?

• [POINT TO FIRST PARAGRAPH, THIRD SENTENCE] This line says that someone who breaks the rules can be barred from SNAP. What does “barred” mean to you?

• [POINT TO SECOND PARAGRAPH, THIRD SENTENCE] This line says not to alter EBT cards. What does that mean to you?

  • (if needed) Can you name an example of how someone might alter an EBT card?

Topic 4 - Fraud reporting

Exhibit 1: AL SNAP Flyer

[RETURN TO INFORMATIONAL FLYER FOR TOPIC 4. POINT PARTICIPANT TO RELEVANT SUBTOPIC EXCERPTS IN FLYER BELOW FOR TOPIC 4 AND PROBE.]

Now I’d like to take a look at one more sections of the flyer.

[POINT PARTICIPANT TO “HOW TO REPORT FRAUD” SECTION.]

• What does this section mean to you?

• Have you ever thought about filing a fraud report related to SNAP or other benefits?

  • (If yes) You don’t have to tell me if you don’t want to, but did you file the complaint?

    • (If no) What made you decide not to?

• If you were to call to report SNAP fraud, would you have any concerns about providing your name if they asked for it?

• What does “SNAP fraud” mean to you?

Those are all the documents. Thank you for sharing your thoughts with me about them.

Debriefing Questions:

Now I just have a few more questions before we are finished.

• Thinking about all the materials we have reviewed today, is there anything you would like to add that we didn’t get a chance to talk about yet?

• In the past, have you had any issues with your benefits from the Food Assistance program?

• If you find yourself having trouble with the program, what do you do at home?

  • Do you call for help? Who tends to help you?

  • Are there other things you do when you have trouble filling out forms?

• Government agencies collect data when people fill out forms for programs, such as applying for food assistance benefits. Do you think this data should be shared with other government agencies?

• Do you have any other last thoughts you’d like to share?

Thank you for your help today!

[ASK PARTICIPANT TO FILL OUT VOUCHER FORM. GIVE PARTICIPANT INCENTIVE]

Thinkaloud Practice Paragraph

Our solar system was formed about 4.5 billion years ago. It consists of our sun and eight main planets: Mercury, Venus, Earth, Mars, Jupiter, Saturn, Uranus and Neptune. Mercury, Venus, Earth and Mars are mostly made of rock and metal and are called “terrestrial planets.” Jupiter, Saturn, Uranus and Neptune are called “gas giants” because they are mostly composed of gas.

6