Form 1 Minnesota Protocol and Materials

Minnesota ABAWD Message Testing Protocol

Participant ID: ______ Interviewer initials: _____

Interview Date: ___ / ___ / 2023 Start Time: ____________ End Time: ____________

Read or paraphrase the following intro text:

Hello. My name is ________________, and I work for the Census Bureau. Thank you for agreeing to participate in our study.

Today we are talking to people about the information they receive from the SNAP food assistance program. The information should be clear and easy to understand, so people who are eligible receive their benefits and know about rules and requirements to continue participating in SNAP. We have found that the best way to test letters and messages is to meet with people in person and get their feedback as they read the materials. You will be helping us test messages and communications that some SNAP recipients have received in the past.

[HAND PARTICIPANT CONSENT FORM]

This is a consent form. It tells you a little bit about the study and gives me permission to audio record this interview. Our session today is completely confidential. Your participation in this study is completely voluntary. You may decline to answer any particular question and you may stop the interview at any time. We do not share information with anyone. Only the people on the project have access to the recording and we do not use any names in our reports. Take a look and let me know if you have any questions. Please sign it when you are finished, and then I will also sign it.

[SIGN CONSENT FORM AFTER PARTICIPANT SIGNS]

Think Aloud: As you read the materials today, I am interested in the thoughts and reactions that go through your mind as you process the messages. So I would like you to tell me everything that you are thinking and feeling as we go through these materials.

Practice: Lets practice before we start. [SHOW PARTICIPANT PRACTICE PARAGRAPH]

Please read this short paragraph and say what comes into your mind as you’re reading it.

Practice probes: In your own words, can you tell me what this paragraph is saying?

Did you find any part of it difficult to understand?

Thanks, that was great. So as we go through the SNAP materials, I will ask you questions like these. I really want to hear your opinions and reactions, so don’t hesitate to speak up whenever something is unclear, or not easy to understand. We are going to look at five letters and I’m going to ask you to read each one first while you think aloud. And then I will ask you questions about each letter after you are done reading it. Do you have any questions before we begin? [TURN ON RECORDER. CHECK BATTERY LEVEL.]

Letter 1 – Notice of Decision

Here is a letter that someone might receive after applying for benefits. Please take a few minutes to read the letter and then we will talk about it. If needed: Please remember to think aloud as you read.

[SHOW RESPONDENT “NOTICE OF DECISION” LETTER]

In your own words, can you tell me what this letter is saying?

Did you find any part of it difficult to understand?

If yes: Which parts were difficult?

What would you expect to happen after receiving this letter?

In your opinion, what is the most important information you saw in this letter, if any?

Was there anything that you expected to see in this letter that was not here?

[POINT TO OR HIGHLIGHT PARAGRAPH AT BOTTOM THAT BEGINS WITH “The household member(s) listed below…” NOTE THAT IT CONTINUES ON PG 2.] Let’s take a look at this paragraph.

[NOTE WHETHER P SEES REFERENCE TO ATTACHMENT AND FAQ’S ON WEBSITE.]

Can you tell me in your own words what this section is saying?

In your opinion, is this section clear?

If no: What about this section is unclear?

This paragraph mentions “SNAP work rules.” Have you heard of that before?

What does “SNAP work rules” mean to you?

[POINT TO OR HIGHLIGHT SENTENCE STARTING WITH “If the people listed below do not follow SNAP work rules…”] What does this sentence mean to you?

[POINT TO OR HIGHLIGHT SENTENCE STARTING WITH “Different household members…”] And how about this sentence, what does that mean to you?

What would you do if you received this letter?

Thank you for sharing your thoughts on that letter. Now let’s take a look at another one.

Letter 2 – Work Rules Notice

[SHOW RESPONDENT “WORK RULES NOTICE” LETTER]

Please take a few minutes to read the letter and then we will talk about it. If needed: Please remember to think aloud as you read.

In your own words, can you tell me what this letter is saying?

Did you find any part of it difficult to understand?

If yes: Which parts were difficult?

Was there anything you expected to see in this letter that was not here?

In your opinion, what is the most important information you saw in this letter, if any?

What would you expect to happen after receiving this letter?

(if needed) What does the recipient of the letter need to do, if anything?

[POINT TO OR HIGHLIGHT “Does everyone need to follow these work rules?” SECTION ON PAGE 1.] What do you think is the point of this section?

Please go through the list and tell me in your own words what each of these points means.

[WHEN R REACHES SEVENTH BULLET “Attending a school, college, or training program at least half-time…” ASK IF NEEDED] What does “attending at least half-time” mean to you?

[WHEN R REACHES NEXT-TO-LAST BULLET “Meeting work rules for another employment program…” ASK IF NEEDED] Have you ever heard of the Minnesota Family Investments Program or MFIP? How about the Diversionary Work Program or DWP? In your opinion is it useful or not useful to include those in this list?

[POINT TO OR HIGHLIGHT “What should you do if…?” SECTION AT BOTTOM OF PAGE 1.] Thanks. How about this section?

[NOTE WHETHER R UNDERSTANDS THE RECIPIENT DOES NOT NEED TO FOLLOW THE GENERAL WORK RULES IF ANY OF THE REASONS APPLY TO THEM.]

In your opinion, is this section clear or not clear? (if not clear) Why do you say that?

[POINT TO OR HIGHLIGHT THE FIRST SECTION UNDER “General work rules” AT TOP OF PAGE 2.] How about this first section on the next page, what does that mean to you?

This section talks about general work rules. Have you heard of that before? What does that mean to you?

In your opinion, is this section clear or not clear? (if not clear) Why do you say that?

The next two sections talk about “good reasons” for not meeting general work rules. What does “good reasons” mean to you?

Can you think of other examples of good reasons?

[POINT TO OR HIGHLIGHT “How long will you lose SNAP benefits…” ON PAGE 2.] What does this section mean to you?

In your opinion, is this section clear or not clear? Why do you say that?

[POINT TO OR HIGHLIGHT “Time-Limited Rules” and “What do you need to do?” ON PAGE 2.] How about this next section, what does this mean to you?

This section talks about time-limited rules. What does that mean to you? Have you heard of time-limited work rules before?

This section also talks about SNAP Employment and Training, also known as SNAP E&T. Have you heard of that before? What does that mean to you?

This section also talks about participating in “another Federal, State, or local work program.” Have you heard of that before? What does that mean to you?

The last paragraph says, “To get started with SNAP E&T you can contact an employment provider.” What does that mean to you? (if needed) What does “employment provider” mean to you?

Would you say this section clear or not clear? Why do you say that?

[POINT TO OR HIGHLIGHT “Does everyone need to meet these Time-Limited Rules?”] How about this next section, what does this mean to you?

In your opinion, is this section clear or not clear? Why do you say that?

[POINT TO OR HIGHLIGHT “What if you do not follow the Time-Limited Rules?”] How about this next section, what does this mean to you?

(if needed) Please tell me in your own words what the first sentence means to you.

(if needed) The second sentence says, “Once we have counted 3 full months…” What does that mean to you?

(if needed) This section also talks about a 36-month time period. What does that mean to you?

Is there anything about this section that is not clear? Why do you say that?

[POINT TO OR HIGHLIGHT “What if you have a good reason for not meeting the Time-Limited Rules?”] How about this next section, what does this mean to you?

In your opinion, is this section clear or not clear? Why do you say that?

[POINT TO OR HIGHLIGHT “If you lose your SNAP benefits, how can you get them back?”] How about this next section, what does this mean to you?

Is there anything about this section that is not clear? Why do you say that?

[POINT TO OR HIGHLIGHT “Do you want help meeting your work requirement by training or a job?”] How about this next section, what does this mean to you?

Do you think this section is clear or not clear? Why do you say that?

[POINT TO OR HIGHLIGHT “What if you need more help?”] How about this next section, what does this mean to you?

This letter describes general work rules and time-limited rules. In your opinion, is it clear why there are two different sets of work rules? Why do you say that?

In your opinion, what are the differences between general work rules and time-limited work rules?

Thank you for reviewing that letter and sharing your thoughts about it. Now let’s take a look at the next letter.

Letter 3 – First Sanction Notice

[SHOW RESPONDENT “FIRST SANCTION NOTICE” LETTER]

Please take a few minutes to read the letter and then we will talk about it. If needed: Please remember to think aloud as you read.

In your own words, can you tell me what this letter is saying?

Did you find any part of it difficult to understand?

If yes: Which parts were difficult?

In your opinion, what is the most important information you saw in this letter, if any?

What would you expect to happen after receiving this letter?

Was there anything you expected to see in this letter that was not there?

[POINT TO OR HIGHLIGHT FIRST TWO LINES: “Your SNAP case will be closed…” and “Jane Doe has failed to cooperate…”] Please take a look at these first couple of lines. What do they mean to you?

[NOTE WHETHER R UNDERSTANDS THAT THE SNAP HOUSEHOLD’S BENEFITS ARE DEPENDENT ON THE PRIMARY WAGE EARNER’S COMPLIANCE.]

This section refers to the principal wage earner. Have you heard that term before? What does that mean to you?

This section also refers to the SNAP household. Have you heard that term before? What does that mean to you?

In your opinion, is that sentence clear or not clear? Why do you say that?

[POINT TO OR HIGHLIGHT “This is the FIRST time…”] How about this next sentence, what does that mean to you?

(if needed) What does “first time” mean to you?

In your opinion, is it clear what this sentence is referring to? Why do you say that?

[POINT TO OR HIGHLIGHT “For your SNAP household to get SNAP…”] How about this sentence, what does that mean to you?

(if needed) This sentence mentions Jane having to attend an orientation. What does that mean to you? Have you heard of SNAP recipients having to attend orientations?

In your opinion, is this clear or not clear? Why do you say that?

[POINT TO OR HIGHLIGHT “If Jane Doe does NOT cooperate…”] And how about this sentence, what does that mean to you?

[NOTE WHETHER R UNDERSTANDS THAT IF JANE DOES NOT COMPLY BY JAN. 31 THE HOUSEHOLD WILL LOSE SNAP FOR FEB. AND MAY HAVE TO REAPPLY IN MAR.]

In your opinion, is this clear or not clear? Why do you say that?

[POINT TO OR HIGHLIGHT “If you don't agree…” AND “If you miss the 10 day deadline…”] Now let’s look at this last section. What does this mean to you?

(if needed) This section states that if the recipient does not agree with the action taken on their case, they can appeal. What does that mean to you?

This section also states that Jane must appeal within 10 days or before the first day of the month when the action takes place. What does this mean to you?

In your opinion, is this clear or not clear? Why do you say that?

[NOTE WHETHER R ATTENDS TO “To keep your benefits until the appeal” AND INTERPRETS CORRECTLY.]

The second paragraph says that if the recipient misses the 10 day deadline, they can appeal within 30 days from the date they get this notice or 90 days for SNAP. What does this mean to you?

In your opinion, is this clear or not clear? Why do you say that?

Thank you for sharing your thoughts on that letter.

Letter 4 – Multi-person Sanction Notice

This next letter is addressed to two people who live together who both receive SNAP benefits. Please take a few minutes to read the letter and then we will talk about it. If needed: Please remember to think aloud as you read.

In your own words, can you tell me what this letter is saying?

In your opinion, what is the most important information you saw in this letter, if any?

Are there any parts of this letter that are confusing or unclear?

If yes: Which parts?

In your own words, what should Jane and John do after receiving this letter?

[POINT TO OR HIGHLIGHT “This is the SECOND time that John Doe has not cooperated with SNAP work requirements.”] What does this sentence mean to you?

(if needed) In your opinion, why does the letter state that this is the second time John has not cooperated with work requirements? What is the significance of that?

[POINT TO OR HIGHLIGHT “For John Doe to get SNAP benefits again…”] And how about this sentence, what does that mean to you?

This sentence mentions John having to develop a work plan. What does that mean to you? Have you heard of SNAP recipients having to develop work plans?

In your opinion, is this clear or not clear? Why do you say that?

[POINT TO OR HIGHLIGHT “If John Doe does NOT cooperate on…”] Now let’s look at this section. What does this mean to you?

In your opinion, is this clear or not clear? Why do you say that?

Do you recall seeing the dates in which Jane and John may each reapply? (Read if necessary) It states that Jane can reapply in March and John can reapply in May. In your opinion, is it clear why Jane and John have two different dates in which they can reapply?

[POINT TO OR HIGHLIGHT “Because Jane Doe is the Principal Wage Earner…”] How about this section. What does this mean to you?

In your opinion, is this clear or not clear? Why do you say that?

[POINT TO OR HIGHLIGHT “If the Principle Wage Earner, Jane Doe cooperates…”] How about this section. What does this mean to you?

In your opinion, is this clear or not clear? Why do you say that?

[POINT TO OR HIGHLIGHT LAST PARAGRAPH ON PAGE 1, “If the Principle Wage Earner, Jane Doe cooperates…”] How about this section. What does this mean to you?

In your opinion, is this clear or not clear? Why do you say that?

Letter 5 – Closing Notice

Ok, this is the last letter we’ll look at. Please take a few minutes to read the letter and then we will talk about it. If needed: Please remember to think aloud as you read.

[POINT TO OR HIGHLIGHT FIRST TWO LINES “Beginning…” and “Starting with…”] What do these lines mean to you?

(if needed) What does it mean when it says the number of eligible members in your household changed from 4 to 3.

In your opinion, is this clear or not clear? Why do you say that?

[POINT TO OR HIGHLIGHT “The following person(s)…”] And how about this section, what does that mean to you?

(if needed) What does it mean when it says they got SNAP for 3 months in the current 36-month period without meeting work requirements

In your opinion, is this clear or not clear? Why do you say that?

[POINT TO OR HIGHLIGHT “You can re-apply…”] And how about this sentence, what does that mean to you?

In your opinion, is this clear or not clear? Why do you say that?

Debriefing Questions

Thank you. Those are all the letters we have to look at today. I just have a few more questions before we are finished.

Is there any other information that you think would be helpful for people who must meet the work requirement?

Have you ever been notified that you were required to comply with the time limit?

[IF YES:] How were you notified?

[IF NEEDED:] Tell me more about what the notice said.

In your opinion, was that notification clear and easy to understand?

Government agencies collect information when people fill out forms for programs. One example is the information given when someone fills out an application for food assistance benefits. Government agencies also collect data using surveys, like the survey that is taken every ten years about the population.

How would you feel if the information that agencies get from the application forms (such as the food assistance benefits) was shared with other government agencies (like the survey about the population) instead of asking the survey questions directly?

[IF NEEDED:] For example the SNAP application that you completed to get benefits asks a question about your date of birth. How would you feel if about your date of birth that you gave to SNAP was used by another agency that was conducting a survey, instead of the other agency asking you for your date of birth?

So the answer that you had given on your SNAP application would be used by the other agency. In your opinion, would that be okay or would that not be okay?

Tell me more about why you answered that way?

Is there anything else you would like to add that we didn’t get a chance to talk about today?

How did you learn about our study? Do you remember where you saw our flyer? Or did someone you know tell you about it?

Thanks for your time. Please remember that nothing that we discussed today will have any impact on any benefits you may receive, and if you have any questions based on anything we discussed, please contact your SNAP caseworker.

[ASK PARTICIPANT TO FILL OUT VOUCHER FORM. GIVE PARTICIPANT INCENTIVE]

Testing materials

Thinkaloud Practice Paragraph

Our solar system was formed about 4.5 billion years ago. It consists of our sun and eight main planets: Mercury, Venus, Earth, Mars, Jupiter, Saturn, Uranus and Neptune. Mercury, Venus, Earth and Mars are mostly made of rock and metal and are called “terrestrial planets.” Jupiter, Saturn, Uranus and Neptune are called “gas giants” because they are mostly composed of gas.

Letter 1 – Notice of Decision (2 pages)

Letter 2 – Work Rules Notice (3 pages)

Letter 3 – First Sanction Notice (1 page)

Letter 4 – Multi-Person Sanction Notice (2 pages)

Letter 5 – Closing Notice (2 pages)

12