Individuals and Households

Appendix E1.1 Colorado Participant Survey Specifications

This page has been left blank for double-sided copying.

OMB Clearance Number: 0584-XXXX

Expiration Date: XX/XX/XXXX

Rapid Cycle Evaluation of Operational Improvements in SNAP E&T Programs

Participant Survey:

Colorado

Public Burden Statement

This information is being collected to assist the Food and Nutrition Service in evaluating operational improvements in Supplemental Nutrition Assistance Program (SNAP) Employment and Training (E&T) programs that aim to improve delivery of services and program outcomes. This is a voluntary collection and FNS will use the information to assess the effectiveness of changes made to the SNAP E&T program. This collection does request any personally identifiable information under the Privacy Act of 1974. According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0584-[xxxx]. The time required to complete this information collection is estimated to average 15 minutes (0.25 hours) per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: U.S. Department of Agriculture, Food and Nutrition Service, Office of Policy Support, 1320 Braddock Place, 5th Floor, Alexandria, VA 22306 ATTN: PRA (0584-xxxx). Do not return the completed form to this address.

2022

Privacy Act Statement

Authority: This information is being collected under the authority of Section 9 of the Food and Nutrition Act of 2008, as amended, (7 U.S.C. 2018). Disclosure of the information is voluntary.

Purpose: The information is being collected to evaluate operational improvements in Supplemental Nutrition Assistance Program (SNAP) Employment and Training (E&T) programs using rapid cycle evaluation.

Routine Use: The information may be shared with SNAP contract researchers and United States Department of Agriculture (USDA) SNAP research and administrative staff.

Disclosure: If all or any part of the information is not provided, interviews may not be admissible in data sets.

I. Introduction

ALL

[SNAP E&T RCE INTERVENTION SITE]

I0. [SNAP E&T RCE INTERVENTION SITE] is participating in a study that the U.S. Department of Agriculture, Food and Nutrition Service (FNS) is sponsoring. This study will help the agency learn more about ways to improve the Supplemental Nutrition Assistance Program (SNAP) Employment and Training (E&T) programs for participants. E&T programs are intended to help SNAP participants gain skills and find work. [SNAP E&T RCE INTERVENTION SITE] is one of eight sites seeking to understand the impact of changes to SNAP E&T program processes on SNAP participants’ engagement with E&T services. Mathematica is leading this study on behalf of FNS. Please read the information below and confirm whether you are willing to participate in the study.

By giving permission to be in the study, you agree to take a short 15 minute survey. The survey asks about barriers to engaging with services and seeking employment, program satisfaction, and reasons for engagement decisions.

Here are some other things to know about the study:

• The study will use your data for research purposes only.

• Study reports will summarize all participants’ findings and will not identify you. None of the reports prepared for this study will include information that identifies you. All confidential information will be stored safely and destroyed at the end of the study.

• Taking the survey is completely voluntary. You can skip any question that you don’t want to answer. If you are unsure of how to answer a question, please give the best answer you can, rather than leaving it blank.

• Participating in the study has no known risks and will not affect your benefits. Your participation will help us learn about how to improve SNAP E&T programs and services to help SNAP participants gain skills and find work.

• You will receive a $30 gift card to thank you for your time completing the survey.

Please indicate below whether you agree to be in the study. If you have any questions about the study or would like a copy of the above information, please contact Mathematica’s survey director, [SURVEY DIRECTOR], at XXX-XXX-XXXX or email [him/her] at [email protected].

 I understand the study description and I agree to participate in the study 1

Electronic Signature

• I do not agree to participate in the study 2

PROGRAMMER:

If I0 = 2, status non-consent and exit survey

I0 = 1

I1. First, we’d like to verify that we are reaching the correct person. What is your date of birth?

| | | / | | | / | | | | |

VALIDATION CHECK:

2 of 3 fields at I1 must match records to continue

IF WEB AND I1_validation check not passed (web mode and DOB does not match)

I1b. Thank you for your time. We need to check our records before continuing. Please contact us at 1-XXX-XXX-XXXX to complete the survey.

PROGRAMMER:

STATUS 1380 FOR SUP REVIEW and EXIT WEB interview

IF PHONE AND I1_validation check not passed (telephone mode and DOB does not match)

I1c. Thank you for your response. I need to check our records before continuing the interview. Please hold on a moment while I get my supervisor.

SUPERVISOR: PLEASE ENTER YOUR ID TO CONTINUE

| | | | |

IF PHONE AND I1_validation check not passed (telephone mode and DOB does not match)

I1d. SUPERVISOR: PLEASE VALIDATE THE RESPONDENT IDENTITY USING ADDRESS OR OTHER CONTACT INFORMATION AVAILABLE

CORRECT RESPONDENT 1 GO TO I1f

WRONG RESPONDENT 0 GO TO I1e

I1d = 0 (wrong respondent)

I1e. Thank you for your response. There may be a problem with some of our records. A representative from Mathematica will give you a call to verify our information.

What is the best number to reach you?

 The caller does not have a phone number 0

When is the best time to reach you?

Select one only

Anytime 1

Weekday mornings 2

Weekday afternoons 3

Weekday evenings 4

Weekend mornings 5

Weekend afternoons 6

Weekend evenings 7

What is your personal email address that you check most often? Please do not provide a school email address, unless it is the only email address you use.

 The caller does not have an email address 0

We need to review and confirm our records before continuing with the interview. Thank you for your help.

PROGRAMMER:

STATUS 1400 FOR locating (wrong respondent) and EXIT interview

I1d = 1 (correct respondent)

I1f. Thank you for your response. I will hand the phone back to the interviewer to continue the interview.

CONTINUE 1 GO TO A1

A. Employment

All

A1. The first questions are about current or recent jobs.

Are you currently working at a job for pay, or self-employed?

Yes 1

No 0

IF NOT AVAILABLE IN SNAP ADMIN DATA

FILL MONTH WITH 3 MONTHS PRIOR TO SURVEY

A2. Were you working at a job for pay, or self-employed, in [MONTH]?

Yes 1

No 0

ALL

A3. Some people have challenges that make it hard to find a new job or keep a current job. First, please think about the challenges you may have had finding or qualifying for a job. Did any of the following make it hard for you to find or keep a job in the last year?

	No	Yes
a. Could not find work or lack of jobs available in the area	0	1
b. Do not have the right schooling	0	1
c. Do not have the right job search skills or experience For example: resume writing, interviewing, or networking	0	1
d. Have difficulty speaking, reading, and/or writing English	0	1

ALL

A4. Next, consider any circumstances that might have made it hard for you to find or keep a job. Did any of the following make it hard for you to find or keep a job in the last year?

	No	Yes
a. Physical or mental health challenges (including a disability)	0	1
b. Housing problems For example: homelessness, unstable housing or no regular place to stay, or no affordable housing	0	1
c. Transportation issues or problems For example: no car or no public transportation available, transportation costs too much, public transportation takes too much time	0	1
d. Family responsibilities, like caring for children, spouse, or a parent	0	1

ALL

A5. Are there any other challenges that made it hard for you to find a new job or keep a current job in the last year?

Yes 1

No 0

IF A5 = 1

A6. What other challenges made it hard for you to find a new job or keep a current job in the last year?

B. Intervention Information (Recruitment)

ALL

FILL SNAP E&T PROGRAM NAME BY SITE

B0. Next, we’re going to ask you some questions about communication you might have received about the [SNAP Employment & Training program/E&T PROGRAM NAME], encouraging you to enroll and participate.

If you are now participating in the [SNAP E&T program/E&T PROGRAM NAME], please answer the following questions thinking about the information you received about the program before you joined.

The [SNAP E&T program / E&T PROGRAM NAME] helps SNAP participants gain skills and find work, providing participants access to employment training and support services.

ALL

FILL TIME RANGE BY SITE

B1. In the last [FILL BY SITE: e.g., two months], did you receive any messages encouraging you to enroll in the [SNAP E&T program/E&T PROGRAM NAME]?

	Yes, received message	No, did not receive message
a. Text message	1	0
b. Email	1	0
c. Mailed postcard	1	0
d. Phone call	1	0

IF RA_STATUS = T AND B1a – B1c = 0 (treatment case and no messages received), or

IF RA_STATUS = C AND B1d = 0 (control case and no phone call received)

FILL BASED ON TREATMENT ARM: texted you / emailed you / called you / sent you mail

B2. [The SNAP E&T program/E&T PROGRAM NAME] recently [texted you at XXX-XXX-XXXX / emailed you at [email protected] / called you at XXX-XXX-XXXX / sent you mail at [address]].

Is that the correct [phone number / email address / address] for you?

Yes 1 GO TO B3

No 0 GO TO B3

IF B1a – B1d = 0

(no message received)

B3. Have you heard of the [SNAP E&T program/E&T PROGRAM NAME]?

Yes 1 GO TO B6

No 0 GO TO B4

IF B3 = 0 (no message received and not aware of program)

B4. [The SNAP E&T program/E&T PROGRAM NAME] helps SNAP participants gain skills and find work, providing SNAP participants access to employment training and support services. To set up an appointment, please call XXX-XXX-XXXX.

PROGRAMMER BOX B4

IF B3 = 0 (no message received and not aware of program):

GO TO QUESTION B14

IF RA_STATUS = T AND ANY B1a – B1c = 1 (treatment and message received), or

IF RA_STATUS = C AND B1d = 1 (control case and phone call received)

FILL “call” IF RA_STATUS = C AND B1d = 1

FILL “text messages” if INTERVENTION_DESC = text and email and [B1a = 1 and B1b = 0], ELSE FILL “emails” if [B1a = 0 and B1b = 1], ELSE FILL “INTERVENTION_DESC” based on treatment arm

B5. These next few questions are about the [call/text messages/emails/INTERVENTION_DESC] you received.

Did you know about [the SNAP E&T program/E&T PROGRAM NAME] before you received any [calls/text messages/emails/INTERVENTION_DESC]?

Yes 1 GO TO B6

No 0 GO TO B7

IF B3 = 1 (no message received but aware of program) OR

B5 = 1 (knew about program before receiving notification)

B6. How did you hear about [the SNAP E&T program/E&T PROGRAM NAME]?

Select all that apply

Referral from SNAP staff member (eligibility worker) 1

Family member, friend, or colleague 2

Another organization in your community 3

Flyer 4

Community event 5

Somewhere else (SPECIFY) 99

IF RA_STATUS = T AND ANY B1a – B1c = 1 (treatment and message received), or

IF RA_STATUS = C AND B1d = 1 (control case and phone call received)

FILL “call” IF RA_STATUS = C AND B1d = 1

FILL “text messages” if INTERVENTION_DESC = text and email and [B1a = 1 and B1b = 0], ELSE FILL “emails” if [B1a = 0 and B1b = 1], ELSE FILL “INTERVENTION_DESC” based on treatment arm

B7. Did you understand that the [calls/text messages/emails/INTERVENTION_DESC] were from [the SNAP E&T program/E&T PROGRAM NAME]?

Yes 1 GO TO B8

No 0 GO TO B8

IF RA_STATUS = T AND ANY B1a – B1c = 1 (treatment and message received), or

IF RA_STATUS = C AND B1d = 1 (control case and phone call received)

FILL “call” IF RA_STATUS = C AND B1d = 1

FILL “text messages” if INTERVENTION_DESC = text and email and [B1a = 1 and B1b = 0], ELSE FILL “emails” if [B1a = 0 and B1b = 1], ELSE FILL “INTERVENTION_DESC” based on treatment arm

B8. Did the [call/text messages/emails/INTERVENTION_DESC] help you understand what next steps you could take to participate in [the SNAP E&T program/E&T PROGRAM NAME]?

Yes 1 GO TO B9

No 0 GO TO B9

IF RA_STATUS = T AND ANY B1a – B1c = 1 (treatment and message received), or

IF RA_STATUS = C AND B1d = 1 (control case and phone call received)

PROGRAMMER: Randomize/rotate options 1 and 3

B9. Did you feel like you were contacted by [the SNAP E&T program/E&T PROGRAM NAME]…

Not frequently enough 1

Just the right amount 2

Too frequently 3

IF RA_STATUS = T AND ANY B1a – B1c = 1 (treatment and message received), or

IF RA_STATUS = C AND B1d = 1 (control case and phone call received)

FILL “call” IF RA_STATUS = C AND B1d = 1

FILL “text messages” if INTERVENTION_DESC = text and email and [B1a = 1 and B1b = 0], ELSE FILL “emails” if [B1a = 0 and B1b = 1], ELSE FILL “INTERVENTION_DESC” based on treatment arm

B10. Did you reach out to [the SNAP E&T program/E&T PROGRAM NAME] in response to the [call/texts/emails/INTERVENTION_DESC] you received?

Yes 1 GO TO B11

No 0 GO TO B13

IF B10 = 1 (reached out in response to notification)

B11. How did you reach out to [the SNAP E&T program/E&T PROGRAM NAME]?

Select all that apply

By phone 1

By text 2

By email 3

IF B10 = 1 (reached out in response to notification)

B12. Were you able to connect with someone from [the SNAP E&T program/E&T PROGRAM NAME]?

Yes 1 GO TO B14

No 0 GO TO B14

IF B10 = 0 (did not reach out in response to notification)

FILL “call” IF RA_STATUS = C AND B1d = 1

FILL “text messages” if INTERVENTION_DESC = text and email and [B1a = 1 and B1b = 0], ELSE FILL “emails” if [B1a = 0 and B1b = 1], ELSE FILL “INTERVENTION_DESC” based on treatment arm

B13. Why did you not respond to the [call/texts/emails/INTERVENTION_DESC] you received?

Select all that apply

You were too busy to respond 1

You thought it was spam 2

You meant to respond but forgot 3

You didn’t know what to say 4

You already had the information they were sending you 5

You weren’t interested in participating in the program 6

You didn’t think program staff would be available to help you 7

Something else (SPECIFY) 99

ALL

B14. What is the best way to contact you or provide you with information about [the SNAP E&T program/E&T PROGRAM NAME]?

Select one only

Text message 1

Email 2

Phone call 3

Mail 4

Some other way (SPECIFY) 99

C. Program Participation

PROGRAMMER BOX C0

IF B3 = 0 (no message received and not aware of program): GO TO QUESTION C2

IF B3 NE 0 (aware of program)

C1a. Which of the following describes your status with the [SNAP Employment &Training program/E&T PROGRAM NAME]?

You are currently receiving services 1 GO TO C2

You are not currently receiving services 2 GO TO C1b

C1a = 2

C1b. Have you received any services from the [SNAP E&T program/E&T PROGRAM NAME] in the last 3 months?

Yes 1 GO TO C2

No 0 GO TO C2

ALL

Fill “Besides the [SNAP E&T program/E&T PROGRAM NAME] are” / “other” IF C1b = 1 or C1a = 1

FILL “Are” IF C1b = 0 OR B3 = 0

FILL appropriate state SNAP E&T program name

C2. [Besides the [SNAP E&T program/E&T PROGRAM NAME] are/Are] you receiving services from any [other] providers to help you further your education or training or help you prepare for or find a job?

Yes 1 GO TO C3

No 0

IF C1a = 1 or C1b = 1 or C2 = 1

IF C1a or C1b = 1, fill “services from [the SNAP E&T program]” or “services from [E&T PROGRAM NAME]”

IF C2 = 1 and C1a NE 1 and C1b NE 1, fill “those services”

PROGRAMMER: Randomize response options

C3. What were the main reasons you decided to receive [services from [the SNAP E&T program/E&T PROGRAM NAME]]/[those services]?

Select all that apply

To keep SNAP benefits 1

To receive help with child care 2

To get help with the costs of training or employment 3

To improve your English 4

To gain job search skills 5

To learn about self-employment (for example: how to start your own business) 6

To earn a certification/credential/license 7

To gain work experience 8

To get promoted 9

To get a raise 10

To get a job 11

To find a better job 12

Some other reason (SPECIFY) 99

IF C1a = 2 AND C1b = 0

PROGRAMMER: Randomize response options

C4. What were the main reasons you haven’t received services from [the SNAP E&T program/E&T PROGRAM NAME]?

Select all that apply

You lacked information about the program 1

The program didn’t match your needs 2

You had transportation issues or problems

For example: no car or public transportation available, transportation costs too much, public transportation takes too much time 3

You didn’t think the program would help you find a job 4

You got a job 5

You had physical or mental health challenges (including a disability) 6

You had housing issues or moved 7

You needed to care for a child or family member 8

Some other reason (SPECIFY) 99

IF C1b = 1

PROGRAMMER: Randomize response options

C5. What were the main reasons you stopped receiving services from [the SNAP E&T program/E&T PROGRAM NAME]?

Select all that apply

The program didn’t match your needs 1

You didn’t think the program would help you find a job 2

You got a job 3

You had transportation issues or problems

For example: no car or public transportation available, transportation costs too much, public transportation takes too much time 4

You had physical or mental health challenges (including a disability) 5

You needed to care for a child or family member 6

You had housing issues or moved 7

You completed the program 8

You did not complete the program, but you no longer needed services 9

Some other reason (SPECIFY) 99

IF C1a = 1 or C1b = 1

If ever received services

C6a. The next questions are about the [SNAP E&T program/E&T PROGRAM NAME] program offerings.

For each category, please rank your satisfaction with the [SNAP E&T program/E&T PROGRAM NAME].

Training location and times

Very satisfied 1

Satisfied 2

Neither satisfied nor dissatisfied 3

Dissatisfied 4

Very dissatisfied 5

IF C1a = 1 or C1b = 1

If ever received services

C6b. Online training or meeting options

Very satisfied 1

Satisfied 2

Neither satisfied nor dissatisfied 3

Dissatisfied 4

Very dissatisfied 5

IF C1a = 1 or C1b = 1

If ever received services

C6c. Support with career planning or job placement services

Very satisfied 1

Satisfied 2

Neither satisfied nor dissatisfied 3

Dissatisfied 4

Very dissatisfied 5

IF C1a = 1 or C1b = 1

If ever received services

C6d. Additional support services, for example transportation assistance or child care

Very satisfied 1

Satisfied 2

Neither satisfied nor dissatisfied 3

Dissatisfied 4

Very dissatisfied 5

IF C1a = 1 or C1b = 1

If ever received services

C6e. Customer service and availability of [SNAP E&T program/E&T PROGRAM NAME] staff

Very satisfied 1

Satisfied 2

Neither satisfied nor dissatisfied 3

Dissatisfied 4

Very dissatisfied 5

IF C1a = 1 or C1b = 1

If ever received services

C6f. The number of [SNAP E&T program/E&T PROGRAM NAME] staff who look like you or who speak your preferred language

Very satisfied 1

Satisfied 2

Neither satisfied nor dissatisfied 3

Dissatisfied 4

Very dissatisfied 5

IF C1a = 2 and C1b = 0 OR B3 = 0

If never received services

C7a. The next questions are about the [SNAP E&T program/E&T PROGRAM NAME] program offerings.

For each category, please indicate whether the item would affect your decision to participate in the [SNAP E&T program/E&T PROGRAM NAME].

More convenient training location and times

Much more likely to participate 1

More likely to participate 2

Unlikely to affect your participation 3

IF C1a = 2 and C1b = 0 OR B3 = 0

If never received services

C7b. More online training or meeting options

Much more likely to participate 1

More likely to participate 2

Unlikely to affect your participation 3

IF C1a = 2 and C1b = 0 OR B3 = 0

If never received services

C7c. More support with career planning or job placement services

Much more likely to participate 1

More likely to participate 2

Unlikely to affect your participation 3

IF C1a = 2 and C1b = 0 OR B3 = 0

If never received services

C7d. Additional support services, for example transportation assistance or additional child care

Much more likely to participate 1

More likely to participate 2

Unlikely to affect your participation 3

IF C1a = 2 and C1b = 0 OR B3 = 0

If never received services

C7e. Additional [SNAP E&T program/E&T PROGRAM NAME] staff training and availability

Much more likely to participate 1

More likely to participate 2

Unlikely to affect your participation 3

IF C1a = 2 and C1b = 0 OR B3 = 0

If never received services

C7f. More [SNAP E&T program/E&T PROGRAM NAME] staff who look like you or who speak your preferred language

Much more likely to participate 1

More likely to participate 2

Unlikely to affect your participation 3

ALL

Fill “consider” IF C1a = 2 OR B3 = 0

Fill “continue” IF C1a = 1

C8. Are there any other program offerings or features not mentioned that would make you more likely to [consider/continue] participating in [the SNAP E&T program/E&T PROGRAM NAME]?

Yes 1

No 2

C8 = 1

Fill “consider” IF C1a = 2 OR B3 = 0

Fill “continue” IF C1a = 1

C9. Tell us more about the program offerings or services that you feel would make you more likely to [consider/continue] participating in [the SNAP E&T program/E&T PROGRAM NAME].

D. Respondent Characteristics

IF ANY QUESTIONS ASKED IN SECTION D

D0. Finally, we have some questions about your background.

IF NOT AVAILABLE IN SNAP ADMIN DATA

D1. What is your gender?

Select all that apply

Male 1

Female 2

Non-binary/third gender 3

You use another term (SPECIFY) 99

You do not wish to answer r

IF NOT AVAILABLE IN SNAP ADMIN DATA

D2. Are you of Hispanic, Latino/a, or Spanish origin?

No, not of Hispanic, Latino/a, or Spanish origin 1

Yes, Hispanic, Latino/a or Spanish origin 2

IF NOT AVAILABLE IN SNAP ADMIN DATA

D3. What is your race?

Select all that apply

American Indian or Alaska Native 1

Asian 2

Black or African American 3

Native Hawaiian or Pacific Islander 4

White 5

Other (SPECIFY) 99

IF NOT AVAILABLE IN SNAP ADMIN DATA

D4. What is the highest degree or level of school you have completed?

Select one only

Less than 8th grade 1

8th to 12th Grade, no diploma 2

High School Diploma or GED 3

Adult Basic Education (ABE) certificate 4

Some college but no degree 5

Vocational/Technical degree or certificate (for example: cosmetology, automotive repair, Certified Nursing Assistant (CNA)) 6

Business degree/certificate 7

Associate’s degree (AA) 8

Bachelor’s degree or equivalent (for example: BA/BS) 9

Master’s degree (for example: MA/MS) or higher (for example: MD, PhD) 10

Other (SPECIFY) 99

E. END

E1. Thank you for participating in this survey.

We would like to confirm your contact information so we can send you your $30 gift card. Please enter your name, address, phone number and email address so we may contact you if we have any questions.

First Name:

Middle Initial:

Last Name:

Street Address 1:

Street Address 2:

City:

State:

Zip:

Telephone:

Email Address: