Supporting Statement Part B
OMB No. 0584-[NEW]
Assessing Equity in Work Requirements
and SNAP Employment and Training
August 7, 2024
Project Officer: Kristen Corey
Office of Policy Support
Food and Nutrition Service
U.S. Department of Agriculture
1320 Braddock Place
Alexandria, VA 22314
Part B. Collection of Information Employing Statistical Methods 1
B.1. Respondent Universe and Selection Methods 1
B.2. Procedures for Collection of Information 6
B.3. Methods to Maximize Response Rates and the Issue of Nonresponse 9
Tables
Table B.1.1. Key Informant Interview State Selection Criteria. 3
Table B.1.2. Respondents and Nonrespondents by Respondent Type 6
Attachments
A. Legal Authority Statutes and Regulations
B. Research Objectives and Approach to Data Collection
C. Survey Instrument
D. SNAP State Agency Interview Protocol
E. SNAP Local Agency Interview Protocol
F. SNAP E&T Provider Interview Protocol
G. Interested Parties Interview Protocol
H.1. Notification Email from FNS to SNAP State Directors
H.2. Overall Project Summary
H.3. Email to SNAP State Directors Launching Survey
H.4. Survey Frequently Asked Questions
H.5. Document Collection Guide
H.6. Phone Scripts for Nonresponse Follow-Up
H.7. Survey Reminder Email
H.8. Final Survey Reminder Email
H.9. Survey Thank-You Email
H.10. Document Reminder Email
H.11. Confirmation Email to Interview States
H.12. Interview Frequently Asked Questions
H.13. Interview Scheduling Email to Interview States
H.14. Interview Consent Form
H.15. Key Informant Interview Thank-You Email
I. Public Comment and Response
J. National Agricultural Statistics Service Comments and Response
K. Pretest Results Memorandum
L. Institutional Review Board Approval Letter
M. Westat Insight Information Security and Confidentiality Pledge
N. Total Public Burden Hours and Cost
Part B. Collection of Information Employing Statistical Methods
Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.
Respondent universe
This is a new Information Collection Request. It includes five primary data collection instruments: (1) Survey Instrument (attachment C.); (2) SNAP [Supplemental Nutrition Assistance Program] State Agency Interview Protocol (attachment D); (3) SNAP Local Agency Interview Protocol (attachment E); (4) SNAP E&T [Employment and Training] Provider Interview Protocol (attachment F); and (5) Interested Parties Interview Protocol (attachment G). The goal of the data collection is to provide the U.S. Department of Agriculture’s (USDA’s) Food and Nutrition Service (FNS) with recommendations on how to assess and monitor equitable administration of work requirements in SNAP and the SNAP E&T program.
The information collected will be evaluated to identify the data needed and the data available to assess equity in policy or other areas where equity issues may arise, such as in the equitable access to SNAP through the administration of work requirements, the equitable participation in education and training programs through SNAP E&T, or the influence of optional and discretionary policies and procedures on participant outcomes. This data collection will help FNS better understand SNAP E&T program operations, current practices, and administrative decision making. The study will employ two data collection efforts:
Web-based survey: The respondent universe for the survey includes all 53 SNAP State agencies; there will be no sampling. Every SNAP State Agency Director, with assistance from their SNAP State E&T Director and a data analyst, is anticipated to respond to the web-based survey. The web-based survey will capture how SNAP State agencies administer SNAP work requirements and their SNAP E&T programs with the purpose of identifying variation in practices across State agencies.
When recruited to complete the survey, SNAP State agencies will be asked to submit relevant documents pertaining to work requirements and their SNAP E&T program. Documents may include SNAP State E&T plans; handbooks or manuals; contracts with providers and partners; documents describing funding allocations; and data dictionaries and reports tracking outcomes, demographics, and other program information. Within each of the 53 SNAP State agencies, the SNAP State Agency Director and a data analyst are anticipated to respond to the document request.
Key informant interviews: USDA FNS will purposively select six SNAP State agencies (and three alternates) based on program and other characteristics and recommendations from SNAP and SNAP E&T program staff and the study team to ensure heterogeneous representation. State agencies will be selected to include variation in FNS Region, SNAP program administration, type of SNAP E&T program, and presence of an Indian Tribal organization. Additional selection factors may include: SNAP E&T program size, types of SNAP E&T providers, and receipt of an E&T technical assistance grant. The alternate study SNAP State agencies will participate in key informant interviews only if one or more of the six identified State agencies decline to participate.
Key informant interviews are designed to provide in-depth information, rather than representative information, of persons with knowledge and experience directly relevant to the study objectives. USDA FNS selects the number of States for in depth data collection based on a variety of factors, including the amount of funding available for the study, the anticipated burden of FNS studies on States at the time of data collection, and the likelihood of getting feedback from the States. Since all States are asked to complete a survey, it can be challenging to get a large number of them to also agree to an in-depth component. FNS has had success, however, in past studies recruiting three to nine States for in-depth work, and has found that these smaller samples provide useful information for program evaluation that captures variation across programs (e.g., regional variations, state administration [county versus state administered], mandatory versus voluntary programs).
Table B.1.1. Key Informant Interview State Selection Criteria.
Within each study State agency, the team will conduct virtual interviews with SNAP State agency staff (e.g., SNAP Director, SNAP E&T Director), SNAP local staff (e.g., supervisors, front-line staff familiar with work requirements and SNAP E&T) from two offices, SNAP E&T providers, and other interested parties (e.g., community-based organizations, community colleges, American Job Centers). The study team will work with each SNAP State Director to identify other agency staff to participate in the SNAP State agency interview and to identify relevant staff at local State offices.
The SNAP State agency interview may include the SNAP State Director, the SNAP E&T Director, and a data analyst familiar with the SNAP eligibility system. The local office interviews may include local office directors and SNAP eligibility workers.
The study team will work with the SNAP State agency to identify the SNAP E&T providers, determine whether any interested parties should be interviewed, and, if so, identify them. Interested parties may include staff from non-SNAP offices, such as the State department of labor, or individuals representing community organizations that work toward increasing workforce participation and self-sufficiency.
The interviews will be informed by each State’s response to the web-based survey. They will facilitate a deep understanding of how SNAP State agencies address equity in administering SNAP work requirements and SNAP E&T, how eligibility workers identify participants who are subject to work requirements, how the provider addresses and ensures equity in SNAP E&T, and how interested parties work with SNAP State agencies to address equity in SNAP E&T.
Estimated number of respondents
The study team anticipates 315 respondents across the survey and key informant interviews. We anticipate three respondents will be responsive to the survey but not the interview component.1 The total number of unique respondents is 293 (243 State, local, and Tribal government staff; 44 staff from businesses; and 6 individuals). The overall response rate to the survey is anticipated to be 1.0; see section B.3 for details on efforts to ensure the intended response rates are achieved. The estimated numbers of respondents per data collection effort follow:
Web-based survey (165 unique respondents): Six individuals pretested the web-based survey; 53 SNAP State Directors will complete the web-based survey. SNAP State Directors may rely on input from a SNAP E&T Director and a data analyst to complete the survey, resulting in an additional 106 respondents. SNAP State Directors may also request the assistance of a data analyst to submit requested documents.
Key informant interviews (138 unique respondents): Key informant interviews were pretested with nine respondents: one SNAP State Director,2 one local SNAP E&T administrator, three local office staff, two persons from an interested party representing a State government organization, and two SNAP E&T providers. To recruit six study SNAP State agencies, a maximum of nine SNAP State Directors will be contacted. Of the nine SNAP State Directors, the study team anticipates three will be nonresponsive (as described at the beginning of this section).
Key informant interviews will be conducted with a total of 138 unique respondents from the six recruited study State agencies: 6 SNAP State Directors, 6 data analysts, 6 SNAP E&T Directors,3 12 local SNAP office directors, 24 SNAP local office staff, and 72 SNAP E&T providers (local government and business); along with 12 interested parties employed by State government agencies or businesses. The study team assumes three interested parties will represent State government agencies, and three interested parties will represent businesses. We anticipate 138 respondents will be responsive, and 0 will be nonresponsive.
Table B.1.2. categorizes respondents and nonrespondents by respondent type and the expected response rate. Attachment N provides a full breakdown of respondents.
Table B.1.2. Respondents and Nonrespondents by Respondent Type
Respondent Type |
Total Sample |
Number of Respondents (unique numbers) |
Number of Nonrespondents (unique numbers) |
Response Rate |
|
Web-based survey |
|||||
State, local, and Tribal government |
SNAP State Director |
53 |
53 (53) |
0 |
1.0 |
SNAP State Data Analyst |
53 |
53 (53) |
0 |
1.0 |
|
SNAP State E&T Director |
53 |
53 (53) |
0 |
1.0 |
|
Individuals |
Survey Pretesters |
6 |
6 (6) |
0 |
1.0 |
Key informant interviews |
|||||
State, local, and Tribal government |
SNAP State Director a, b |
10 |
7 (0) |
3 (0) |
0.7 |
SNAP State Data Analystb |
6 |
6 (0) |
0 |
1.0 |
|
SNAP State E&T Directorb |
6 |
6 (0) |
0 |
1.0 |
|
Local SNAP Office Director |
13 |
13 (13) |
0 |
1.0 |
|
Local SNAP Office Staff |
27 |
27 (27) |
0 |
1.0 |
|
SNAP E&T Provider |
38 |
38 (38) |
0 |
1.0 |
|
Interested Parties (e.g., American Job Centers staff) |
8 |
8 (8) |
0 |
1.0 |
|
Business |
SNAP E&T Provider |
36 |
36 (36) |
0 |
1.0 |
Interested Parties (e.g., community-based organizations, community colleges) |
6 |
6 (6) |
0 |
1.0 |
|
Total |
315 |
312 (293) |
3 (0) |
1.0 |
|
a The three interview nonrespondents will respond to the survey, so they are not counted as unique respondents.
b SNAP State Directors, SNAP E&T Directors, and data analysts participating in interviews and the SNAP State Director who pretested the interview protocol will respond to the survey, so they are not counted as unique respondents.
Describe the procedures for the collection of information including:
Statistical methodology for stratification and sample selection
Estimation procedure
Degree of accuracy needed for the purpose described in the justification
Unusual problems requiring specialized sampling procedures
Any use of periodic (less frequent than annual) data collection cycles to reduce burden
Following Office of Management and Budget approval, the study team will conduct the following activities.
Survey recruitment
FNS will initiate recruitment by contacting the SNAP Regional Office and SNAP State Directors via email (attachment H.1) to explain the overall project and provide a study summary document (attachment H.2). The study team will recruit SNAP State Directors via email (attachment H.3). The email will include a frequently asked questions (FAQ) document to help answer anticipated questions about the web‑based survey (attachment H.4).
Web-based survey
The survey (attachment C) will be sent to all 53 SNAP State agencies via email (attachment H.3) with an embedded web link; the study team anticipates a 100 percent response rate. We anticipate a 4-month field period, beginning with recruitment. We will send email reminders 2 weeks and 4 weeks after receipt of the initial invitation (attachments H.7 and H.8, respectively).
Ten weeks after receipt of the initial invitatio*n, the study team will begin following up by phone with nonrespondents to encourage them to respond to the survey via the website or by phone with trained data collectors; nonrespondents will also be encouraged to upload the requested documents (attachment H.6). We anticipate approximately 4 weeks of telephone follow-up for the survey and document aggregation. The study team will send participants a thank-you email for participating in the survey (attachment H.9).
When recruited for participation in the survey, the 53 SNAP State agencies will be asked to submit relevant files pertaining to work requirements and SNAP E&T programs (attachment H.5). Examples of relevant documents include strategic plan progress tracking, documents describing funding allocations, outreach materials, contracts with providers and partners, and any analyses or reports related to equity that SNAP State agencies have produced.
All SNAP State Directors will receive instructions on the types of documents the study team is requesting and instructions on submission of the files through a web portal. We anticipate all 53 SNAP State agencies will be responsive and provide the requested documents within 4 months of receiving the invitation email.
The study will produce univariate descriptive analyses of the survey items. For categorical and noncontinuous survey variables, the team will present frequencies and percentages/proportions. For continuous variables, the team will present means, medians, ranges, and measures of variation (e.g., 95 percent confidence intervals or interquartile ranges). Survey items with high nonresponse (missing rates greater than 5 percent) will be noted. The team will conduct these analyses on all observations in the survey data.
Key informant interviews
For the interview component, the study team anticipates needing to reach out to up to nine States to participate, with the goal of including six States. State agencies selected for interviews will be invited to participate in the interviews in the initial notification email about the study (attachment H.1) and in a confirmation email (attachment H.11) with an interview FAQ (attachment H.12) at the start of the interview data collection. The States that agree to participate in the interviews will receive a scheduling email (attachment H.13).
All interviews will be conducted virtually and recorded with the permission of the respondents to aid with note-taking (attachments D–G). Interview participants will receive a consent form prior to participation (attachment H.14). Interviews with SNAP State agency staff will last 90 minutes, and interviews with SNAP local agency staff, SNAP E&T providers, and interested parties will last 60 minutes each. Participants will receive a thank-you email for their time (attachment H.15). The study team anticipates a 2-month field period, beginning 1 month after the close of the survey data collection.
To reduce burden on SNAP State agencies, the study team will review survey responses prior to conducting key informant interviews and tailor interview protocols based on what we learn from the State agencies. For example, we will probe on survey answers about the types of data State agencies gather to assess case manager assignment. Interviews will collect more in-depth information about how SNAP State agencies determine who is subject to work requirements, and we will ask participants to share challenges or considerations for measuring equity in SNAP E&T and work requirements.
The selected States and persons within SNAP State agencies participating in the key informant interviews will be purposively selected as a sample intended to gather information on some State agencies’ data collection practices associated with the application of SNAP work requirements and SNAP E&T programs that may be useful in determining measurements of equitable access to programs. The key informant interviews are not meant to produce generalizable or representative information.
No statistical sampling methodology will be employed, no estimation of the number of data sources or systems used will be required, and no unusual problems requiring specialized sampling procedures have been identified. This is a one-time data collection, so periodic data collection cycles are not applicable.
Describe methods to maximize response rates and to deal with issues of nonresponse. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.
The study team expects the planned methods of data collection will result in the accurate and reliable data needed for the planned analysis. The study team will take several steps to ensure the highest possible response rates and highest quality data possible. The study team will select data collectors based on their experience administering web-based surveys and collecting SNAP State agency documents or conducting interviews with SNAP State agency and SNAP E&T provider staff, depending on their assignment.
Data collector training sessions will cover the study objectives and the research questions, review the relevant data collection protocols, and recommend practices for eliciting a high level of response. Additional steps to ensure the highest possible response rates and highest quality data possible follow.
Web-based survey
To encourage respondents to complete the survey, the study team will—
Ask the FNS National Office to send a prerecruitment email explaining the study to the Regional Offices 4 weeks prior to the survey launch.
Ask the FNS National Office to send an email to SNAP State agencies explaining the study (attachment H.1) with a study summary (attachment H.2) 2 weeks prior to the survey launch. The email will emphasize how participants can save their responses and complete the survey later, and they can invite additional people to respond in the survey.
Provide instructions via email launching the survey (attachment H.3) with the FAQ document about the survey (attachment H.4).
Call any SNAP State agencies that have not yet completed the survey to ask if they need assistance (attachment H.6).
Send two follow-up emails reminding SNAP State agencies to complete the survey after launch (attachments H.7–H.8).
Schedule a three-quarters-day training for the web-based survey and invite document data collectors. Training will focus on providing phone support for survey respondents (survey help desk), monitoring all aspects of survey data collection using a survey management system tailored to the needs of this project, and conducting phone follow-up with slow-to-respond SNAP State agencies. The training will discuss strategies to address respondent questions and achieve a 100 percent response rate.
Because the surveys are not designed to provide results representative of a larger population, nonresponse will not affect the study’s ability to address primary research objectives.
Document review material
To encourage response, the study team will—
Provide instructions on the submission of files for the document review with the material sent for the web-based survey (attachment H.5).
Provide contact information for a study team member should the participant have any questions or need assistance with uploading documents.
Send one follow-up email reminding SNAP State agencies to complete the document review (attachment H.10).
Key informant interviews
To encourage response, the study team will—
Identify three alternate SNAP State agencies to ensure completion of the target number of States (six) recruited for interviews.
Ask the FNS National Office to initiate the request for participation in the interviews (attachment H.1).
Prioritize discussions with participating SNAP State agencies to identify appropriate SNAP E&T providers and interested parties that will be responsive.
Work with respondents to schedule interviews when convenient for SNAP State agency staff, local staff, SNAP E&T providers, and other interested parties to ensure their availability.
Share the FAQ (attachment H.12).
Schedule a full-day training for the interview data collectors. Training will focus on scheduling phone interviews, conducting phone interviews, and monitoring key informant interview data collection. Training will include practice role-plays to help each interviewer become comfortable with the protocols and demonstrate readiness.
Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.
The contractor pretested data collection instruments in July 2023 to minimize burden and improve the quality of the data collected. See Attachment K, Pretest Results Memorandum, for further details on the pretesting findings.
Web-based survey
The study team recruited six former administrators and staff from SNAP State agencies to pretest the survey instrument (attachment C) using a cognitive testing approach. Cognitive testing is employed to ensure instructions are clear and survey question are interpreted as intended. After receiving institutional review board approval for data collection, we emailed a PDF version of the survey with a template for providing feedback. After each participant sent their feedback, we scheduled and conducted interviews to review their input and gain additional insight into the interview protocol and inquire about specific survey questions.
The interviews lasted no more than 90 minutes each. The study team received generally positive feedback about the survey. Participants thought the planned protocol would work well and SNAP State Agency Directors would take the task seriously. Most participants believed staff at the agencies could answer all survey questions, though they all agreed it would be a collaborative effort. Participants found very few questions in the survey confusing, but feedback did reveal several areas for improvement. Changes were related to restructuring of questions, adding additional recommended response options, and adjusting wording for clarity.
Some participants thought the 40- to 60-minute estimate seemed accurate, while others thought survey completion would take between 90 minutes and 2 hours. Because of the wide variation in anticipated completion times, we increased our anticipated time to complete the survey to 75 minutes. For specific feedback by question and changes made to the web-based survey, refer to attachment K.
Key informant interviews
FNS emailed the SNAP State Director in New Jersey’s Department of Human Services to request the State agency’s participation in pretesting the key informant interview protocols (attachments D–G). The SNAP State agency confirmed its willingness to participate. The contractor followed up with the SNAP State Director via email to provide additional information about the pretest and worked with the SNAP State Director to identify and recruit local SNAP office staff, SNAP E&T providers, and other interested parties to participate in pretesting. The six study SNAP State agencies selected for key informant interviews will not include New Jersey.
The primary objectives of pretesting the key informant interview protocols were to identify problems related to communicating the meaning of questions, determine whether respondents could provide the information requested, and assess the amount of time needed to complete the interviews. The contractor allotted 15 extra minutes after each interview for follow-up questions and feedback on the protocols. The contractor took detailed notes during each interview and observed which questions respondents had difficulty answering or needed additional clarification to answer.
Pretesting of the interview protocols minimized the burden for respondents by validating the length of the interviews and improved the quality of the data through minor edits clarifying the wording of questions. Overall, the interview protocols flowed well, and all interviews were completed in the allotted time. Based on findings from the pretest, no adjustments to the anticipated time required to complete any of the interviews were necessary.
State Agency Interview Protocol (attachment D): For the SNAP State agency interview, the interviewer skipped questions related to SNAP E&T provider selection and management because that responsibility falls under the purview of the New Jersey Department of Labor and Workforce Development. To ensure the protocol can be completed within the allotted timeframe for all respondents, the study team streamlined questions related to the general and able-bodied adults without dependents (ABAWD) work requirements. More specific feedback and changes made to the State Agency interview protocol can be found in attachment K.
SNAP Local Agency Interview Protocol (attachment E): Local agency respondents had difficulties answering questions related to how to define equitable access and equitable outcomes in SNAP and SNAP E&T. The study team restructured these questions to make them clearer. Local office respondents also had difficulties answering questions related to the general work requirements, ABAWD requirements, and data collected. Local agency staff noted they do not screen for general work requirements because the SNAP State agency no longer has a mandatory SNAP E&T program.
Local agency staff also struggled to recall processes related to ABAWD work requirements because the time limit has not been in place since the onset of the COVID-19 public health emergency in 2020. However, because the landscape of work requirements in SNAP may be different by the time data collection begins next summer, the study team will retain these questions as written. Local offices in other States may be better able to speak to these topics because administrative processes vary across SNAP State agencies. More specific feedback and changes made to the SNAP local agency interview protocol can be found in attachment K.
SNAP E&T Provider Interview Protocol (attachment F): The respondents were able to answer all questions and did not have any additional feedback. The respondents had some difficulty answering questions related to how to define equitable access and equitable outcomes in SNAP and SNAP E&T. The study team restructured these questions to make them clearer. More specific feedback and changes made to the SNAP E&T provider interview protocol can be found in attachment K.
Interested Parties Interview Protocol (attachment G): The respondents were able to answer all the questions we asked and did not have any feedback on the protocol questions. Minor structuring and word changes were made to the questions related to organization and staffing to promote clarity. The study team also added SNAP E&T provider-related questions to the other interested parties protocol. More specific feedback and changes made to the Interested Parties Interview protocol can be found in attachment K.
Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.
The following individuals from USDA were involved in the design of this project:
Kristen Corey, Social Science Research Analyst, USDA, FNS, 703.305.2517
Rachel Gragg, Program Analyst, USDA, FNS, 703.605.3213
Kari Goddard, Program Analyst, USDA, FNS, 303.571.6176
Andrew Dau, Mathematical Statistician, USDA, National Agricultural Statistics Service
FNS has contracted with Westat Insight (now Westat) to conduct this study. Table B.5.1 lists the names and contact information for Westat Insight staff who will be responsible for the collection and analysis of the study data. The Project Officer for the contract providing funding for the evaluation, Dr. Kristen Corey, will be responsible for receiving and approving all contract deliverables; the table also includes her contact information.
Name |
Title (Project Role) |
Organizational Affiliation and Address |
Telephone Number |
Kristen Corey |
Project Officer |
Office of Policy Support Food and Nutrition Service, USDA 3101 Park Center Drive Alexandria, VA 22302 |
703.305.2517 |
Jonathan Blitstein |
Project Director |
Westat Insight 1310 North Courthouse Road Suite 880 Arlington, VA 22201 |
571.347.1033 |
April Fales |
Survey Methodologist |
Westat 1600 Research Boulevard Rockville, MD 20850 |
301.738.3597 |
Kevin Baier |
Quantitative Analysis Lead |
Westat 1600 Research Boulevard Rockville, MD 20850 |
301.279.4593 |
Kathy Wroblewska |
Qualitative Data Collection Lead |
Westat Insight 1310 North Courthouse Road Suite 880 Arlington, VA 22201 |
571.758.5029 |
1 SNAP State Directors invited to participate in interviews are a subset of the directors participating in the survey. The study team expects nine directors will be invited to participate in interviews and three will decline to participate. The three nonrespondents are noted in attachment N but not double-counted in the sample size column.
2 The SNAP State Director who pretested the interview guide is not included in the total count of unique respondents because they are included as a survey respondent.
3 The SNAP State Directors, data analysts, and SNAP E&T Directors participating in interviews are not included in the total count of unique respondents because they are included as survey respondents.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 0000-00-00 |