2. MET OMB Part B

Supporting Statement Part B

OMB No. 0584-[NEW]

Assessment of Mandatory E&T Programs

Month XX, 2018

Project Officer: Jordan Younes

Office of Policy Support

Food and Nutrition Service

U.S. Department of Agriculture

3101 Park Center Drive

Alexandria, VA 22303

703-305-2935

[email protected]

Contents

Part B. Collection of Information Employing Statistical Methods 1

B.1 Respondent Universe and Selection Methods 1

B.2 Procedures for the Collection of Information 3

B.3 Methods To Maximize Response Rates and the Issue of Nonresponse 4

B.4 Tests of Procedures 5

B.5 Consultants 7

Tables

Table B.1.1. Breakout of Respondents and Nonrespondents by Respondent Type 3

Table B.3.1. Expected Response Rates 5

Table B.5.1. Consultants 7

Appendices

A. Legal Authority Statutes and Regulations

B.1 Study Objectives Response to Supporting Statement Question 2

B.2 Research Objectives and Questions by Data Source

C.1 Recruitment Email from FNS to States

C.2 Follow Up Email From Study Director to States To Schedule Call

D. Study Overview and Frequently Asked Questions

E. State SNAP Director and E&T Director Interview Protocol

F. Local SNAP Office Director Interview Protocol

G. Local SNAP Office Eligibility Worker Interview Protocol

H. Process Mapping Protocol

I. Observation Checklist

J. Client Consent Form for Observations

K. E&T Provider Interview Protocol

L. Administrative Data Submission Instructions and Variable List

M.1. FTP Site Instructions for Submitting Administrative Data

M.2. Email to States with FTP Password

N. Federal Register Notice Comments

O. FNS Responses to Federal Register Comments

P.1. National Agricultural Statistics Service Comments

P.2. FNS Responses to NASS Comments

Q. Insight Policy Research Confidentiality Pledge

R. Pretest Methods and Summary of Findings

S. Total Public Burden Hours and Respondent Costs: Excel

Part B. Collection of Information Employing Statistical Methods

1. Respondent Universe and Selection Methods

Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Respondent Universe

This study has two components: (1) site visits to six States with mandatory E&T programs; and (2) analysis of extant administrative data collected from the six States. The six study States will be selected from among 17 States currently operating mandatory E&T programs. Three of 17 States were dropped from consideration because they are part of the FNS Evaluation of SNAP E&T Pilots. Among the remaining 14 mandatory E&T States, a two-tier framework of primary and secondary criteria was developed to select the six States. Selection criteria included the following:

Primary criteria

• Ratio of SNAP E&T participants to SNAP recipients

• Ratio of SNAP E&T participants to SNAP work registrants

• Number of E&T components offered

• Percentage of participants receiving each component in fiscal year (FY) 2016

• Number of participants anticipated in each E&T component in FY 2018

• Number of E&T exemptions

• Type of exemption offered

• Race of household head

Secondary criteria

• Geographical area offering E&T

• Type of SNAP administration

• FNS region

• Pledge State

• SNAP E&T target population

• Able-bodied adult without dependents (ABAWD) time limit waiver

The study team compiled data on the remaining 14 mandatory E&T States based on these selection criteria and evaluated the variation in States across the criteria. The six study States will be chosen to reflect variation across the selection criteria, including variation in the number of participants served through the SNAP E&T programs, adoption of SNAP E&T policy options, racial diversity, and geography. Each of the six selected States will also have a recommended backup State to recruit if the primary State declines to participate in the study. Backup States will be selected to most closely match the variation each State provides to the group.

The study team will work with each of the six study States to select two local SNAP offices and three E&T providers to participate in the study. The study team aims to visit both urban and rural providers and those providers that serve the largest number of participants and provide services representative of those available to participants.

Estimated Number of Respondents

All six States will be involved in both the site visit and administrative data collection components of the study. Within each State, the study team will visit and interview staff at the State SNAP office, two local SNAP offices, and three local E&T providers. Of the three E&T providers, the study team assumes one will be a State or local government E&T provider, one a business or other for-profit provider, and one not-for-profit provider. The study team will collect extant administrative data from the State SNAP office and one E&T provider per State. Individuals will be involved in the observations at both the local SNAP office and the E&T providers.

The total estimated number of respondents is 207, which includes 119 State and local government staff, 29 business or other for-profit staff, 29 not-for-profit staff, and 30 individuals. Out of the 207 contacted, 198 are estimated to be responsive and 9 are estimated to be nonresponsive. Table B.1.1 provides the breakout of respondents and nonrespondents by respondent type.

Table B.1.1. Breakout of Respondents and Nonrespondents by Respondent Type

Respondent Type		Total Contacted	Number of Respondents	Number of Nonrespondents
State and local government	State SNAP staff	12	12	0
	State database administrators	6	6	0
	State E&T provider staff	29	26	3
	Local office staff	72	72	0
Business or other for-profit	E&T provider staff	29	26	3
Not-for-profit	E&T provider staff	29	26	3
Individuals	SNAP participants	30	30	0

2. Procedures for the Collection of Information

Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection

• Estimation procedure

• Degree of accuracy needed for the purpose described in the justification

• Unusual problems requiring specialized sampling procedures

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden

Advance materials will be sent to the SNAP director at each State SNAP office via email. A call will then be scheduled with each SNAP director to describe the study’s purpose and data collection activities, answer questions, and assess the State’s capacity and willingness to participate. Advance materials will also be sent to local office staff, State and local E&T providers, and database administrators (see appendices C.1, C.2 and D for advance materials). Site visits will be scheduled with all participating States, and data will be gathered through semistructured interviews, observations, and process-mapping discussions (see appendices E, F, G, H, I, and K for site visit protocols). Administrative data for the six States and E&T providers will be collected via the contractor’s secure FTP site (see appendix L for the administrative data submission instructions and list of variables).

No statistical sampling methodology will be employed, no estimation of the number of data sources or systems used will be required, and no special sampling procedures will be used. Communication will consist of email, phone calls, and in-person site visits.

No unusual problems requiring specialized sampling procedures have been identified. This is a one-time data collection, so periodic data collection cycles are not applicable.

3. Methods To Maximize Response Rates and the Issue of Nonresponse

Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

The study team expects the planned methods of data collection will result in the accurate and reliable data needed for the planned analysis. Table B.3.1 shows anticipated response rates. To ensure the highest response rates and highest-quality data possible, the study team will—

• Send advance materials (appendix C.1 – Recruitment Email from FNS to States and appendix C.2 – Follow Up Email to States from Study Director to Schedule Call) and schedule site visits in advance to answer all questions and ensure the data collection takes place at convenient times.

• Schedule site visits when convenient to State, local, and provider staff to ensure availability of staff for data collection.

• Send advance instructions and list of variables (appendix L – Administrative Data Submission Instructions and Variable List) to States and E&T providers for preparing administrative data and answer any questions.

• Track respondents who have agreed to participate in the site visit and send reminders to those who have not.

• Contact respondents who are slow in responding to administrative record and site visit requests via phone to answer their questions and remind them of the importance of the study.

• Be staffed with qualified, well-trained professional interviewers.

Table B.3.1. Expected Response Rates

Respondents	Research Activity	Expected Response Rate
State SNAP agency	Onsite interviews	100
	Administrative data collection	100
Local SNAP office	Onsite interviews and process-mapping discussion	100
E&T providers	Onsite interviews	67
	Administrative data collection	100
SNAP participants	Observations of the Intake Process	100

We anticipate all selected States will participate. However, in the event a State selected to participate is unable to do so, the study team will match each study State with a backup State. Backup States will be selected to most closely match the variation each State provides to the group. Choosing a well-paired backup will help maintain the level of variation created by the original group of six States.

4. Tests of Procedures

Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

The contractor pretested all interview data collection instruments for the study to evaluate the clarity of the questions asked, identify possible modifications to question wording or order that could improve the quality of the data, and estimate respondents’ burden. Staff at the State SNAP office, one local SNAP office, and a SNAP E&T provider in Minnesota participated in the pretest of the interview instruments. The contractor also conducted a pretest of the administrative data submission instructions with a staff member at the Minnesota State SNAP office to assess clarity of the instructions and obtain feedback on the list of variables requested. See appendix R for results of the pretest of the site visit protocols and data submission instructions.

Pretest Findings and Changes to Interview Protocols

1. State SNAP and E&T Director Protocol. Based on the results of the pretest, the study team recommended removing three questions from the protocol that were not directly related to the research questions or were better answered by other staff to ensure that the interviews were reduced from 75 minutes to an average of 60 minutes. The study team revised three questions to increase clarity and reduce burden.

2. Local Office Director Protocol. The interviewee had difficulty answering some questions on the spot, such as questions regarding the local economy or the effect of the new policies on the E&T program. The study team revised these questions to improve clarity, and will provide a bulleted list of discussion topics to the local office in advance of the visit to allow respondents to prepare as needed. The study team also reworded a question on the E&T referral process to reduce potential misunderstanding.

3. Local Office Eligibility Worker Protocol. The study team reworded a question on the E&T referral process to reduce potential misunderstanding.

4. E&T Provider Protocol: Supervisor. The interviewee deferred some of the questions about data reporting and tracking to the E&T frontline staff. As a result, the study team revised the protocol to direct the high-level data reporting and tracking questions to supervisors and the more detailed questions to frontline staff.

5. E&T Provider Protocol: Frontline Staff. Based on the interviewee’s responses, the study team added language to clarify four questions.

Administrative Data Findings and Changes to Data Collection Instructions

Based on the results of the pretest, the study team made the following revisions to the administrative data collection materials:

• Limited the ages of participants included in the data to focus only on the universe of individuals potentially affected by E&T requirements and reduce the volume of data requested.

• Limited the data to include variables only for months in which the participants are in the State caseload system.

• Added two new variables to the data file: (1) individual-level earnings, and (2) sanction start and end date.

• Made three minor edits to existing variables.

5. Consultants

Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

A review by the USDA National Agricultural Statistics Service (NASS) was conducted on July 10, 2018 (see appendix P.1). No other individuals outside the evaluation project were consulted on statistical aspects of the design. FNS has contracted with Insight Policy Research to conduct this study. Table B.5.1 lists the NASS statistician who reviewed these materials and the Insight staff members who will be responsible for the collection and analysis of the study’s data. The Project Officer for the contract providing funding for the evaluation, Jordan Younes, will be responsible for receiving and approving all contract deliverables. Her contact information is also included in Table B.5.1.

Table B.5.1. Consultants

Name	Title (Project Role)	Organizational Affiliation and Address	Phone Number
Brittany McGill	Project Director	Insight Policy Research, Inc. 1901 North Moore Street, Suite 1100 Arlington, VA 22209	703.504.9485
Carole Trippe	Co-Principal Investigator	Insight Policy Research, Inc. 1901 North Moore Street, Suite 1100 Arlington, VA 22209	703.504.9498
Claire Wilson	Qualitative Analysis Lead	Insight Policy Research, Inc. 1901 North Moore Street, Suite 1100 Arlington, VA 22209	703.504.9484
Brian Estes	Site Visit Lead and Project Manager	Insight Policy Research, Inc. 1901 North Moore Street, Suite 1100 Arlington, VA 22209	703.504.9492
Gretchen Rowe	Co-Principal Investigator	Mathematica Policy Research 1100 1st Street, NE, 12th Floor Washington, DC 20002	202.484.4221
Linette Lanclos	Mathematical Statistician	USDA-NASS 1400 Independence Ave., SW Washington, DC 20250	202.720.2641
Jordan Younes	FNS Project Officer	Office of Policy Support SNAP Analysis Branch Food and Nutrition Service, USDA 3101 Park Center Drive Alexandria, VA 22302	703.305.2935