OMB Part B OCIO comments 3-12-2020

Supporting Statement Part B

OMB No. 0584-[NEW]

Job Search as a Supplemental Nutrition Assistance Program (SNAP) Employment and Training (E&T) Component

January 14, 2021

Project Officer: Danielle Deemer

Office of Policy Support

Food and Nutrition Service

U.S. Department of Agriculture

1320 Braddock Place

Alexandria, VA 22314

703.305.2952

[email protected]

Contents

Part B. Collection of Information Employing Statistical Methods 1

B.1. Respondent Universe and Selection Methods 1

B.2. Procedures for the Collection of Information 4

B.3. Methods to Maximize Response Rates and the Issue of Nonresponse 5

B.4. Tests of Procedures 6

B.5. Consultants 7

References 9

Tables

Table B.1.1. Breakout of Respondents and Nonrespondents by Respondent Type 4

Table B.5.1. Consultants 8

Attachments

A. Legal Authority Statutes and Regulations

B. Research Objectives and Approach to Data Collection

C. FTP Site Instructions for Submitting Administrative Data

D. Email to States With FTP Password

E. State SNAP Director and E&T Director Protocol

F.1. Local SNAP Office Director Protocol

F.2. Local SNAP Office Frontline Staff Protocol

G. Process-Mapping Protocol

H. Observation Checklist

I.1. E&T Provider Director Protocol

I.2. E&T Provider Frontline Staff Protocol

J. Other Stakeholder Protocol

K. Administrative Cost Table and Transmittal Email

L. Participant Protocol

M.1. Federal Register Comment 1

M.2. FNS Response to Federal Register Comment 1

M.3. Federal Register Comment 2

M.4. FNS Response to Federal Register Comment 2

N.1. NASS Comments

N.2. FNS Response to NASS Comments

O. Use of Tokens of Appreciation

P. Client Consent Form for Observations

Q. Insight Policy Research Confidentiality Pledge

R. Total Public Burden Hours and Respondent Costs: Excel

S. Sample MOU

T. Administrative and Wage Data Collection Instructions

U. Participant Advance Letter

V. Description of Scheduling Tool

W. Participant Call Script

X. FNS Advance Letter to States

Y. Study Description

Z. Letter from State SNAP Office to Local Offices

AA. Pretest Memorandum

Part B. Collection of Information Employing Statistical Methods

1. Respondent Universe and Selection Methods

Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Respondent Universe

This is a new information collection request. This study will collect qualitative and administrative data from four States (Connecticut, Michigan, Oregon, and Tennessee) that offer job search as a Supplemental Nutrition Assistance Program (SNAP) employment and training (E&T) component. The U.S. Department of Agriculture’s (USDA) Food and Nutrition Service (FNS) identified three categories of States from which to select the four study States. Selection criteria included the following:

• States where job search is the largest E&T component and is typically operated through a workforce agency, such as through One Stop Job Centers

• States with robust E&T programs where job search is combined with other approaches

• States that are currently improving their E&T programs by partnering with third-party training providers to offer job search services

Using these criteria, FNS identified four study States and confirmed their willingness to participate in the study in early 2019.

Within each study State, the team will visit the State office, two local SNAP offices, and two E&T providers. The study team will work with each of the four study States to select the local SNAP offices and E&T providers to participate in the study. The team plans to visit both urban and rural providers. The team will also work with the State agency to identify any other key stakeholder that works closely with the State to implement job search activities as applicable. In-person stakeholder interviews will vary depending on the type of stakeholder the State identifies during the planning process; interviews with these respondents are intended to inform the design and implementation of SNAP E&T job search services in the State. In some States, for example, the State department of labor may be a key stakeholder in designing job search or other E&T services; tracking participant outcome data; or even offering job search services.

The study team will aim to use the linked administrative and State Unemployment Insurance (UI) wage record datafiles to purposively select SNAP E&T job search participants from each State to recruit and complete interviews with up to 50 SNAP participants (English speakers) in each State¹. However, because the timeline for implementing the necessary memoranda of understanding (Attachment S: Sample MOU) and collecting linked SNAP UI datafiles (Attachment T: Administrative and Wage Data Collection Instructions) may vary by State, the study team may recruit SNAP participants as needed from the unlinked SNAP administrative data (OMB Control Number: 0584-0594; Expiration Date: 01/31/2020 currently at OMB under review) for the most recent month available to minimize schedule delays in the interview data collection process.

SNAP participants will be purposively selected to provide a range of perspectives that may affect experiences with SNAP job search. To the extent education and employment data are available, the study team will stratify based on work experience (evidence of recent work experience versus little to no recent work experience) and education (high school diploma or equivalent versus less than high school education). Within these categories, the team will sort the SNAP participants by location of residence (urban/rural), age, and current/former job search participation status when available. The study team will establish approximate recruitment targets for each subgroup based, in part, on the characteristics of each State’s overall SNAP E&T job search population (e.g., aim to recruit similar proportions of urban and rural respondents as are in the State’s overall SNAP E&T population).

The study team will use these targets to purposively select SNAP participants for recruitment. Purposeful sampling increases the likelihood that the study team will interview a range of participants whose job search experiences and outcomes may differ, thereby enhancing opportunities for analytic generalization. The research team will monitor responses to ascertain whether particular subgroups are under- or overrepresented.²

Estimated Number of Respondents

All four States will be involved in both the site visit and administrative data collection components of the study. Within each State, the study team will visit and conduct in-person interviews with staff at the State SNAP office, two local SNAP offices, two local E&T providers, and a stakeholder organization when applicable (see Attachments E, F.1, F.2, I.1, I.2, and J: State SNAP Director and E&T Director Protocol, Local SNAP Office Director Protocol, Local SNAP Office Frontline Staff Protocol, E&T Provider Director Protocol, E&T Provider Frontline Staff Protocol, and Other Stakeholder Protocol). After (or during) the local SNAP office and local E&T provider interviews, the team will conduct observations to detect key steps in the process (see Attachment H: Observation Checklist). The team will also facilitate a process-mapping exercise with staff at the two local SNAP offices (see Attachment G: Process-Mapping Protocol). The team will collect extant administrative data from the State SNAP office and work with State UI agencies to obtain UI wage records (see Attachment T: Administrative and Wage Data Collection Instructions). SNAP E&T participants will be involved (with their informed consent) in the observations at both the local SNAP office and the E&T providers, and the study team will conduct in-depth phone interviews with up to 50 SNAP job search participants in each of the four study States (see Attachments P and L: Client Consent Form for Observations and Participant Protocol).

Including the pretest (see attachment AA: Pretest Memorandum), the total expected number of respondents is 460, which will include 101 State and local government staff, 12 business or other for-profit staff, 19 nonprofit staff, and 328 individuals. Out of the 460 to be contacted, 375 are expected to be responsive, and 85 are expected to be nonresponsive. FNS anticipates 100 percent participation from Business or other for profits, non-profits, and State, Local, and Tribal governments. Table B.1.1 provides the breakout of respondents and nonrespondents by respondent type.

Table B.1.1. Breakout of Respondents and Nonrespondents by Respondent Type

Respondent Type		Total to Be Contacted	Expected Number of Respondents	Expected Number of Nonrespondents
State and Local Government	State SNAP staff	9	9	0
	State database administrators	5	5	0
	UI agency database administrator	4	4	0
	UI agency DUA liaison	4	4	0
State and Local Government (continued)	SNAP E&T job search stakeholder agency	4	4	0
	State E&T provider staff	18	18	0
	Local office staff	57	57	0
Business or Other for Profit	E&T provider staff	12	12	0
Nonprofit	E&T provider staff	19	19	0
Individuals	SNAP participants	328	243	85

2. Procedures for the Collection of Information

Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection

• Estimation procedure

• Degree of accuracy needed for the purpose described in the justification

• Unusual problems requiring specialized sampling procedures

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden

Following OMB approval, administrative data and UI wage data from the four States will be collected via the contractor’s secure FTP site (see Attachment T: Administrative and Wage Data Collection Instructions) and site visits will be scheduled with all participating States. Site visit data will be gathered through semi-structured interviews, observations, and process-mapping discussions (see Attachments E, F.1, F.2, G, H, I.2, I.2, and J: State SNAP Director and E&T Director Protocol, Local SNAP Office Director Protocol, Local SNAP Office Frontline Staff Protocol, Process-Mapping Protocol, Observation Checklist, E&T Provider Director Protocol, E&T Provider Frontline Staff Protocol, and Other Stakeholder Protocol).

Job search participants will be purposively selected from the administrative data files provided by the States. As described in section B.1, participants will be purposively selected for recruitment to reflect a range of perspectives on SNAP job search, based on a number of characteristics including work experience, education, location of residence (urban/rural), age, and current/former job search participation status when available.

No statistical sampling methodology will be employed, no estimation of the number of data sources or systems used will be required, and no unusual problems requiring specialized sampling procedures have been identified. This is a one-time, voluntary data collection, so periodic data collection cycles are not applicable.

3. Methods to Maximize Response Rates and the Issue of Nonresponse

Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

The study team expects the planned methods of data collection will result in the accurate and reliable data needed for the planned analysis. To ensure the highest response rates and highest quality data possible for the site visit interviews and administrative data collection, the study team will—

• Coordinate with State staff well in advance of the site visits to answer all questions and ensure the data collection takes place at convenient times.

• Work with States to schedule site visits when convenient to State, local, and provider staff to ensure their availability for data collection.

• Send the advance instructions and list of variables (see Attachment S: Administrative and Wage Data Collection Instructions) to States for preparing administrative data and participate in a consultative discussion with each study State to discuss the request and answer any questions.

To maximize the response rates of SNAP participants for telephone interviews, an advance letter will be mailed to selected participants to introduce the study and request their participation in a telephone interview and to assure recipients of the study’s legitimacy (see Attachment U: Participant Advance Letter). The letter will inform recipients the data collection is voluntary and there will be no penalties or loss of benefits if they decide not to participate. The letter will also notify recipients a trained interviewer from the study team may follow up by telephone to schedule an interview, or they can schedule an interview by calling a toll-free number or by visiting Calendly.com where they can select a date and time for their interview (see Attachment V: Description of Scheduling Tool for details on the use of this tool). The study team will attempt to contact nonrespondents once and will leave a brief voicemail describing the study if there is no answer (see Attachment W: Participant Call Script). FNS will send additional advance letters, as necessary, to the replacement sample members. The research team will determine the number of letters that need to be sent based on the initial response rate and recruitment progress.

To encourage a high response rate, a $30 gift card will be used as a token of appreciation for participating in the interview will be offered to SNAP Participants (individuals/households) to offset any expenses incurred during the telephone interviews. (See Attachment O: Use of Tokens of Appreciation). To further bolster the perceived legitimacy of the study, the research team will ensure the SNAP offices and E&T providers are aware of and have information on the study to share with participants who receive the letter. The research team will provide to States a letter from FNS recruiting their participation in the study, as well as study overview for dissemination (see Attachments X & Y: FNS Advance Letter to States and Study Description). Because selected sample members may reside in different areas of the State, the team will explore the feasibility of having each State’s SNAP office send a letter to all local offices to promote awareness of the study (see Attachment Z: Letter from State SNAP Office to Local Offices.

We anticipate all selected States will respond to the information collection because the study States have confirmed their ability and willingness to participate.

4. Tests of Procedures

Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

FNS pretested the following interview data collection instruments for the study to evaluate the clarity of the questions asked, identify possible modifications to question wording or order that could improve the quality of the data, and estimate respondents’ burden:

• State SNAP director and E&T director interview protocol

• Local SNAP office director interview protocol

• Local SNAP office staff interview protocol

• E&T provider interview protocol

• Process-mapping protocol

• Administrative cost table

• Participant interview protocol

The study team pretested each of these instruments with a respondent from Maryland, which is a State not included in the study. After the pretest, the study team made minor revisions to the instruments, including changing the order of some questions, deleting some questions to reduce redundancy, and making minor edits to clarify wording. See Attachment AA: Pretest Memorandum for details.

5. Consultants

Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

FNS consulted with a mathematical statistician from USDA’s National Agricultural Statistics Service (NASS), who reviewed the study methodology and statistical procedures. The review from NASS and the study team’s response to NASS’s comments appear in Attachments N.1 and N.2, respectively.

FNS has contracted with Insight Policy Research to conduct this study. Table B.5.1 lists the Insight staff members who will be responsible for the collection and analysis of the study data. The Project Officer for the contract providing funding for the evaluation, Dr. Danielle Deemer, will be responsible for receiving and approving all contract deliverables. Her contact information is also included in table B.5.1.

Table B.5.1. Consultants

Name	Title (Project Role)	Organizational Affiliation and Address	Telephone Number
Brittany McGill	Project Director	Insight Policy Research, Inc. 1901 North Moore Street, Suite 1100 Arlington, VA 22209	703.504.9485
Carole Trippe	Quantitative Analysis Lead	Insight Policy Research, Inc. 1901 North Moore Street, Suite 1100 Arlington, VA 22209	703.504.9498
Brian Estes	Qualitative Analysis Lead	Insight Policy Research, Inc. 1901 North Moore Street, Suite 1100 Arlington, VA 22209	703.504.9492
Kathy Wroblewska	Project Manager	Insight Policy Research, Inc. 1901 North Moore Street, Suite 1100 Arlington, VA 22209	571.758.5029
Peter Mueser	Consultant	University of Missouri-Columbia 118 Professional Building Columbia, MO 65211	573.882.6427
Danielle Deemer	FNS Project Officer	Food and Nutrition Service, USDA Office of Policy Support SNAP Analysis Branch 1320 Braddock Place Alexandria, VA 22314	703.305.2952

References

Mason, M. (2010). Sample size and saturation in PhD studies using qualitative interviews. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research, (11)3.

1 All interviews will be conducted in English, including with SNAP participants. This will ensure the relatively small sample size will yield themes that can be analyzed across the sample to ensure the validity of research findings.

2 Although the study team chose this approach to minimize sample bias that could occur with a straightforward convenience sample, regardless, the sample sizes in each stratum will be too small to make general claims about representativeness. Small sample sizes are customary in qualitative research; to ensure rigor, the team will need to take reasonable steps to reduce obvious sources of bias and determine the limitations of the sample in the analysis. Because the goal of qualitative research is meaning rather than generalizability, the representativeness of the sample is a lower priority than it would be in quantitative research (see Mason, 2010).