Attachment B. Research Objectives and Approach to Data Collection
OMB No. 0584-[NEW]
Job Search as a Supplemental Nutrition Assistance Program (SNAP) Employment and Training (E&T) Component
Project Officer: Danielle Deemer
Office of Policy Support
Food and Nutrition Service
U.S. Department of Agriculture
3101 Park Center Drive
Alexandria, VA 22303
703.305.2952
Research Objectives and Approach to Data Collection
Research Objectives
Objective 1. For each study State, document and describe job search activities (including independent job search, job search assistance, and job search training) offered at the State level (and local level where available) in fiscal years 2017 and 2018. Relevant State- and local-level E&T policies that may affect job search activities and participant outcomes broadly (e.g., mandatory or voluntary E&T) as well as requirements specific to job search services (e.g., minimum application submissions required) should also be described.
Objective 2. For each study State, conduct a process evaluation documenting in depth the implementation and operation of job search activities in the study States.
Objective 3. For each study State, conduct an outcomes evaluation by assessing shorter- and longer-term impacts of these job search activities on SNAP E&T participants. Quarterly data on participant outcomes for fiscal years 2017 and 2018 should be examined. To isolate the relationship between job search components and participant outcomes, this analysis may take the form of a multivariate analysis that accounts for program characteristics, participant characteristics, local- and State-level policy contexts, and local economic conditions. The Contractor should also conduct in-depth participant interviews to understand client perceptions of job search outcomes.
Purpose of Data Collection
To achieve these study objectives, FNS will gather data through (1) site visits to four States, (2) the collection of SNAP administrative caseload data and employment and wage record data, and (3) telephone interviews with job search participants. A description of the purpose of each data collection component, along with a description of the data collection process, follows:
Site Visits to Four SNAP E&T States
The study team will begin site visit recruitment and planning shortly after OMB approval and will conduct site visits with four SNAP E&T States beginning approximately 4 weeks after receiving OMB clearance. The sampled States will be chosen to reflect variation in E&T program implementation, such as whether job search is combined with other E&T components, or if the E&T program is operated though a State workforce agency. Participating States are not intended to be nationally representative; cross-State comparisons will not be made.
State SNAP office staff
In advance of each site visit, the study team will ask the State SNAP director or E&T director to complete a cost table prepopulated with cost information available in the State E&T plan (see Attachment K: Administrative Cost Table and Transmittal Email). In-person interviews with State SNAP office staff in each of the four study States will provide high-level information on the design of job search offerings within the State’s SNAP E&T program, integration of job search activities with other services, estimated costs, the definition of “success” for job search participants, and challenges and promising practices related to job search (see Attachment E: State SNAP Director and E&T Director Protocol).
Local SNAP office staff
In-person interviews with two local SNAP offices in each of the four study States will provide a detailed understanding of the SNAP E&T job search processes and procedures and the challenges and promising practices for implementing job search programs (see Attachments F.1 and F.2: Local SNAP Office Director Interview Protocol and Local SNAP Office Frontline Staff Protocol). During the local office visit, the site team will facilitate an in-person process-mapping exercise to map the job search service delivery process (e.g., intake, referral, orientation, service delivery); if possible, the team will also observe day-to-day interactions with job search clients or job search activities (see Attachments G and H: Process-Mapping Protocol and Observation Checklist).
SNAP E&T provider staff
The site visit team will conduct in-person interviews with two SNAP E&T providers in each State to further understand client flow through the SNAP E&T process, available services, and participant barriers and outcomes (see Attachments I.1 and I.2: E&T Provider Director Protocol and E&T Provider Frontline Staff Protocol). If possible, the study team will also conduct observations of job search activities to observe key steps in the process, including screening, client intake, orientation, or job search service delivery (see Attachment H: Observations Checklist).
Other stakeholders
The study team may interview other stakeholders in each State, depending on the nature of the State’s job search program, partners, providers, and participant outcome data tracking processes (see Attachment J: Other Stakeholders Protocol). These stakeholders may include, for example, the State department of labor or other primary partners in implementing the SNAP job search program. These in-person interviews will focus on the stakeholder’s role in SNAP job search, descriptive information about the stakeholder organizations, and challenges and promising practices for job search activities.
Administrative Data Collection
The study team will obtain SNAP administrative caseload (OMB Control Number: 0584-0594; Expiration Date: 01/31/2020) and employment and wage data from each of the four study States to conduct a quantitative assessment of the relationship between participation in job search activities and employment outcomes (see Attachment T: Administrative and Wage Data Collection Instructions).
SNAP administrative caseload data
The study team will collect SNAP caseload data from each State to identify and track participant patterns of job search and SNAP participation and provide information on participant characteristics (see Attachment T: Administrative and Wage Data Collection Instructions). The data will provide information on demographic and income characteristics, participation in job search activities, and reasons for leaving SNAP.
Employment and wage data
The study team will obtain employment and wage data from the State UI agency that collects this information from employers (see Attachment T: Administrative and Wage Data Collection Instructions). These data will provide information on employment and earnings for all SNAP participants in the administrative SNAP data for up to 8 calendar quarters. The employment and wage data will be merged with administrative caseload data to assess the relationship between job search activities and subsequent employment and earnings.
Telephone Interviews With SNAP E&T Job Search Participants
Interviews with current and former job search participants will provide firsthand accounts of the respondents’ job search experiences, including their perceptions of benefits, challenges, and program effectiveness (see Attachments U, W, and L: Participant Advance Letter, Participant Call Script, and Participant Protocol).
The data collection activities for this study will be conducted in the summer of 2020.
Data Collection Process
FNS sent a letter via email to the State SNAP directors to inform them of their States’ selection for the study and encourage their participation (see Attachment W: FNS Advance Letter to States). This email included a description of the study. FNS will answer any preliminary questions States may have about the study. After the States committed to FNS to participate in the study, the study team followed up with an email to each of the SNAP directors to schedule a call to further discuss the study. (The State selection process is described in Supporting Statement Part B.) The objective of this initial call was to introduce the study team, describe the study’s data collection timeline, and determine the State’s process for obtaining wage data. The study team is in the process of negotiating a memorandum of understanding (MOU) (see Attachment S: Sample MOU) with the necessary State agencies as needed.
Site Visits to SNAP E&T States
The study team will conduct site visits to four States that offer job search as a SNAP E&T component. In each State, the study team will meet in person with the following personnel:
State SNAP office staff
The study team will conduct a 90-minute (1.5-hour) interview with the State SNAP director and E&T program director together in each of four States (for a total of four interviews; see Attachment E: State SNAP Director and E&T Director Interview Protocol). The interviews will provide high-level information on the design of job search offerings within the State’s SNAP E&T program, integration of job search activities with other services, program monitoring, estimated costs, and challenges and promising practices related to job search.
Local SNAP office staff
The study team will conduct interviews and process-mapping exercises with, as well as observations of, staff at two local SNAP offices per State (see Attachments F.1, F.2, G, and H: Local SNAP Office Director Protocol, Local SNAP Office Frontline Staff Protocol, Process-Mapping Protocol, and Observation Checklist). The study team will select one rural office and one urban office per State.
Local office director and frontline staff interviews. The study team will conduct 60-minute (1-hour) interviews with the directors of 2 local offices per State for a total of 8 interviews (see Attachment F.1: Local SNAP Office Director Protocol) and 1-hour interviews with up to 2 other staff in 2 local offices in each State for a total of 16 interviews (see Attachment F.2: Local SNAP Office Frontline Staff Protocol). These interviews will increase the study team’s understanding of the office’s operations and job search activities offered, types of providers used and how they are selected, challenges and promising practices for implementing job search programs, and how the local labor market informs services offered or needs of participants.
Process-mapping protocol with local office staff. The study team will facilitate a 90- to 120-minute (1.5- to 2-hour) process-mapping exercise (see Attachment G: Process-Mapping Protocol) with up to five staff in two local offices per State for a total of eight exercises. Using whiteboards to illustrate, the study team will use this exercise to identify each step in the job search process; the staff involved in each step; the time required for each step; and the methods of communication among staff, clients, and providers.
Observations. The study team will observe (see Attachment H: Observation Checklist) intake or eligibility workers in each local office as they conduct screening, client intake, orientation, job search activities, and compliance determinations and/or issue notices of noncompliance or sanctions. These “real-world” observations will follow the guidelines of a standardized observation tool and enrich and provide context for the interview findings.
SNAP E&T provider staff
The study team will conduct 60-minute (1-hour) interviews with and observations of staff E&T providers1 (see Attachments I.1, I.2, and H: E&T Provider Director Protocol, E&T Provider Frontline Staff Protocol, and Observation Checklist). In each State, selected E&T providers may include community colleges, Workforce Innovation and Opportunity Act agencies, for-profit or nonprofit vocational training providers, or community- or faith-based programs. The study team will aim to select providers in both rural and urban areas and ensure variety across provider types.
SNAP E&T provider director and frontline staff interviews. The study team will conduct 60- minute (1-hour) interviews with the directors of two E&T providers in each study State (see Attachment I.1: E&T Provider Director Protocol) for a total of eight interviews. The study team will also interview up to four frontline staff from two E&T providers in each study State (see Attachment I.2: E&T Provider Frontline Staff Protocol). These 60-minute interviews will include up to 2 staff for a total of 16 interviews. These frontline staff may include intake personnel, case managers, job search assistance instructors, job developers, or administrative staff who provide data to the SNAP agency.
Observations. The study team will also observe key steps of the SNAP E&T process at each of the two E&T providers per State to further their understanding of the key steps in the process, including screening, client intake, orientation, or job search service delivery (see Attachment H: Observation Checklist).
Other stakeholder interviews
During the initial telephone call with the State SNAP director and the SNAP E&T program director to discuss the site visit, the study team will discuss whether other stakeholders should be interviewed during the visit, depending on the nature of the State’s job search program, partners, providers, and participant outcome data tracking processes. When an interview is deemed appropriate, State SNAP leadership will facilitate a virtual introduction. Stakeholder interviews will vary depending on the type of stakeholder the State identifies during the planning process. The team will conduct a 60-minute (1-hour) interview with one staff member in each stakeholder agency for a total of four interviews (see Attachment J: Other Stakeholder Protocol).
The study team will send instructions and a list of variables requested to the State staff members familiar with the State and E&T provider data systems and State UI agency wage record systems (see Attachment T: Administrative and Wage Data Collection Instructions). The team will then arrange a consultative discussion to discuss the administrative and wage data to be requested (OMB Control Number: 0584-0594; Expiration Date: 01/31/2020), assess the level of detail and quality of the data the State can provide on E&T participants and their outcomes, and answer any questions the State has. The team will collect 1 year of SNAP administrative data from each of the four States. These data will be used to identify and track participant patterns of job search and SNAP participation and to provide information on participant characteristics to the extent data availability and quality allow. The team will also work with State UI agencies to obtain 2 years of wage records for the four study States.
The administrative data will be used to assess the relationship between SNAP job search activities and participant outcomes such as short- and long-term employment, earnings, and employment stability. The study team will calculate (1) State-level descriptive statistics; (2) cross-tabulations to identify how outcomes of job search participation vary by E&T practices and policies, participant characteristics, and State and local economic conditions; and (3) results of multivariate regressions to examine the relationship between job search participation and outcomes while accounting for E&T practices and policies, participant characteristics, and State and local economic conditions.
SNAP Participant Telephone Interviews
To the extent possible, the study team will aim to use linked administrative and wage record datafiles to select a stratified random sample of job search participants from each State to recruit and complete interviews with up to 50 participants. The study team will draft an advance letter from FNS to be mailed to all selected participants to introduce the study, request their participation in a telephone interview, and assure recipients of the study’s legitimacy (see Attachment U: Participant Advance Letter). The letter (see attachment U: Participant Advance Letter) will inform recipients that an interviewer from the study team may follow up by telephone to schedule an interview or that they can schedule an interview directly by calling a toll-free number or by visiting a website called www.Calendly.com through which they can select a date and time for the interview (see Attachment U: Description of Scheduling Tool). The study team will reach out to nonrespondents by telephone and leave a brief voicemail describing the study if there is no answer (see Attachment W: Participant Call Script) . Advance letters (see attachment U: Participant Advance Letter) will be mailed in batches, and the team will continue to work through the sample until approximately 50 interviews per State are completed or the data collection period ends. Because the telephone interviews are meant to supplement the analysis, a relatively small (n=50) number of interviews will be conducted in each State in a common language. All interviews will be conducted in English to ensure themes can be identified and analyzed across the entire sample.
The agency pretested the site visit protocols in one State (see Attachment AA: Pretest Memorandum).
1 The E&T providers are assumed to be a mix of State, business or other for-profit, and nonprofit organizations (i.e., three State, two for-profit, and three nonprofit E&T providers).
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Allyson Corbo |
| File Modified | 0000-00-00 |
| File Created | 2021-01-14 |