BEES Supporting Statement B

The Building Evidence on Employment Strategies for Low-Income Families Project (BEES)

OMB Information Collection Request

0970-0356

Supporting Statement

Part B

November 2017

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers:

Tiffany McCormack

Carli Wulff

The purpose of the current Generic Information Collection (Gen IC) is to conduct a scan focused on identifying high priority interventions and sites that could participate in rigorous evaluation. Information will be gathered via tailored, semi-structured discussions with national, regional, state, and local informants about employment-based interventions for TANF recipients and other disadvantaged groups, including those facing substance abuse (including abuse of opioids) and mental health challenges. The scan will inform the research design, recruitment, and sampling strategies used for the first phase of a research and evaluation study sponsored by the Office of Planning, Research, and Evaluation (OPRE) called The Building Evidence on Employment Strategies for Low-Income Families Project (BEES).

B1. Informant Universe and Sampling Methods

This Gen IC for the BEES project seeks to gather information from key informants in the fields of workforce development, substance abuse, and mental health. First, the evaluation team will develop a list of possible informants based on recommendations from the Contracting Officer’s Representatives (CORs) and other staff in OPRE, referrals from other informants, and Internet searches of employment interventions. Informants in this Gen IC will include:

• Researchers and policy experts in the fields of workforce development, substance abuse, and mental health (16 respondents)

• State and local administrators of systems such as Temporary Assistance for Needy Families (TANF), Supplemental Nutrition Assistance Program (SNAP), Workforce Innovation and Opportunity Act (WIOA), behavioral health, and criminal justice (27 respondents)

• Program staff of employment interventions for TANF recipients and other disadvantaged groups, including those facing substance abuse and mental health challenges (12 respondents).

Next, the evaluation team will use a purposeful, snowball sampling strategy to determine which informants to engage and when. The informants will be selected to participate in an iterative fashion based on their expertise and the study’s need for information from different sources and localities. The team will return consistently to the list of potential informants to determine which individuals with select expertise should be engaged next given the evaluation team’s remaining gaps in knowledge. Overall, we expect that 95 percent of the individuals we contact will participate in a discussion.

B2. Procedures for Collection of Information

Informal, semi-structured discussions with the field assessment participants will be conducted by telephone and in-person. The evaluation team will use the approved discussion guides to help ensure that appropriate topics are covered, given the type of stakeholder being interviewed. Using semi-structured discussions to collect data will ensure that each discussion covers the key topics, while allowing the interviewer to tailor the conversation as needed to efficiently elicit information from each individual respondent. Discussions will be led by senior members of the evaluation team. The evaluation team will obtain relevant written materials that are readily available, and prepare written summaries of each discussion for submission to ACF. These written materials will be for internal purposes.

Discussions will be conducted commencing with the receipt of OMB clearance and will occur from January to December 2018. The discussions will require up to 100 burden hours total from those being interviewed.

B3. Methods to Maximize Response Rates and Deal with Nonresponse

Expected Response Rates

Participation in this Gen IC is voluntary. In our prior efforts to gather insights and information from key informants in the early design phases of studies, we have found that informants are eager to participate in these types of discussions. We therefore expect to obtain a very high response rate (95 percent) among stakeholders who are contacted for an interview.

Several factors will help ensure a high rate of cooperation among informants. First, senior members of the evaluation team who are familiar with the issues will conduct the discussions. Second, it is expected that staff conducting the discussions will personally know many of the informants through their professional connections. Finally, many states and localities are interested in improving efforts to support employment for low-income families and are actively seeking ideas and evidence to respond to the opioid epidemic, and, therefore, we expect that they will be interested in identifying interventions to be tested in the study.

Dealing with Nonresponse

If a potential informant does not respond to our initial email request (see Attachment D) for a meeting, the evaluation team will send a second, similar email. If we do not receive a response to our second email, the team may follow up with a phone call. After three attempts, the evaluation team will not continue to pursue a response.

Maximizing Response Rates

The evaluation team will be accommodating of informants’ schedules and will be flexible about the meeting dates and times. The informants will be contacted by a senior member of the evaluation team. These senior members have had significant experience in working closely with programs and their stakeholders on previous evaluations and have had a high level of success in engaging programs in preliminary discussions such as these.

B4. Tests of Procedures or Methods to be Undertaken

The discussion guides are similar to guides that have been approved and used successfully in prior OPRE studies, including the Job Search Assistance Strategies Evaluation (OMB #0970-0440), Subsidized and Transitional Employment Demonstration (OMB #0970-0413) and the Assets for Independence Evaluation (OMB #0970-0414).

B5. Individual(s) Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

This data collection effort will be designed and carried out by senior members of the evaluation team, on behalf of ACF, with ACF oversight and approval. The analysis of the interview data will be qualitative, and statistical methods will not be employed. The key individuals involved in this effort follow:

• Dan Bloom, Policy Area Director, MDRC

• Sue Scrivener, Senior Research Associate, MDRC

• Megan Millenky, Research Associate, MDRC

• Karin Martinson, Principal Associate, Abt Associates

• Mike Fishman, President, MEF Associates

• Tiffany McCormack, Senior Social Science Research Analyst, ACF

• Carli Wulff, Social Science Research Analyst, ACF

4