ACF Behavioral Interventions to Advance Self-Sufficiency Next Generation (BIAS-NG) Project
Generic Information Collection Request for Qualitative and Descriptive Quantitative Data Collection for a Child Welfare Site
0970-0502
SUPPORTING STATEMENT
PART B
June 2018
Submitted By:
Office of Planning, Research, and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
Mary E. Switzer Building
330 C Street, SW
Washington, D.C. 20201
B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS
TABLE OF CONTENTS
1. Respondent Universe and Sampling Methods 2
2. Procedures for the Collection of Information 3
3. Methods to Maximize Response Rates and Deal with Non-response 5
Behavioral Interventions to Advance Self-Sufficiency-Next Generation (BIAS-NG) consists of a series of mixed-methods studies to identify, develop, and test interventions related to social service and benefit receipt. This Generic Information Collection (GenIC) pertains to a site in the program area of Child Welfare, the Allegheny County Department of Human Services.
The target respondents to be included in this GenIC include:
Customers or individuals receiving services from state and local ACF programs in the domain of Child Welfare.
Staff working in state and local programs or agencies in the domain of Child Welfare.
This IC pertains to activities that support the impact evaluation and implementation research.
For the implementation study, this data collection will involve formal interviews as well as surveys, which will be sent to the full universe of staff and clients participating in the study. For the impact study, data collection will rely on administrative records on case outcomes and MIS data on agency processes, as described below.
Universe of Potential Respondents
The universe of clients for client surveys is any adult client who has a case open during the study period and has an active cell phone. We are randomizing all cases that open during our 12- to 15-month study period. We expect about 1,500 cases to open during this time. We anticipate extending the client text survey to 1,500 clients – at least one adult per case. Of these, we expect that 600 clients (40 percent) will respond, with a burden estimate of 0.04 hours for four responses each for a total of 0.16 burden hours per respondent.
The universe for client interviews is any adult client who can be reached by the agency during the month of the interview. We estimate about 100 cases open per month, and intend to approach at least three new cases in each of the six regional offices. We expect a maximum of 18 respondents total at 1 hour of burden per respondent.
The universe for staff and administrator surveys is any case-carrying caseworker or case supervisor dealing with case openings during the month of the survey. This does not include adoption caseworkers. We anticipate there will be 180 individuals receiving the staff survey. We expect a response rate of about 60 percent (108 respondents) at 0.25 burden hours per respondent.
The universe for staff interviews is any active caseworker who can be reached by the agency during the month of the interview plus some supervisors. We expect a maximum of 18 respondents at 1 burden hour per respondent.
For descriptive or implementation research purposes, the study will field online self-administered surveys to caseworker and case supervisor staff. Staff can refuse to complete the survey, or refuse to answer any of the questions on the survey, and will not be penalized in any way. In addition, the study will field text message surveys to study participants (clients). We view the text message surveys as a way to collect data similar to what we might collect from focus groups—we can collect information about experiences and perceptions in brief encounters at multiple time points, for lower overall burden than an in-person visit. The instruments are provided in Appendices A-D.
To collect qualitative data, study teams composed of at least two members will make the telephone and in-person meeting contacts. These staff members are experienced in the process of implementation research. Each interview will begin with an introduction that explains the purpose and goals of the BIAS-NG project. The facilitator will inform participants that the conversation will be audio-recorded but that we will stop the recording at any time they do not feel comfortable being recorded. For individual phone interviews, the facilitator will simply begin the conversation with the participant after receiving verbal consent. For in-person interviews, the facilitator will begin after receiving the signed consent form. At the end of the interview, participating clients will receive a gift card worth $20.
For outcomes data for the impact analysis, we should have very little missing data from the administrative records because the county is required to track cases.
For qualitative implementation research data collection, the research team will work closely with administrators and staff to recruit families and staff for us to interview. We will aim to gather informants from each of the 6 regional offices, and request informants that reflect a mix of case experiences. To further increase the likelihood of participation, we will offer clients participating in interviews gift cards as incentives, as discussed in Supporting Statement Part A.
One aspect of our quantitative implementation research data collection from clients is text message surveys. Response rates for text message surveys are typically 40% or lower. We do not expect this population’s response rate to be different. As a result, we do not view the text message surveys as large-scale data collection but rather as opportunistically gathered data to help us understand how the intervention was delivered. In the Building Bridges and Bonds study funded by HHS, response rates via text message have ranged from 20-30 percent. Follow-up contact with non-responders does not materially increase response rates via text message. We will not send reminders for the client survey.
Staff working with the program and control groups will be asked to complete an online survey several months after the launch of the intervention. Based on the response rates for the staff surveys in the Enhanced Transitional Jobs Demonstration, and responsiveness from caseworkers to existing county processes, we expect around 85 percent of staff to open the survey but only 60 percent to complete the survey. For the staff survey, we will use common follow-up methods, such as sending reminders, setting completion deadlines, attempting to reach participants by phone after several failed attempts to obtain a response, and working closely with staff to maintain an accurate list of contact information (see Appendix C).
B4. Tests of Procedures or Methods to be Undertaken
Formative focus groups and interviews with fewer than 10 people informed the development of interview protocols for data collection and the development of survey instruments. The data collection instruments will not be pre-tested at scale.
B5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
Jean Grossman, MDRC, Project Director
Rekha Balu, MDRC, Senior Research Associate
Kim Clum
Senior Social Science Research Analyst
Office of Planning, Research, and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
330 C St. SW, 4th Floor
Washington, DC 20201
(202) 205-4922
Victoria Berk Kabak
Social Science Research Analyst
Office of Planning, Research, and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
330 C St. SW, 4th Floor
Washington, DC 20201
(202) 401-5871
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Clum, Kimberly (ACF) |
| File Modified | 0000-00-00 |
| File Created | 2021-01-20 |