Alternative Supporting Statement for Information Collections Designed for
Research, Public Health Surveillance, and Program Evaluation Purposes
Strengthening the Implementation of Responsible Fatherhood Programs (SIRF)
Formative Data Collections for ACF Research
0970 - 0356
Supporting Statement
Part B
February 2020
Submitted By:
Office of Planning, Research, and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
4th Floor, Mary E. Switzer Building
330 C Street, SW
Washington, D.C. 20201
Project Officers:
Katie Pahigiannis
Kriti Jain
Part B
B1. Objectives
Study Objectives
The Strengthening the Implementation of Responsible Fatherhood Programs (SIRF) Study will use iterative learning methods to identify and test promising practices for addressing critical implementation challenges in Responsible Fatherhood programs. To achieve this goal, the SIRF team will identify common implementation challenges and potential solutions, select programs to undertake iterative learning activities, work with sites on iterative learning activities, and strengthen implementation, and increase capacity for participating in summative evaluations. The results from this project are intended to inform future large-scale impact evaluations of programs that adopt them.
The current information data collection request is to hold discussions with relevant stakeholders and programs to gather information about their current practices and potential ways to improve fatherhood program implementation. These activities will inform the development of iterative learning methods and tests, which will be documented in a later information request package.
Generalizability of Results
This study is intended to present an internally valid description of Responsible Fatherhood programs, the implementation challenges they face, and innovative practices to address those challenges in chosen sites, not to promote statistical generalization to other sites or service populations.
Appropriateness of Study Design and Methods for Planned Uses
The activities outlined in this information request are optimal for minimizing respondent burden as SIRF answers the following questions in advance of designing the study:
What are the key implementation challenges facing Responsible Fatherhood programs?
What are successful or innovative efforts to solve implementation challenges that Responsible Fatherhood programs and participants face?
The information to answer these questions will be collected through semi-structured phone discussions with stakeholders and program staff, and semi-structured follow up - in-person or virtual - discussions with select program leaders, staff, and program participants. Because responses can vary, semi-structured discussions allow for flexibility during the discussion, such as restricting the discussion length to the minimum required or holding discussions in groups whenever possible. In addition, this mode prevents programs from having to write lengthy responses and the need for follow-up to clarify the written responses. Each protocol is designed to collect the minimum information necessary to allow the study team to understand the variation of programming in the field, the range of perspectives on SIRF and assess what particular study design options will be feasible given the structure of a range of fatherhood programs.
As noted in Supporting Statement A, this information is not intended to be used as the principal basis for public policy decisions and is not expected to meet the threshold of influential or highly influential scientific information.
B2. Methods and Design
Target Population
The target population for this data collection are staff at up to 25 programs serving fathers (up to 27 by phone only; up to 48 by phone and in-person; up to 112 in-person only), program participants (160 in person), and fatherhood program stakeholders (40 virtually).
Stakeholders are relevant researchers, technical assistance providers (national, state, and local), curriculum developers, and program funders. After these phone calls, the study team will conduct follow-up discussions with approximately 16 local programs serving fathers through in-person visits or virtual conference calls . During follow-up with each program, the study team expects to talk with up to 10 program staff and up to 10 program participants.
Sampling
To ensure that SIRF findings reflect the full breadth of fatherhood programs, individuals will be purposefully invited to SIRF discussions to vary perspectives by affiliation, role, geography, demographics, service approaches, etc. For example, stakeholders will be selected for their knowledge of fathers, their service needs, and the services provided to them. Likewise, the selection of programs to visit in-person will be based on the target population they serve (i.e. work with fathers or a specific subgroups of fathers), program size, services offered, program structure, geography, past success engaging fathers in services, along with recommendations from the project’s experts, other stakeholders, and consultation with the federal offices overseeing SIRF and Responsible Fatherhood programming.
B3. Design of Data Collection Instruments
Development of Data Collection Instrument(s)
Each data collection instrument was designed to streamline efforts by only asking the necessary questions to achieve the objectives of the data collection and to allow flexibility in our approach given that the availability of individuals may vary. The data collection instruments will not be pre-tested as previous large-scale evaluations have successfully used similar discussion topics, such as the Building Bridges and Bonds Evaluation (OMB Control No.: 0970-0485), during information gathering processes.
B4. Collection of Data and Quality Control
Contractors to ACF, led by MDRC and including researchers from MEF Associates and Insight Policy Research, will gather information. In consultation with OPRE, the study team will identify up to 40 stakeholders and 75 program staff to conduct phone discussions with, and up to 16 programs to conduct discussions with program staff and participants in-person or through virtual conference calls.
The study team will send fatherhood program staff a request for a one-hour telephone or video conference call via email (see Appendix A: SIRF Phone Meeting Email Template). It will describe the study and its goals, introduce the team that is doing the study on ACF’s behalf, offer suggested times for a phone meeting, and will state that the phone meeting is voluntary. Attached to the email will be the project description (see Appendix B: SIRF Project Description) and a summary of the topics we hope to cover during the phone call (see Appendix C: SIRF Phone Meeting Topics for Program Staff and Appendix D: SIRF Phone Meeting Topics for Stakeholders). Each semi-structured discussion will be guided by a protocol for quality assurance and consistency of the data collected.
With a select group of approximately 16 programs, the study team will conduct follow-up in-person visits or virtual conference calls with organizations serving fathers for further discussion with program staff and fathers. If the follow-up is in person, the study team will use an agenda to guide the site visits (see Appendix E: SIRF Site Visit Agenda). The follow-up discussions will be guided by a semi-structured protocol (see Instrument 3). If the follow-up is in person, the study team will also observe program activities to gain insight into how programs operate in practice and the strengths and potential areas of improvement for the program. Observations will not require any involvement from staff and do not impose burden. The study team will also speak to fathers, either one-on-one or in small groups when possible. These discussions will be guided by a semi-structured protocol (see Instrument 4).
After identifying program challenges from staff and participant perspectives, the study team will engage staff and participants to participate in a group brainstorming session about how to address the challenges. All staff and participants we held discussions with will be invited to join. This format allows each person to build on others’ thoughts and leverage multiple perspectives to generate the most innovate ideas. This discussion will be guided by a protocol (see Instrument 5, described in Table 1 below) that will allow the study team to pose a series of “how might we…” questions about the challenges facing the program and participants based on what we learn from observations and discussions. Brainstorming participants will be encouraged to generate as many ideas as they can in response to each question, and then, through discussion, identify which ideas are most likely to solve the problem in an innovative way. Through this process, the study team will potentially gather dozens of ideas from each site that participates in this exercise and will learn how much convergence there is for support for similar ideas across programs.
All discussions will be completed or overseen by senior researchers with extensive experience conducting discussions. All team members will receive background about Responsible Fatherhood programming and a training on the semi-structured protocols to ensure that programs are engaged in a consistent manner. After a call or a visit, if there is missing information, the study team does not intend to follow-up with stakeholders or staff, unless the information is essential.
B5. Response Rates and Potential Nonresponse Bias
Site and Respondent Selection
The data collection activities in this request are not designed to produce statistically generalizable findings and participation is wholly at the respondent’s discretion. Response rates will not be calculated or reported.
Because there is minimal burden involved with the initial phone call, we expect nearly all individuals contacted will participate. The purpose of this study is to learn about program services and operations in the field, and we expect that most program directors and fathers will be eager to share this information with the study team. The programs selected for follow-up discussions, either in-person or virtually, will be based in part on their willingness to have an additional discussion and their availability. For this reason, we expect that most programs selected will participate in the in-person visits.
To maximize participation, the study team will be accommodating of respondents’ schedules, including flexibility about the meeting times and date. The program will be contacted by their assigned study team liaisons, which will include at a minimum one senior and one junior staff member from the study team. The senior member has had significant experience in working closely with programs and their stakeholders on previous evaluations and has had a high level of success in engaging programs in preliminary discussions such as these.
Non-Response
As participants will not be randomly sampled and findings are not intended to be representative, non-response bias will not be calculated.
When members of the project team have conducted tasks like this for other similar projects, participation has been high. For example, the stakeholders, many of whom have long careers researching, advocating for, or working in fatherhood programs, have been eager to contribute their expertise to inform projects; we expect this to be the case with SIRF.
Similarly, we expect that most program staff and participants whom we contact will agree to participate. A small number may decline. When members of the project team have gathered information about fatherhood programs previously for other research projects, nonparticipation was mostly due to scheduling conflicts.
If we encounter no response to the initial email request for a meeting, we may send a second request by e-mail or follow up with a phone call, during which a member of the research team will use the e-mail template as a script. After these attempts, we will not continue to pursue a response. Should anyone decline our invitation, we will ask if there are any other suitable staff at their organization who might be able to participate instead.
B6. Production of Estimates and Projections
Estimates will not be produced for this data collection.
B7. Data Handling and Analysis
Data Handling
Key points of each discussions with stakeholders, program staff, and participants will be documented in Microsoft Word by a note-taker who will be present during each discussion. The information gathered will be kept private and protected securely. The study team is required to use encrypted laptops, secure storage locations, secure transfer mechanisms (when necessary), and access to secure locations will only be granted to those on a need-to-know basis.
We are not producing a data set for research purposes. The information gathered will inform priorities for the iterative learning tests, which will be explained in a future information data collection request submission.
Data Analysis
Notes from the semi-structured discussion protocols will be organized into categories for review and summarized in an internal memo.
Data Use
The information gathered will inform priorities for the iterative learning tests, which will be explained in a future information data collection request submission. Findings from the current information collection request will not be used for the primary purpose of publication, but may be incorporated into documents or presentations that are made public, such as research design documents or reports; research or technical assistance plans; background materials for technical workgroups; or contextualization of research findings from a follow-up data collection that has full PRA approval. In sharing findings, we will describe the study methods and limitations with regard to generalizability and as a basis for policy.
B8. Contact Person(s)
Primary Project Contacts
Charles Michalopoulos, MDRC
SIRF Project Director
(510) 844 – 2235
Dina Israel, MDRC
SIRF Project Deputy Director
(212) 340 – 8606
Michelle Manno, MDRC
SIRF Information Gathering Task Lead
(212) 340 – 8873
Data Collection and Analysis Staffing
Research activities with program staff, participants, and stakeholders will be conducted by trained researchers at MDRC, MEF Associates, and Insight Policy Research. Study teams will be composed of at least one senior member and one junior member for every telephone and in-person contact. These staff members are experienced in the process of gathering information for purposes of designing research projects, such as SIRF. In addition, all team members will receive a training to ensure that programs are engaged in a consistent manner. A smaller number of team members will engage in analysis; junior staff will be supervised by senior staff with previous experience in analyzing qualitative information.
Attachments
Appendix A_SIRF Phone Meeting Email Template
Appendix B_SIRF Project Description
Appendix C_SIRF Phone Meeting Topics for Program Staff
Appendix D_SIRF Phone Meeting Topics for Stakeholders
Appendix E_SIRF Site Visit Agenda
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Pahigiannis, Katie (ACF) |
| File Modified | 0000-00-00 |
| File Created | 2021-01-13 |