Alternative Supporting Statement Instructions for Information Collections Designed for
Research, Public Health Surveillance, and Program Evaluation Purposes
Fatherhood and Marriage Local Evaluation and Cross-Site Services Components
Formative Data Collections for ACF Research
0970 - 0356
Supporting Statement
Part B
February 2020
Submitted By:
Office of Planning, Research, and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
4th Floor, Mary E. Switzer Building
330 C Street, SW
Washington, D.C. 20201
Project Officers:
Katie Pahigiannis
Senior Social Science Research Analyst
Pooja Gupta Curtin
Social Science Research Analyst
Office of Planning, Research, and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
Part B
B1. Objectives
Study Objectives
An important objective of the ACF Office of Planning, Research, and Evaluation’s (OPRE) Fatherhood and Marriage Local Evaluation (FaMLE) Cross-Site project, contracted to Mathematica, is to strengthen the capacity of grantees, working with their own local evaluators. OPRE supports grantees in conducting evaluations that add to the body of evidence on HMRF program effectiveness, operations, and outcomes. The final report templates (the information collection instruments submitted through this request) will help the federal government and FaMLE Cross-Site Mathematica staff provide research and evaluation support to grantees. Specifically, as part FaMLE Cross-Site project, we are submitting this information collection request for the following:
To ask HMRF grantees conducting local impact evaluations to complete final report templates that discuss their analyses and main findings. The information request includes:
Impact Evaluation Final Report Template (Instrument 1)
Impact Evaluation Final Report Table Shells (Instrument 2)
2. To ask HMRF grantees conducting local descriptive evaluations to complete a final report template that discusses their analyses and main findings. The information request includes:
Descriptive Evaluation Final Report Template (Instrument 3)
Descriptive Evaluation Final Report Table Shells (Instrument 4)
The templates have accompanying instructions to guide the grantees in completing them. We have provided these instructions as Appendices A and B.
Generalizability of Results
The set of descriptive evaluation final reports will describe the service populations, their outcomes, and the implementation of the interventions to add to the evidence base on ways to engage and serve populations involved with HMRF programs. These reports are intended to present internally-valid descriptions of the service population and/or the implementation of an intervention in chosen sites, not to promote statistical generalization to other sites or service populations
The set of impact evaluation final reports will produce estimates of impacts of HMRF programs on a range of outcomes, such as attitudes about relationships and knowledge of conflict resolution strategies, parenting behavior, and employment. They will also provide information about the implementation of the programs. These reports are intended to present internally-valid estimates of the intervention’s causal impact, not to promote statistical generalization to other sites or service populations.
Appropriateness of Study Design and Methods for Planned Uses
The templates for the final reports will provide a structure for presenting information about the methods and findings of the local evaluations. This structure will help ACF and the HMRF field to assess whether the impact evaluations were well-executed and the data were analyzed appropriately so they can be confident about the reported findings. It will also help ACF and the HMRF field to readily identify the key findings from the impact and descriptive evaluations, to better understand lessons and insights for the field about the outcomes, implementation, and impacts of healthy marriage and responsible fatherhood programs implemented through the HMRF grant program since 2015.
As noted in Supporting Statement A, this information is not intended to be used as the principal basis for public policy decisions and is not expected to meet the threshold of influential or highly influential scientific information.
B2. Methods and Design
Target Population
The target population for this information collection is 33 HMRF grantees with local evaluations: 16 descriptive evaluations and 18 impact evaluations (one grantee is conducting both an impact and a descriptive local evaluation). These 33 grantees will be asked to respond to the information collection.
Sampling
All grantees will be asked to respond because they are each conducting unique evaluations of their specific programs and will require tailored technical assistance based on their evaluation findings.
B3. Design of Data Collection Instruments
Development of Data Collection Instruments
The final report templates, the data collection instruments submitted under this request, were informed by HMRF grantees’ accepted analysis plans and reflect best practices in evaluation reporting.
B4. Collection of Data and Quality Control
ACF will send the final report template and instructions to grantees upon OMB approval. ACF and the contractor’s evaluation team technical assistance liaison are in regular communication with grantees and will monitor completion of the final report template. The liaisons will work closely with grantees and their local evaluators to address any challenges they may face in completing their final reports. Both ACF and the contractor’s evaluation team technical assistance liaison will review each final report and provide feedback. Grantees will submit completed final report templates to ACF via email.
B5. Response Rates and Potential Nonresponse Bias
Response Rates
ACF expects all HMRF grantees to produce a final report.
NonResponse
As participants will not be randomly sampled and findings are not intended to be representative, non-response bias will not be calculated.
B6. Production of Estimates and Projections
The data will not be used to generate population estimates, either for internal use or dissemination. ACF may make final reports available on its website.
B7. Data Handling and Analysis
Data Handling
Grantees will complete the templates using the instructions as guidance, and then ACF and FaMLE Cross-Site Mathematica staff will review them and provide detailed feedback designed to mitigate or correct detectable errors.
Data Analysis
As part of the cross-site FaMLE Cross-Site project, ACF may use impact estimates presented in the final impact reports in a meta analysis of impacts of HMRF programs, as approved in a prior OMB package (OMB control number 0970-0460).
Data Use
ACF may make final reports available on its website. In the data collection guidance, grantees are asked to summarize and present content in a table or figure when possible for clear interpretation. While reviewing final reports, the FaMLE Cross-Site technical assistance liaisons will also provide feedback to grantees about whether data analyses were conducted correctly, findings were interpreted correctly, and the appropriate conclusions are drawn based on the analysis presented in the report, and suggestions for how to correct any inaccurately conveyed information.
B8. Contact Person(s)
ACF staff and its contractor, Mathematica Policy Research, will work with grantees to collect and review the data. No consultation was necessary for either sampling or data collection, as no statistical sampling procedures will be employed.
Table 1. Individuals who will collect, process, and analyze the information collected
Name |
Affiliation |
Megan Reid |
Office of Planning, Research, and Evaluation, Administration for Children and Families |
Seth Chamberlain |
Office of Family Assistance, Administration for Children and Families |
Millicent Crawford |
Office of Family Assistance, Administration for Children and Families |
Phyllis Carroll |
Office of Family Assistance, Administration for Children and Families |
Tanya Howell |
Office of Family Assistance, Administration for Children and Families |
Meghan Heffron |
Office of Family Assistance, Administration for Children and Families |
Toya Joyner |
Office of Family Assistance, Administration for Children and Families |
Barbara Spoor |
Office of Family Assistance, Administration for Children and Families |
John Allen |
Office of Family Assistance, Administration for Children and Families |
Jacqueline Proctor
|
Office of Family Assistance, Administration for Children and Families |
Christine Ross |
Mathematica |
Elizabeth Brown |
Mathematica |
Julieta Lugo-Gil |
Mathematica |
Sarah Avellar |
Mathematica |
Adam Dunn |
Mathematica |
Angela D’Angelo |
Mathematica |
Derekh Cornwell |
Mathematica |
Jane Choi |
Mathematica |
Armando Yanez |
Mathematica |
Miranda Kharsa |
Mathematica |
Attachments
Instrument 1: Final Impact Evaluation Report Text Template for Healthy Marriage and Responsible Fatherhood Grantees
Instrument 2: Final Impact Evaluation Report Tables Template for Healthy Marriage and Responsible Fatherhood Grantees
Instrument 3: Final Descriptive Evaluation Report Text Template for Healthy Marriage and Responsible Fatherhood Grantees
Instrument 4: Final Descriptive Evaluation Report Tables Template for Healthy Marriage and Responsible Fatherhood Grantees
Appendix A: Instructions for the Final Impact Evaluation Report for Healthy Marriage and Responsible Fatherhood Grantees
Appendix B: Instructions for the Final Descriptive Evaluation Report for Healthy Marriage and Responsible Fatherhood Grantees
File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
Author | Sarah Avellar |
File Modified | 0000-00-00 |
File Created | 2023-10-26 |