Supporting Statement A_HMRF Local Eval Final Rpts_21Nov2024_Clean - final

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

Healthy Marriage and Responsible Fatherhood Local Evaluation Final Report Templates

OMB Information Collection Request

New Collection

Supporting Statement

Part A

December 2024

Type of Request: New

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers:

Rebecca Hjelm

Kathleen McCoy

Harmanpreet Bhatti

Part A

Executive Summary

• Type of Request: This Information Collection Request is for a new data collection. We are requesting one year of approval.

• Description of Request: The Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services (HHS) seeks approval of the Healthy Marriage and Responsible Fatherhood (HMRF) Final Report Templates. HMRF grant recipients carrying out local evaluations of their programs are required to submit a final evaluation report to ACF at the end of their grant. This request includes templates for grant recipients to use to document their evaluation’s analysis and findings. In addition, the information collected in the final report templates will inform technical assistance provided to grant recipients as they develop the final reports to ACF to fulfill the grant requirement. We do not intend for this information to be used as the principal basis for public policy decisions.

• Time Sensitivity: ACF would like to distribute the final report template and instructions to HMRF grant recipients as soon as possible to allow them sufficient time to complete and submit their final reports to ACF before the end of the grant funding period in September 2025.

A1. Necessity for Collection

Per the 2020 Healthy Marriage and Responsible Fatherhood (HMRF) Notice of Funding Opportunities (NOFOs) issued by Administration for Children and Families (ACF), HMRF grant recipients are required to submit a final report to ACF at the end of their evaluations. ^1,2,3 To increase the consistency of the final reports to ACF, HMRF grant recipients will use structured final evaluation report templates to provide critical information on the rigor and appropriateness of their research approaches and present the findings clearly. The information collected will also be used to inform the provision of technical assistance (TA) on the final reports grant recipients will submit to ACF to meet the grant requirement.

Background

Since 2005, Congress has authorized dedicated funding for discretionary awards from ACF’s Office of Family Assistance (OFA) to support HMRF programs designed to develop the skills people need to form and sustain stable, high-quality relationships. For more than 10 years, ACF’s Office of Planning, Research, and Evaluation (OPRE) has partnered with OFA and led a sustained effort within the federal government to develop, document, and evaluate HMRF programs, particularly those serving populations with low incomes. OPRE and OFA have a long and proven commitment to supporting research on healthy relationships and family stability through federally funded HMRF programs. Through several prior and ongoing evaluations, ACF has sought to assess and improve HMRF programs.

OPRE is conducting the Healthy Marriage and Relationship Education (HMRE) Local Evaluation Technical Assistance (LETA) and the Responsible Fatherhood (RF) LETA projects, hereafter referred to jointly as the HMRF-LETA project.⁴ The HMRF-LETA project teams provide evaluation TA to support federally funded grant recipients that are evaluating their own HMRF programs providing services to couples, adult individuals, fathers, and youth. The final report templates provide guidance to grant recipients to develop the required final reports and the information provided will inform the provision of TA by the HMRF-LETA teams to grant recipients on developing and submitting the final reports to ACF.

OPRE has contracted with Mathematica and MDRC to complete this work.

Legal or administrative requirements that necessitate the collection

There are no requirements that necessitate the collection. ACF is undertaking the collection at the discretion of the agency.

A2. Purpose

The purpose of the current information collection request (ICR) is to provide standardized templates (for descriptive or impact evaluations) and accompanying instructions to grant recipients for them to tailor to their evaluation and document their research questions, measures, study design, planned and actual implementation of the program, analytic methods for their evaluation, and evaluation findings. A structured final report template will facilitate grant recipients’ efficient and effective reporting of evaluation findings in their final reports. The completed reports will be reviewed by the HMRF-LETA team for the purpose of identifying strengths and weaknesses of the analytic approach, determining whether the reports meet ACF’s standards of rigor, developing recommendations for improvement, and informing TA on the development of the report before final submission to ACF. The templates will be used by grant recipients upon OMB approval until September 2025, which will allow them to submit draft versions of their final reports before revising and finalizing their reports by September 2025. Specifically, this ICR includes the following templates:

1. Descriptive Evaluation Final Report Template (Instrument 1): HMRF grant recipients conducting local descriptive evaluations will complete a final report template (which includes table shells to display quantitative results) that discusses their analyses and main findings.

2. Impact Evaluation Final Report Template (Instrument 2) and Table Shells (Instrument 3): HMRF grant recipients conducting local impact evaluations will complete a final report template (accompanied by separate table shells⁵ to display quantitative results) that discusses and documents their analyses and main findings.

Grant recipients will complete the final report template (and table shells, for impact studies) using accompanying tailored instructions which we have provided (Appendices A and B).

The information collected is meant to contribute to the body of knowledge on ACF programs. It is not intended to be used as the principal basis for a decision by a federal decision-maker and is not expected to meet the threshold of influential or highly influential scientific information.

Information Collection Processes

Upon OMB approval, the HMRF-LETA team will send an email to each HMRF grant recipient conducting a local evaluation. There are a total of 69 grant recipient organizations conducting 79 local evaluations (40 HMRE studies and 39 RF studies). Ten grant recipient organizations were awarded two grants which ACF considers to be separate HMRF programs and local evaluations; each evaluation must meet the final report requirement. The emails (Appendices C and D) will introduce the templates, the instructions, and the timeline for submission. The emails will be tailored, such that grant recipients will only receive the final report templates for the type of study they are conducting (e.g., descriptive study grant recipients will only receive descriptive final report templates). Ten grant recipient organizations will receive two emails tailored to share the applicable materials for the local evaluation they are conducting for each grant award. Grant recipients will submit their draft and final reports to their OFA Family Assistance Program Specialist (FPS) and HMRF-LETA TA team. The HMRF-LETA team will review completed draft reports from all grant recipients and provide TA to grant recipients so they can further improve the clarity and quality of the report to meet the ACF grant requirements for the final submission. Ultimately, OFA will approve the final reports and notify the grant recipients that the report meets the ACF grant requirements.

The HMRF-LETA team will primarily provide support by offering one-on-one TA calls with grant recipients while they develop the reports. The cadence of these calls (such as monthly, bimonthly, or quarterly) will be mutually decided by OPRE, the TA contractors, and grant recipients. The evaluation TA liaisons for each grant recipient will invite the grant recipient’s project director and key staff, the local evaluator, and the FPS from OFA to join these calls. The TA contractors will determine whether to provide additional TA, such as training webinars or other resources to support reporting, in consultation with OPRE.

Table A2.1 shows the data collection activities, instruments, respondents, content, purpose of collection, as well as the mode and expected duration to complete the instruments.

As outlined in the NOFO, the final reports must be developed and refined as a condition of grant recipients’ funding, thus we expect a response rate of 100 percent among those responding to the data collection activities outlined in this package.

Table A2.1. Data Collection Activities

Instrument and instructions	Respondent, Content, Purpose of Collection	Mode and Duration
Descriptive Evaluation Final Report Template (Instrument 1)	Respondents: Grant recipients conducting local evaluations with a descriptive design. Content: A descriptive final report describes the program design; the research questions; and the analytic approach for the outcome study, the implementation study, or both (depending on the studies the grant recipient is conducting). In addition, for the outcome study, the report includes a description of the evaluation enrollment process, data collection procedures, and outcome measures; for the implementation study, the report also discusses the research questions and the data used to answer the research questions. The final report concludes with the evaluation’s findings and implications. Purpose: To document grant recipients’ final descriptive evaluation findings and inform subsequent TA before submission of the report to ACF.	Mode: Shared and collected via email Duration: 40 hours
Impact Evaluation Final Report Template (Instrument 2) Impact Evaluation Final Report Table Shells (Instrument 3)	Respondents: Grant recipients conducting local evaluations with an impact design. Content: The impact final report describes the research questions, the selected outcome measures, the program design and counterfactual conditions, the evaluation study design, the analytic approaches to gauge the effect or impact of the intervention, and the findings and implications for both the impact and implementation studies. Purpose: To document grant recipients’ final impact evaluation findings and inform subsequent TA before submission of the report to ACF.	Mode: Shared and collected via email Duration: 40 hours

Other Data Sources and Uses of Information

The information collection described above is the only source of information about the findings resulting from the local evaluations conducted by each grant recipient.

A3. Use of Information Technology to Reduce Burden

The burden on HMRF grant recipients is reduced where possible. Grant recipients can complete the reports on their own computers, at a time of their choosing, and submit the final reports electronically, via email to ACF. ACF does not require grant recipients to submit or follow-up on the report in-person and they do not require hardcopies of the report.

A4. Use of Existing Data: Efforts to reduce duplication, minimize burden, and increase utility and government efficiency

The purpose of the final report templates is to provide grant recipients guidance on structuring a comprehensive and accessible final report. Grant recipients will be encouraged to build on their local evaluation plans^{6, 7} and their analysis plans⁸ to complete the final report template. Grant recipients can draw many of the report sections, such as the research questions, description of the intervention, and study design, directly from their previously accepted evaluation and analysis plans. This will reduce the burden in writing the report and support consistency between the analysis plans and final reports. Consistency between the final report and prior approved plans promotes credibility and transparency by demonstrating grant recipients’ commitment to a prespecified and systematic approach to the focus and execution of the study and data analysis methods. No unnecessary information is being requested of grant recipients. None of the report templates will ask for information that can be reliably obtained through other sources by the study team.

A5. Impact on Small Businesses

Data collection activities could affect small organizations that are under subcontract to the grant recipient since HMRF grant recipients contract out the execution of evaluations to local evaluators who may be affiliated with small businesses. Grant recipients may task the local evaluator with writing portions of the final report. The proposed templates are designed to minimize the burden on all organizations involved, including small businesses and entities, by collecting only critical information using the standardized templates. Grant recipients can complete the template on their own time depending on what is most convenient to them. Additionally, the TA provided to grant recipients to help with completing the template will also minimize the burden on all organizations involved.

A6. Consequences of Less Frequent Collection

This is a one-time data collection.

A7. Now subsumed under 2(b) above and 10 (below)

A8. Consultation

Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on October 4, 2024 (89 FR 80904) and provided a sixty-day period for public comment. No substantive comments were submitted during the comment period.

Consultation with Experts Outside of the Study

ACF consulted the evaluation TA contractors, Mathematica and MDRC, to prepare the final report templates.

A9. Tokens of Appreciation

No tokens of appreciation for respondents are proposed for this information collection.

A10. Privacy: Procedures to protect privacy of information, while maximizing data sharing

Personally Identifiable Information

No Personally Identifiable Information (PII) will be collected.

Information will not be maintained in a paper or electronic system from which data are actually or directly retrieved by an individuals’ personal identifier.

Data Security and Monitoring

Grant recipients will send completed final reports to ACF and the HMRF-LETA team by email. The completed final reports templates will be stored on the Contractors’ computer system in a file folder that can only be accessed by project staff. Project staff will record in a tracking file the dates they receive the completed (and revised) reports and dates when the reports are returned to grant recipients for revision.

The Contractors shall protect respondent privacy to the extent permitted by law and will comply with all Federal and Departmental regulations for private information.

The security procedures for each contractor are described next.

Mathematica

1. Data stored on the secure network drive reside within a project-specific folder that is only accessible to Mathematica staff who have a business-related need to know, as restricted by identity-based policies and access control lists. The Mathematica project director, or designee, approves access to the data stored on the network drive.

2. Data is encrypted as it is stored on the server with an AES 256-bit key, which is Federal Information Processing Standards (FIPS) 140-2 compliant.

3. All ingress and egress points on the Mathematica network are limited to authorized channels via a monitored firewall. There is no route for public access into the Mathematica network.

These are supplemented by: 1) an annual security awareness training, employee review and signature of the company’s Rules of Behavior and confidentiality pledge; and 2) resources on Mathematica’s intranet web page, such as links to the Mathematica Security Manual which outlines HHS-specified procedures for identifying, reporting, and resolving security incidents.

MDRC

1. Access to information on a need-to-know basis, supported by multi-factor authentication factors

2. End-to-end encryption, in-transit and at-rest, using TLS 1.2+ and AES256 via FIPS 140-2 modules for systems integrity, systems and communications protection, and media protection

3. Continuous monitoring of application and transport-level traffic for inbound and outbound flows

These are supplemented by 1) employee nondisclosure agreements and annual data security training, 2) IT support teams well-versed in cyber security, and 3) policies for responding to data security incidents.

A11. Sensitive Information ⁹

There are no sensitive questions in this data collection.

A12. Burden

Explanation of Burden Estimates

Table A12.1 provides the estimated annual reporting burden and cost calculations for the three instruments included in this request. As described in section A2, 10 grant recipient organizations were funded to conduct two HMRF evaluations, thus they are counted as separate respondents in the estimates below because the organizations must meet the final report requirement for each grant.

Explanation of Burden Estimates

The total annual burden is estimated based on the following assumptions:

• Instrument 1: Descriptive Evaluation Final Report Template for HMRF Grant Recipients. At most, 50 grants funded to conduct a descriptive evaluation (24 HMRE and 26 RF) will complete the descriptive report template and the embedded table shells. On average, it will take 40 hours to complete this template, which includes edits to the report identified during the review process and final submission of the report.

• Instrument 2: Impact Evaluation Final Report Template for HMRF Grant Recipients. At most, 29 grants funded to conduct an impact evaluation (16 HMRE and 13 RF) will complete the impact report template. On average, it will take 30 hours to complete this template, which includes edits to the report identified during the review process and final submission of the report.

• Instrument 3: Impact Evaluation Final Report Table Shells for HMRF Grant Recipients. Grant recipients completing the impact report template will complete the accompanying table shells. On average, it will take 10 hours to complete these table shells, which includes edits to the tables identified during the review process and submission of the tables with the final report.

Estimated Annualized Cost to Respondents

For cost calculations for the labor associated with completing the final reports, we estimate the average hourly wage for program directors and managers to be the average hourly wage for “Social and Community Services Manager” ($37.03), taken from the U.S. Bureau of Labor Statistics, Occupational Employment Statistics, 2023.¹⁰ To account for fringe benefits and overhead, we multiplied this amount by two ($74.06).

Table A12.1. Total Burden Requested Under this Information Collection

Instrument	No. of Respondents (total over request period)	No. of Responses per Respondent (total over request period)	Avg. Burden per Response (in hours)	Total/ Annual Burden (in hours)	Average Hourly Wage Rate	Total Annual Respondent Cost
Instrument 1: Descriptive Evaluation Final Report Template	50	1	40	2,000	$74.06	$148,120.00
Instrument 2: Impact Evaluation Final Report Template	29	1	30	870	$74.06	$64,432.20
Instrument 3: Impact Evaluation Final Report Table Shells	29	1	10	290	$74.06	$21,477.40
Total				3160		$234,029.60

A13. Costs

There are no additional costs to respondents.

A14. Estimated Annualized Costs to the Federal Government

Table A14.1 presents the annualized costs to the federal government estimated based on the contractors’ staff costs for review of the final reports and TA support to grant recipients to develop and submit their final reports. The total estimated cost to the federal government for the data collection activities under this ICR will be $353,148.72.

Table A14.1. Annualized Costs to Federal Government

Cost Category	Estimated Costs
Review of and TA support for final reports	$353,148.72
Total costs over the request period	$353,148.72

A15. Reasons for changes in burden

This is a new information collection request.

A16. Timeline

Table A16.1 presents the timeline for the data collection activities. Upon OMB approval, the dissemination of materials and the collection of final reports will begin. The goal is to begin use of the templates in February 2025. Templates will continue to be used through September 2025, as we anticipate that final reports will be completed by the end of the grant recipients’ funding period.

Table A16.1. Timeline for Activities Under the Information Collection

Activity	Time Period*
Evaluation TA contractors distribute final report templates and instructions	As soon as OMB approval is received (approximately February 2025)
Grant recipients complete and submit final report drafts	February – September 2025
Evaluation TA contractors review final report drafts	March – September 2025
Grant recipient revises and/or clarifies information in the final report	March – September 2025
ACF approves grant recipient final reports	April– September 2025

*Start dates for use of templates is dependent on OMB approval of this information collection request.

A17. Exceptions

No exceptions are necessary for this information collection.

Attachments

Instrument 1: Descriptive Evaluation Final Report Template for HMRF Grant Recipients

Instrument 2: Impact Evaluation Final Report Template for HMRF Grant Recipients

Instrument 3: Impact Evaluation Final Report Table Shells for HMRF Grant Recipients

Appendix A: Instructions for the Descriptive Evaluation Final Report Template for HMRF Grant Recipients

Appendix B: Instructions for the Impact Evaluation Final Report Template for HMRF Grant Recipients

Appendix C: Descriptive Evaluation Final Report Dissemination Email for HMRF Grant Recipients

Appendix D: Impact Evaluation Final Report Dissemination Email for HMRF Grant Recipients

1The NOFO for Family, Relationship, and Marriage Education Works - Adults (FRAMEWorks) can be accessed at: https://www.acf.hhs.gov/sites/default/files/documents/ofa/HMRF_FRAMEWorks_NOFO.pdf

2 The NOFO for Relationships, Education, Advancement, and Development for Youth for Life (READY4Life) can be accessed at: https://www.acf.hhs.gov/sites/default/files/documents/ofa/HMRF_READY4Life_NOFO.pdf

3 The NOFO for Fatherhood—Family—focused, Interconnected, Resilient, and Essential

(Fatherhood FIRE) can be accessed at: https://www.acf.hhs.gov/sites/default/files/documents/ofa/HMRF_FIRE_NOFO.pdf

4 OPRE awarded two contracts to conduct this work – one to Mathematica to work with HMRE grant recipients and the other to MDRC to work with RF grant recipients.

5 For the impact studies, the table shells are in a separate document for ease of use by grant recipients because the table shell document is long and if integrated into the final template (as is the case with the descriptive studies) would make it difficult to navigate.

6 HMRE evaluation plans were collected under a generic clearance request for formative data collections for research and evaluation in December 2020 (Strengthening the Implementation of Marriage and Relationship Programs [SIMR]; OMB #0970-0356).

7 HMRF evaluation plans were collected under a generic clearance request for formative data collections for research and evaluation in the spring of 2021 (Fatherhood Family-Focused, Interconnected, Resilient, and Essential [Fatherhood FIRE] Grantee Local Evaluation Plan Template; OMB #0970-0356).

8 HMRE analysis plans were collected under a generic clearance request for formative data collections for program improvement in November 2023 (Strengthening the Implementation of Marriage and Relationship Programs [SIMR]; OMB #0970-0531).

9 Examples of sensitive topics include (but not limited to): social security number; sex behavior and attitudes; illegal, anti-social, self-incriminating and demeaning behavior; critical appraisals of other individuals with whom respondents have close relationships, e.g., family, pupil-teacher, employee-supervisor; mental and psychological problems potentially embarrassing to respondents; religion and indicators of religion; community activities which indicate political affiliation and attitudes; legally recognized privileged and analogous relationships, such as those of lawyers, physicians and ministers; records describing how an individual exercises rights guaranteed by the First Amendment; receipt of economic assistance from the government (e.g., unemployment or WIC or SNAP); immigration/citizenship status.

10 Bureau of Labor Statistics, U.S. Department of Labor, Occupational Outlook Handbook, Social and Community Service Managers, on the Internet at https://www.bls.gov/ooh/management/social-and-community-service-managers.htm (visited July 2024).

10