Supporting Statement - Formative GenIC Program Support - PREP PYP

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

Collection of Information for Local Evaluations as part of the Personal Responsibility Education Program (PREP): Promising Youth Programs (PYP)

Formative Data Collections for Program Support

0970 – 0531

Supporting Statement

Part A

February 2021

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officer:

Selma Caal

Part A

Executive Summary

• Type of Request: This Information Collection Request is for a generic information collection (GenIC) under the umbrella generic, Formative Data Collections for Program Support (0970-0531).

• Description of Request: This data collection will inform evaluation support provided to Personal Responsibility Education Program (PREP) grantees for their local evaluations. The request includes two final evaluation report templates that ask grantees to describe their analyses and findings from local evaluations to help ensure the reports meet the ACF standards for rigor. The data collected through the templates are not intended to generalize to a broader audience or to be used as the principal basis for public policy decisions.

• Time Sensitivity: Grants are currently scheduled to end in September 2021. Some, but not all, grantees are likely to apply for a no-cost extension beyond that. Therefore, the template needs to be disseminated no later than March 2021 to allow grantees who will not be applying for a no cost extension to begin work on completing their reports by the end of the grant period.

A1. Necessity for Collection

To improve the life course of adolescents and reduce the risks related to sexual activity, Congress authorized the Personal Responsibility Education Program (PREP) as part of the 2010 Affordable Care Act. It was reauthorized in 2015 for an additional two years of funding through the Medicare Access and CHIP Reauthorization ACT of 2015, which also mandates that the Secretary evaluate the programs and activities carried out with funds made available through PREP. Section 513 of the Social Security Act (42 USC 713)--as amended by Section 215 of the Medicare Access and CHIP Reauthorization Act of 2015--authorizes funding for PREP grants, including the Personal Responsibility Education Innovative Strategies Program (PREIS), and Tribal PREP grants (see Appendix A for the legislation).

This generic information collection (GenIC) request pertains to PREIS and Tribal PREP grants. PREIS supports the development of innovative strategies to prevent teen pregnancy for high-risk, vulnerable, and culturally underrepresented youth populations. Tribal PREP supports programming designed to reduce teen pregnancy and birth rates and the spread of STIs for American Indian/Alaska Native (AI/AN) youth.

In 2016, ACF contracted with Mathematica to conduct the PREP: Promising Youth Programs (PYP) project, with the goal of supporting further development of the evidence base for teen pregnancy prevention programs. The PYP project has two broad objectives: (1) to provide evaluation support to grantees and their local evaluators and (2) to develop curricula for underserved youth to address sexual health and other PREP-related priorities. The data collected with the instruments in this GenIC are necessary to meet the first objective.

A2. Purpose

Purpose and Use

This proposed GenIC meets the following goals of ACF’s generic clearance for formative data collections for program support (0970-0531):

• Delivery of targeted assistance related to program implementation or the development or refinement of program and grantee processes, and the development and refinement of recordkeeping and communication systems.

• Planning for provision of programmatic and evaluation-related technical assistance (T/TA).

The purpose of the structured final evaluation report templates include the following:

• Ensure that grantees’ local evaluators provide critical information on the rigor and appropriateness of their analysis and provide a structure to clearly present findings.

• Give ACF and Mathematica information needed to provide technical assistance to PREIS and Tribal PREP grantees on their analysis and final evaluation reports.¹ The final evaluation reports provide a record of the results of the five-year evaluations funded by ACF. Mathematica’s and ACF’s reviews are designed to provide feedback on the clarity and quality of the presentation and analyses.

The information collected is meant to contribute to the body of knowledge on ACF programs. It is not intended to be used as the principal basis for a decision by a federal decision-maker, and is not expected to meet the threshold of influential or highly influential scientific information.

Research Questions/Tests

The information being collected using these instruments is for program and evaluation monitoring and technical assistance only for the 12 PREIS and 8 Tribal PREP grantees. There are no research questions or tests.

Final report templates: In accordance with grant requirements, grantees must produce rigorous evaluation reports.  The templates are a means of ensuring that analyses demonstrate a commitment to objectivity and a systematic, scientific approach, as well as that final reports are comprehensive in describing the interventions, evaluation designs, and findings. Grantees will complete the templates using the instructions as guidance, and then the government and contractor will review them and provide detailed feedback in order to help grantees strengthen the rigor and relevance of the reporting of their evaluation findings. Once finalized, the information reported by grantees in the templates will serve as their final evaluation report.

Please note that the final report templates include two parts: Part a) text template and Part b) table template. The templates also have accompanying instructions to guide the grantees in completing them (See Appendices B and C).

Data Collection Activity	Instrument(s)	Respondent, Content, Purpose of Collection	Mode and Duration
Final report templates	Instrument 1a: PREIS Impact Report Template Instrument 1b: PREIS Impact Report Table Template Instrument 2a: Tribal PREP Descriptive Report Template Instrument 2b: Tribal PREP Descriptive Report Table Template	Respondents: For the PREIS grantees, 36 individuals will respond; for the Tribal PREP grantees, 24 individuals will respond Content: Requests information on study design and methods grantees used to analyze outcomes and implementation data for their local evaluations, and evaluation findings. Purpose: To ensure final reports are comprehensive and provide critical information on the rigor and appropriateness of their research approaches and present the findings clearly.	Mode: Electronic written document Duration: 25 hours

Other Data Sources and Uses of Information

The project received OMB approval for the following activities, which all serve as a foundation to help the grantee complete the final evaluation report template.

• Collection of grantees’ evaluation design plans (approved under the Formative Generic for ACF Research (0970-0356))

• Collection of information related to grantees’ CONSORT diagrams and baseline equivalence and a project abstract, along with topic guides related to the second objective of the PYP project-- developing curricula for underserved youth to address sexual health and other PREP-related priorities (approved under the full information collection, Local Evaluations as part of the PREP: PYP) (0970-0504) and the Formative Generic for ACF Program Support (0970-0531))

• Collection of information about grantees’ analysis plans and about key lessons learned from conducting local evaluations (approved under the Formative Generic for ACF Program Support (0970-0531)).

Grantees can also draw on those materials to complete some sections of their final report. We also plan to request approval in the near future for data collection instruments in service of the second objective of the PYP project, as described in section A1.

A3. Use of Information Technology to Reduce Burden

ACF and its contractors will employ information technology as appropriate and whenever possible to reduce the burden of respondents who agree to participate by making use of electronic templates for completion and submission.

A4. Use of Existing Data: Efforts to reduce duplication, minimize burden, and increase utility and government efficiency

This GenIC is one of several related information collections to carrying out the current round of grants funded by PREP, but this is the only one that will address final reports for the grantees in question. See section A2 for additional information about related ICs.

Grantees will be able to use other documents (such as their design plans, abstracts and analysis plans) to complete their final report templates. This will minimize their burden in filling out the final report template. However, none of the existing documentation on this project has collected all of the information collected in the final report template and some prior information may be out of date. It will be the final record of the results of the rigorous evaluations and needs to provide an overview of the design, analysis and findings for any future readers of the final reports. The final reports are the only documents we expect to be available to the public. Therefore, no unnecessary information is being requested of program staff in the PREIS and Tribal PREP grants.

A5. Impact on Small Businesses

The potential exists for data collection activities to affect small entities, that is the grantees themselves or organizations associated with the grantees. Grantee organizations may be small businesses or may hire local evaluators that are part of a small business. Proposed data collection efforts are designed to minimize the burden by collecting only critical information through the use of standardized templates.

A6. Consequences of Less Frequent Collection

All grantees will need to complete the final evaluation report template at least once. Grantees will likely need to revise this template between two to four times based on feedback received from the evaluation team. If grantees did not submit revisions, we would not have a complete and finalized report for each evaluation. Finally, ACF may publish the evaluation reports on its website and fewer revisions may result in dissemination of reports that lack clarity or contain an analytic issues that mean the reports do not meet the overall rigor set by ACF.

A7. Now subsumed under 2(b) above and 10 (below)

A8. Consultation

Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of the overarching generic clearance for formative information collection. This notice was published on October 13, 2020, Volume 85, Number 198, page 64480, and provided a sixty-day period for public comment.

Consultation with Experts Outside of the Study

ACF consulted with the following staff from Mathematica Policy Research when preparing the templates:

• Jean Knab, PYP Project Director

• Christine Ross, PYP Principal Investigator

• Juliette Henke, PYP Deputy Project Director

• Kristin Hallgren, Senior Researcher

A9. Tokens of Appreciation

No tokens of appreciation for respondents are proposed for this information collection.

A10. Privacy: Procedures to protect privacy of information, while maximizing data sharing

Personally Identifiable Information

The only expected personally identifiable information in the templates will be associated with the authors of the report (such as names and email addresses). The final reports will not include any personally identifiable information on individual participants in the local evaluation.

Assurances of Privacy

As specified in the contract, Mathematica (the Contractor) shall protect grantee privacy to the extent permitted by law and will comply with all Federal and Departmental regulations for private information. Grantees will be informed about the plans on how the data will be used, and that their information will be kept private to the extent permitted by law.

Data Security and Monitoring

The Contractor shall ensure that all of its employees, including employees of all subcontractors, who perform work under this contract are trained on data privacy issues and comply with the above requirements. All Mathematica staff are required to sign the Mathematica Staff Confidentiality Agreement and participate in annual security awareness training.

The Contractor will use a data access plan consistent with OMB Policy and the Foundations for Evidence-based Policy Making Act of 2018. The data collected through the templates will not be shared outside of the federal and contractor staff directly involved with the grantees.

A11. Sensitive Information

There are no sensitive questions in this data collection.

A12. Burden

Explanation of Burden Estimates

Table A12.1 provides the estimated annual reporting burden calculations for the two instruments included in this request. The total annual burden is estimated to be 750 hours for a two-year period. Assumptions by instrument follow.

• PREIS Impact final report template (Instruments 1a and 1b). Twelve PREIS grantees (three respondents per grantees) will complete the full impact final report template. On average it will take each person 25 hours to complete this template. The estimated total burden for this effort will be 900 hours (36 people * 25 hours). This burden includes the initial submission and responding to comments from ACF and Mathematica.

• Tribal PREP Descriptive final report template (Instruments 2a and 2b). Eight Tribal grantees (three respondents per grantee) will complete the full descriptive final report template.. On average it will take each person 25 hours to complete this template. The estimated total burden for this effort will be 600 hours (24 people * 25 hours). This burden includes the initial submission and responding to comments from ACF and Mathematica.

Estimated Annualized Cost to Respondents

The estimated annualized cost to respondents is $24,210. For cost calculations for the labor associated with completing the final report and local evaluation information collection templates, we estimate the average hourly wage for program directors and managers to be the average hourly wage for “Social and Community Services Manager” ($32.28), taken from the U.S. Bureau of Labor Statistics, Occupational Employment Statistics, 2019.²

Table A12.1

Instrument	No. of Respondents (total over request period)	No. of Responses per Respondent (total over request period)	Avg. Burden per Response (in hours)	Total Burden (in hours)	Annual Burden (in hours)	Average Hourly Wage Rate	Total Annual Respondent Cost
Impact final report template	36	1	25	900	450	$32.28	$14,526
Descriptive final report template	24	1	25	600	300	$32.28	$9,684
Total				1,500	750		$24,210

A13. Costs

There are no additional costs to respondents.

A14. Estimated Annualized Costs to the Federal Government

The total cost for the data collection activities under this current request will be $329,811 over 2 years. We estimate the cost for each study activity in the table below. Estimated costs include contractor staff labor hours; operational expenses including equipment, overhead, printing, staff support, and travel; and any other expenses which would not have been incurred without this collection of information.

Cost Category	Estimated Costs
Review of final reports	$329,811
Total costs	$329,811

A15. Reasons for changes in burden

This is a request for a new GenIC under the Formative Generic for ACF Program Support (0970-0531).

A16. Timeline

Activity	Date
Receive OMB approval	Winter 2021
Grantees will receive initial drafts of templates	Upon OMB approval
First draft of final report due	Spring 2021 – Spring 2022 (approximately 6 months from the end of the grant, accounting for no-cost extensions)
Final revisions of the final report due	Summer 2021 – Summer 2022 (at the end of the grant, accounting for no-cost extensions)

A17. Exceptions

No exceptions are necessary for this information collection.

Attachments

• Instruments:

  • Instrument 1a: PREIS Impact Report Template

  • Instrument 1b: PREIS Impact Report Table Template

  • Instrument 2a: Tribal PREP Descriptive Report Template

  • Instrument 2b: Tribal PREP Descriptive Report Table Template

• Appendices:

  • Appendix A: Section 513 of Social Security Act

  • Appendix B: FYSB Personal Responsibility Education Program (PREP): Promising Youth Programs (PYP)— PREIS Impact Report Guidance

  • Appendix C: FYSB Personal Responsibility Education Program (PREP): Promising Youth Programs (PYP)—Tribal PREP Descriptive Report Guidance

1 The data to be collected in this GenIC through the final report templates will build on data also collected under 0970-0531. Under 0970-0531 approved data collection, grantees will pre-specify their analysis in analysis plan templates. The grantees will execute those analysis plans and the results summarized in the final reports.

2 Bureau of Labor Statistics, U.S. Department of Labor, Occupational Outlook Handbook, Social and Community Service Managers, on the Internet at https://www.bls.gov/ooh/management/social-and-community-service-managers.htm (visited Jul 21, 2020).

9