Supporting Statement A: PYP

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

Justification for the Collection of Information for Local Evaluations as part of the Personal Responsibility Education Program (PREP): Promising Youth Programs (PYP)

Formative Data Collections for Program Support

0970 – 0531

Supporting Statement

Part A

October 2020

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officer:

Selma Caal

Part A

Executive Summary

• Type of Request: This Information Collection Request (ICR) is for a generic information collection under the umbrella generic, Formative Data Collections for Program Support (0970-0531). It includes a request for renewal of two previously approved instruments to support local evaluation information collection for a two year period. Those instruments were approved originally under ICR (0970-0504), which expires November 30, 2020.

• Description of Request: This data collection will inform evaluation support provided to Personal Responsibility Education Program (PREP) grantees for their local evaluations. The request includes two local evaluation information instruments -- CONSORT and baseline equivalence templates -- that we will use to provide targeted evaluation support around recruitment, data collection and plans for analysis. The data collected through the templates are not intended to generalize to a broader audience or to be used as the principal basis for public policy decisions.

• Time Sensitivity: The existing ICR for the CONSORT and baseline equivalence templates expires November 30, 2020 (0970-0504).

A1. Necessity for Collection

To improve the life course of adolescents and reduce the risks related to sexual activity, Congress authorized the Personal Responsibility Education Program (PREP) as part of the 2010 Affordable Care Act. It was reauthorized in 2015 for an additional two years of funding through the Medicare Access and CHIP Reauthorization ACT of 2015, which also mandates that the Secretary evaluate the programs and activities carried out with funds made available through PREP. Section 513 of the Social Security Act (42 USC 713)--as amended by Section 215 of the Medicare Access and CHIP Reauthorization Act of 2015--authorizes funding for PREP grants, including the Personal Responsibility Education Innovative Strategies Program (PREIS), and Tribal PREP grants.

This information collection request pertains to PREIS and Tribal PREP grants. PREIS supports the development of innovative strategies to prevent teen pregnancy for high-risk, vulnerable, and culturally underrepresented youth populations. Tribal PREP supports programming designed to reduce teen pregnancy and birth rates and the spread of STIs for American Indian/Alaska Native (AI/AN) youth.

In 2016, ACF contracted with Mathematica to conduct the PREP: Promising Youth Programs (PYP) project, with the goal of supporting further development of the evidence base for teen pregnancy prevention programs. The PYP project has two broad objectives: (1) to provide evaluation support to grantees and their local evaluators and (2) to develop curricula for underserved youth to address sexual health and other PREP-related priorities. The data collected with the instruments in this ICR relates to the first objective. The local evaluation information instruments, specifically CONSORT and baseline equivalence templates, ask grantees to report on sample flow and characteristics. We use this information to provide targeted evaluation support around recruitment, data collection and plans for analysis. The data collected will give ACF and Mathematica information needed to provide technical assistance to PREIS and Tribal PREP grantees on their analysis.

A2. Purpose

Purpose and Use

This proposed ICR meets the following goals of ACF’s generic clearance for formative data collections for program support (0970-0531):

• Delivery of targeted assistance and workflows related to program implementation or the development or refinement of program and grantee processes, and the development and refinement of recordkeeping and communication systems.

• Planning for provision of programmatic and evaluation-related technical assistance (T/TA).

The information collected is meant to contribute to the body of knowledge on ACF programs. It is not intended to be used as the principal basis for a decision by a federal decision-maker, and is not expected to meet the threshold of influential or highly influential scientific information.

Research Questions/Tests

The information being collected using these instruments is for program and evaluation monitoring and technical assistance only for the 12 PREIS and 8 Tribal PREP grantees. There is no independent study being conducted using the information collected.

Local evaluation information templates: This request includes the renewal of two currently approved instruments (0970-0504, expiration date 11/30/2020) that will help grantees complete requirements for their rigorous local evaluations as specified in their cooperative agreements and prepare for their final report analysis and final report write-up. The CONSORT sample flow and baseline equivalence documents will continue to allow the evaluation support team to determine if the study has any potential methodological issues (such as high sample attrition or a lack of equivalence in the baseline sample) that the team should be aware of prior to the grantee conducting final report analyses. Grantees will submit a CONSORT diagram and in some cases a baseline equivalence template (depending on their proposed design) in Spring 2021; and for those grantees who receive a no-cost extension, one additional time during the no-cost extension period in Winter 2022.

The templates also have accompanying instructions to guide the grantees in completing them. We have provided these instructions as Appendices C through E.

Data Collection Activity	Instrument(s)	Respondent, Content, Purpose of Collection	Mode and Duration
Local evaluation information templates	Instrument 1a: Impact CONSORT Diagram Template Instrument 1b: Descriptive CONSORT Diagram Template	Respondents: 12 PREIS grantees and 8 Tribal PREP grantees Content: Requests information on sample flow on a semi-annual basis Purpose: To monitor sample size changes and resulting implications for rigorous evaluations	Mode: Electronic template Duration: 30 hours
Local evaluation information templates	Instrument 2: Baseline Equivalence Template	Respondents: 12 PREIS grantees Content: Requests information on sample characteristics of the treatment and control group on a semi-annual basis Purpose: To monitor baseline equivalence of the analytic sample(s) and resulting implications for rigorous evaluations	Mode: Electronic template Duration: 36 hours

Other Data Sources and Uses of Information

The project received OMB approval in September 2016 under the generic clearance for Formative Data Collections (0970-0356) to collect grantees’ evaluation design plans. The project also received OMB approval in 2017 (0970-0504; expires November 30, 2020) for collection of information related to grantees’ CONSORT diagrams and baseline equivalence and a project abstract, along with topic guides related to the second objective of the PYP project-- developing curricula for underserved youth to address sexual health and other PREP-related priorities. The project also received OMB approval in 2020 (0970-0531; expires July 31, 2022) for collection of information about grantees’ analysis plans and about key lessons learned from conducting local evaluations.

The two local evaluation information instruments we are seeking to renew were originally approved in the 2017 package (ICR 0970-0504). These instruments provide a foundation for helping grantees conduct their final report analyses and write-up and allow for continued, up-to-date training and technical evaluation assistance to grantees.

We will be seeking approval under a separate package for grantee final report templates. We also plan to request approval in the near future for data collection instruments in service of the second objective of the PYP project, as described in section A1.

A3. Use of Information Technology to Reduce Burden

ACF and its contractors will employ information technology as appropriate and whenever possible to reduce the burden of respondents who agree to participate by making use of electronic templates for completion and submission.

A4. Use of Existing Data: Efforts to reduce duplication, minimize burden, and increase utility and government efficiency

The CONSORT and Baseline equivalence templates, which have been previously approved, build on existing materials and previous experience from similar evaluation TA efforts by the Contractor. The grantees will use the same template at each reporting period which will minimize grantee burden for reporting periods once they are familiar with the request and have any necessary statistical programs developed.

A5. Impact on Small Businesses

The potential exists for data collection activities to affect small entities, that is the grantees themselves or organizations associated with the grantees. Grantee organizations may be small businesses or may hire local evaluators that are part of a small business. Proposed data collection efforts are designed to minimize the burden by collecting only critical information through the use of standardized templates.

A6. Consequences of Less Frequent Collection

The CONSORT and Baseline equivalence templates will be collected once from all grantees and a second time for the estimated half of the grantees that receive no-cost extensions during the reporting period associated with this ICR. This reporting will involve updates to previously submitted materials. If we do not collect updates, we will not be able to appropriately guide grantees on the implications of sample attrition and a lack of baseline equivalence for their study rigor and analysis on an ongoing basis during the full length of their evaluation.

A7. Now subsumed under 2(b) above and 10 (below)

A8. Consultation

Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of the overarching generic clearance for formative information collection. This notice was published on October 11, 2017, Volume 82, Number 195, page 47212, and provided a sixty-day period for public comment. A subsequent notice, updated with more specific information, was published on June 18, 2019, Volume 84, Number 117, page 28307, and provided a thirty-day period for public comment.

Consultation with Experts Outside of the Study

ACF consulted staff from Mathematica Policy Research when preparing the templates. See Attachment F for a list of the persons consulted.

A9. Tokens of Appreciation

No tokens of appreciation for respondents are proposed for this information collection.

A10. Privacy: Procedures to protect privacy of information, while maximizing data sharing

Personally Identifiable Information

There is no personally identifiable information in the local evaluation information collection instruments.

Assurances of Privacy

As specified in the contract, Mathematica (the Contractor) shall protect grantee privacy to the extent permitted by law and will comply with all Federal and Departmental regulations for private information. Grantees will be informed about the plans on how the data will be used, and that their information will be kept private to the extent permitted by law.

Data Security and Monitoring

The Contractor shall ensure that all of its employees, including employees of all subcontractors, who perform work under this contract are trained on data privacy issues and comply with the above requirements. All Mathematica staff are required to sign the Mathematica Staff Confidentiality Agreement and participate in annual security awareness training.

The Contractor will use a data access plan consistent with OMB Policy and the Foundations for Evidence-based Policy Making Act of 2018. The data collected through the templates will not be shared outside of the federal and contractor staff directly involved with the grantees.

A11. Sensitive Information

There are no sensitive questions in this data collection.

A12. Burden

Explanation of Burden Estimates

Table A12.1 provides the estimated annual reporting burden calculations for the two instruments included in this request. The total annual burden is estimated to be 762 hours for a two year period. Assumptions by instrument follow.

• CONSORT Diagram Template (Instrument 1a and 1b). All grantees will submit the template in Spring 2021. Grantees receiving no-cost extensions (estimated to be half of the grantees -- 6 PREIS and 4 Tribal PREP) will submit the template one more time during the no-cost extension period (Winter 2022). On average, it will take one hour to complete this template. The estimated total burden for this effort is 30 hours.

• Baseline Equivalence Template (Instrument 2). Twelve PREIS grantees will submit the template in Spring 2021. Six grantees receiving a no-cost extension will submit the template one more time during the no-cost extension period (Winter 2022). On average, it will take two hours to complete this template. The estimated total burden for this effort is 36 hours.

Estimated Annualized Cost to Respondents

The estimated annualized cost to respondents is $1,065.24. For cost calculations for the labor associated with completing the final report and local evaluation information collection templates, we estimate the average hourly wage for program directors and managers to be the average hourly wage for “Social and Community Services Manager” ($32.28), taken from the U.S. Bureau of Labor Statistics, Occupational Employment Statistics, 2019.¹

Table A12.1

Instrument	No. of Respondents (total over request period)	No. of Responses per Respondent (total over request period)	Avg. Burden per Response (in hours)	Total Burden (in hours)	Annual Burden (in hours)	Average Hourly Wage Rate	Total Annual Respondent Cost
CONSORT Diagram Template	20	1.5	1	30	15	$32.28	$484.20
Baseline Equivalence Template	12	1.5	2	36	18	$32.28	$581.04
Total	32	48		66	33		$1,065.24

A13. Costs

There are no additional costs to respondents.

A14. Estimated Annualized Costs to the Federal Government

The total cost for the data collection activities under this current request will be $45,609 over 2 years. We estimate the cost for each study activity in the table below. Estimated costs include contractor staff labor hours; operational expenses including equipment, overhead, printing, staff support, and travel; and any other expenses which would not have been incurred without this collection of information.

Cost Category	Estimated Costs
Review of CONSORT and Baseline Equivalence	$45,609
Total costs	$45,609

A15. Reasons for changes in burden

This is a new information collection request.

A16. Timeline

Activity	Date
Receive OMB approval	Fall 2020
CONSORT Diagram and Baseline Equivalence semi-annual templates due – all grantees	Spring 2021
CONSORT Diagram and Baseline Equivalence semi-annual templates due – - grantees with no-cost extensions	Winter 2022

A17. Exceptions

No exceptions are necessary for this information collection.

Attachments

• Instruments:

  • Instrument 1a: Impact CONSORT Diagram Template

  • Instrument 1b: Descriptive CONSORT Diagram Template

  • Instrument 2: Baseline Equivalence Template

• Appendices:

  • Appendix A: Section 513 of Social Security Act

  • Appendix B: List of experts consulted in designing templates

  • Appendix C: Instructions for CONSORT Diagram for Clusters and for Youth (Impact Studies)

  • Appendix D: Instructions for CONSORT Diagram for Clusters and for Youth (Descriptive Studies)

  • Appendix E: Instructions for Baseline Equivalence Template

1 Bureau of Labor Statistics, U.S. Department of Labor, Occupational Outlook Handbook, Social and Community Service Managers, on the Internet at https://www.bls.gov/ooh/management/social-and-community-service-managers.htm (visited Jul 21, 2020).

8