0990-PAF Supporting Statement A_to OMB_082614 revised

Part A: Justification for the Collection of Design and Implementation Survey Data and Baseline Data - Pregnancy Assistance Fund (PAF) Feasibility and Design Study (FADS)

April 2014 (Revised August 2014)

Submitted to: U.S. Department of Health and Human Services Office of Adolescent Health Office of the Director Department of Health and Human Services 1101 Wootton Parkway, Suite 700 Rockville, MD 20852 Project Officer: Submitted by: Mathematica Policy Research P.O. Box 2393 Princeton, NJ 08543-2393 Telephone: (609) 799-3535 Facsimile: (609) 799-0005 Project Director: Amy Farb

Part A: Justification for the Collection of Design and Implementation Survey Data and Baseline Data - Pregnancy Assistance Fund (PAF) Feasibility and Design Study (FADS) April 2014 (Revised August 2014)

CONTENTS

PART A: INTRODUCTION 1

A1. Circumstances Making the Collection of Information Necessary 1

1. Legal or Administrative Requirements that Necessitate the Collection 1

2. Study Objectives 2

A.2. Purpose and Use of the Information Collection 3

A.3. Use of Information Technology to Reduce Burden 4

A.4. Efforts to Identify Duplication and Use of Similar Information 5

A.5. Impact on Small Businesses 5

A.6. Consequences of Not Collecting the Information/Collecting Less Frequently 5

A.7. Special Circumstances 6

A.8. Federal Register Notice and Consultation Outside the Agency 6

A.9. Payments to Respondents 6

A.10. Assurance of Confidentiality 6

A.11. Justification for Sensitive Questions 7

A.12. Estimates of the Burden of Data Collection 8

1. Annual Burden for Grantees 8

2. Annual Burden for Youth Participants 9

3. Overall Burden 11

A.13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers 11

A.14. Annualized Cost to Federal Government 11

A.15. Explanation for Program Changes or Adjustments 11

A.16. Plans for Tabulation and Publication and Project Time Schedule 12

1. Analysis Plan 12

2. Time Schedule and Publications 12

A.17. Reason(s) Display of OMB Expiration Date is Inappropriate 13

A.18. Exceptions to Certification for Paperwork Reduction Act Submissions 13

SUPPORTING REFERENCES FOR INCLUSION OF SENSITIVE QUESTIONS OR GROUPS OF QUESTIONS 14

TABLES

A.11.1 Summary of Sensitive Questions to be Included on the Baseline Survey and Their Justification 8

A.12.1 Calculations of Burden Hours and Cost for Grantees 9

A.12.2 Revised Calculations of Burden Hours and Cost for Youth Participants 10

A.12.3 Calculations of Annual Burden Hours and Costs 11

ATTACHMENTS

ATTACHMENT A: OVERVIEW OF THE PAF EVALUATION

ATTACHMENT B: EMAIL TEMPLATE FOR INTERVIEW REQUEST

ATTACHMENT C: GRANTEE INTERVIEW GUIDE

ATTACHMENT D: PAF BASELINE SURVEY

ATTACHMENT E: QUESTION BY QUESTION SOURCE LIST FOR THE BASELINE SURVEY

ATTACHMENT F: SOURCES REFERENCED FOR THE BASELINE SURVEY

ATTACHMENT G: BASELINE PRETEST MEMO

ATTACHMENT H: 60 DAY FEDERAL REGISTER NOTICE

ATTACHMENT I: PERSONS CONSULTED ON INSTRUMENT DEVELOPMENT AND/OR ANALYSIS OF THE PAF BASELINE SURVEY

ATTACHMENT J: CONSENT LETTERS AND FORMS AND YOUTH ASSENT FORM

ATTACHMENT K: CONFIDENTIALITY PLEDGE

ATTACHMENT L: ANALYSIS PLAN

ATTACHMENT M: OVERVIEW DOCUMENT FOR GRANTEES

ATTACHMENT N: 30 Day FEDERAL REGISTER NOTICE

PART A: INTRODUCTION

In March 2010, Congress authorized the Pregnancy Assistance Fund (PAF) Competitive Grants Program as part of the Patient Protection and Affordable Care Act (ACA). The PAF grant program is a key element of the federal strategy to support teens and young adults who are having or raising a child and their families. Administered by the Office of Adolescent Health (OAH), the PAF grant program funded a second cohort of 17 grantees—states, tribes, and tribal entities—in summer 2013 to develop and implement programs focused on an array of outcomes, including increasing access to and completion of secondary and postsecondary education, improving child and maternal health, reducing the likelihood of repeat teen pregnancies, increasing parenting and co-parenting skills, decreasing intimate partner violence, and raising awareness of available resources. To promote positive outcomes, grantees may implement a wide variety of services for expectant and parenting teens, women, fathers, and their families.

The PAF evaluation will have two core components: a rigorous assessment of program impacts and implementation of two grantees, and a descriptive examination of program design of all 17 grantees. The PAF evaluation will help the federal government, grantees, and local service providers learn more about program design, implementation, and impacts.

Preliminary PAF evaluation efforts, including instrument development, will be conducted through the PAF Feasibility and Design Study (FADS). The purpose of the FADS is to design rigorous impact evaluations in two sites, develop data collection materials for all aspects of a future evaluation, and conduct telephone interviews with grantees about the program design decisions and early implementation experiences. Information collected through the FADS will also be used to provide funding agencies with information to inform the structure and components of programs for expectant and parenting teens and their families, so that future rigorous program evaluation will also be possible.

The FADS comprises a design and implementation analysis and an impact study as its two primary data collection components. The design and implementation analysis will describe grantees’ program design and factors that influenced their decision making. The impact study will use an experimental design to test the effectiveness of PAF-funded services on outcomes related to education, sexual behaviors, parenting, and health. An overview of the PAF Evaluation is found in Attachment A.

This proposed information collection activity focuses on collecting (a) program design and early implementation data collected through telephone interviews with grantees and (b) baseline data collection in up to three impact sites.

A1. Circumstances Making the Collection of Information Necessary

1. Legal or Administrative Requirements that Necessitate the Collection

On March 23, 2010 the President signed into law the Patient Protection and Affordable Care Act (ACA), H.R. 3590 (Public Law 111-148, Sections 10211-10214). In addition to its other requirements, the act authorizes $25 million for each of fiscal years 2010 through 2019 and authorizes the Secretary of HHS, in collaboration and coordination with the Secretary of Education, to “establish a Pregnancy Assistance Fund to be administered by the Secretary, for the purpose of awarding competitive grants to States to assist expectant and parenting teens and women.¹”

The Office of Management and Budget has requested an evaluation of the PAF program as a condition of the program funding (per conversations with OAH Director, Evelyn Kappeler), recognizing that there is a unique opportunity to contribute to the field by designing a rigorous evaluation of PAF-funded programs that can overcome previous challenges in doing so.

2. Study Objectives

There is currently little rigorous program evaluation published in the expectant and parenting youth literature. This is due, in part, to lack of federal funding to evaluate programs until very recently. Additionally, there are methodological difficulties inherent in conducting evaluations of programs for these teens. For example, the sample sizes available for evaluation within any one program are generally small. In addition, low program enrollment and low retention rates reflect the complex social profiles and needs of this population.

Within OAH there is a unique opportunity to contribute to the field by designing a rigorous evaluation that overcomes these challenges. The PAF grants are made to entire states and tribal entities; the grantees are implementing programs across large geographic areas, which increases the numbers of youth that will be served by any one programmatic approach. Many grants are supporting existing programs that have a demonstrated history of recruiting, engaging, and retaining expectant and parenting youth for the intended program duration.

The objective of the Pregnancy Assistance Fund (PAF) Feasibility and Design Study (FADS) is to establish a foundation for a rigorous impact and implementation evaluation. Specifically, FADS will: (1) assess design options for implementation and impact evaluation, (2) document how PAF programs are operationalized in the field, (3) identify and enter into agreements with two sites for the evaluation, (4) provide assistance to sites to support a rigorous evaluation framework, (5) develop all evaluation instruments and obtain clearance, and (6) pilot baseline data collection. As described below, the evaluation will have two main components: the Design and Implementation Analysis and the Impact Study.

Design and Implementation Analysis. During the FADS, all PAF grantees will participate in telephone interviews about program design and early implementation experiences. These interviews will be conducted toward the end of the first grant year.

Impact Study. Using an experimental design, the PAF evaluation will test the effectiveness of services to impact educational, health, sexual behavior, and parenting outcomes. During the FADS, the study team will identify and work with two grantees to decide which service components will be evaluated, which participants will be included, and which outcomes will be measured. In addition, the FADS team will work with grantees to develop a plan for random assignment. Finally, the FADS team will work with the selected sites to design a process for collecting study data, including evaluation consent, a baseline survey, and two follow-up surveys.

The two sites selected for the impact evaluation will also participate in, during a future evaluation, a more in-depth implementation study. The in-depth implementation study will take a detailed look at program operations along four key aspects: (1) inputs required for implementation to succeed and be sustained, (2) contextual factors that influence implementation, (3) quality of program implementation, and (4) participants’ responsiveness to service.

OAH is currently requesting OMB approval for the collection of (a) program design and early implementation data collected through telephone interviews with 17 grantees and (b) baseline data in two impact sites. Approval for additional data collection activities through follow-up surveys and the in-depth implementation study will be requested in later submissions.

A.2. Purpose and Use of the Information Collection

Design and Implementation Analysis. If this request is approved, the data collection is intended to inform OAH and the field of the design and initial implementation experiences of all 17 PAF grantees. It will build on knowledge about the grantees and their program plans gathered from a review of their grant applications and implementation plans. This effort will be very similar to one conducted for the PREP evaluation across all 45 of the State PREP grantees (OMB Control Number: 0970-0398; approved on March 7, 2012). A public report on the grantees’ plans and early implementation experiences will provide a rich source of varying approaches for supporting pregnant and parenting teens in different contexts. Additionally, as we have learned from the similar PREP evaluation component, this descriptive effort can help to identify sites for the impact study; grantee plans evolve since initial grant application, and the later data collection effort for this component can identify back up sites for the impact evaluation, should previously identified sties prove infeasible. Attachment B contains a copy of the email template that will be used to request an interview. A copy of the grantee interview protocol is found in Attachment C.

Impact Study. Data collected on the PAF baseline survey will be used as a central component to the impact study. Specifically, the data will be used to establish baseline equivalence of the treatment and control groups and thus to confirm the integrity of the random assignment process. Baseline data will also be used to define subgroups for which impacts will be estimated, and to adjust impact estimates to account for survey non-response. Many baseline measures will be measured again at follow-up; their baseline values can be used to improve the precision of impact estimates by their inclusion as covariates in the impact models.

Many of the items included on the baseline survey are taken directly from similar surveys OMB has already approved for use in the ongoing Evaluation of Adolescent Pregnancy Prevention Approaches (PPA), the Teen Pregnancy Prevention Replication Study and the Personal Responsibility Education Program (PREP) Multi-Component Evaluation². To date, the PPA baseline survey has been administered to approximately 6,270 adolescents, including 1,535 expectant and parenting young women; the Replication Study baseline has been administered to 7,945 adolescents; and the PREP baseline has been administered to 1,414 youth, including 148 expectant and parenting young women.

HHS has made a priority of aligning measures being used in other federal evaluations of similar programs. For PAF, the evaluation team adapted the OMB-approved PPA, Replication Study, and PREP baseline instruments in two primary ways. First, certain measures were added to reflect PAF’s’s authorizing legislation—specifically, the legislation’s focus on continuing and completing secondary and postsecondary education, improving maternal and child health, and increasing awareness of available resources. Second, measures were added or adapted to reflect the characteristics of expectant and parenting teens, and to be able to understand and describe their risk profile. Finally, measures were added to reflect many programs’ goals of delaying a subsequent pregnancy. To accommodate additional measures, other items of lower priority for PAF were dropped from the survey, such as measures that are predictive of early sexual initiation (which is not relevant for a sexually active sample). Attachment D contains a copy of the PAF baseline survey, Attachment E includes a question by question source list for items on the baseline survey, Attachment F includes a description of each of the sources referenced, and Attachment G includes the pretest memo, summarizing adjustments made to the survey as a result of the pretest.

A.3. Use of Information Technology to Reduce Burden

Design and Implementation Analysis. The data collection plan for the phone interviews reflects sensitivity to issues of efficiency, accuracy, and respondent burden. We will collect data once, through a two hour telephone interview. Mathematica’s team will be well prepared for each interview following the review of two sets of documents - grantee’s applications and implementation plans submitted to the OAH. Interview protocols will be customized to verify information found in these documents; the remaining questions will obtain information not able to be gathered from the document reviews.

Impact Study. The data collection plan for the baseline survey also reflects sensitivity to issues of efficiency, accuracy, and respondent burden. We will offer various modes for completing the baseline survey. These modes are likely to be computer-assisted telephone interviewing (CATI), a web-survey, or a self-administered paper and pencil instrument (PAPI). We will work with grantees to assess the best baseline survey mode given the context, with sensitivity to respondent literacy levels and access to technology.

For CATI completes, data collectors will be assigned a project cell phone that will be handed off to respondents during the intake process. Respondents will use the phone to call Mathematica’s Survey Operations Center (SOC) and complete the survey over the phone with a trained interviewer. Data collectors will also be given a toll-free number for the Survey Operations Center that they can give to respondents to call in and complete the survey over the phone at their convenience. Additionally, SOC staff can make calls to the respondent to complete the survey over the phone. When completing the survey through CATI, the interviewer (and data collector, when applicable) will ensure the respondent is in a secure, private place to respond to the survey questions. We have used this method successfully in the past for similar evaluations that similarly ask sensitive questions. CATI has the added benefit of ensuring higher rates of survey completion, with a “live person” walking through the survey with respondents.

We will also offer the option of completing the survey over the web. If a web-based survey is used, respondents will be provided a unique PIN/password. We will also provide them with a toll-free number to call should they have any issues with the web survey.

A paper and pencil (PAPI) version of the baseline survey will also be available for anyone wishing to complete the survey using the self-administered instrument. Data collectors will keep PAPI versions on-hand and distribute them to respondents, as needed. The completed surveys will be returned to staff in sealed envelopes

A.4. Efforts to Identify Duplication and Use of Similar Information

OAH has carefully reviewed the information collection requirements for PAF to avoid duplication with existing and ongoing studies of programs to support expectant and parenting teens, and in particular those that are federally funded. The PAF evaluation will contribute to a very slim knowledge base on effective approaches for improving outcomes for expectant and parenting teens. In the past few decades, many social policy efforts have focused on the prevention of teen and unplanned pregnancy. When prevention efforts are absent or failed, we must consider how to support young people facing these daunting challenges. This evidence base for doing so is slim. The PAF evaluation will add two effectiveness studies (through evaluation contracts to be completed this summer) to this literature, and will provide a detailed description of grantees’ programmatic approaches.

The PAF evaluation is also unique in that it studies a competitive grant program awarded to entire states or tribal entities to support expectant and parenting teens. Such a large-scale grant effort is unique across federal programs to support expectant and parenting teens, and has not previously been rigorously evaluated, nor have the lessons learned from approaches grantees are implementing across multiple sites and large geographic areas been fully assessed and described.

A.5. Impact on Small Businesses

Design and Implementation Analysis. Because this data collection effort will focus exclusively on grantee-level PAF administrators, it will not impact small businesses or other small entities.

Impact Study. Programs are likely to be operated by community-based organizations and have ongoing enrollment. The baseline data collection plan is designed to minimize burden on sites by primarily collecting baseline data via telephone or the web, with little involvement of the site staff. These approaches minimize requirements for “sample pursuit” by site staff.

A.6. Consequences of Not Collecting the Information/Collecting Less Frequently

Design and Implementation Analysis. Under the FADS, information will be collected only once, thus no repetition of effort is planned. Not collecting the information at all would substantially limit our understanding of the PAF program and the value of the investment OAH is making in this study and in the PAF program itself. Each grantee is operating in a very different context, and employing various means to get programming to expectant and parenting teens, making it critical to collect this information across the 17 grantees. In the absence of such data, the decisions made by grantees regarding the design and initial implementation of PAF programs in their various contexts will be unclear, and future funding and operational decisions about pregnancy assistance programs will be based on insufficient information.

Impact Study. Baseline data are essential to conducting a rigorous evaluation of PAF programs supported under Public Law 111-148. Specifically, without these baseline data, we would not be able to monitor whether random assignment was conducted correctly and created two very similar research groups. In addition, we would not be able to estimate impacts for key subgroups or to improve the precision of our impact estimates by including baseline covariates in our statistical models used to estimate program impacts.

A.7. Special Circumstances

There are no special circumstances for the proposed data collection efforts.

A.8. Federal Register Notice and Consultation Outside the Agency

The 60-day Federal Register Notice was posted on February 7, 2014. No public comments were received. A copy of the 60-day Federal Register Notice is found in Attachment H. A draft of the 30-day Federal Register Notice is found in Attachment N.

Attachment I provides contact information of the federal entities and persons outside the agency consulted on the instrument development and/or the analysis of the baseline survey.

A.9. Payments to Respondents

Respondents to the baseline survey will be given a $25 gift card. This is consistent with payments provided to respondents in similar sites for the PPA Evaluation (OMB Control Number: 0990-0382) where the programs are recruiting adolescent mothers who have recently given birth and may be completing the baseline survey in their homes, most likely with their newborns present. As with the previous studies, providing a gift card as a thank you is essential for obtaining high response rates and encouraging participation in future rounds of data collection.

A.10. Assurance of Confidentiality

Design and Implementation Analysis. The data collected through the grantee interview will be reported in two ways, and assurances of privacy will reflect those two ways.

First, a summary profile will be created for each grantee that will contain PAF program design decision facts – for example, the annualized PAF grant amount, the identification of the state-level grantee, the mechanism for distributing finds, the total funding given to program providers, the programs funded, target populations, implementation settings, and the number of youth expected to be served. This information, which is very similar to the information reported for PREP grantees, will be reported for each grantee in a PAF evaluation report. Therefore, while we will not attribute such factual information to a specific respondent, it will be attributed to a specific grantee. Respondents will be made aware of how this factual information will be reported.

Second, the reports will discuss themes emerging from responses regarding “how” and “why” PAF program decisions were made. While these responses may not contain confidential information, they are likely to contain sensitive topics. In particular, responses will be sensitive if state political or other contextual factors influenced the PAF program design decisions. In project reports, such responses will not be attributed to any one state, but will instead be analyzed and reported as part of overall trends across all states. Respondents will be made aware that responses to these questions will not be attributed to themselves or their state.

Beyond these two ways in which data will be reported, information will be kept private to the fullest extent of the law.

Impact Study. Mathematica Policy Research will secure IRB approval for the impact study and will be responsible for securing any additional local IRB approvals for each site prior to information collection, as necessary. Prior to collecting baseline data, we will seek consent from a parent or legal guardian if the respondent is a minor or from respondents themselves if they are 18 or older. An explanation of the data being collected and its use, which answers will be kept private and not seen by anyone outside of the study team, that participation is voluntary, and that they may refuse to participate at any time without penalty will be provided. Participants and their parents/guardians will be told that, to the extent allowable by law, individual identifying information will not be released or published; rather, data collection will be published only in summary form with no identifying information at the individual level. Attachment J provides a copy of the consent and assent forms.

All Mathematica data collection staff are required to sign a confidentiality pledge when hired by Mathematica (see Attachment K). The survey administration protocol for CATI, web, and PAPI completes provides reassurance that we take the issue of privacy seriously. Participants will be informed that all of their answers will be kept private, that identifying information will be kept separate from baseline data, and that no one outside of the study team will see their responses.

Telephone surveys are completed by trained interviewers recording respondent’s answers into a computer program. Prior to beginning the survey, a statement of confidentiality and the student assent is read aloud and respondents are given a chance to verbally opt out of the survey. No identifying information is attached to the data; only a unique study ID will be included on the questionnaire.

For the web surveys, a unique password and PIN will be sent to respondents to log into the survey. A statement of confidentiality will be presented at the beginning of the survey, and we will have a screen where respondents can choose to opt out of the survey. No names will be attached to the data – only the student’s unique study ID.

For administration of hard copy surveys, on the day of the survey administration data collectors will distribute an assent form, providing respondents with a chance to opt out of the baseline data collection, should they want to do so. The questionnaire will be distributed in a sealed envelope, and the questionnaire and distribution envelope will have a label with a unique ID number. Before turning completed questionnaires in to the data collectors, respondents will place them in blank return envelopes and seal them; no identifying information will appear on the questionnaire or the return envelope. Data collectors are trained to keep all data collection forms in a secure location and are instructed not to share any materials with anyone outside of the study team. Completed surveys are immediately shipped via FedEx to Mathematica’s Survey Operations Center for receipting. Any forms with identifying information (consent and assent forms) will be shipped separately from the surveys.

All electronic data will be stored in secure files, with identifying information kept in a separate file from survey and other individual-level data. Survey responses will be stored on a secure, password-protected computer shared drive.

A.11. Justification for Sensitive Questions

Design and Implementation Analysis. There are no sensitive questions in this data collection.

Impact Study. A key objective of PAF programs is to provide assistance to teens and young adults who face challenges associated with pregnancy and parenthood. Because this is the primary focus of the programs, some questions on the baseline survey are necessarily related to sensitive issues.

Table A.11.1 provides a list of the sensitive questions found on the PAF baseline survey, along with a justification for their inclusion.

Table A.11.1. Summary of Sensitive Questions to be Included on the Baseline Survey and Their Justification

Topic	Justificationa
Drug and alcohol use and violence (3.1 - 3.8)	There is a substantial body of literature linking various high-risk behaviors of youth, particularly drug and alcohol use, early sexual intercourse, and risky sexual behavior. The effectiveness of various program strategies is expected to differ for youth who are and are not experimenting with or using drugs and alcohol (Tapert et al., 2001; Li et al., 2001; Boyer et al., 1999; Fergusson and Lynskey, 1996; Sen, 2002; Dermen et al., 1998; Santelli et al., 2001.)
Sexual activity, incidence of pregnancy and STDs, and contraceptive use (5.5 - 5.12, 6.3, - 6.12	Sexual activity, contraceptive use, incidence of STDs, number of pregnancies, and healthcare during pregnancies are all powerful predictors of outcomes relevant to the PAF programs. Having data at baseline increases the precision of our estimates of impacts on sexual behaviors, contraceptive use, and maternal and child health at follow-up.

^a Full references for sources cited in table may be found at the end of Supporting Statement A.

Sensitive questions are drawn from previously-successful youth surveys and evaluations including PPA, the Replication Study, and PREP. The items have been carefully selected, and we have been guided by past experience in determining whether or not the benefits of measures may outweigh concerns about the heightened sensitivity among sample members, parents, and program staff to specific issues. Although these questions are sensitive, they are commonly and successfully asked of youth similar to those who will be in the PAF study.

A.12. Estimates of the Burden of Data Collection

Tables A12.1 and A12.2 provide the estimated annual reporting burden calculations for the grantee interview and baseline survey. These are broken out separately as burden for PAF grantees (Table A12.1) and for PAF youth participants (Table A12.2).Table A12.3 provides a summary of combined burden hours and costs for both components of the PAF FADS.

1. Annual Burden for Grantees

Design and Implementation Analysis. For the telephone interviews of all PAF grantees, we are requesting three years of clearance. The annual burden was estimated from a total number of 17 interviews (one respondent from each of the 17 current PAF grantee), and an estimate of two-hours per interview.

Table A.12.1. Calculations of Burden Hours and Cost for Grantees

Instrument	Type of respondent	Total Number of Respondents	Annual Number of Respondents	Number of Responses Per Respondent	Average Burden Hours Per Response	Total Burden Hours	Average Hourly Wage	Total Annual Cost
Design and Implementation Analysis
Grantee interview	Grantee administrator	17	6	1	2	12	$37.45 (wage for “Social Scientists and Related Workers, All Other”)	$449
Estimated Annual Burden Hours for Grantee Administrators						12		$449

Average hourly wages for Grantee-Level Administrators were estimated from the latest—May 2011—National Occupational Employment and Wage Estimates, Bureau of Labor Statistics, Department of Labor website for those fields (see table). The total annualized cost is estimated at $449.

Impact Study. There is no grantee burden associated with administration of the baseline survey. Data collectors from Mathematica Policy Research will be responsible for the baseline survey data collection.

2. Annual Burden for Youth Participants

Design and Implementation Analysis. There are no youth participants, and therefore no youth burden, associated with the design and implementation study.

Impact Study. The impact evaluation sample will be composed of females, as expectant and parenting young women are the large majority of participants served by the PAF programs, and the services provided are designed to primarily meet the needs of these young women.

Revised Burden Hours

Since submitting the original ICR, our sample size estimates have decreased. The revised table updates the sample size estimates, associated burden, and costs.

Table A.12.2. Revised Calculations of Burden Hours and Cost for Youth Participants

Instrument	Type of respondent	Total Number of Respondents	Annual Number of Respondents	Number of Responses per Respondent	Average Burden Hours per Response	Total Annual Burden Hours	Total Annual Burden Hours for Youth Age 18 or Older	Hourly Wage Rate	Total Costs
Impact Study
Baseline survey of impact study participants	Participating program females and control group females	2,019	673	1	0.5	337	67	$7.25		$485.75
Estimated Annual Burden for Youth Participants						337				$485.75

Note: We assume that 95 percent of the enrolled sample of 2,125 across the two sites will complete the baseline survey. We also assume that 20 percent of the sample will be 18 or older at the time of baseline survey administration.

It is expected that 2,125 young women will be enrolled in the evaluation sample across the two evaluation sites. Sample intake will take place over three years. The expected response rate for the baseline survey is 95 percent, for a total of 2,019 completed baselines, and an average of 673 per year. Based on previous experience with similar questionnaires, it is estimated that it will take youth 30 minutes (0.50 hour) to complete the baseline survey, on average. The total annual burden for this data collection is estimated to be 673 x 0.5 = 337 hours. The cost of this burden is estimated to be 337 hours x 0.20 (proportion of youth age 18 or older) x $7.25 = $485.75.

3. Overall Burden

Table A.12.3 detail the overall burden requested for data collection associated with the PAF FADS. A total of 349 hours (and a cost of $934.75) is requested in this ICR.

Table A.12.3. Calculations of Annual Burden Hours and Costs

Data collection instrument	Type of Respondent	Annual number of respondents	Number of responses per respondent	Average burden hours per response	Total burden hours	Total Burden Hours for Youth Age 18 or Older	Hourly Wage Rate	Total costs
Design and Implementation Analysis
Grantee Interview Protocol	Grantee Administrator	6	1	2	12	N/A	$37.45	$449
Impact Study
Baseline survey of impact study participants	Participating program females and control group females	673	1	0.5	337	67	$7.25	$485.75
Estimated Total Annual Burden					349		$934.75

A.13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

These information collection activities do not place any capital cost or cost of maintaining requirements on respondents.

A.14. Annualized Cost to Federal Government

Design and Implementation Analysis. Data collection will be carried out by Mathematica Policy Research as part of the PAF FADS. The annual and total cost for completion of the Design and Implementation Analysis is $270,765. This includes the cost of coordination between Mathematica and OAH, OMB applications, protocol development, conducting interviews

Impact Study. The federal government has contracted with Mathematica Policy Research for a pilot of the baseline survey under the PAF FADS. The annual and total cost for the baseline data collection under the PAF FADS is $322,632. This includes the cost of coordination between Mathematica and OAH, OMB applications, survey development, and survey administration.

If this proposed ICR is approved, the total cost to the federal government for this part of the PAF FADS is $270,765 + $322,632 = $593,397.

A.15. Explanation for Program Changes or Adjustments

There are no program changes or adjustments at this time.

A.16. Plans for Tabulation and Publication and Project Time Schedule

1. Analysis Plan

Design and Implementation Analysis. The Design and Implementation Analysis is a broad, descriptive study documenting the design and implementation across the 17 PAF grantees. As part of this study, Mathematica will review all states’ applications for funding, as well as other documents available to OAH (such as grantees’ implementation plans), and has engaged in clarifying conversations with federal staff. The next step—which is the focus of this information collection request—will be to conduct interviews with an administrator for each grantee, focused on the overall design and implementation of their PAF programs.

The report produced based on the data collection effort will document the general design and implementation of the PAF grant programs. The data will be reported in two ways: First, a summary profile will be created for each state that will contain PAF program design decision facts – for example, the annualized PAF grant amount, the identification of the grantee organization, the mechanism for distributing funds, the total funding given to program providers, the programs funded, target populations, implementation settings, and the number of youth serviced. This information will be reported for each PAF grantee. Second, the reports produced will discuss themes emerging from responses regarding “how” and “why” PAF program decisions were made, the degree to which state PAF program plans have been implemented, what any challenges that may have emerged during early program implementation.

Impact Study. Data from the baseline survey will be used for two initial purposes. First, OAH will use the data to describe the study sample. This step will enable OAH to compare the characteristics of youth in the study with youth nationwide and provide guidance on how the study sample and findings might generalize to a broader policy setting. Second, OAH will assess whether random assignment resulted in similar baseline characteristics of youth, on average, for the treatment and control groups.

Ultimately, the baseline data will also be used in estimating program impacts on youth outcomes. The program impact estimates will rely primarily on data from the two planned follow-up surveys, which OAH will submit for OMB approval later after site selection has progressed. With a random assignment design, unbiased impact estimates can be obtained by comparing mean outcomes for the treatment and control group based on follow-up data alone. However, we can improve precision of impact estimates by controlling in our regression model for baseline covariates, especially baseline measures of outcomes. Regression adjustment can also address any differences between the treatment and control groups in baseline characteristics that arose by chance or from survey nonresponse. Baseline data will also be used for subgroup analysis, to assess whether program impacts vary by baseline characteristics.

A detailed analysis plan is found in Attachment L.

2. Time Schedule and Publications

OAH expects that the PAF evaluation will be conducted over a five-year period, beginning with the PAF FADS. This request is for a three year period and subsequent packages will be submitted as necessary for new collections, or to extend collection periods. Below is a schedule of the data collection efforts for Design and Implementation Analysis and the Baseline Survey for the Impacts Analysis, the focus for this ICR:

Instrument	Date of 30-Day Submission	Date Clearance Needed	Date for Use in Field
Design and Implementation Analysis
Grantee protocol	April 2014	September 2014	October 2014
Impact and In-Depth Implementation Study
Baseline survey	April 2014	September 2014	October 2014

Design and Implementation Analysis. OAH anticipates one report detailing findings from the 17 PAF grantee interviews. The report shall detail how grantees have designed their PAF programs and how these programs will implemented. The report will also contain a profile of PAF program facts for each grantee. The report is expected in 2015.

Impact Study. OAH expects one or more sites to begin enrolling young women and administering baseline surveys in December 2014. Printing and preparation is expected to begin in October 2014. The other site may begin later (for a total of two sites), and because OAH plans to analyze each site separately (discussed in Section A.3); it is acceptable for the data collection schedule to vary across sites. The current project schedule assumes that all sites will begin enrolling young women and administering baseline surveys by December 2014. To generate sufficient sample sizes for the impact study, the project schedule allows for sample enrollment to continue for up to three years, through 2017. No separate publications are planned for the baseline survey data. The data will be used for internal descriptive and analytic purposes.

A.17. Reason(s) Display of OMB Expiration Date is Inappropriate

All instruments and consent and assent forms will display the OMB Control Number and expiration date.

A.18. Exceptions to Certification for Paperwork Reduction Act Submissions

No exceptions are necessary for this information collection.

SUPPORTING REFERENCES FOR INCLUSION OF SENSITIVE
QUESTIONS OR GROUPS OF QUESTIONS

Boyer, Cherrie B., Jeanne M. Tschann, and Mary-Ann Shafer. "Predictors of Risk for Sexually Transmitted Diseases in Ninth Grade Urban High School Students." Journal of Adolescent Research, vol. 14, no. 4, 1999, pp. 448-65.

Dermen, K. H., M. L. Cooper, and V. B. Agocha. "Sex-Related Alcohol Expectancies as Moderators of the Relationship between Alcohol use and Risky Sex in Adolescents." Journal of Studies on Alcohol, vol. 59, no. 1, 1998, pp. 71.

Fergusson, David M. and Michael T. Lynskey. "Alcohol Misuse and Adolescent Sexual Behaviors and Risk Taking." Pediatrics, vol. 98, no. 1, 1996, pp. 91.

Li, Xiaoming, Bonita Stanton, Lesley Cottrell, James Burns, Robert Pack, and Linda Kaljee. "Patterns of Initiation of Sex and Drug-Related Activities among Urban Low-Income African-American Adolescents." Journal of Adolescent Health: Official Publication of the Society for Adolescent Medicine, vol. 28, no. 1, 2001, pp. 46.

Santelli, John S., Leah Robin, Nancy D. Brener, and Richard Lowry. "Timing of Alcohol and Other Drug use and Sexual Risk Behaviors among Unmarried Adolescents and Young Adults." Family Planning Perspectives, vol. 33, no. 5, 2001.

Sen, Bisakha. "Does Alcohol-use Increase the Risk of Sexual Intercourse among Adolescents? Evidence from the NLSY97." Journal of Health Economics, vol. 21, no. 6, 2002, pp. 1085.

Tapert, Susan F., Gregory A. Aarons, Georganna R. Sedlar, and Sandra A. Brown. "Adolescent Substance use and Sexual Risk-Taking Behavior." Journal of Adolescent Health : Official Publication of the Society for Adolescent Medicine., vol. 28, n3, 2001, pp.181.

www.mathematica-mpr.com

Improving public well-being by conducting high quality, objective research and surveys

Princeton, NJ ■ Ann Arbor, MI ■ Cambridge, MA ■ Chicago, IL ■ Oakland, CA ■ Washington, DC

Mathematica® is a registered trademark of Mathematica Policy Research

1 See http://www.gpo.gov/fdsys/pkg/PLAW-111publ148/html/PLAW-111publ148.htm; Section 10212.

2 ACF received initial OMB approval for the PPA baseline survey on July 26, 2010 (OMB Control Number 0970-0360). In summer 2011, oversight of PPA was transferred to the Office of Adolescent Health (OAH) within the Office of the Assistant Secretary, and the project is now tracked with a different OMB Control Number (0990-0382). The OMB Control Number for the Teen Pregnancy Prevention Replication Study is 0990-0394. OMB approval for the PREP baseline survey was received on March 12, 2013 (OMB Control Number 0970-0398).