PPA OMB Package - Supporting Statement Part A - 9-8-08

Evaluation of Pregnancy Prevention Approaches

(PPA)

Supporting Statement

for OMB Clearance Request

PART A

Submitted by

U.S. Department of Health and Human Services

Administration for Children and Families

Office of Planning, Research, and Evaluation

7^th floor, West, Aerospace Building

370 L’Enfant Promenade, SW

Washington, DC 20447

Project Officer:

Seth Chamberlain

Part A

Justification

This section provides support for the collection of information in the Pregnancy Prevention Approaches (PPA) demonstration and evaluation. This study will be carried out in two steps:

1. informal interviews with stakeholders – including researchers, policy experts, coordinators (e.g. state-level coordinators), programs (including directors, staff, and focus groups with participants), and school administrators – about interventions to prevent pregnancy, in order to identify and select approximately eight sites to be included in the study; and

2. randomization of individuals (or classrooms, or schools, etc.) to intervention and control groups in the selected study sites, data collection, analysis, and reporting.

This submission seeks clearance for the first step of information collection activities. A subsequent OMB submission will seek clearance for second step.

A1. Circumstances Making the Collection of Information Necessary

The Administration for Children and Families (ACF) in the U.S. Department of Health and Human Services (HHS) seeks to identify and evaluate promising approaches to reduce teen pregnancy. Preliminary data for 2005 show that teen birth rates have recently increased after several years of decline. The negative outcomes associated with teen pregnancy for both parents and their children are well documented in the research literature. The purpose of this data collection is to provide information on pregnancy prevention approaches, so that the contractor may make well-informed recommendations, and ACF may make well-informed decisions, as to the best programs to include in this evaluation. The PPA evaluation will utilize an experimental research design, involving treatment and control groups, to assess the effectiveness of a range of promising approaches to pregnancy prevention among high school age teens.

To accomplish this objective, ACF seeks approval of the proposed discussion guides (including a focus group guide) to be used in telephone and in-person informal discussions with a range of experts and stakeholders such as researchers, policy experts, coordinators (e.g. state-level coordinators), pregnancy prevention program directors and staff, and, as appropriate, program participants and school administrators.

The goal of the Pregnancy Prevention Approaches (PPA) evaluation is to determine the effectiveness of these approaches in affecting key outcomes related to pregnancy prevention (e.g. sexual debut, pregnancy, STD infection, etc.). Ultimately, the purpose of the evaluation is to provide stakeholders – including practitioners and federal and other policymakers – with sound scientific findings on a range of existing and promising approaches to preventing teen pregnancy. This section provides an overview of the overall study and discusses its objectives and the need for the proposed information collection.

Legal or Administrative Requirements that Necessitate the Collection

Public Law 110-161, which set FY 08 appropriations levels, included the following language: “$4,500,000 shall be available from amounts available under section 241 of the Public Health Service Act to carry out evaluations (including longitudinal evaluations) of adolescent pregnancy prevention approaches.” The current proposed collection will inform the site selection process of the PPA evaluation.

Study Objectives

As noted above, the overall objective of the PPA is to test selected promising approaches to preventing teen pregnancy among high school age teens. For decades, policy makers as well as the general public have remained concerned about the prevalence of sexual intercourse among adolescents. Although adolescents today are waiting somewhat longer before having sex than they did in the 1990s, 60 percent of teenage girls and more than 50 percent of teenage boys report having had sexual intercourse by their 18^th birthday.¹ Approximately one in five adolescents has had sexual intercourse before his or her 15th birthday.² While rates of teenage pregnancy declined by 38 percent from 1990 to 2004, the rate of teen births followed a similar decline³ until recently, when the rate of teen births rose by 3% from 2005 to 2006. The adverse outcomes associated with teenage sexual activity remain a major concern. Approximately 80 percent of teen pregnancies are unplanned.⁴ While the direction of causality is not always clear, for those who give birth teenage motherhood is associated with a number of adverse conditions. Teen mothers are less likely to complete school and more likely to have large families and to be single parents.⁵ They are also more likely to live in poverty—over 60 percent live in poverty at the time their first child is born, and over 80 percent live in poverty at some point in their lives.⁶ The children of teenage mothers are also likely to contend with a number of hardships. Compared with infants of older mothers, children born to teens are more likely to be premature and underweight; they have a higher mortality rate and a higher rate of illness and injuries. On average, they live in homes that are of poorer overall quality, and they are likely to suffer higher rates of abuse and neglect.⁷ Teen parents and their children are less likely to graduate from high school. In fact, less than four of 10 teen mothers who begin their families before age 18 ever complete high school.⁸ Lastly, the costs of teen pregnancy and childbirth to the nation are substantial. According to a 2006 report from the National Campaign to Prevent Teen Pregnancy,⁹ even after a one-third decline in the United States (U.S.) teen birth rate since the 1990s, teen childbearing in the U.S. still cost taxpayers (Federal, State and local) at least $9.1 billion in 2004. The estimated cumulative public costs of teen childbearing between 1991 and 2004 total $161 billion.

From 1999 to 2007, the federal Abstinence Education program funded an evaluation of programs funded by Title V, Section 510 of the Social Security Act; this evaluation was overseen by the Office of the Assistant Secretary for Planning and Evaluation (ASPE) in HHS and was carried out by Mathematica Policy Research, Inc. This study concluded that students who participated in the four abstinence education programs evaluated were no more likely to abstain from sexual intercourse than those who received services as usual (i.e. the control group). The study also found that students who participated in the abstinence education programs were no more likely to engage in unprotected sex than those that received services as usual.¹⁰

The current study would build on the Mathematica study in at least three ways. First, the sites selected for the Mathematica study focused on elementary and middle school-age youth; the sites selected for the current PPA study will focus primarily on high school-age youth. Second, the Mathematica study focused solely on programs funded through Title V, Section 510, often called “abstinence-only” or “abstinence-until-marriage” (AUM) programs; the current PPA study will focus on a range of programs whose goal is to reduce teen pregnancy, including AUM and other programs such as abstinence-based education (programs which focus on abstinence but also include other elements of sex education), comprehensive sex education (programs which instruct students on safer sex practices and abstinence) and STD/HIV education and prevention programs. Lastly, the Mathematica study involved four sites; ACF expects to select up to eight sites for this evaluation.

Although the current data collection will be used solely for site selection, the research questions that ACF will attempt to answer after site selection bear mention. The questions may be divided into two groups:

• Questions on program implementation: What are the interventions (their settings, their content, the manner in which they are delivered, etc.)? What are the goals and objectives of the strategies and what attitudes, knowledge, and behaviors do they intend to affect? What are the characteristics of the populations served? What are the issues and challenges associated with implementing and operating each approach?

• Questions on program impact (i.e. effectiveness): Are these approaches effective in meeting their objectives? Are these approaches effective in reducing adolescent pregnancy? What are their effects on related outcomes such as postponing sexual activity and reducing or preventing sexual risk behaviors and STDs? Do these approaches work better for some groups of adolescents?

The study’s design and activities also bear mention (although, again, the current collection will be used solely for site selection). The study will use an experimental design in approximately eight sites to compare outcomes for randomly assigned treatment and control group members (note that random assignment may be at the individual, classroom, school, or other level). ACF is interested in evaluating programs/strategies that are fairly intensive, involving participants over a period of time (e.g., curriculum covering one or more semesters, sequenced courses provided during different years in high school, year-long community programs, etc.). Major study activities will include:

• Identifying promising strategies and programs both through review of the literature as well as interviews with the field (i.e. researchers and policy experts, programs, and, as appropriate, participants and school administrators) in order to recommend which types of interventions are of the highest interest to the field and show the most promise for achieving reduced rates of teen sexual activity and pregnancy. This clearance request is specifically for making contacts with the field.

• Recruiting sites to participate in the demonstration and evaluation of the selected interventions, and providing technical guidance on evaluation support activities.

• Collecting data on the research sample at baseline and at follow-up, i.e. post-random assignment and post-treatment (the specific timing of follow-up may be site-dependent).

• Analyzing data collected and preparing reports with the results.

This OMB submission seeks clearance for the information collection activities from key stakeholders (i.e. the first bullet above). A subsequent submission will seek clearance for information collection activities related to the later activities.

A2. Purpose and Use of the Information Collection

The purpose of this information collection is to help ACF identify and select teen pregnancy prevention intervention programs for evaluation. The information gathered will be used by contractor staff to make recommendations to ACF about interventions to be considered for inclusion in the evaluation and to focus the selection process on the programs that are of most interest to practitioners and policymakers. Information collected will also be used to identify major research themes and questions that are most pressing to the field that the PPA evaluation can answer.

Program/intervention selection for the PPA evaluation will begin with the review of existing documentation on current programs. Shortly thereafter, the process will necessitate the engagement of key stakeholders via the proposed guides/protocols. The intervention identification process must engage stakeholders broadly to ensure that the most promising and relevant interventions have been identified. We propose to engage in discussions and informal interviews the following groups of stakeholders:

• Researchers, Policy Experts, and Coordinators—individuals who study and/or are experts on issues related to pregnancy prevention – including experts in the fields of abstinence education, comprehensive sex education, STD/HIV education and prevention, positive youth development, etc.; policy analysts and policy makers who are instrumental in applying research on pregnancy prevention approaches to inform and, in some cases, craft public policies related to such approaches; and state-level or other coordinators;

• Practitioners—directors or staff of pregnancy prevention programs, including school- and community-based programs as well as local and state agencies, as appropriate; we also propose to observe program activities, e.g. classroom activities, as appropriate;

• As appropriate, participants—students or youth (or, in some cases, parents) who participate in programs: these discussions will include focus groups.

• As appropriate, school administrators—individuals who coordinate, oversee, or otherwise work with pregnancy prevention programs within educational settings.

Appendices A through E present the proposed information collection instruments. A combined assent/consent form for the focus group participants is presented in Appendix F.

A3. Use of Improved Information Technology and Burden Reduction

The information will be collected through informal discussions (or, in the case of program participants, focus groups) that are not conducive to information technology, such as computerized interviewing.

A4. Efforts to Identify Duplication and Use of Similar Information

The information collection requirements for activities in this step of the evaluation have been carefully reviewed to determine what information is already available, and what will need to be collected. Literature reviews and recent studies have been and will continue to be consulted (and, where necessary, additional review will be conducted) to assess what is currently known about pregnancy prevention programs. For example, a descriptive study currently underway, Exploration of Approaches to Preventing Sexual Risk Behaviors among High School Students,¹¹ is expected to provide useful information for this study: this is a descriptive study that is documenting the range of programs designed to reduce teen pregnancy and developing a classification system to categorize those approaches. This exploratory study will also include an implementation evaluation on a subset of individual programs. All relevant information from the exploratory study will be used and not duplicated. While the information from previous studies as well as that produced under the exploratory study described above can be expected to provide very useful information, they do not cover the full range of experts and stakeholders. Gaps in the knowledge base require further discussions. Discussions related to any programs included in the exploration study will build on, not duplicate, information already available and will focus on missing and supplementary information. No existing data sources can provide this complete information.

A5. Impact on Small Businesses or Other Small Entities

Not applicable. No small businesses are expected to be involved.

A6. Consequences of Collecting Information Less Frequently

During this step of the evaluation, information will be collected only once, thus no repetition of effort is planned. Not collecting the information at all would substantially limit the value of the investment ACF will make in this study. Identifying interventions of most interest to the field is crucial to ensuring that findings from the study are relevant to federal, state and local policymakers and program administrators.

A7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances for the proposed data collection.

A8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13 and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on June 25, 2008, Volume 73, Number 123, page 36088, and provided a sixty-day period for public comment. A copy of this notice is attached as Appendix G.

During the notice and comment period, the government received 2 requests for copies of the instruments. All requests were fulfilled. One request was followed by the question, “Is this study funded by ACF?” The requester was directed to the RFP associated with the study.

The government also received one letter from a think tank (the Guttmacher Institute) which “urged ACF to reach out to a wide range of pregnancy prevention programs and experts.” A reply letter was sent that confirmed that we will reach out to a wide range of programs and experts. The letter, and response to the letter, are included in Appendix H.

No further public comments on the proposed information collection instruments were received.

A discussion guide for program administrators and staff, researchers, and policymakers, developed for ACF by Abt Associates Inc., for the Innovative Strategies for Increasing Self-Sufficiency (ISIS) evaluation project, was consulted in the development of the proposed information collection. This instrument was approved by OMB¹² and is being effectively used to gather information to improve site selection for the ISIS project. The proposed information collection reflects adapted elements of the ISIS protocols to a pregnancy prevention perspective.

Instruments were drafted by staff in the Office of Planning, Research, and Evaluation in ACF, and further developed and reviewed by staff from the Family and Youth Services Bureau (Division of Abstinence Education) in ACF; the Office of Planning, Research, and Evaluation in ACF; and the Office of the HHS Assistant Secretary for Planning and Evaluation (ASPE).

A9. Explanation of Any Payment or Gift to Respondents

No payments to respondents are proposed for this information collection.

A10. Assurance of Confidentiality Provided to Respondents

For informal discussions, no personal identifying information beyond name and professional affiliation (e.g., name of the academic/research institution, name of the school or program, etc.) will be sought. Discussants will be told that their conversations are not confidential, but that it is expected that their name and affiliation will only be included in summary information provided to ACF. ACF staff may participate in telephone or on-site discussions. Discussants will be told that, to the extent allowable by law, individual identifying information will not be disseminated publicly.

For focus groups, consent forms will be collected, and no personal identifying information beyond name will be sought. The consent form will instruct participants that, beyond the consent form, no identifying information will be collected, and during the focus group, answers will not be connected with any identifying information. The participants will be told that their answers will be kept confidential to the greatest extent allowable by law.

At this time, ACF does not anticipate that a report for public dissemination will be produced using the information obtained. ACF anticipates that the information gathered through the proposed information collection will be used only for internal purposes related to program/site selection for the evaluation.

A11. Justification for Sensitive Questions

There are no personally sensitive questions in this data collection.

A12. Estimates of Annualized Burden Hours and Costs

This proposed information collection does not impose a financial burden on respondents. Respondents will not incur any expenses other than the time spent answering the questions contained in the discussion guides.

Exhibit A12.1 summarizes the reporting burden on respondents. Response times were estimated from informal pre-tests with ACF staff and prior experience. The annual burden is estimated from the total number of completed discussions proposed and the time required to complete the discussions. The total annual burden is expected to be 388 hours. Note that data collection is expected to occur over two years, so the total burden will be double the annual burden.

Exhibit A12.1 Annual Information Collection Burden Estimate
Instrument	Number of Respondents	Number of Responses Per Respondent	Average Burden Hours Per Response	Total Burden Hours
Discussion Guide for Use with Researchers, Policy Experts, and Macro-Level Coordinators	50	1	1	50
Discussion Guide for Use with Program Directors	25	1	2.5	63
Discussion Guide for Use with Program Staff	50	1	2	100
Focus Group Discussion Guide for Use with Program Participants	100	1	1.5	150
Discussion Guide for Use with School Administrators	25	1	1	25
Total	250	-	-	388

Estimates of Annualized Costs

Survey respondents will be researchers, policy experts, coordinators, program directors, program staff, participants, and school administrators. To compute the total estimated annual cost, the total burden hours were multiplied by the average hourly wage, according to the closest professional grouping in the National Compensation Survey, 2005. The total estimated annual cost is $6,496.¹³

Exhibit A12.2 Annual Information Collection Costs Estimate
Instrument		Total Burden Hours	Average Hourly Wage	Total Annual Cost
Discussion Guide for Use with Researchers, Policy Experts, and Macro-Level Coordinators		50	$28.99 (wage for “Social Scientists”)	$1,450
Discussion Guide for Use with Program Directors		63	$30.55 (wage for “Managers, Service Organizations”)	$1,925
Discussion Guide for Use with Program Staff		100	$19.13 (wage for “Social Workers”)	$1,913
Focus Group Discussion Guide for Use with Program Participants	Students	135	$0.00	$0
	Parents	15	$18.62 (wage for “All Occupations”)	$279
Discussion Guide for Use with School Administrators		25	$37.21 (wage for “Administrators, Education and Related Fields”)	$930
Total		338	-	$6,49614

A13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

Not applicable. These information collection activities do not place any capital cost or cost of maintaining capital requirements on respondents.

A14. Annualized Cost to the Federal Government

Instruments were drafted by staff in the Office of Planning, Research, and Evaluation in ACF, and further developed and reviewed by staff from the Office of Abstinence Education in ACF, the Office of Planning, Research, and Evaluation in ACF, and the Office of the HHS Assistant Secretary for Planning and Evaluation (ASPE). Costs of federal government employees’ time is estimated at $1,925, per the following table:

Exhibit A14.1 Costs of Creating Data Collection Instruments Estimate
Government Employee	Description of Work	Total Hours Estimated	GS-Rate	Estimated Cost
Research Analyst	Drafting and Revising Instruments	40	GS-13: $32.88	$1,315
Program Analyst	Reviewing Instruments	2	GS-12: $27.65	$55
(Another) Research Analyst	Reviewing Instruments	2	GS-13: $32.88	$66
Program Director	Reviewing Instruments	2	GS-14: $38.86	$78
Division Director	Reviewing Instruments	2	GS-15: $45.71	$91
Senior Research Analyst	Reviewing Instruments	6	GS-15: $45.71	$274
Office Director	Reviewing Instruments	1	GS-15: $45.71	$46
Total	-	55	-	$1,925

Data collection will be carried out by the contractor selected through the competitive procurement process to contract the Pregnancy Prevention Approaches evaluation. Information collection may continue for up to two years after award of contract, though most information is expected to be obtained in the first year. Total contractor costs associated with the information collection, compilation, analysis and reporting to ACF have yet to be determined. Based on our experience with other similar information collection activities carried out by contractors, total costs to the government are estimated to be approximately $190,080.¹⁵ Combined with the costs estimated in the table above, total costs are estimated to be $192,005. Since information collection will occur over two years, annualized costs are estimated to be $96,001.

A15. Explanation for Program Changes or Adjustments

This submission to OMB is a new request for approval.

A16. Plans for Tabulation and Publication and Project Time Schedule

A16.1 Analysis Plan

This phase of the Pregnancy Prevention Approaches (PPA) demonstration and evaluation involves collecting information that will be used for selecting sites, specifically identifying promising approaches and selecting the interventions and demonstration programs for the study. As discussed in questions A1 and A2 above, the PPA evaluation is designed to test promising approaches to preventing pregnancy among high school age teens and, specifically, testing approaches that are of greatest interest to key stakeholders. This requires a strong intervention identification and development process, involving researchers, policy experts, coordinators, programs (directors, staff, and participants), and, where appropriate, school administrators.

The informal discussions that will be conducted as part of this data collection will be used for internal study purposes, to help inform the types of interventions the study should examine and to assist in the identification and selection of demonstration sites. Individual summaries of all discussions and overarching summaries of information obtained from each source/type of stakeholder (e.g., researchers, policymakers, program administrators and staff) shall be prepared by the contractor. The information will be analyzed to identify common themes and specific recommendations of types of approaches to be evaluated and specific programs to be considered.

The information obtained in the interviews will be combined with information obtained from the reviews of existing research (and, as appropriate, from attendance at relevant conferences and meetings) to produce initial recommendations regarding intervention strategy and, as possible, site recommendations.

A16.2 Time Schedule and Publications

This study is expected to be conducted over a five-year period beginning September 30, 2008. This OMB submission seeks approval for information collection as part of initial activities of the study, which will occur in the first two years of the project. No formal publications are planned from this information collection.

A17. Reason(s) Display of OMB Expiration Date is Inappropriate

All instruments will display the expiration date for OMB approval.

A18. Exceptions to Certification for Paperwork Reduction Act Submissions

No exceptions are necessary for this information collection.

1 Abma JC et al., Teenagers in the United States: sexual activity, contraceptive use, and childbearing, 2002, Vital and Health Statistics, 2004, Series 23, No. 24.

2 Albert B, Brown S, Flannigan C, eds. 14 and Younger: The Sexual Behavior of Young Adolescents. Washington, DC: National Campaign to Prevent Teen Pregnancy; 2003.

3 Teen birth rates declined by 34% from 1991-2005. See: Hamilton BE, Martin JA, Ventura SJ. Births: Preliminary data for 2006. National vital statistics reports; vol 56 no 7. Hyattsville, MD: National Center for Health Statistics. 2007..

4 Finer, Lawrence B & Stanley K Henshaw. “Disparities in Rates of Unintended Pregnancy In the United States, 1994 and 2001”. Perspectives on Sexual and Reproductive Health, 38(2), 90-6. (2006)

5 Brown, SS, & Eisenberg, L. (Eds). The best intentions: unintended pregnancy and the well-being of children and families. Washington, DC: National Academy Press. (1995)

6 Maynard, R. (ed). Kids Having Kids.: Economic costs and social consequences of teen pregnancy. Washington, D.C.: The Urban Institute. (1996)

7 George, RM, & Lee, BJ. Abuse and neglect of children. In R.A. Maynard (Ed.), Kids Having Kids: Economic costs and social consequences of teen pregnancy (pp 205-230). Washington, D.C.: The Urban Institute. (1996)

8 Holtz, V, McElroy S, & Saunders, S. The impacts of teenage childbearing on mothers and the consequences of those impacts for the government. In RA Maynard (Ed.), Kids having kids: Economic costs and social consequences of teenage pregnancy (pp. 55-95). Washington, DC: Urban Institute Press. (1997)

9 Hoffman, S.  By the Numbers: The Public Costs of Teen Childbearing.  Washington, DC: National Campaign to Prevent Teen Pregnancy, 2006.

10 Trenholm, C., et al. (2007.) Impacts of Four Title V, Section 510 Abstinence Education Programs: Final Report. Retrieved on June 2, 2008 from http://www.mathematica-mpr.com/publications/PDFs/impactabstinence.pdf

11 This study is currently underway; Abt Associates Inc. is conducting the study, under contract to the Office of the HHS Assistant Secretary for Planning and Evaluation (ASPE). Draft materials will be shared with the yet-to-be-determined winner of the Pregnancy Prevention Approaches contract.

12 The OMB number is 0970-0343.

13 Source: Bureau of Labor Statistics, National Compensation Survey, 2005. Retrieved on July 31, 2008 from http://www.bls.gov/ncs/ocs/

14 Total may slightly differ due to rounding.

15 This estimate was calculated using the costs of a similar effort: discussions with experts as part of the Innovative Strategies for Increasing Self-Sufficiency project. The estimated costs of that effort were $152,064. Since that project anticipated discussions with 400 persons – but this project with 500 – the total costs were increased by 25%.