SSA Working with Public Centers Teen Preg ICRO Final

Performance Monitoring of “Working with Publicly Funded Health Centers to Reduce Teen Pregnancy among Youth from Vulnerable Populations”

OMB Control Number 0920-1156

Revision Request

Supporting Statement

Part A

December 2, 2019

Contact Person:

Heather Tevendale, PhD

Deputy Associate Director for Science

Telephone: 770-488-6147

Fax: 770-488-6291

E-mail: [email protected]

Division of Reproductive Health

National Center for Chronic Disease Prevention and Health Promotion

Centers for Disease Control and Prevention

Atlanta, Georgia

CONTENTS

List of Attachments……………………………………………………………………………………………………………………………………………………………3

A. Justification

1. Circumstances Making the Collection of Information Necessary…………………………………………………………...5

2. Purpose and Use of the Information Collection……………………………………………………………………………………..7

3. Use of Improved Information Technology and Burden Reduction……………………………………………………….…9

4. Efforts to Identify Duplication and Use of Similar Information…………………………………………………………..…10

5. Impact on Small Businesses or Other Small Entities……………………………………………………………………………..10

6. Consequences of Collecting the Information Less Frequently…………………………………………………………..….10

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5……………………………………………………….12

8. Comments in Response to the Federal Registrar Notice and Efforts to Consult Outside the Agency….…12

9. Explanation of any Payment or Gift to Respondents…………………………………………………………………………….13

10. Protection of the Privacy and Confidentiality of Information Provided by Respondents……………………...13

11. Institutional Review Board and Justification for Sensitive Questions……………………………………………………16

12. Estimates of Annualized Burden Hours and Costs………………………………………………………………………………..17

13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers……………………………….20

14. Annualized Cost to the Government…………………………………………………………………………………………………...20

15. Explanation for Program Changes or Adjustments……………………………………………………………………………….21

16. Plans for Tabulation and Publication and Project Time Schedule……….………………………………………………..21

17. Reason(s) Display of OMB Expiration Date Is Inappropriate……………………..………………………………………….22

18. Exceptions to Certification for Paperwork Reduction Act submissions…………………………………………………22

Attachments

Attachment 1. Section 301 of the Public Health Service Act (42 U.S.C 241)

Attachment 2: Crosswalk of Objectives and Data Sources

Attachment 3a: Health Center Organizational Assessment, paper

Attachment 3b: Health Center Organizational Assessment, online

Attachment 4a: Health Center Provider Survey, paper

Attachment 4b: Health Center Provider Survey, online

Attachment 5a: Quarterly Health Center Performance Measure Reporting Tool, paper

Attachment 5b: Quarterly Health Center Performance Measure Reporting Tool, online

Attachment 6a: Annual Health Center Performance Measure Reporting Tool, paper

Attachment 6b: Annual Health Center Performance Measure Reporting Tool, online

Attachment 7: Awardee Training and Technical Assistance Tool, Excel

Attachment 8a: Awardee Performance Measure Reporting Tool, paper

Attachment 8b: Awardee Performance Measure Reporting Tool, online

Attachment 9: Project Logic Model

Attachment 10a: 60-Day Federal Register Notice

Attachment 10b: Response to Public Comments on the 60-Day Federal Register Notice

Goal of the Study: To continue to evaluate and improve efforts of three organizations under the cooperative agreement “Working with Publicly Funded Health Centers to Reduce Teen Pregnancy among Vulnerable Populations” that work with health centers to enhance implementation of best practices in adolescent reproductive health care. As part of the cooperative agreement, the three organizations also have worked with youth-serving organizations (YSOs) to develop systematic approaches to referring youth at risk for a teen pregnancy to reproductive health care.

Intended use: Information collected will be used to continue monitoring awardee performance and determine training and technical assistance needs to address any performance issues.

Methods to Collect: The three awardee organizations have been planning, coordinating, and leading efforts to improve access to high quality reproductive health care in collaboration with approximately 25 health centers in total. Reflecting expected changes in policies and provider practices, awardees will continue to conduct assessments of health center organizations, as well as surveys of providers at the health center partner organizations. The three awardees also will continue to report on their own activities including training and technical assistance provided to partners.

Subpopulations:

Subpopulations include health center and YSO partner organizations participating in the cooperative agreement “Working with Publicly Funded Health Centers to Reduce Teen Pregnancy among Vulnerable Populations,” along with their staff members and youth receiving services.

How the data will be analyzed: Analyses will consist of summary statistics and paired t-tests

comparing change from baseline to each annual follow-up assessment.

1. Justification

1. Circumstances Making the Collection of Information Necessary

The Centers for Disease Control and Prevention (CDC), Division of Reproductive Health (DRH) is requesting a Revision for the currently approved information collection Performance Monitoring of “Working with Publicly Funded Health Centers to Reduce Teen Pregnancy among Youth from Vulnerable Populations” (OMB No. 0920-1156, Exp. 1/31/2020); this submission requests an extension to 9/30/2020. Three awardees were funded under the “Working with Publicly Funded Health Centers to Reduce Teen Pregnancy among Vulnerable Populations” cooperative agreement (DP15-1508). The information collection has been and will continue to be used for performance monitoring and improvement of the three funded projects. An extension is requested in order to allow for information collection through the end of project funding (9/30/2020).

Although the 2017 US rate of 18.8 births per 1,000 female teens aged 15-19 represents a continued decline [1], the United States has one of the highest teen birth rates of all Western industrialized countries [2]. Teen pregnancy and childbearing are estimated to cost tax payers approximately $11 billion each year [3]. Furthermore, significant racial/ethnic and geographic disparities in teen birth rates persist.

Access to reproductive health services and the most effective types of contraception have been shown to reduce the likelihood that teens become pregnant [4-6]. Nevertheless, recent research [6] and lessons learned through a previous teen pregnancy prevention project implemented through CDC in partnership with the Office of Adolescent Health (2010-2015; OMB No. 0920-0952, Exp. Date 12/31/2015) demonstrate that many health centers serving adolescents in communities with high teen birth rates are not engaging in youth friendly best practices that may enhance access to care and to the most effective types of contraception. Additionally, youth at highest risk of experiencing a teen pregnancy are often not connected to the reproductive health care that they need, even when they are part of a population that has been identified as being at high risk for a teen pregnancy (e.g., youth in foster care) [7].

An revision of this information collection (OMB No. 0920-1156, Exp. 01/31/2020) is requested to continue program monitoring and management of three funded projects under the cooperative agreement “Working with Publicly Funded Health Centers to Reduce Teen Pregnancy among Vulnerable Populations” until the end of the funding period. The three funded awardees are Mississippi First, Inc.; Sexual Health Initiatives For Teens North Carolina (SHIFT NC); and the Georgia Association for Primary Health Care, Inc. To carry out these projects, each of these three organizations has worked with 6-9 health centers and 6-13 youth serving organizations (YSO) in their target communities. Awardees selected these partners during the first year of the project.

These projects seek to decrease teen pregnancy by:

• Improving the quality of adolescent sexual and reproductive health (ASRH) services at partner health centers.

• Increasing use of ASRH services among sexually active teens in the target communities.

• Increasing the number of sexually active teens at partner health centers who receive effective methods of contraception.

The awardees are seeking to achieve these aims by engaging in three main activities:

• They are working with publicly funded health centers to make organizational changes and engage in clinical provider training to increase health centers’ implementation of best practices in adolescent reproductive health care. The overall goal of these efforts is to improve the quality of and access to youth-friendly reproductive health services.

• They are working with YSOs that serve youth at risk for a teen pregnancy to make organizational changes and provide staff training so that youth at those agencies are assessed for whether they are at risk for a teen pregnancy (i.e., sexual health assessment) and, when indicated, referred to reproductive health care.

• They have developed communication campaigns intended to increase awareness of the partner health centers’ services for teens.

Youth-friendly best practices included in this program are supported by evidence in the literature and recommended by major medical associations, such as the American Academy of Pediatrics, the Society for Adolescent Health and Medicine, and the American College of Obstetricians and Gynecologists. Each of the components of the current project has been implemented as part of past teen pregnancy prevention efforts. Consistent with CDC’s mission of using evidence to improve public health programs, conducting an evaluation of combined best practices, in concert with community-clinical linkage of youth to services to increase their access to reproductive health care, can provide information that will inform future teen pregnancy prevention efforts.

The three awardees planned, coordinated, and led the efforts in collaboration with 25 health centers (6-9 per awardee) and 27 youth serving organizations (6-13 per awardee). Reflecting expected changes to health center and YSO partners’ policies, to staff practices, and to youth health care seeking behaviors, awardees conducted assessments of health center and YSO partners’ organizational factors, staff members, and youth served by the partner organizations.

The continuous quality improvement model used by the health center partners suggests that additional changes in health center policies and practices can reasonably be expected, thus supporting continued health center partner assessments. Given an expectation that additional significant change to YSO policies and practices would not be found in the final year of the project, data collection burden is reduced by not conducting YSO-related assessments during the final year of the project. The Health Center Youth Survey will be conducted late in 2019 and will not be conducted again during the course of the project. Thus, it also is not included in this information collection revision request.

CDC is authorized to conduct this information collection under Section 301 of the Public Health Service Act (42 U.S.C 241) Authority of the Secretary (See Attachment 1). For the purposes of this information collection request, a “respondent” is a “Working with Publicly Funded Health Centers to Reduce Teen Pregnancy among Youth from Vulnerable Populations” awardee or staff at a health center partner.

2. Purpose and Use of Information Collection

Information has been requested from awardees, their health center and YSO partners, as well as the staff and youth served by the health center partners, to evaluate program performance and identify needs for training and technical assistance in support of quality improvement. As part of the grant application process, all awardees were required to propose working in communities with a 2013 teen birth rate above the national average (26.6 births per 1,000 female adolescents ages 15-19) and those communities had to be located in states with a 2013 birth rate above the national average. To carry out these projects, each of these three organizations has worked with 6-9 health centers and 6-13 youth serving organizations in their target communities. Mississippi First, Inc., a non-profit focused on child well-being and educational achievement, was funded to work in Coahoma, Quitman and Tunica counties in Mississippi. Sexual Health Initiatives For Teens North Carolina (SHIFT NC), a non-profit organization focused on the sexual health of adolescents, was funded to work in Durham County, North Carolina. The Georgia Association for Primary Health Care, Inc, which represents all of Georgia’s Federally Qualified Health Centers, was funded to work in Chatham County, Georgia.

To evaluate the progress of awardees and their partner organizations in making structural changes that have been shown to lead to improvements in the quality and use of adolescent sexual and reproductive health (ASRH) services, along with the number of sexually active teens who receive effective methods of contraception, it has been necessary to assess performance on a number of short-term objectives. These short-term objectives include:

1. Improve health center organizational factors (e.g., financing, clinical policies, protocols, practices, medication formulary, appointment scheduling) that support high quality ASRH services

2. Increase health center staff knowledgeable about and support of implementation of evidence-based guidelines for reproductive health services, youth-friendly best practices, and the provision of the most effective types of contraception to sexually active teens

3. Increase frequency with which sexually active youth are screened for pregnancy intention, counseled on the full range of FDA approved methods of birth control, and provided method that youth selects without delay at partner health centers

4. Increase the extent to which ASRH services at partner health centers are provided in a youth-friendly manner

5. Increase in implementation of organizational policies at partner YSOs that support efforts to link youth to SRH services

6. Increase in YSO staff support of efforts to link youth to SRH services

7. Increase in YSO staff knowledge about screening youth to determine need for SRH services and provide referrals

8. Increase in number of youth at partner YSOs receiving referral to SRH services

These short-term objectives and the intermediate project objectives they may lead to are listed in a table in Attachment 2. The table also provides a crosswalk demonstrating how each data collection tool has provided the data necessary for judging whether grantees and their partners are meeting project objectives.

The information already collected from the tools has been summarized by CDC and reported back to the grantees as part of a process to manage program performance and drive a quality improvement process. A data driven quality improvement process is resulting in high quality projects being conducted that make the best possible use of the funds provided to awardees.

For example, in one health center, declines in the number of youth being served were observed in data collected from the Quarterly Performance Measure Reporting Tool. This led the grantee and health center to work together to examine and address scheduling issues that were contributing to the decline in the number of youth served, as well as consider how to modify the flow of adolescents visits (i.e., the order in which providers were seen and procedures were conducted) to increase efficiency and allow more patients to be seen. In another example, when data from a Health Center Youth Survey indicated that some youth at a health center were not receiving time alone to speak with their medical provider, the grantee worked with the health center to develop procedures that made time alone more likely while also developing parent education materials about the value of adolescents being able to speak privately with their provider. Grantees also use their data to motivate change in their communities. For example, one grantee developed one page data briefs with 4-6 data points on a focused topic that they shared with health center providers when working to improve a specific activity (e.g., reporting percent of teens who were screened for pregnancy risk when working to improve the frequency with which providers engage in screening).

The current revision request will allow us to continue measurement of expected changes in health center policies and practices, as well monitor the performance of 3 grantees and approximately 25 health centers (21 private sector and 4 public sector) The data collection tools that will be used during the extension period include a Health Center Organizational Assessment (Attachment 3a and 3b), Health Center Provider Survey (Attachment 4a and 4b), and Quarterly and Annual Health Center Performance Measurement Tools (Attachment 5a, 5b, 6a, and 6b). Two data collection tools are the responsibility of the three awardees funded to establish partnerships with Health Centers and Youth Serving Organizations: the Awardee Training and Technical Assistance Tool (Attachment 7) and the Awardee Performance Measure Reporting Tool (Attachment 8a and 8b).

3. Use of Improved Information Technology and Burden Reduction

The Awardee Training and Technical Assistance Tool (Attachment 7) is an Excel file that awardees can complete electronically in Excel each time they provide training or technical assistance to a partner. At the end of each month, they submit the file via email.

All of the other data collection instruments can be completed though the use of an electronic data collection system and have been programed using SPSS Data Collection Author software. These data collection instruments are available through a dedicated and secure server managed by CDC information technology staff. The data collection instruments are designed with automatic skip patterns depending on the participant’s responses, thus reducing burden on the respondents.

OMB approved paper versions of all of the data collection instruments except the Awardee Training and Technical Assistance Tool (available only in an Excel format) will be maintained for use by grantees and their partners who feel that paper versions of the data collection instruments are easier for them to use. In the current project and a similar prior project, awardees have found that completing the data collection instruments on paper to be more convenient for many of their partners.

All data collection instruments were reviewed multiple times by CDC staff to ensure that all possible approaches were taken to minimize respondent burden. Each data collection instrument was designed to be brief, easy to use, and understandable. The instruments were designed such that questions that are not applicable to a respondent based on an answer to a previous question are passed over via formatting and skip patterns. CDC staff have carefully considered the content, appropriateness and phrasing of each question. Furthermore, awardee staff including those implementing programmatic efforts (e.g., project directors, project coordinators, technical assistance providers) and those evaluating the projects from all three awardees have provided feedback on the data collection instruments. Awardee feedback was incorporated into the instruments to ensure that the instruments meet the needs of the awardees for determining the technical assistance needs of their health center partners and that the burden of collecting the data is kept to an absolute minimum.

4. Efforts to Identify Duplication and Use of Similar Information

Once every month, the evaluation lead for the current project participates in a weekly evaluation workgroup conference call with staff from other HHS agencies to discuss federal evaluations of teen pregnancy prevention efforts. This workgroup involves staff from the Office of Adolescent Health, the Office of the Assistant Secretary for Planning and Evaluation (ASPE), and the Administration for Children and Families, who are evaluating other teen pregnancy prevention efforts. This high level of coordination among agencies assures that federal staff engaged in or supporting evaluations of federal teen pregnancy prevention efforts collaborate when appropriate and assures that there is no duplication of efforts. The proposed evaluation is a unique effort.

A review of the literature indicates singular efforts of evaluating implementation of youth-friendly clinical best practices in reproductive health care; however, this project is unique in its strategy of implementing multiple best practices, in concert with community-clinical linkage of youth from youth-serving organizations to reproductive health care as a strategy to increase access and utilization of services. Thus, we have determined that no similar data is available. Furthermore, given that data are collected to assess the awardees’ performance in implementing a new initiative, previously collected data from other teen pregnancy prevention projects would not offer information about our awardees or their partners’ performance.

5. Impact on Small Business or Other Small Entities

Data will be collected from awardees and their health center partners in the public and private sector. Most health center partners are either part of a larger network of health centers (e.g., federally qualified health centers [FQHC]) or are local health departments; one FQHC has only one site and is fairly small. However, the questions have been held to the absolute minimum required for the intended use of the data.

6. Consequences of Collecting the Information Less Frequently

CDC proposes to collect most data on an annual basis. Most data collection instruments from partner Health Centers, including the Organizational Assessment (Attachments 3a and 3b) and Health Center Provider Survey (Attachments 4a and 4b) are completed annually as the topics they measures are not expected to change as often. Also data are collected annually from the Awardee Performance Measure Reporting Tool (Attachment 8a and 8b). Annual tracking is still important for reaching project goals and collecting this data. CDC relies on the data to understand project progress, as well as lack of progress.

However, data from the Awardee Training and Technical Assistance Tool (Attachment 7) are collected monthly. Given experience on two prior 5-year projects, completing the tool less often would result in unreliable data given that awardees tend to complete the tool at the time that it is due rather than at the time that training or technical assistance is provided. Good recall over more than a month is unlikely, thus completing the tool quarterly would result in incomplete and incorrect information. Therefore, the Awardee Training and Technical Assistance Tool is collected monthly for ongoing continuous monitoring.

Also, health center partners have reported on key performance measures quarterly, using the Quarterly Health Center Performance Measure Tool (Attachment 5a and 5b) for the first three quarters of the year and the Annual Health Center Performance Measure Tool (Attachment 6a and 6b) for the fourth quarter (including data on the fourth quarter and the overall fiscal year). We have found that having a small number of performance measures collected more than annually has allowed for significant issues to be addressed quickly, as opposed to waiting a year to discover and address problems. During the revision period, we plan to use the Quarterly Health Center Performance Measure Tool (Attachment 5a and 5b) to collect data from the first and second quarters of the year. We further plan to use the Annual Health Center Performance Measure Tool (Attachment 6a and 6b) to collect data on the third quarter which would also include year-to-date data. Although we would prefer to have annual data for all of FY 2020, the project ends just prior to the end of the fiscal year and, thus, leaves no time for awardees to work with their partners to collect quarter four or full-year data. The Annual Health Center Performance Measure Reporting Tool includes some unique items (e.g., service provision by race/ethnicity) that would allow us to estimate some changes over the course of the project (e.g., changes in service provision by race/ethnicity from the start to the end of the project). Having a smaller number of performance measures collected and reported on quarterly has allowed for significant issues to be addressed quickly, while still minimizing the burden on awardees and their partners by limiting more detailed data collection to the Annual Health Center Performance Measure Reporting Tool. These tools were designed to be as brief as possible so that the most important indicators of grantee and partner progress are captured, while minimizing the burden on awardees. We also solicited input from all of our awardees on how best to reduce burden. As part of this process and to further reduce burden, we identified items that are sufficient to be completed only annually. Most quality improvement efforts involve more frequent data collection (e.g., monthly), but quarterly seemed to be a reasonable compromise.

The frequency of each data collection instrument for the revision is shown in Table A.2. Without the proposed information collection, CDC’s oversight of activities will be based on anecdotal and nonsystematic information; CDC’s ability to provide technical assistance and recommend corrections will be diminished. Additionally, the awardees would be unable to carry out one of the key components of the project, which is to engage their partners in a data driven continuous quality improvement process.

Table A.2. Frequency of Assessment Tool Use^a

Assessment Tool	Frequency
Health Center Organizational Assessment (Attachment 3a and 3b)	Annually
Health Center Provider Survey (Attachment 4a and 4b)	Annually
Quarterly Health Center Performance Measure Reporting Tool (Attachment 5a and 5b)	2 times per year
Annual Health Center Performance Measure Reporting Tool (Attachment 6a and 6b)	Annually
Awardee Training and Technical Assistance Tool (Attachment 7)	Monthly
Awardee Performance Measure Reporting Tool (Attachment 8a and 8b)	Annually

^aDuring extension period

To fully understand performance as well as the outcome of awardee efforts, CDC needs to understand what training and technical assistance was provided to health center and YSO partners, what organizational changes were made once training and technical assistance was provided, and whether health center providers and YSO staff then change their practices (see Attachment 9 for a logic model displaying how the overall program is expected to work). Tracking change at these multiple levels results in a more complicated data collection than would be necessary if awardees were simply implementing a program with youth and only needed to collect information about the impact on youth. Limiting assessment to only the impact on youth, however, would not provide any information on whether performance issues need to be addressed at the awardee, health center partner, and health center provider levels in order to have the biggest positive impact on youth.

In summary, the negative consequences of not collecting information as proposed would be the limited ability to understand awardee and partner performance and, thus, the limited ability to help awardees improve performance and ensure that awardee funding is well-spent. We also would be unable to examine the extent of changes made by awardees and their partners without collecting follow-up assessment data at the end of the project.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances. This request complies with the regulation of 5 CFR 1320.5.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A 60-day notice was published in the Federal Register on September 5, 2109, Vol. 84, No. 172, pp. 46733-46735 with the title “Performance Monitoring of Working with Publicly Funded Health Centers to Reduce Teen Pregnancy among Youth from Vulnerable Populations” (Attachment 10a). CDC received three comments related to this notice (Attachment 10b). All three comments were sent by individuals outside of CDC. CDC has responded to the comments all of which expressed support for the proposed extension of the data collection and the associated programmatic efforts (Attachment 10b). No changes in the information collection were made in response to these comments.

CDC consulted with the Office of Population Affairs (OPA) on the project framework and the development of data collection instruments. CDC also consulted partners on a different teen pregnancy prevention initiative for their feedback on the development of data collection instruments for this project. As noted previously, CDC has consulted closely with the awardees to this project as to the design of the proposed data collection instruments.

Where possible, CDC has used previously developed instruments from existing CDC- and OPA-developed instruments. Previously developed and tested data collection instruments from “Monitoring Changes in Attitudes and Practices among Family Planning Providers and Clinics” (OMB No. 0920-0969, Exp. 5/31/2014) were used and incorporated into the Health Center Organizational Assessment (Attachment3a and 3b) and Health Center Provider Survey (Attachment 4a and 4b). All data and lessons learned resulting from this project will be shared with OPA and other publicly funded health centers (including FQHCs).

Year Consulted	Name, Title, Agency	Email/Phone #
2016	Emily Feher, MPH, Project Director, Mississippi First	[email protected]/(601) 398-9008 x105
2016	Kathleen Ragsdale, PhD, Associate Research Professor Social Science Research Center, Mississippi State University	[email protected]/662-325-9168
2016	Melissa Reams, Project Coordinator, Georgia Association for Primary Health Care	[email protected]/912-527-1011
2016	Christine Ley, PhD, Evaluator, Georgia Association for Primary Health Care	[email protected]/412-953-9544
2016	Cathy Bowden, Information Management Coordinator, Georgia Association for Primary Health Care	[email protected]/404-270-2178
2016	Joy Sotolongo, Director of Evaluation, SHIFT NC	[email protected]/919-226-1880 ext 102
2016	Sally Swanson, MSW, MPH, Chief Program Officer, SHIFT NC	[email protected]/919-226-1880
2016	Sarah Davis, MPH, Senior Technical Advisor, SHIFT NC	[email protected]/919-226-1880 x114
2016	Lorrie Gavin, Office of Population Affairs	[email protected]/240-453-2826
2016	Dawn Middleton, Cicatelli Associates Inc	[email protected]/212-594-7741
2016	Karen Schlanger, Cicatelli Associates Inc	[email protected]/404-521-2151
2014	Deborah U’huru, NYC Department of Health and Mental Hygiene	[email protected]/718-299-0169
2014	Kristen Plastino, University of Texas Health Science Center at San Antonio	[email protected]/210-567-7036
2014	Michelle Reese, Adolescent Pregnancy Prevention Campaign of North Carolina	[email protected]/919-226-1880

9. Explanation of Any Payment or Gift to Respondents

Not applicable.

10. Protection of the Privacy and Confidentiality of Information Provided by Respondents

Privacy Act Determination

This submission has been reviewed by CDC’s National Center for Chronic Disease Prevention and Health Promotion Information Systems Security Officer who determined that the Privacy Act does not apply. The Privacy Act does not apply because CDC does not collect or receive any information in identifiable form (IIF).

While the Privacy Act is not applicable, the appropriate security controls and rules of behavior will be incorporated to protect the confidentiality of information, proprietary, sensitive, and personally identifiable information (PII) the awardee may come in contact with.

Overview of the Data Collection System

The Awardee Training and Technical Assistance Tool (Attachment 7) is an Excel file that awardees can complete electronically in Excel each time they provide training or technical assistance to a partner. At the end of each month, they submit the file via email.

All of the other tools can be completed though the use of an electronic data collection system. All measures have been programed using SPSS Data Collection Author software. Surveys are available through a dedicated and secure server managed by CDC information technology staff.

OMB-approved paper versions of all of the measures except the Awardee Training and Technical Assistance Tool (available only in an Excel format) are maintained for use by grantees and their partners who feel that paper versions of the measure are easier for them to use. In the current project and a similar prior project, awardees found that completing the measure on paper to be more convenient for many of their partners.

The specific procedures for collecting information using the annual Health Center Provider Survey (Attachment 4a and 4b) are that each health center partner will have a staff member who is identified as the primary contact for the awardee. The awardee contacts that person and asks him/her to provide all involved providers at the health center with a link to the online survey. The providers complete the survey online or, if the health center or providers prefer to use paper surveys, providers are given a paper survey to complete and asked to return the survey in a sealed envelope. The staff member identified as the primary contact submits the anonymous surveys to the awardee. The awardee enters the paper surveys into the online data collection system in order to submit to CDC. The names of those staff or providers are not shared.

The Health Center Organizational Assessment (Attachment 3a and 3b), as well as the Quarterly and Annual Health Center Performance Measure Reporting Tool (Attachments 5a, 5b, 6a and 6b), are the responsibility of the awardee’s primary contact at the health center. The awardee enters the paper surveys into the online data collection system in order to submit to CDC. No individual’s name is recorded on the tools.

The Awardee Training and Technical Assistance Tool (Attachment 7) and the Awardee Performance Measurement Tool (Attachment 8a and 8b) are completed by the three awardees. The project coordinator for each awardee is responsible for seeing that these two tools are completed. The project coordinator or other staff member enters the responses into the online data collection system. No individual’s name is recorded on the tools.

Items of Information to be Collected

The Health Center Provider Survey (Attachment 4a and 4b) asks providers to report on their knowledge of and practices consistent with evidence-based guidelines for reproductive health services, youth-friendly best practices, and the provision of contraception to sexually active teens. The Health Center Organizational Assessment (Attachment 3a and 3b) examines health center policies, procedures, and practices associated with providing reproductive health care to adolescents. The Annual and Quarterly Health Center Performance Measure Reporting Tools (Attachments 5a, 5b, 6a and 6b) include aggregate data on the number of youth served and the number of youth receiving certain types of services from health centers. The Awardee Training and Technical Assistance Tool (Attachment 7) asks awardees to report on the training and technical assistance provided to partners. The Awardee Performance Measurement Tool (Attachment 8a and 8b) asks the grantee to report on partnerships developed and on communication efforts.

How Information Will be Shared and For What Purpose

Results from the tools are summarized in written reports provided to each of the awardees in order that awardees may work with their health center partners to adjust training and technical assistance plans for partners who are not making sufficient progress or, if needed due to ongoing lack of progress, end the partnership with the health center. The data collected is also used by grantees as part of their evaluation plans with annual evaluation reports submitted to CDC. CDC is using the data to evaluate the overall program and determine the impact of this program’s approach to improving adolescent access to reproductive health care. CDC has presented an overview of the project including baseline data summarizing organizational and provider factors that influence access and quality of adolescent reproductive health care at the Society for Adolescent Health and Medicine Annual Meeting. Grantees have made similar presentations providing an overview of their individual projects and baseline data from their health center partners at state and national conferences (e.g., Conference on Adolescent Health) [8].

Impact of the Proposed Collection on Respondents’ Privacy

The Health Center Provider Survey (Attachment 4a and 4b) is completed anonymously. In health centers or practice settings with only one provider (i.e., in rural areas), the provider survey data is only released in aggregate to assure that an individual provider’s responses cannot be identified.

The data collection instrument completed on behalf of health centers do not include confidential business information. However, services associated with adolescent reproductive health care or referral to such care can be sensitive topics. As such, CDC will not identify health centers when findings are disseminated.

Opportunities to Consent

Before respondents answer questions on the Health Center Provider Survey (Attachment 5a and 5b), they will be informed about the voluntary nature of participation in a brief description at the top of each instrument. The brief description also will assure potential respondents that their answers will be maintained in a secure manner.

Health center partners participate voluntarily in the overall project but, if they decide to participate, are required to complete the Health Center Organizational Assessment (Attachment 3a and 3b), as well as the Quarterly and Annual Health Center PMRT (Attachment 5a, 5b, 6a and 6b). This agreement to complete the measures as part of participating in the project is formalized ahead of measurement completion through a memorandum of understanding between the health center partner and the awardee. Awardees are required to complete the Awardee Training and Technical Assistance Tool (Attachment 7) and the Awardee PMRT (Attachment 8a and 8b) in order to receive grant funding.

How Information Will Be Secured

Safeguards exist to minimize possibility of unauthorized access, use or dissemination of information being collected. Paper files received by the awardee organization are stored in a locked file cabinet, with only access allowed for identified staff. Data is submitted to CDC through the online data collection system. All electronic data are stored on secured servers protected with firewalls and passwords. All employees are trained on data security measures by taking appropriate HHS courses online. The data collection system adheres to HHS and CDC IT policies and procedures.

11. Institutional Review Board (IRB) and Justification for Sensitive Questions

IRB approval was not required as this project was approved as public health practice. A portion of respondents may view race and ethnicity data, as well as receipt and provision of reproductive health care services, as sensitive. The only data collection instrument that collects race/ethnicity data during the extension period is the Annual Health Center Performance Measure Reporting Tool (Attachment 6a and 6b). The tool asks partners to report on the race/ethnicity of the youth that they serve in aggregate using administrative data. This data from the Annual Health Center Performance Measure Reporting Tool is important for tracking provision of contraceptives by the race/ethnicity of the youth served; particularly because minority women in the past have experienced coercion related to birth control, it is important to monitor contraceptive provision by race/ethnicity.

Reporting using the HHS OMB categorizations for race/ethnicity for youth served at the health center is not practical nor feasible given the use of administrative data and would represent a much larger burden on the awardee, the partners and many youth users of reproductive health care services. Therefore administrative data, which is only available in a combined format with no separate reporting for race and Hispanic ethnicity, is used. Additionally, the grantees are all working in counties with either White or African American majorities. In one of the communities, 15% of the population identifies as Hispanic. The percentage of the population in the participating communities that is from other racial groups is very small. Given this background, we determined that the smallest burden on the health centers was for them to report using a combined format and reporting only for Black/African American, White, Hispanic/Latino, and all other races. If data for all other races were separated into distinct categories, the number of youth included would be very small and the data would be unreliable and not meaningful for tracking change. In addition, this may increase the risk for breach in confidentiality due to small cell sizes.

Data collected from the health centers is not expected to result in liability or competitive disadvantage. As noted previously, services associated with adolescent reproductive health care may be considered sensitive topics. CDC will not identify health centers when findings are disseminated. These data are needed to track the impact of the activities to increase youth-friendly best practices in provision of reproductive health services.

12. Estimates of Annualized Burden of Hours and Costs

Table A.4 below summarizes the estimated burden hours for the revision period, which pertains only to health center and awardee assessments. The estimated burden is based on the total number of target respondents multiplied by the number of times that each assessment (Attachments 3a-8b) will be administered during the extension period (Table A.3). The total estimated burden hours for all respondents is 485 hours. The total estimated burden hours for private sector participants is 415 hours and the total estimated burden hours for State and Local Government participants is 70 hours.

The three awardees will complete the Awardee Training and Technical Assistance Tool (Attachment 7) each month through the remainder of the project (February 2020 to August 2020) resulting in 8 completed tools. The tool assesses training and technical assistance provided by the awardee to health center and youth-serving system partners and is intended to track grantee implementation of planned efforts. We estimate that it will take 2 hours per response to complete the tool. This estimation is based on use of this tool in the current project and in a previous project that tracked training and technical assistance provided to health centers and YSOs (OMB No. 0920-0952, Exp. 12/31/2015).

The 3 awardees also will complete the Awardee Performance Measure Reporting Tool (Attachment 8a and 8b) during the final year of the project. This tool is intended to track awardee performance with respect to building required partnerships and reaching youth with communication campaigns. We estimate that it will take 1 hour per response based on the time it has been taking during this project and the time it took for grantees in a previous project to complete items from a similar tool (OMB No. 0920-0952, Exp. 12/31/2015).

We anticipate that there will be 25 partner health centers (21 private and 4 state/local government) Health Center Organizational Assessment (Attachment 3a and 3b) is administered. Completion of the measure will be required of all partner health centers once during the final year of the project. The measure assesses health center policies, procedures, and practices associated with providing reproductive health care to adolescents and will be used to identify training and technical assistance needed to sustain project changes and track progress in provision of youth friendly reproductive health care. Completing the assessment will take an estimated 2 hours per response; the response time estimate is also based on experience during this project and from a similar data collection instrument used in a previous project (OMB no. 0920-0952, exp. Date 12/31/2015).

Each of the anticipated25 partner health centers (21 private and 4 state/government) will complete the Quarterly Health Center Performance Measure Reporting Tool (PMRT, Attachment 5a and 5b) for quarter one and two of the final year of the project. This tool assesses health care service use by adolescents and contraception provided to adolescents to determine progress toward project objectives. Data to be reported in the tool is already collected by health center partners through their electronic medical record systems. We estimate that completing the Quarterly Health Center PMRT will take approximately 4 hours. We anticipate that this will include the time health centers will need to program a report to be run within their electronic medical records system; most health centers use programs developed at the beginning of the project to run the report each quarter, substantially decreasing the time to complete.

Each of the anticipated 25 partner health centers (21 private and 4 state/local government) also will complete the Annual Health Center PMRT which assesses health care service use by adolescents and contraception provided to adolescents in somewhat greater detail that does the Quarterly Health Center PMRT. Partner health centers will complete one time after the third quarter is completed, and the information gathered will be used to determine the extent to which the partners reached project objectives. Although we would prefer to have annual data for all of FY 2020, the project ends just prior to the end of the fiscal year and, thus, leaves no time for awardees to work with their partners to collect quarter four or full year data. We propose to use the Annual Health Center Performance Measure Reporting Tool (Attachment 6a and 6b) rather than the Quarterly version because the Annual version includes some unique items (e.g., service provision by race/ethnicity) that would allow us to estimate some changes over the course of the project (e.g., changes in service provision by race/ethnicity from the start to the end of the project). The Annual Health Center PMRT is estimated to take an average of 6 hours to complete. The estimated time to complete the Annual and Quarterly PMRTs is based on our experience during the current project and with a similar PMRT used in a previous project (OMB no. 0920-0952, exp. Date 12/31/2015).

Providers at the health center partners are asked to complete a Health Center Provider Survey (Attachment 4a and 4b) annually. The survey assesses clinical provider attitudes and practices associated with providing reproductive health care to adolescents and will be used to assess awardee and health center performance in encouraging clinical providers to engage in youth friendly best practices when providing adolescent reproductive health care. Based on grantees’ understanding of the number of providers at anticipated health center partners, we estimate that an average of 4 providers will complete the Health Center Provider Survey at each of the 25 health centers (21 private and 4 state/local government) one time during the final year, resulting in a total of 100 providers (84 from private and 16 from state/local government entities) completing the survey. A time estimate of 20 minutes to complete the survey is based on experience during the current project. The assessment is a straight forward measure asking providers to report on a topic with which they are very familiar (i.e., their own practice behaviors).

Table A.3. Estimated Burden to Respondents

Respondents	Form Name	Number of Respondents	Number of Responses per Respondent	Average Burden per Response (in hours)	Total Burden (in hours)
Private Sector	Health Center Organizational Assessment	21	1	2	42
	Quarterly Health Center Performance Measure Reporting Tool	21	2	4	168
	Annual Health Center Performance Measure Reporting Tool	21	1	6	126
	Health Center Provider Survey	84	1	20/60	28
	Awardee Training and Technical Assistance Tool	3	8	2	48
	Awardee Performance Measure Reporting Tool	3	1	1	3
State and Local Government	Health Center Organizational Assessment	4	1	2	8
	Quarterly Health Center Performance Measure Reporting Tool	4	2	4	32
	Annual Health Center Performance Measure Reporting Tool	4	1	6	24
	Health Center Provider Survey	16	1	20/60	6
TOTAL					485

The table below summarizes the estimated burden costs for the eight month extension period. The estimate of hourly wages were obtained from the Department of Labor based on May 2018 data (http://www.bls.gov/oes/current/oes_nat.htm ). The total estimated cost to respondents is $27, 525.18. The estimated cost for private sector respondents is $23,338.18; the estimated cost for state and local government respondents is $4,187.02.

Table A.4. Estimated Annualized Cost to Respondents

Respondents	Form Name	Number of Respondents	Total Burden (in hours)	Average Hourly Wage	Total Cost
Private Sector	Health Center Organizational Assessment	21	42	$ 54.68	$ 2,296.56
	Quarterly Health Center Performance Measure Reporting Tool	21	168	$ 54.68	$ 9,186.24
	Annual Health Center Performance Measure Reporting Tool	21	126	$ 54.68	$ 6,889.68
	Health Center Provider Survey	84	28	$ 114.58	$ 3,208.24
	Awardee Training and Technical Assistance Tool	3	48	$ 34.46	$ 1654.08
	Awardee Performance Measure Reporting Tool	3	3	$ 34.46	$ 103.38
State and Local Government	Health Center Organizational Assessment	4	8	$ 54.68	$ 437.44
	Quarterly Health Center Performance Measure Reporting Tool	4	32	$ 54.68	$ 1,749.76
	Annual Health Center Performance Measure Reporting Tool	4	24	$ 54.68	$1,312.32
	Health Center Provider Survey	16	6	$ 114.58	687.48
TOTAL					$27,525.18

13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

There is no other total annual cost burden to respondents.

14. Annualized Cost to the Government

The ICR is being funded as one part of a cooperative agreement between DRH and the three awardees. In total, the three awardees receive $1,950,000 annually with estimated costs of $1,300,000 for the extension period (February 2020 to September 2020). Personnel costs of federal employees involved in the oversight of the data collection under the cooperative agreement, technical assistance, and analysis of data (i.e., direct costs to the federal government) includes 4 DRH staff. Costs to the government are based on 2019 fiscal year costs and total 144,309.56.

The total estimated cost to the government of collecting the requested information during the requested extension period is $1,444,309.56.

Table A.5: Total Annualized Cost to the Government

Expense type	Expense Explanation			Annual Costs (dollars)
Federal government staff salaries	Health Scientist	GS-13	.30FTE	48,270.88
	Project Officer	GS-13	.20FTE	32,443.08
	Statistician/Data Manager	GS-13	.30FTE	48, 943.67
	Team Lead	GS-14	.05FTE	14,651.93
Funds provided to awardees for extension period (February 2020 to September 2020)				1,300,000
TOTAL				1,444,309.56

15. Explanation for Program Changes or Adjustments

This is a request for a Revision of a currently approved data collection (Exp. 1/31/2020) to continue data collection for health center partners and awardees until the end of the funding period (9/30/2020).

Included in the initial request, but not included in this revision request, are an YSO Organizational Assessment, a YSO Staff Survey, and a YSO Performance Measurement Tool, as well as a Health Center Youth Survey. As noted previously, we do not expect further change in YSO policies and practices with respect to linking adolescents to care beyond what has occurred already. We determined that the burden of collecting additional data from the YSO partners outweighed the benefit. Under the currently approved information collection request, the Health Center Youth Survey will be conducted late in 2019 and will not be conducted again during the course of the project. Thus, it is not included in this information collection extension request. The burden was reduced from 1150 hours annually to 485 hours due to taking out instruments that are no longer necessary and a shorter period of collection.

16. Plans for Tabulations and Publication and Project Time Schedule

Data from the Quarterly Health Center PMRTs (Attachment 5a and 5b) that are submitted quarterly will be summarized in a written report and the CDC project officer will meet with each of the grantees to review the data and discuss their partners’ performance. Grantees will then be expected to meet with their partners and adjust training and technical assistance plans for partners who are not making sufficient progress or, if needed due to ongoing lack of progress, end the partnership with the health center. The final Annual Health Center PMRT (Attachment 6a and 6b) will also be summarized in a written report so that grantees can utilize that data (along with previously collected data) to complete their project evaluations. Similarly, the Health Center Provider Survey data (Attachment 4a and 4b) will be summarized in a written report so that grantees can use the data for their project evaluations.

Six months after the project ends, a summary report using data from all of the data collection instruments will be authored that describes the extent to which grantees successfully met their objectives. Analyses will consist of descriptive statistics and, for examining change from baseline, paired t-tests. The summary report will be shared with CDC leadership, as well as with other federal agencies working to prevent teen pregnancy (e.g., Office of Population Affairs). Results shared in the summary report may also be submitted for presentation at a conference that reaches government agency staff and their grantees who work on teen pregnancy prevention.

Table A.6: Project Time Schedule

Activity	Time Schedule
Data Collection
Awardee Training and Technical Assistance Tool	Monthly (February 2020 through August 2020)
Quarterly Health Center Performance Measure Reporting Tool	February and May 2020
Health Center Organizational Assessment	March 2020
Heather Center Provider Survey	June 2020
Annual Health Center Performance Measure Reporting Tool	August 2020
Awardee Performance Measure Reporting Tool	August 2020
Data Reporting
Health Center PMRT Report to Awardees	March, June and August 2020
Health Center Provider Survey Report to Awardees	July 2020
Final Report	March 2021

17. Reason(s) Display of OMB Expiration is Inappropriate

The expiration date for OMB approval will be displayed on all data collection instruments for which approval is being sought.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certifications.

Bibliography

1. Martin, JA, Hamilton BE, Osterman, MJK. Births in the United States, 2017. NCHS data brief, no 318. Hyattsville, MD: National Center for Health Statistics. 2018.

2. United Nations Department of Economic and Social Affairs, Statistics Division. Demographic yearbook 2017. Table 10. Live births by age of mother and sex of child, general and age-specific fertility rates: Latest available year, 2008–2017. New York, NY: United Nations. 2018.

3. Counting it up: The public costs of teen childbearing: Key data, Pregnancy, Editor. 2013, The National Campaign to Prevent Teen and Unplanned Pregnancy: Washington, DC.

4. Santelli, J.S., et al., Explaining recent declines in adolescent pregnancy in the United States: the contribution of abstinence and improved contraceptive use. Am J Public Health, 2007. 97(1): p. 150-6.

5. Secura, G., et al., Provision of no-cost, long-acting contraception and teenage pregnancy. New England Journal of Medicine, 2014. 371: p. 1316-23.

6. Kavanaugh, M.L., et al., Meeting the Contraceptive Needs of Teens and Young Adults: Youth-Friendly and Long-Acting Reversible Contraceptive Services in U.S. Family Planning Facilities. Journal of Adolescent Health, 2013. 52(3): p. 284-292.

7. Carpenter, S.C. et al., The association of foster care or kinship care with adolescent sexual behavior and first pregnancy. Pediatrics, 2001. 108(3): E46.

8. Mueller, T., et al., Supporting implementation of youth-friendly best practices through the Teen Access and Quality Initiative: Baseline data from health centers in three communities. Poster presented at the Society for Adolescent Health and Medicine 2018 Annual Meeting, Seattle, WA.

17