REV SS Part A_Working with Public Centers TPP_03_7_2017_clean

p.19

Performance Monitoring of “Working with Publicly Funded Health Centers to Reduce Teen Pregnancy among Youth from Vulnerable Populations”

Information Collection Request

New

Supporting Statement

Part A

June 16, 2016

Submitted by:

Heather Tevendale

Health Scientist

Division of Reproductive Health, CDC

4770 Buford Highway, NE MS F-74

Atlanta, GA

Telephone: 770-488-6147

Fax: 770-488-6291

Email: [email protected]

SECTION A: TABLE of CONTENTS

Part A. Justification

A.1 Circumstances making the collection of information necessary

A.2 Purpose and use of the information collection

A.3 Use of improved information technology and burden reduction

A.4 Efforts to identify duplication and use of similar information

A.5 Impact on small businesses or other small entities

A.6 Consequences of collecting the information less frequently

A.7 Special circumstances relating to the guidelines of 5 CFR 1320.5

A.8 Comments in response to the Federal Registrar Notice and efforts to consult outside the agency

A.9 Explanation of any payment or gift to respondents

A.10 Assurance of confidentiality provided to respondents

A.11 Justification of sensitive questions

A.12 Estimates of annualized burden hours and costs

A.13 Estimates of other total annual cost burden to respondents or record keepers

A.14 Annualized cost to the government

A.15 Explanation for program changes to adjustments

A.16 Plans for tabulation and publication and project time schedule

A.17 Reason(s) display of OMB expiration date is inappropriate

A.18 Exceptions to certification for Paperwork Reduction Act submissions

Tables

Table A.1. Description and Purpose of Each Measure

Table A.2. Frequency of Assessment Tool Use

Table A.3. Estimated Annualized Burden to Respondents

Table A.4. Estimated Annualized Cost to Respondents

Table A.5. Total Annualized Cost to the Government

Table A.6. Data collection time schedule

List of Attachments

Attachment 1. Section 301 of the Public Health Service Act (42 U.S.C 241)

Attachment 2a: 60-day Federal Register Notice

Attachment 2b: Public Comment Received

Attachment 3: Crosswalk of Objectives and Data Sources

Attachment 4: Health Center Organizational Assessment, paper

Attachment 4a: Health Center Organizational Assessment, online

Attachment 5: Health Center Provider Survey, paper

Attachment 5a: Health Center Provider Survey, online

Attachment 6: Health Center Youth Survey, paper

Attachment 6a: Health Center Youth Survey, online

Attachment 7: Quarterly Health Center Performance Measure Reporting Tool, paper

Attachment 7a: Quarterly Health Center Performance Measure Reporting Tool, online

Attachment 8: Annual Health Center Performance Measure Reporting Tool, paper

Attachment 8a: Annual Health Center Performance Measure Reporting Tool, online

Attachment 9: Youth Serving Organization (YSO) Organizational Assessment, paper

Attachment 9a: Youth Serving Organization (YSO) Organizational Assessment, online

Attachment 10: YSO Staff Survey, paper

Attachment 10a: YSO Staff Survey, online

Attachment 11: YSO Performance Measure Reporting Tool, online

Attachment 11a: YSO Performance Measure Reporting Tool, online

Attachment 12: Awardee Training and Technical Assistance Tool, Excel

Attachment 13: Awardee Performance Measure Reporting Tool, paper

Attachment 13a: Awardee Performance Measure Reporting Tool, online

Attachment 14: Project Logic Model

ACRONYMS

cdc = centers for disease control and prevention

DRh = division of reproductive health

TPP = TEen pregnancy prevention

CWI = COMMUNITY-WIDE INITIATIVE

YFS =yOUTH-FRIENDLY SERVICES

QFP = Quality family planning

opa = office of population affairs

oah = Office of Adolescent health

acog = american college of obstetricians and gynecologists

aap = american academy of pediatrics

larc = long acting reversible contraception

iud = interuterine device

Goal of the Study: To evaluate and improve efforts of three organizations under the cooperative agreement “Working with Publicly Funded Health Centers to Reduce Teen Pregnancy among Vulnerable Populations” to 1) work with health centers to enhance implementation of best practices in adolescent reproductive health care and 2) work with youth-serving organizations (YSOs) to develop systematic approaches to referring youth at risk for a teen pregnancy to reproductive health care.

Intended use: Information collected will be used to monitor awardee performance and determine training and technical assistance needs to address any performance issues.

Methods to Collect: The three awardee organizations will plan, coordinate, and lead the efforts in collaboration with approximately 25 health centers and 35 YSOs, in total. Reflecting expected changes in policies, staff practices, and youth health care seeking and teen pregnancy prevention behaviors, awardees will conduct assessments of health center and YSO partners’ organizations, staff members at the partner organizations, and youth served by the health center partner organizations. Most assessments will be conducted annually throughout the five year projects.

Subpopulations: Subpopulations include health center and YSO partner organizations participating in the cooperative agreement “Working with Publicly Funded Health Centers to Reduce Teen Pregnancy among Vulnerable Populations,” along with their staff members and youth receiving services.

How the data will be analyzed: Analyses will consist of summary statistics and paired t-tests

comparing change from baseline to each annual follow-up assessment.

1. Justification

1. Circumstances Making the Collection of Information Necessary

The Centers for Disease Control and Prevention (CDC) is seeking OMB review and approval for a new information collection to carry out performance monitoring and improvement of projects funded by the “DP15-1508 Working with Publicly Funded Health Centers to Reduce Teen Pregnancy among Vulnerable Populations” cooperative agreement. Three awardees have been funded under this cooperative agreement. Approval is being requested for a new information collection for three years (Years 1-3) of a 5-year project. During Year 3, a request will be made for an extension of information collection to cover Years 4-5.

The 2014 US rate of 24.2 births per 1,000 female teens aged 15-19 is the highest of all Western industrialized countries [1]. Teen pregnancy and childbearing are estimated to cost tax payers $11 billion each year [2]. Significant racial/ethnic and geographic disparities in teen birth rates persist. Given the magnitude of this public health issue and perceptions that significant improvements can be made, teen pregnancy was chosen as one of CDC’s seven winnable battles, making it clear that teen pregnancy prevention is an agency priority (http://www.cdc.gov/winnablebattles).

Access to reproductive health services and the most effective types of contraception have been shown to reduce the likelihood that teens become pregnant [3-5]. Nevertheless, recent research [5] and lessons learned through a previous teen pregnancy prevention project implemented through CDC in partnership with the Office of Adolescent Health (2010-2015; OMB no. 0920-0952, exp. Date 12/31/2015) demonstrate that many health centers serving adolescents in communities with high teen birth rates are not engaging in youth friendly best practices that may enhance access to care and to the most effective types of contraception . Furthermore, youth at highest risk of experiencing a teen pregnancy are often not connected to the reproductive health care that they need, even when they are part of a population that has been identified as being at high risk for a teen pregnancy [6] (e.g., youth in foster care).

Information collection is proposed for program monitoring and management of three funded projects under the cooperative agreement “Working with Publicly Funded Health Centers to Reduce Teen Pregnancy among Vulnerable Populations”. The three funded awardees are Mississippi First, Inc., Sexual Health Initiatives For Teens North Carolina (SHIFT NC), and the Georgia Association for Primary Health Care, Inc. To carry out these projects, each of these three organizations will work with 5-10 health centers and 10-15 youth serving organizations in their target communities. Awardees will select these partners during the first year of the project.

These projects seek to decrease teen pregnancy by:

• Improving the quality of adolescent sexual and reproductive health (ASRH) services at partner health centers.

• Increasing use of ASRH services among sexually active teens in the target communities.

• Increasing the number of sexually active teens at partner health centers who receive effective methods of contraception.

The awardees will seek to achieve these aims by engaging in three main activities.

• They will work with publicly funded health centers to make organizational changes and engage in clinical provider training to increase health centers implementation of best practices in adolescent reproductive health care. The overall goal of these efforts is to improve the quality of and access to youth-friendly reproductive health services.

• They will work with youth-serving organizations (YSOs) that serve youth at risk for a teen pregnancy to make organizational changes and provide staff training so that youth at those agencies are assessed for whether they are at risk for a teen pregnancy (i.e., sexual health assessment) and, when indicated, referred to reproductive health care.

• They will develop communication campaigns intended to increase awareness of the partner health centers’ services for teens.

Youth friendly best practices included in this program are supported by evidence in the literature and recommended by major medical associations. Each of the components of the current project has been implemented as part of past teen pregnancy prevention efforts. Consistent with CDC’s mission of using evidence to improve public health programs, conducting an evaluation of combined best practices, in concert with community-clinical linkage of youth to services to increase their access to reproductive health care, can provide information that will inform future teen pregnancy prevention efforts.

The three awardees will plan, coordinate, and lead the efforts in collaboration with approximately 25 health centers (5-10 per awardee) and 35 youth serving organizations (10-15 per awardee). Reflecting expected changes to health center and YSO partners’ policies, to staff practices, and to youth health care seeking behaviors, awardees will conduct assessments of health center and YSO partners’ organizational factors, staff members, and youth served by the partner organizations. The proposed information collection will not involve the collection of personal information in identifiable form.

CDC is authorized to conduct this information collection under Section 301 of the Public Health Service Act (42 U.S.C 241) Authority of the Secretary (See Attachment 1).

2. Purpose and Use of Information Collection

Information is being requested from awardees, their health center and YSO partners, as well as the staff and youth served by the health center partners to evaluate performance and training needs for quality improvement. As part of the grant application process, all awardees were required to propose working in communities with a 2013 teen birth rate above the national average (26.6 births per 1,000 female adolescents ages 15-19) and those communities had to be located in states with a 2013 birth rate above the national average. To carry out these projects, each of these three organizations will work with 5-10 health centers and 10-15 youth serving organizations in their target communities. Awardees will select these partners during the first year of the project. Mississippi First, Inc., a non-profit focused on child well-being and educational achievement, was funded to work in Coahoma, Quitman and Tunica counties in Mississippi. Sexual Health Initiatives For Teens North Carolina (SHIFT NC), a non-profit organization focused on the sexual health of adolescents, was funded to work in Durham County, North Carolina. The Georgia Association for Primary Health Care, Inc, which represents all of Georgia’s Federally Qualified Health Centers, was funded to work in Chatham County, Georgia.

Information collected in the first year of the project will determine the types of training and technical assistance that awardees need to provide to their health center and youth-serving organization partners. In subsequent years, the information collected will be used by CDC to monitor whether awardee and partners are meeting project objectives and determine what further training and technical assistance is needed. Additionally, the three awardees will meet annually with health center and YSO leaders and staff to review data collected and to address any problem areas. Health center partners will use the Quarterly Health Center Performance Measure Tools (Attachment 7) for the first three quarters of the year and the Annual Health Center Quarterly Performance Measure Tool (Attachment 8) for the fourth quarter. YSO partners will use the YSO Performance Measure Reporting Tool quarterly (Attachment 11). Awardees also will meet quarterly with partner leadership to review these key performance measures. Having a smaller number of performance measures collected quarterly will allow for significant issues to be addressed quickly, as opposed to waiting a year to discover and address problems, while minimizing the burden on awardees and their partners by limiting more detailed data collection to the Annual Health Center Performance Measure Reporting Tool We anticipate that a data driven quality improvement process will result in high quality projects being conducted that make the best possible use of the funds provided to awardees.

The negative consequences of not having the information that we propose to collect would be limited ability to understand awardee and partner performance and, thus, limited ability to help awardees improve performance and ensure that awardee funding is well spent.

To evaluate the progress of awardees and their partner organizations in making structural changes that have been shown to lead to improvements in the quality and use of adolescent sexual and reproductive health (ASRH) services, along with the number of sexually active teens who receive effective methods of contraception, it is necessary to assess performance on a number of short-term objectives. These short term objectives include:

1. Improve health center organizational factors (e.g., financing, clinical policies, protocols, practices, medication formulary, appointment scheduling) that support high quality ASRH services

2. Increase health center staff knowledgeable about and support of implementation of evidence-based guidelines for reproductive health services, youth friendly best practices, and the provision of the most effective types of contraception to sexually active teens

3. Increase frequency with which sexually active youth are screened for pregnancy intention, counseled on the full range of FDA approved methods of birth control, and provided method that youth selects without delay at partner health centers

4. Increase the extent to which SRH services at partner health centers are provided in a youth-friendly manner

5. Increase in implementation of organizational policies at partner youth serving organizations (YSOs) that support efforts to link youth to SRH services

6. Increase in YSO staff support of efforts to link youth to SRH services

7. Increase in YSO staff knowledge about screening youth to determine need for SRH services and provide referrals

8. Increase in number of youth at partner YSOs receiving referral to SRH services

These short-term objectives and the intermediate project objectives they may lead to are listed in a table in Attachment 3. The table also provides a crosswalk demonstrating how each data collection tool will provide the data necessary for judging whether grantees and their partners are meeting project objectives. The data collection tools include a Health Center Organizational Assessment (Attachment 4), Health Center Provider Survey (Attachment 5), Health Center Youth Survey (Attachment 6) and Quarterly and Annual Health Center Performance Measurement Tools (Attachment 7 and 8), as well as a Youth Serving Organization (YSO) Organizational Assessment (Attachment 9), a YSO Staff Survey (Attachment 10), and a YSO Performance Measurement Tool (Attachment 11). Two data collection tools are the responsibility of the three awardees funded to establish partnerships with Health Centers and Youth Serving Organizations: the Awardee Training and Technical Assistance Tool (Attachment 12) and the Awardee Performance Measure Reporting Tool (Attachment 13).

3. Use of Improved Information Technology and Burden Reduction

The Awardee Training and Technical Assistance Tool (Attachment 12) is an Excel file that awardees can complete electronically in Excel each time they provide training or technical assistance to a partner. At the end of each month, they submit the file via email.

All of the other tools can be completed though the use of an electronic data collection system. All measures have been programed using SPSS Data Collection Author software. Surveys are available through a dedicated and secure server managed by CDC information technology staff. The surveys are designed with automatic skip patterns depending on the participant’s responses, thus reducing burden on the respondents.

OMB approved paper versions of the measures will be maintained for use by grantees and their partners who feel that paper versions of the measure are easier for them to use. For example, the Health Center and the YSO Organizational Assessments (Attachment 4 and 9) and the Performance Measure Reporting Tools (Attachments 7, 8, 11 and 13) each require input from multiple staff members at the health centers or YSOs. Our previous experience on past project using similar measures indicated that the awardees from the past project found that completing the measure on paper was more convenient for their partners who prefer to be able to hand off a paper copy.

All data collection tools were reviewed multiple times by CDC staff to ensure that all possible approaches were taken to minimize respondent burden. Each assessment tool was designed to be brief, easy to use, and understandable. Questions that are not applicable to a respondent based on an answer to a previous question will be passed over via formatting and skip patterns. CDC staff have carefully considered the content, appropriateness and phrasing of each question. Furthermore, awardee staff including those implementing programmatic efforts (e.g., project directors, project coordinators, technical assistance providers) and those evaluating the projects from all three awardees have provided feedback on the tools. Awardee feedback was incorporated into the tools to ensure that the tools meet the needs of the awardees for determining the technical assistance needs of their health center and YSO partners and that the burden of collecting the data is kept to an absolute minimum.

4. Efforts to Identify Duplication and Use of Similar Information

Once every two weeks, the evaluation lead for the current project participates in a weekly evaluation workgroup conference call with staff from other HHS agencies to discuss federal evaluations of teen pregnancy prevention efforts. This workgroup involves staff from the Office of Adolescent Health, the Office of the Assistant Secretary for Planning and Evaluation (ASPE), and the Administration for Children and Families, who are evaluating other teen pregnancy prevention efforts. Additionally, CDC participates in a leadership group meeting held once every two months which assures coordination between upper level staff at each of the agencies participating in the evaluation workgroup. This high level of coordination among agencies assures that federal staff engaged in or supporting evaluations of federal teen pregnancy prevention efforts collaborate when appropriate and assures that there is no duplication of efforts. The proposed evaluation is a unique effort.

A review of the literature indicates singular efforts of evaluating implementation of youth-friendly clinical best practices in reproductive health care, however this project is unique in its strategy of implementing multiple best practices, in concert with community-clinical linkage of youth from youth-serving organizations to reproductive health care as a strategy to increase access and utilization of services. Thus, we have determined that no similar data is available. Furthermore, given that data will be collected to assess the awardees’ performance in implementing a new initiative, previously collected data from other teen pregnancy prevention projects would not offer information about our awardees or their health center and YSO partners’ performance.

5. Impact on Small Business or Other Small Entities

Data will be collected from awardees and their clinical and YSO partners in the public and private sector. Health center and YSO partners have not been finalized, but awardees are encouraged to partner with larger organizations in order to reach more youth in their communities which will minimize the involvement of small entities. Some of the partner organizations, particularly community-based organizations, may be classified as small businesses. The questions have been held to the absolute minimum required for the intended use of the data.

6. Consequences of Collecting the Information Less Frequently

CDC proposes to collect most data on an annual basis. However, data from the Awardee Training and Technical Assistance Tool (Attachment 12) will be collected monthly. The health center and YSO partners will report on key performance measures quarterly. Health center partners will use the Quarterly Health Center Performance Measure Tools (Attachment 7) for the first three quarters of the year and the Annual Health Center Quarterly Performance Measure Tool (Attachment 8) for the fourth quarter. YSO partners will use the YSO Performance Measure Reporting Tool quarterly (Attachment 11). Having a small number of performance measures collected more than annually will allow for significant issues to be addressed quickly, as opposed to waiting a year to discover and address problems.

The frequency of each assessment tool is shown in Table A.2. Without the proposed information collection, CDC’s oversight of activities will be based on anecdotal and nonsystematic information; CDC’s ability to provide technical assistance and recommend mid-course corrections will be diminished. Additionally, the awardees would be unable to carry out one of the key components of the project which is to engage their partners in a data driven continuous quality improvement process.

Table A.2. Frequency of Assessment Tool Use

Assessment Tool	Frequency
Health Center Organizational Assessment (Attachment 4)	Annually
Health Center Provider Survey (Attachment 5)	Annually
Health Center Youth Survey (Attachment 6)	Annually
Quarterly Health Center Performance Measure Reporting Tool (Attachment 7)	3 times per year
Annual Health Center Performance Measure Reporting Tool (Attachment 8)	Annually
YSO Organizational Assessment (Attachment 9)	Annually
YSO Staff Survey (Attachment 10)	Annually
YSO Performance Measure Reporting Tool (Attachment 11)	Quarterly
Awardee Training and Technical Assistance Tool (Attachment 12)	Monthly
Awardee Performance Measure Reporting Tool (Attachment 13)	Annually

To fully understand performance as well as the outcome of awardee efforts, CDC needs to understand what training and technical assistance was provided to health center and YSO partners, what organizational changes were made once training and technical assistance was provided, whether health center providers and YSO staff then change their practices, and finally whether there is an impact on youth (see Attachment 14, for a logic model displaying how the overall program is expected to work). Tracking change at these multiple levels results in a more complicated data collection than would be necessary if awardees were simply implementing a program with youth and only needed to collect information about the impact on youth. Limiting assessment to only the impact on youth, however, would not provide any information on whether performance issues need to be addressed at the awardee, health center partner, YSO partner, health center provider, or YSO staff level in order to have the biggest positive impact on youth.

The Awardee Training and Technical Assistance Tool (Attachment 12) will be completed monthly. Given experience on two prior 5-year projects, completing the tool less often would result in unreliable data given that awardees tend to complete the tool at the time that it is due rather than at the time that training or technical assistance is provided. Good recall over more than a month is unlikely, thus completing the tool quarterly would result in incomplete and incorrect information. Therefore, the Awardee Training and Technical Assistance Tool is collected monthly for ongoing continuous monitoring.

Given the vital importance of close monitoring of awardee and partner progress, CDC has developed brief Health Center and YSO Performance Measure Reporting Tools (Attachments 7, 8 and 11) to be completed quarterly by the Health Center and YSO partners. This tool was designed to be as brief as possible so that the most important indicators of grantee and partner progress are captured, while minimizing the burden on awardees. We also solicited input from all of our awardees on how best to reduce burden. As part of this process and to further reduce burden, we identified items that are sufficient to be completed only annually. Most quality improvement efforts involve more frequent data collection (i.e., monthly), but quarterly seemed to be a reasonable compromise.

As noted, most measures from partner Health Centers and YSOs are completed annually as these measures are not expected to change as often. However, annual tracking is still important for reaching project goals and collecting this data. CDC relies on the data to understand project progress, as well as lack of progress.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This request complies with the regulation of 5 CFR 1320.5

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A. Comments in Response to the FRN

A 60-day Federal Register Notice was published in the Federal Register on February 5, 2016, vol. 81, No. 24, p. 6271 (see Attachment 2a). CDC received one non-substantive comment (Attachment 2b) and replied with a standard CDC response.

2. Efforts to Consult Outside the Agency

CDC has consulted with the Office of Population Affairs (OPA) on the project framework and the development of assessment tools. CDC has also consulted partners on a different teen pregnancy prevention initiative for their feedback on the development of tools for this project. As noted previously, CDC has consulted closely with the awardees to this project as to the design of the proposed data collection instruments.

Where possible, CDC has used previously developed measures from existing CDC and OPA developed tools. Previously developed and tested survey measures from the ‘Monitoring Changes in Attitudes and Practices among Family Planning Providers and Clinics’ (OMB no.0920-0969, expiration date 5/31/2014) were used and incorporated into the Health Center Organizational Assessment (Attachment 4) and Health Center Provider Survey (Attachment 5). All data and lessons learned resulting from this project will be shared with OPA and other publicly funded health centers (including FQHCs).

Year Consulted	Name, Title, Agency	Email/Phone #
2016	Emily Feher, MPH, Project Director, Mississippi First	[email protected]/(601) 398-9008 x105
2016	Kathleen Ragsdale, PhD, Associate Research Professor Social Science Research Center, Mississippi State University	[email protected]/662-325-9168
2016	Melissa Reams, Project Coordinator, Georgia Association for Primary Health Care	[email protected]/912-527-1011
2016	Christine Ley, PhD, Evaluator, Georgia Association for Primary Health Care	[email protected]/412-953-9544
2016	Cathy Bowden, Information Management Coordinator, Georgia Association for Primary Health Care	[email protected]/404-270-2178
2016	Joy Sotolongo, Director of Evaluation, SHIFT NC	[email protected]/919-226-1880 ext 102
2016	Sally Swanson, MSW, MPH, Chief Program Officer, SHIFT NC	[email protected]/919-226-1880
2016	Sarah Davis, MPH, Senior Technical Advisor, SHIFT NC	[email protected]/919-226-1880 x114
2016	Lorrie Gavin, Office of Population Affairs	[email protected]/240-453-2826
2016	Dawn Middleton, Cicatelli Associates Inc	[email protected]/212-594-7741
2016	Karen Schlanger, Cicatelli Associates Inc	[email protected]/404-521-2151
2014	Deborah U’huru, NYC Department of Health and Mental Hygiene	[email protected]/718-299-0169
2014	Kristen Plastino, University of Texas Health Science Center at San Antonio	[email protected]/210-567-7036
2014	Michelle Reese, Adolescent Pregnancy Prevention Campaign of North Carolina	[email protected]/919-226-1880

9. Explanation of Any Payment or Gift to Respondents

Assessment tools will be completed by awardees, health center and YSO partners and youth served by the health center partners. Awardees and their health center or YSO partners are being funded to complete this project. No additional payments or gifts will be provided to the awardees or their health center and YSO partners. The Health Center Youth Survey (Attachment 6) will be completed by youth at the end of their visit to the health center. Youth who participate in the survey may receive items such as water bottles or notebooks branded with project logos and/or teen pregnancy prevention messages developed by awardees as part of their funded initiatives. The items will be worth no more than five dollars. The role of incentives has been widely documented to increase survey participation among youth [7, 10], and not offering incentives can result in low response rates [8, 9, 11] and, thus, results that are not generalizable to the target population. Based on the experience of our research team, the amount of the proposed incentive is quite modest compared to other projects that collect data from teens.

10. Protection of the Privacy and Confidentiality of Information Provided by Respondents

This submission has been reviewed by the National Center for Chronic Disease Prevention and Health Promotion Director for Science Office and a determination was made that the Privacy Act does not apply. Data collected by awardees and partner organizations will not include names or other information in identifiable form (IIF)

Health center provider and YSO staff surveys, as well as health center youth surveys will assure potential respondents that their answers will be maintained in a secure manner. All responses will be voluntary. Before respondents answer questions on any of the assessment tools, they will be informed about the voluntary nature of participation in a brief description at the top of each instrument.

Safeguards will exist to minimize possibility of unauthorized access, use or dissemination of information being collected. Paper files received by the awardee organization will be stored in a locked file cabinet, with only access allowed for identified staff. Data will be submitted to CDC through the online data collection system.

The specific procedures for collecting information using the annual Health Center Provider Survey (Attachment 5) and the YSO Staff Survey (Attachment 10) will be that each health center and YSO partner will have a staff member who is identified as the primary contact for the awardee. The awardee will contact that person and ask him/her to provide all involved providers at the health center or staff at the YSO with a link to the online survey. The staff and providers will complete the measures online. The staff contact at each organization will inform the awardee of how many total staff or providers were asked to complete a survey so that a response rate can be calculated; the names of those staff or providers will not be shared.

The specific procedures for collecting information using the Health Center Youth Survey (Attachment 6) will be that at the end of each visit a provider will ask youth if they would be willing to complete an optional survey about their experience at the clinic, stressing that participating is voluntary, choosing not to participate will not impact services received in the future, that confidentiality will be maintained, and that the survey is anonymous. Data will be collected starting with the first 15-19 year old provided services on a given date and continuing with each successive 15-19 year old served until completed surveys have been collected from 30 youth from each participating practice setting (e.g., family planning, pediatrics), in each health center. Youth will be provided with a tablet or laptop on which to complete the survey and data will be directly submitted to CDC. If the health center prefers to use paper surveys, youth will be provided with a paper survey to complete and will be asked to return the survey in a sealed envelope to the provider or to the staff member who checks them out of the clinic. The identified staff member will submit the anonymous surveys to the awardee who will submit to CDC. Awardees will be encouraged to engage an academic institution’s ethical review board to review the awardee and health center partners’ planned procedures for collecting data with the Health Center Youth Survey.

The Awardee Performance Measurement Tool (Attachment 13) is to be completed by the three awardees. The project coordinator for each awardee is responsible for seeing that the Performance Measurement Tool is completed, but more than one awardee staff member may be involved in completing the measure. No individual’s name is recorded on the tools. The measure asks the grantee to report on partnerships developed and on communication efforts.

Similarly, the Health Center and YSO Organizational Assessments (Attachment 4 and 9), as well as the Quarterly and Annual Health Center Performance Measure Reporting Tool (Attachments 7 and 8) and YSO Performance Measure Reporting Tools (Attachment 11), will be the responsibility of the awardee’s primary contact at the health center or YSO, but multiple individuals may be involved in completing a single measure. No individual’s name is recorded on the tools. Health center partners and YSO’s will participate voluntarily in the overall project but, if they decide to participate, will be required to complete organizational assessments and will need to agree to this ahead of time through a memorandum of understanding with the awardee. The Health Center Organizational Assessment examines health center policies, procedures, and practices associated with providing reproductive health care to adolescents and the YSO Organizational Assessment examines YSO partner policies, procedures and practices associated with conducting sexual health assessments and providing referrals to reproductive health services. The Health Center and YSO Performance Measure Reporting Tools include aggregate data on the number of youth served and the number of youth receiving certain types of services from health centers and YSOs.

The measures completed on behalf of health centers and YSOs do not include confidential business information. However, services associated with adolescent reproductive health care or referral to such care can be sensitive topics. As such, CDC will not identify health centers or YSO’s when findings are disseminated.

11. Justification for Sensitive Questions

The proposed data collection was approved as non-research, public-health practice by the National Center for Chronic Disease Prevention and Health Promotion, and thus institutional review board (IRB) approval is not required.

A portion of respondents may view race and ethnicity data, as well as receipt and provision of reproductive health care services as sensitive. The only data collection instruments that collect race/ethnicity data are the Annual Health Center Performance Measure Reporting Tool (Attachment 8) and the YSO Organizational Assessment (Attachment 9). Both measures ask partners to report on the race/ethnicity of the youth that they serve in aggregate using administrative data. This data from the Annual Health Center Performance Measure Reporting Tool is important for tracking provision of contraceptives by the race/ethnicity of the youth served; particularly because minority women in the past have experienced coercion related to birth control, it is important to monitor contraceptive provision by race/ethnicity. Data on race/ethnicity of youth served at the YSO’s is important for establishing that the awardees and their partners are reaching a diverse group of youth including those at higher risk for a teen pregnancy.

For both measures, reporting using the HHS OMB categorizations for race ethnicity for youth served at the health center and YSO partners is not practical nor feasible given the use of administrative data and would represent a much larger burden on the awardee, the partners and many youth users of reproductive health care services. Therefore administrative data, which is only available in a combined format with no separate reporting for race and Hispanic ethnicity, is used. Additionally, the grantees are all working in counties with either White or African American majorities. In one of the communities, 15% of the population identifies as Hispanic. The percentage of the population in the participating communities that is from other racial groups is very small. Given this background, we determined that the smallest burden on the health centers was for them to report using a combined format and reporting only for Black/African American, White, Hispanic/Latino, and all other races. If data for all other races were separated into distinct categories, the number of youth included would be very small and the data would be unreliable and not meaningful for tracking change. In addition, this may increase the risk for breach in confidentiality due to small cell sizes.

Data collected from the Health Center Youth Survey (Attachment 6) may be considered sensitive as it asks about reproductive health services received. Although this activity is considered public health practice, all necessary elements of informed consent are included in the preamble to the survey, including notification that participation is voluntary, there will be no loss of services otherwise entitled to, and data would be kept confidential. To protect confidentiality, youth primarily will be provided with access to an online survey that collects no identifiable information. Some health center may ask youth to complete a paper survey that also collects no identifiable information. In that case, surveys will be returned in a sealed envelope for transmission to CDC. Data will only be reported in aggregate. Collection of data on youth-reported receipt of reproductive health services is essential to track the impact of the activities to increase youth-friendly best practices in provision of reproductive health services.

Data collected from the health centers or youth-serving systems is not expected to result in liability or competitive disadvantage. As noted previously, services associated with adolescent reproductive health care or referral to such care may be considered sensitive topics. CDC will not identify health centers or YSO’s when findings are disseminated. These data are needed to track the impact of the activities to increase youth-friendly best practices in provision of reproductive health services.

12. Estimates of Annualized Burden of Hours and Costs

A. Table A.4 below summarizes the estimated burden hours. The estimated burden is based on the total number of target respondents multiplied by the number of times that each assessment (Attachments 4-13) will be administered annually. The total estimated annualized burden hours for all respondents is 1150 hours. The total estimated annualized burden hours for private sector participants is 687 hours; the total estimated annualized burden hours for individuals is 175 hours; and the total estimated annualized burden hours for State and Local Government participants is 288 hours.

Information collection instruments are included as Attachments 4-13. Information on the frequency of data collection on each of the assessment is found in Table A.3.

Each of the three awardee organizations will work with 5-10 health centers and 10-15 youth serving organizations in their target communities. The three awardees will complete the Awardee Training and Technical Assistance Tool (Attachment 12) monthly. The measure assesses training and technical assistance provided by the awardee to health center and youth-serving system partners and is intended to track grantee implementation of planned efforts. We estimate that it will take 2 hours per response to complete the measure. This estimation is based on use of the same tool in a previous project that tracked training and technical assistance provided to health centers and YSOs (OMB no. 0920-0952, exp. Date 12/31/2015).

The 3 awardees also will complete the Awardee Performance Measure Reporting Tool (Attachment 13) one time per year. This tool is intended to track awardee performance with respect to building required partnerships and reaching youth with communication campaigns. We estimate that it will take 1 hour per response based on the time it took for grantees in a previous project to complete items from a similar tool (OMB no. 0920-0952, exp. Date 12/31/2015).

In the first three months after OMB approval, the awardees will collaborate with their health center partners to see that the Health Center Organizational Assessments (Attachment 4), Annual Performance Measure Reporting Tool (PMRT; Attachment 8), Providers Surveys (Attachment 5), and Youth Surveys (Attachment 6) are completed. The data from these measures will be used to determine the extent to which each health center is engaging in best practices in adolescent reproductive health care which, in turn, will determine the types of training and technical assistance that awardees need to provide to their health center partners.

Grantees have identified the health center partners with which they would like to work and are in the process of forming partnerships. The estimates of the number of health center partners used to determine the burden estimates is based on the number of health center partners with which awardees hope to work, allowing for the possibility that a few more partners will be identified in the future. Most of the identified potential health center partners are private, nonprofit entities; though a few are part of state or local government (i.e., health departments). Based on the information we have, we estimate that there will be 21 private entities and 4 state/local government entities that participate as health center partners.

Each of the anticipated 25 partner health centers (21 private and 4 state/local government) will complete a Health Center Organizational Assessment once per year. The measure assesses health center policies, procedures, and practices associated with providing reproductive health care to adolescents and will be used to identify needed training and technical assistance and track progress in provision of youth friendly reproductive health care. Completing the measure will take an estimated 2 hours per response; the response time estimate is also based on experience from a similar measure used in a previous project (OMB no. 0920-0952, exp. Date 12/31/2015).

Each of the anticipated 25 partner health centers (21 private and 4 state/local government) also will complete the Annual Health Center Performance Measure Reporting Tool (PMRT) which assesses health care service use by adolescents and contraception provided to adolescents to determine progress toward project objectives. Data to be reported in the tool is already collected by health center partners through their electronic medical record systems. The first time the Health Center PMRT is used, health center partners will report data (e.g., number of teen clients served, number provided contraception) from Fiscal Year (FY) 2015, the year prior to the beginning of funding. This data will provide a baseline from which change will be measured. The Annual Health Center Performance Measure Reporting Tool will then be completed each November with data from the preceding fiscal year. The Annual Health Center PMRT is estimated to take an average of 6 hours to complete. We anticipate that this will include the time health centers will need to program a report to be run within their electronic medical records system; completed health centers will be able to use the already developed program to run the report each year, substantially decreasing the time to complete.

One to two months after OMB approval is received, each of the anticipated 25 partner health centers (21 private and 4 state/government) also will begin the regular quarterly completion of the Quarterly Health Center PMRT (Attachment 7) for the first, second, and third quarters of the year (the Annual PMRT provides information for the fourth quarter). We estimate that completing the Quarterly Health Center PMRT, a shortened version of the Annual PMRT also intended to track progress but with a shorter measure, will take approximately 4 hours. We expect a similar decrease in burden once the health center becomes accustomed to running the same report each quarter. The estimated time to complete the Annual and Quarterly PMRTs is based on our experience with a similar PMRT used in a previous project (OMB no. 0920-0952, exp. Date 12/31/2015).

Providers at the health center partners will be asked to complete a Health Center Provider Survey (Attachment 5) annually. The survey assesses clinical provider attitudes and practices associated with providing reproductive health care to adolescents and will be used to assess awardee and health center performance in encouraging clinical providers to engage in youth friendly best practices when providing adolescent reproductive health care. Based on grantees’ understanding of the number of providers at anticipated health center partners, we estimate that 6 providers will complete the Health Center Provider Survey at each of the 25 health centers (21 private and 4 state/local government) one time per year resulting in a total of 150 providers (126 from private and 24 from state/local government entities) completing the survey. A time estimate of 20 minutes to complete the survey is based on feedback from grantees and from the similarity of the measure to one used by the training and technical assistance provider to the project. The assessment is a straight forward measure asking providers to report on a topic with which they are very familiar (i.e., their own practice behaviors).

Youth from each of the health centers will be asked to complete the Health Center Youth Survey (Attachment 6). The survey measures adolescent clients’ perception of the services that they received from partner health centers and will be used to track health center performance in improving provision of youth-friendly care. Some of the health centers have more than one practice setting that serve youth (e.g., a pediatric practice setting and a family planning practice setting). In order to assess the youth friendliness of each practice setting, 30 youth will be surveyed from each practice setting. Data will be collected annually starting with the first 15-19 year old provided services on a given date and continuing with each successive 15-19 year old served until completed surveys have been collected from 30 youth from each practice setting (e.g., family planning, pediatrics), in each health center. We estimate that 10 of the 25 anticipated health centers will include two practice settings in the project resulting in a total of 35 practice settings. With 30 youth from each of 35 practice settings participating, we anticipate a total of 1050 youth participants. A time estimate of 10 minutes to complete is based on feedback from grantees and from the similarity of the measure to one used by the training and technical assistance provider to the project.

In the first three months after OMB approval, the awardees also will collaborate with their YSO partners to see that the YSO Organizational Assessments (Attachment 9) and YSO Staff Surveys (Attachment 10) are completed. The results will be used to inform the awardee as to the types of training and technical assistance needed by their YSO partners, as well as partner progress in meeting project objectives.

Grantees have identified the YSO partners with which they would like to work and are in the process of forming partnerships. The estimates of the number of YSO partners used to determine the burden estimates is based on the number of YSO partners with which awardees hope to work, allowing for the possibility that a few more partners will be identified in the future. More of the identified potential partners are part of state or local government (e.g., school, foster care agency), though many are private entities (e.g., youth serving community-based organization). Based on the information we have, we estimate that there will be 20 state/local government entities and 15 private entities that participate as YSO partners.

Each of the anticipated 35 YSOs partners (20 state/local government and 15 private entities) will complete a YSO Organizational Assessment (Attachment 9) once per year at an estimated 1 hour per response. This measure asks respondents to report on YSO partner policies, procedures and practices associated with conducting screening to determine need for reproductive health services and providing referrals to reproductive health services.

Staff members from each YSO will be asked to complete the YSO Staff Survey (Attachment 10). This measures asks about knowledge, attitudes and practices related to screening to determine need for and provision of referrals to reproductive health services and is estimated to take 20 minutes to complete. Based on information provided by grantees, we estimated that on average each YSO would have 15 staff members conducting assessment and referral activities as part of the project. Those 15 providers will be asked to complete the YSO Staff Survey at each of the 35 YSOs (20 state/local government and 15 private entities) one time per year. As a result, 525 staff are expected to complete the measure annually (300 from state/local government and 225 from private entities). The response time estimate for both the YSO Organizational Assessment and Staff Survey are based on review and feedback from awardee staff, all of whom work closely with YSOs and some of whom are former YSO staff.

We anticipate that the YSO Performance Measure Reporting Tool will be completed for the first time in February 2017. The YSO screening and referral efforts will not begin until October 2016 (the beginning of the second year of the grant). The first data collection will be a report on services provided during the first quarter of fiscal year 2017 (October to December 2016). The YSO PMRT will be completed every quarter thereafter. Each of the anticipated 35 YSOs partner (20 state/local government and 15 private entities) will complete the YSO Performance Measure Reporting Tool quarterly at an estimated one hour per responses. The response time estimate is based on feedback from awardee staff.

Table A.3. Estimated Annualized Burden to Respondents

Respondents	Form Name	Number of Respondents	Number of Responses per Respondent	Average Burden per Response (in hours)	Total Burden (in hours)
Private Sector	Health Center Organizational Assessment	21	1	2	42
	Quarterly Health Center Performance Measure Reporting Tool	21	3	4	252
	Annual Health Center Performance Measure Reporting Tool	21	1	6	126
	Health Center Provider Survey	126	1	20/60	42
	Youth Serving Organization (YSO) Organizational Assessment	15	1	1	15
	YSO Performance Measure Reporting Tool	15	4	1	60
	Youth Serving Organization (YSO) Staff Survey	225	1	20/60	75
	Awardee Training and Technical Assistance Tool	3	12	2	72
	Awardee Performance Measure Reporting Tool	3	1	1	3
Individual	Health Center Youth Survey	1050	1	10/60	175
State and Local Government	Health Center Organizational Assessment	4	1	2	8
	Quarterly Health Center Performance Measure Reporting Tool	4	3	4	48
	Annual Health Center Performance Measure Reporting Tool	4	1	6	24
	Health Center Provider Survey	24	1	20/60	8
	Youth Serving Organization (YSO) Organizational Assessment	20	1	1	20
	YSO Performance Measure Reporting Tool	20	4	1	80
	Youth Serving Organization (YSO) Staff Survey	300	1	20/60	100
TOTAL					1150

B. The table below summarizes the estimated annualized burden costs. The estimate of hourly wages were obtained from the Department of Labor based on May 2015 data (http://www.bls.gov/oes/current/oes_nat.htm ). The total estimated annualized cost to respondents is $39,941.00. The estimated annualized cost for private sector respondents is $30,755.64; the estimated annualized cost for state and local government respondents is $9,185.36. There are no costs for individual respondents for youth to complete the youth survey.

Table A.4. Estimated Annualized Cost to Respondents

Respondents	Form Name	Number of Respondents	Total Burden (in hours)	Average Hourly Wage	Total Cost
Private Sector	Health Center Organizational Assessment	21	42	$ 48.65	$ 2,043.30
	Quarterly Health Center Performance Measure Reporting Tool	21	252	$ 48.65	$ 12,259.80
	Annual Health Center Performance Measure Reporting Tool	21	126	$ 48.65	$ 6,129.90
	Health Center Provider Survey	126	42	$ 106.92	$ 4,490.64
	Youth Serving Organization (YSO) Organizational Assessment	15	15	$ 22.19	$ 332.85
	YSO Performance Measure Reporting Tool	15	60	$ 22.19	$1,331.40
	Youth Serving Organization (YSO) Staff Survey	225	75	$ 22.19	$ 1,664.25
	Awardee Training and Technical Assistance Tool	3	72	$ 33.38	$ 2,403.36
	Awardee Performance Measure Reporting Tool	3	3	$ 33.38	$ 100.14
Individual	Health Center Youth Survey	1250	208	$ 0	$ 0
State and Local Government	Health Center Organizational Assessment	4	8	$ 48.65	$ 389.20
	Quarterly Health Center Performance Measure Reporting Tool	4	48	$ 48.65	$ 2,335.20
	Annual Health Center Performance Measure Reporting Tool	4	24	$ 48.65	$1,167.60
	Health Center Provider Survey	24	8	$ 106.92	$ 855.36
	Youth Serving Organization (YSO) Organizational Assessment	20	20	$ 22.19	$ 443.80
	YSO Performance Measure Reporting Tool	20	80	$ 22.19	$ 1,775.20
	Youth Serving Organization (YSO) Staff Survey	300	100	$ 22.19	$ 2,219.00
TOTAL					$39,941.00

13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

There are no costs to respondents other than their time.

14. Annualized Cost to the Government

The ICR will be funded as one part of a cooperative agreement between the CDC Division of Reproductive Health and the three awardees. In total, the three awardees will receive $1,950,000 annually. Personnel costs of federal employees involved in the oversight of the cooperative agreement, technical assistance, and analysis of data (i.e., direct costs to the federal government) will include those of 4 CDC Division of Reproductive Health staff, and one contract staff who will manage and analyze evaluation data. Costs to the government are based on 2015 fiscal year costs.

The total estimated cost to the government of collecting the requested information is $232,215.

Table A.5: Total Annualized Cost to the Government

Expense type	Expense Explanation			Annual Costs (dollars)
Federal government staff salaries	Health Scientists	GS-13	.50FTE	$68,743
	Project Officers	GS-13	.50FTE	$66,351
	Team Lead	GS-14	.10FTE	$23,768
Contract staff	Statistician/Data Manager		.75FTE	$73,353
TOTAL				$232,215

15. Explanation for Program Changes or Adjustments

This is a request for a new data collection.

16. Plans for Tabulations and Publication and Project Time Schedule

Data from the Health Center and YSO PMRTs that are submitted quarterly will be summarized and the CDC project officer will meet with each of the grantees to review the data and discuss their partners’ performance. Grantees will then be expected to meet with their partners and adjust training and technical assistance plans for partners who are not making sufficient progress or, if needed due to ongoing lack of progress, end the partnership with the health center or YSO.

In March of 2017 (and again in March of 2018 and 2019), CDC will provide awardees with a written report of the awardee and the partners’ performance based on the data collected up to that point (using from the PMRTs, as well as data collected from organizational assessments and provider/staff/youth surveys). Data analysis for this report will consist of summary statistics indicating awardee and partner progress on each measure and, once multiple years of data have been collected, paired t-tests will be conducted to examine change from baseline in the indicators of performance. The CDC project officer will review the data with grantees who will in turn use the data in their quality improvement process with their health center and YSO partners.

Two months later, a summary report describing the extent to which grantees are meeting their objectives will be completed. Analyses will consist of descriptive statistics and, for examining change from baseline, paired t-tests. The summary report will be completed and shared with CDC leadership, as well as with other federal agencies working to prevent teen pregnancy (e.g., Office of Population Affairs). Results shared in the summary report will also be submitted for presentation at two key federally sponsored conferences that reach government agency staff and their grantees who work on teen pregnancy prevention. These conferences are the HHS Teen Pregnancy Prevention Grantee Conference and the National Reproductive Health Conference.

OMB approval is being requested for Years 1-3 of the 5-year program. An additional request will be made to continue data collection for the last two years of program implementation.

Table A.6: Project Time Schedule

Activity	Time Schedule
Data Collection
Health Center Organizational Assessment	1-3 month after OMB approval 10-14 months after OMB approval 22-26 months after OMB approval
Heather Center Provider Survey	1-3 month after OMB approval 10-14 months after OMB approval 22-26 months after OMB approval
Health Center Youth Survey	1-3 month after OMB approval 10-14 months after OMB approval 22-26 months after OMB approval
Quarterly Health Center Performance Measure Reporting Tool	August of 2016; February, May, August of 2017; February, May, August of 2018; February and May of 2019
Annual Health Center Performance Measure Reporting Tool	1-2 months after OMB approval and then November of 2016, November of 2017 and November of 2018
YSO Organizational Assessment	1-3 month after OMB approval 10-14 months after OMB approval 22-26 months after OMB approval
YSO Staff Survey	1-3 month after OMB approval 10-14 months after OMB approval 22-26 months after OMB approval
YSO Youth Survey	1-3 month after OMB approval 10-14 months after OMB approval 22-26 months after OMB approval
YSO Performance Measure Reporting Tool	February, May, August, and November of 2017; February, May, August and November of 2018; February and May of 2019
Awardee Training and Technical Assistance Tool	1 month after OMB approval, Monthly after that
Awardee Performance Measure Reporting Tool	Every December following OMB approval (December 2016, December 2017, December 2018)
Data Reporting
Annual Quality Improvement Report to Awardees	March 2017, March 2018, March 2019
Performance Measure Report to CDC Leadership	May 2017, May 2018, May 2019

17. Reason(s) Display of OMB Expiration is Inappropriate

The expiration date for OMB approval will be displayed on all data collection instruments for which approval is being sought.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certifications.

Bibliography

1. Martin JA, Hamilton BE, Osterman MJK. Births in the United States, 2014. NCHS data brief, no 216. Hyattsville, MD: National Center for Health Statistics. 2015.

2. Counting it up: The public costs of teen childbearing: Key data, T.N.C.t.P.T.a.U. Pregnancy, Editor. 2013, The National Campaign to Prevent Teen and Unplanned Pregnancy: Washington, DC.

3. Santelli, J.S., et al., Explaining recent declines in adolescent pregnancy in the United States: the contribution of abstinence and improved contraceptive use. Am J Public Health, 2007. 97(1): p. 150-6.

4. Secura, G., et al., Provision of no-cost, long-acting contraception and teenage pregnancy. New England Journal of Medicine, 2014. 371: p. 1316-23.

5. Kavanaugh, M.L., et al., Meeting the Contraceptive Needs of Teens and Young Adults: Youth-Friendly and Long-Acting Reversible Contraceptive Services in U.S. Family Planning Facilities. Journal of Adolescent Health, 2013. 52(3): p. 284-292.

6. Sara C. Carpenter, M., MSPH, Robert B. Clyman, MD, Arthur J. Davidson, MD, MSPH, John F. Steiner, MD, MPH and The Association of Foster Care or Kinship Care With Adolescent Sexual Behavior and First Pregnancy. Pediatrics, 2001. 108(3).

7. Church AH. Estimating the effect of incentives on mail survey response rates: a meta-analysis. Public Opinion Quarterly. 1993; 57:62–79.

8. Groves RM, Singer E, Corning AD. A leverage-saliency theory of survey participation: description and illustration. Public Opinion Quarterly. 2000;64:299–308

9. Shettle C, Mooney G. Monetary incentives in government surveys. Journal of Official Statistics.1999; 15:231–250.

10. Singer E, Van Hoewyk J, Gebler N, Raghunathan T, McGonagle K. The effect of incentives on response rates in interviewer-mediated surveys. Journal of Official Statistics. 1999; 15(2):217–230.

11. Singer E. The use of incentives to reduce nonresponse in household surveys. In: Groves RM, Dillman DA, Eltinge JL, Little RJA, editors. Survey Nonresponse. Wiley-Interscience; New York: 2002. pp. 163–178.