Supporting Statement Part A -- OASH_Final REVISED 011917

The purpose of the Guide is to provide an easy to follow tool that community leaders can use to 1) establish a community coalition with broad membership, and 2) develop a community plan for improving adolescent health and well-being that includes multi-impact strategies. To understand whether and how community leaders are able to use the Guide to achieve these two goals, OASH needs information about the Guide’s utility and effectiveness. The evaluation of the Second Decade Project Community Planning Guide (“the evaluation”) is intended to support the goals of OASH’s Second Decade Project of helping community leaders incorporate the needs of children, adolescents and young adults in community growth and development plans, and to improve outcomes of young adults and adolescents.

The second decade of life, between ages 10 and 19, is a critical period when people establish healthy or risky habits that can last a lifetime.¹ At the same time, peer influences increase² and parental controls decrease,³ often before adolescents’ prefrontal cortexes mature enough for them to make the healthiest choices.⁴ These choices include food selection and physical activity⁵; alcohol, tobacco, or other drug use or misuse⁶; early sexual activity that could lead to sexually transmitted diseases, pregnancy, or unhealthy relationships⁷; and school achievement.⁸ Some young people risk poorer outcomes than others, when individual susceptibility interacts with negative community environments.⁹ Both the ecological model of child development¹⁰ and the social determinants of health model¹¹ emphasize the strength of community influences on individual health outcomes and the potential of community-level interventions.

Many evidence-based initiatives can prevent adolescent health risks and promote resilience. Information about programs that reduce risk and promote healthy development in adolescents is widely available. For example, the National Prevention Strategy includes sources of its evidence based recommendations in a detailed appendix, ¹² and the Office of Adolescent Health offers a searchable list of evidence-based teen pregnancy prevention programs (http://www.hhs.gov/ash/oah/oah-initiatives/teen_pregnancy/db/). The challenge for community groups trying to build an environment that promotes adolescent health is not a lack of evidence based approaches, but navigating this plethora of programs. Communities need “prevention architecture” to help them develop a coordinated and integrated approach to the multitude of challenges adolescents face. The public health prevention system, or architecture, that has proven so effective for infants and young children can serve as a model for similar guidance for adolescents. ¹³ It is within this context that the Office of the Assistant Secretary for Health (OASH) developed Promoting Health and Healthy Development in the Second Decade of Life: A Planning Guide for Communities (“the Guide”).

Five communities will participate in the piloting and evaluation of the Guide. The evaluation will provide OASH with critical information regarding the components of the Guide that community leaders found most useful and effective in accomplishing their goals of improving adolescent health and wellbeing; the compilation and inclusiveness of the coalitions implementing the Guide; and the demographic and environmental context of these communities. While secondary data will be collected from sources such as the U.S. Census Bureau American Community Survey and Youth Risk Behavior and National Health Interview Surveys, these sources do not provide nuanced information needed by OASH to understand the contexts in which the Guide is most effective.

This supporting statement is for a new Information Collection Request (ICR) for the evaluation that requires OMB approval. As a preliminary step toward OMB review and approval a 60-day FRN was initiated on May 2, 2016.

2. Purpose and Use of Information

The evaluation will use a range of complementary data sources to determine the extent to which the Guide was a useful and effective tool in communities’ development of 1) a coalition with broad membership, and 2) a long-range plan with action steps to create an optimal environment for healthy development among adolescents. The evaluation will gather input from community leaders, coalition members and secondary stakeholders (adolescent health experts and state/local health department officials) to provide OASH with a comprehensive picture of communities’ experiences using the Guide.

Specifically, the evaluation will help OASH answer the following 9 evaluation questions:

Which sites/communities are appropriate to pilot the Guide?
How useful is the Guide to key leaders in establishing a broad and inclusive coalition?
How effective is the Guide in helping coalition members develop a comprehensive long-term plan?
How comprehensive is the Guide?
How easy is the Guide to follow?
How can the Guide be improved?
To what extent did communities develop a quality plan using the Guide?
How effective is the Guide in identifying a process and issues to inform development of comprehensive community plans?
To what extent have communities taken concrete actions based on the plans they developed?

This evaluation is being conducted by OASH through its contractor, ICF International, pursuant to OASH’s statutory authority (Section 1707 of the Public Health Service Act, 42 U.S.C. § 300u-6).Data from the Evaluation will provide OASH with both qualitative and quantitative information from each of the communities that have piloted OASH’s Second Decade Project Community Planning Guide. This information will enable OASH to understand how community leaders use the Guide to identify high-priority, high-health-impact action steps that will promote healthy development among adolescents. Information from the evaluation will also help OASH revise and improve the Guide as needed to ensure that it is useful and effective for communities, thus achieving OASH’s goal of providing direction for communities working to improve adolescent health.

To achieve the goal of this project the following two qualitative and two quantitative data collection activities will be implemented. All instruments will be administered following receipt of OMB approval.

The qualitative activities include the Community Leader Interviews (CLI) and Coalition Member Focus Groups (CFG):

Community Leader Interviews (CLI)—Community Leader Interviews (CLI) will be conducted with up to 10 identified leaders in each of the pilot communities to examine how they have defined a shared vision, and operationalized the integration of health and social services, coordinated governance, youth involvement, and community culture of health and safety. The CLI will explore how key leaders support development of a diverse coalition, raise awareness, and educate the community about the issues facing adolescents. Additionally, understanding how leaders assess community needs and readiness for change, and how they propose to coordinate and integrate existing programs, services, and supports for adolescents will be examined. The CLI will provide information on how the use of the Guide facilitated discussions and activities related to these components and on the steps taken to convene appropriate stakeholders to realize the vision (see Attachment B).
Coalition Member Focus Groups (CFG)—Coalition Member Focus Groups (CFG) will be conducted virtually via web-conferencing software (Adobe Connect) with coalition members from each of the five pilot sites (with approximately eight people per group). Groups will include: the broader members of the coalition and other stakeholders (e.g., public health officials, school officials, social service agencies, the faith community, the private sector, parents). Additional groups may be conducted (up to two), depending on the size of the site’s coalition and number of stakeholders, to capture as many perspectives as possible. The CFG will assess how the Guide facilitated the work of the coalition to develop a comprehensive plan, using multi-impact measures and identifying high-impact, issue-specific areas that are critically important to address in the site. Groups will assess the ease of following the Guide (e.g., step-by-step approach) to help shape community change (see Attachment D).

The quantitative activities include the Coalition Assessment Surveys (CAS) and Secondary Stakeholder Surveys (SSS):

Coalition Assessment Surveys (CAS)—A web-based coalition assessment survey (CAS) will be administered through Survey Monkey with personalized links for each site’s coalition. All coalition members and primary stakeholders are eligible to participate in the survey; the number recruited will depend on the size of the site’s coalitions and partnerships but we assume up to 50 respondents per site. The CAS will assess coalition members’ perspectives on the coalition membership (diversity, composition, and inclusiveness), member profile (membership length, engagement), usefulness and ease of implementing the Guide, and the activities related to planning the adolescent health comprehensive plan based on the Guide. Coalition success factors will also be examined (recruitment, objectives, shared understanding, resource identification, and committee establishment). Survey questions will be identified from validated coalition assessment instruments and piloted surveys from previous evaluations. CAS administration will take place several weeks after the CFG, so that coalition members who participate in both activities will not feel burdened by the data collection activities (see Attachment F).

Secondary Stakeholder Surveys (SSS)—Secondary stakeholders, specifically adolescent health experts and state/local health department officials, have a stake and interest in adolescent health, but may not necessarily be engaged with a community coalition. The secondary stakeholder survey (SSS) will engage adolescent health researchers and practitioners through a web-based survey to garner additional feedback and assessment of the Guide. The selected panel will provide an expert review of the Guide on the following indicators:

Utility /Capacity Building – The extent to which the Guide provides the user with tools and practical steps for developing a community action plan
Guidance Quality – Comprehensiveness of the guidance and adequacy of the direction given to the communities; whether the steps to guide the coalition in developing a plan tailored to meet the needs and priorities of their community were appropriate
Flexibility – The extent to which the Guide allows community-specific priorities

Secondary stakeholders will complete the web-based SSS administered online via Survey Monkey. Follow-up calls will be completed with Adolescent Health Experts to obtain additional details and clarification on their feedback as needed (see Attachment G).

3. Use of Improved Information Technology

A mixed-methods data collection approach consisting of semi-structured telephone interviews, virtual focus groups, and online surveys will be conducted. Every effort will be made to limit burden on individual respondents who participate in the evaluation through the use of technology. Semi-structured telephone interviews with community leaders will be conducted at convenient times. Focus groups supported by Adobe Connect web conferencing software will be conducted with coalition members. Using Adobe Connect will reduce burden because participants do not have to travel to be a part of the focus group and instead can join from their own computers (and telephones). Two Web-based surveys (the CAS and SSS) will be administered online via Survey Monkey with coalition members and with secondary stakeholders, respectively. Nearly half of all total evaluation participants will be submitting their responses electronically, through Survey Monkey, thus reducing burden.

4. Efforts to Identify Duplication

This project will be the first time that communities have piloted the complete version of the Guide; therefore, the information collected as part of the evaluation is not duplicative. OASH, in partnership with ICF, conducted internet searches and document reviews to ensure that information collected from coalition members and community leaders through the evaluation could not be gathered elsewhere.

5. Involvement of Small Entities

No small businesses will be involved in this study.

6. Consequences if Information Collected Less Frequently

This evaluation includes one-time data collection.

7. Special Circumstances

This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d)(2). No special circumstances apply.

8. Federal Register Notice and Outside Consultations

8.a. Federal Register Notice

As required by 5 CFR 1320.8(d), a 60-day notice was published in the Federal Register on May 2, 2016 Vol. 81, No. 84, pages 26237-26238. No comments were received.

8.b. Outside Consultations

No persons outside the agency were consulted.

9. Payments/Gifts to Respondents

No payment or incentive is provided to individual respondents or community members for participating in the data collection activities associated with the evaluation.

10. Assurance of Confidentiality

This information collection will not involve any personal health information. The personally identifiable information (PII) that will be collected will be minimal (only the contact information for the respondents) and no PII will be publicly disclosed. Identifiers such as name, email address, telephone number, and position will be collected only to facilitate survey and telephone interview administration. The contact information will be entered into a password-protected database that can only be accessed by the limited number of individuals who require access (e.g., selected ICF staff conducting telephone interviews and focus groups, sending emails with links to web-surveys). Identifying information collected to facilitate the administration of surveys will not be stored with survey responses. Once data collection is complete, respondent contact information will be destroyed. Data gathered from the CLI and CFG will be recorded and transcribed into a text document. All transcripts will be stored in a restricted-access network folder, accessible only by members of ICF’s team. A profile of each communities’ activities will be highlighted in the final evaluation report but the name/identity of POCs and specific coalition members will not be included.

All information collected as part of the evaluation will be kept private to the extent allowed by law. The protection of human subjects, the data collection instruments has been reviewed by the ICF institutional review board (IRB). The ICF IRB holds a Federal wide Assurance (FWA00000845; Expiration, April 13, 2019) from the HHS Office for Human Research Protections (OHRP). This review ensures compliance with the spirit and letter of HHS regulations governing such projects; the ICF approved the evaluation on May 18, 2016 (see Attachment H for a copy of the ICF IRB approval documentation)

11. Questions of a Sensitive Nature

There are no questions of a sensitive nature included in any of the data collection activities.

12. Estimates of Annualized Burden Hours and Costs

Exhibit 1 shows the estimated annualized burden hours for the respondents’ time to participate in data collection activities. An estimated 430 respondents will participate. Completing the CLI will take approximately 60 minutes, the CFG 60 minutes, the CAS 15minutes, and the SSS 30 minutes. The total annual burden hours are estimated to be 218.

Up to 10 community leaders from each of the 5 pilot sites will participate in the CLI. Up to 2 focus groups consisting of up to 8 participants per group will be conducted for each of the 5 pilot sites (up to 80 respondents total). Up to 50 secondary stakeholders will participate in the SSS.

Exhibit 1. Estimated annualized burden hours

Form Name	Number of respondents	Number of responses per respondent	Hours per response	Total burden hours
Community Leader Interview (CLI)	50	1	1	50
Coalition Member Focus Group (CFG)	80	1	1	80
Coalition Assessment Survey (CAS)	250	1	.25	63
Secondary Stakeholder Survey (SSS)	50	1	.5	25
Total	430			218

Exhibit 2 shows the estimated annualized cost burden based on the respondents’ time to complete the various data collection activities. The cost burden is estimated to be $$5689 annually.

Exhibit 2. Estimated annualized cost burden

Form Name	Total burden hours	Average hourly wage rate*	Total Respondent cost
Community Leader Interview (CLI)	50	$33.38	$1669
Coalition Member Focus Group (CFG)	80	$21.33	$1706
Coalition Assessment Survey (CAS)	63	$21.33	$1344
Secondary Stakeholder Survey (SSS)	25	$38.78	$970
Total	218		$5689

*The wage rate in Exhibit 2 is based on April 2016 National Industry-Specific Occupational Employment and Wage Estimates, Bureau of Labor Statistics, U.S. Dept. of Labor. The mean hourly wage for Social and Community Service Managers, Community and Social Service Specialists, and Social Scientists (Social and Community Service Managers, $33.38, Occupation Code, 11-9151; Community and Social Service Specialists, $21.33, Occupation Code 21-1099; Social Scientists, $38.78, Occupation Code 19-3099) is located at http://www.bls.gov/oes/current/naics4_999200.htm#11-0000.

13. Estimates of Annualized Respondent Capital and Maintenance Costs

Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection. There are no direct costs to respondents other than their time to participate in the data collection activities associated with the Evaluation.

14. Estimates of Annualized Cost to the Government

Exhibit 3 shows the estimated annualized cost to the government for developing, fielding, and reporting the results of the CLI, CFG, CAS and SSS.

Exhibit 3. Estimated Annualized Cost

The evaluation contract has been awarded to ICF International for evaluation of the Guide. The total cost to the federal government is $271,176. The estimated average annual cost of the contract will be $180,784. Included in these costs are the expenses related to developing and monitoring the evaluation including, but not limited to, developing the evaluation design; developing the evaluation instrumentation; data collection; and data processing and analysis; report development and publication of results. The costs also include overall project development and program management.

15. Changes in Hour Burden

This is a new collection of information.

16. Time Schedule, Publication and Analysis Plans

As soon as OMB approval is received, data collection via the Community Leader Interviews, Coalition Member Focus Groups, and Secondary Stakeholder Surveys will begin. Administration of the Coalition Assessment Survey will take place several weeks after the Coalition Member Focus Groups to reduce burden for coalition members who participate in both data collection activities. Information for the evaluation will be collected by OASH through its contractor, ICF International. The estimated time to conduct these activities is shown below:

Finalize introductory/recruitment letters (drafts shown as Attachment A) to community leaders, coalition members and secondary stakeholders (adolescent health experts and state/local health department officials) (1 week following OMB approval)
Recruit participants for each data collection activity (3 weeks following OMB approval)
Data collection and analysis (3 months following OMB approval)
Produce evaluation of the Second Decade Project Community Planning Guide Report (5 months following OMB approval). A PowerPoint presentation summarizing evaluation findings will be produced. Electronic copies of the report and the PowerPoint presentation will distributed to OASH and the pilot sites. OASH will further distribute the final report to other federal and non-federal adolescent health stakeholders.
Develop manuscript for peer review (6 months following OMB approval).

Quantitative data elements from the SSS and the CAS will be saved and exported to SPSS for analysis; primarily descriptive statistics (e.g., mean, median, distribution) will be performed. Qualitative, narrative data from the CLI and CFG will be analyzed for major themes that are directly related to specific evaluation questions, both by site and at the aggregate level. These data will then be used to produce an evaluation of the Second Decade Project Community Planning Guide Report, PowerPoint presentation and manuscript presenting evaluation findings.

17. Exemption for Display of Expiration Date

OASH does not seek this exemption.

List of Attachments:

Attachment A: OASH Introductory Letters
Attachment B: Community Leader Interview (CLI)
Attachment C: List of interview topics for Community Leader Interview (CLI)
Attachment D: Coalition Member Focus Group (CFG)
Attachment E: List of topics for Coalition Member Focus Group (CFG)
Attachment F: Coalition Assessment Survey (CAS)
Attachment G: Coalition Assessment Survey - Web Screenshots
Attachment H: ICF IRB Approval Documentation
Attachment I: Secondary Stakeholder Survey (SSS)
Attachment J: Secondary Stakeholder Survey - Web Screenshots

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	OMB Clearance Application
Author	hamlin-ben
File Modified	0000-00-00
File Created	2021-01-23

Supporting Statement Part A -- OASH_Final REVISED 011917