Supporting Statement A. v07.rev

INFORMATION COLLECTION REQUEST

NEW

National Evaluation of the DP18-1815 Cooperative Agreement Program: Improving the Health of Americans through Prevention and Management of Diabetes and Heart Disease and Stroke

CATEGORY B: CARDIOVASCULAR DISEASE PREVENTION AND MANAGEMENT

Supporting Statement Part A

Program Official/Contact

Name: Joanna Elmi

Telephone: 770-488-5979

Email: [email protected]

National Center for Chronic Disease

Prevention and Health Promotion

Centers for Disease Control and Prevention

Atlanta, Georgia

September 20, 2019

Resubmitted from NCCDPHP to ICRO 11/13/2019

Reformatted by ICRO 11/18/2019

Additional comments 12/02/2019

Revised and submitted to ICRO for review: 02/05/2020

Revised and submitted to ICRO for review: 04/09/2020

Revised and submitted to ICRO for review: 12/01/2020

November 24, 2020

TABLE OF CONTENTS

A. JUSTIFICATION

1. Circumstances Making the Collection of Information Necessary

2. Purposes and Use of the Information Collection

3. Use of Improved Information Technology and Burden Reduction

4. Efforts to Identify Duplication and Use of Similar Information

5. Impact on Small Businesses or Other Small Entities

6. Consequences of Collecting the Information Less Frequently

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

9. Explanation of Any payment or Gift to Respondents

10. Protection of the Privacy and Confidentiality of Information Provided by Respondents

11. Institutional Review Board (IRB) and Justification for Sensitive Questions

12. Estimates of Annualized Burden Hours and Costs

13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

14. Annualized Cost to the Federal Government

15. Explanation for Program Changes or Adjustments

16. Plans for Tabulation and Publication and Project Time Schedule

17. Reason(s) Display of OMB Expiration Date is Inappropriate

18. Exceptions to Certification for Paperwork Reduction Act Submissions

ATTACHMENTS

Attachment 1.

• Centers for Disease Control and Prevention. National Center for Chronic Disease Prevention and Health Promotion. Notice of Funding Opportunity: Improving the Health of Americans through Prevention and Management of Diabetes and Heart Disease and Stroke – Financed in part by the 2018 Prevention and Public Health Funds. CDC-RFA-DP18-1815PPHF18

Attachment 2.

• Authorizing Legislation Section 301(a) of the Public Health Services Act [42.U.S.C. 242k]

Attachment 3. 1815 Awardees and Information Collection Plan

1. 1815 List of Health Department Awardees

2. 1815 Strategies for Preventing and Controlling Diabetes and Heart Disease and Stroke

3. 1815 Logic Model

4. Category B 1815 Crosswalk of Evaluation Components and Data Collection Tools

5. Category B 1815 Summary of Annualized Respondents

6. Category B 1815 Evaluation Gantt Chart

Attachment 4. Category B Case Studies

1. CQM Health Department Interview Guide

2. CQM Group Discussion Guide

3. CQM Partner Site-Level Interview Guide

4. TBC Health Department Interview Guide

5. MTM Health Department Interview Guide

6. TBC Group Discussion Guide – TBC/MTM

7. TBC Partner Site-Level Interview Guide: Strategy B3

8. MTM Partner Site-Level Interview Guide: Strategy B4

9. MTM Partner Site-Level Interview Guide: Strategy B4 – Pharmacy Managers/Pharmacists

10. CCL Health Department Interview Guide

11. CCL Group Discussion Guide

12. CCL Partner Site-Level Informant Interview Guide

Attachment 5. Category B Cost Study

1. Cost Study Resource Use and Cost Inventory Tool (Category B) – HD Level

2. Cost Study Resource Use and Cost Inventory Tool (Category B) – Partner Site Level

Attachment 6. Category B Recipient-led Evaluations

1. Category B Recipient-Led Evaluation Annual Report Templates – Year 1 Implementation Brief

2. Category B Recipient-Led Evaluation Annual Report Templates – Year 2 Efficiency Strategy Map Report

3. Category B Recipient-Led Evaluation Annual Report Templates – Year 3 Effectiveness Brief

4. Category B Recipient-Led Evaluation Annual Report Templates – Year 4 Sustainability Action Report

5. Category B Recipient-Led Evaluation Annual Report Templates – Year 5 Health Impact Statement

Attachment 7. 30 Day Federal Register Notice Template

Attachment 8.

1. Institutional Review Board Approval Notification or Exemption Determination Part A

2. Institutional Review Board Approval Notification or Exemption Determination Part B

Attachment 9. Introductory/Follow-Up Letters

1. Category B Case Study HD Invitation E-mail

2. Category B Case Study Partner Site-Level Invitation E-mail

3. Category B Case Study Confirmation E-mail

4. Category B Case Study Reminder E-mail

5. Category B Case Study Follow-Up E-mail

List of Acronyms

CCL	Community Clinical Linkages
CDC	Centers for Disease Control and Prevention
CHWs	Community Health Workers
CQM	Clinical Quality Measures
CVD	Cardiovascular Disease
DDT	Division of Diabetes Translation
DHDSP	Division for Heart Disease and Stroke Prevention
DOL	Department of Labor Bureau
EHR	Electronic Health Record
EPET	Evaluation and Program Effectiveness Team
EPMP	Evaluation and Performance Measurement Plan
HBC	High Blood Cholesterol
HBP	High Blood Pressure
HD	Health Department
HIT	Health Information Technology
ICR	Information Collection Request
LCP	Lifestyle Change Program
MTM	Medication Therapy Management
NCCDPHP	National Center for Chronic Disease Prevention and Health Promotion
OMB	Office of Management and Budget
TBC	Team-Based Care

National Evaluation of the DP18-1815 Cooperative Agreement Program - Category B: Cardiovascular Disease Prevention and Management

Overview

This is a three-year information collection request for an evaluation of a five-year Cooperative Agreement program CDC-RFA-DP18-1815PPHF18: Improving the Health of Americans Through Prevention and Management of Diabetes and Heart Disease and Stroke, “1815”. 1815 is a collaboration between two Divisions at the Centers for Disease Control and Prevention (CDC) and is structured into two categories: (1) Category A: Diabetes Management and Type 2 Diabetes Prevention, and (2) Category B: Cardiovascular Disease (CVD) Prevention and Management. This information collection request package focuses on data collection activities for the Category B CVD assessment.

• Goal of the assessment: The purpose of the assessments is to determine all 50 state health departments’ and the Washington, D.C. health department’s progress of utilizing CDC’s DP18-1815 cooperative agreement funds to implement evidence-based strategies and how the efforts are contributing to state- and health-system level changes in support cardiovascular disease.

• Intended use of the resulting data: The data collected from this assessment will be used to (1) determine health departments’ progress using 1815 funds to implement evidence-based strategies; (2) identify practices that have shown promise in preventing and managing cardiovascular disease to share across programs; and (3) to determine how these efforts are contributing to state level, health system or other organizational level changes and outcomes to support the prevention and management of cardiovascular disease.

• Methods to be used to collect data: The assessment is comprised of the following primary data collection methods: (1) site visits and program observations; (2) key informant interviews and group discussions; and (3) a cost inventory tool. Secondary data collection is comprised of a systematic review of health departments’ program records and respective evaluation reports.

• The subpopulation to be studied: Recipients from all 50 state health departments and the Washington, D.C. health department receiving non-research funding from the CDC National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP) through the DP18-1815 Cooperative Agreement and the recipients’ partner sites where 1815-funded activities are implemented.

• How data will be analyzed: Qualitative data collected from primary and secondary sources will be analyzed for key and emerging themes to measure specific constructs related to the implementation of the 1815 strategies for cardiovascular disease. The data will be triangulated with quantitative data collected from the cost study tool, which will be analyzed using descriptive statistics.

1. Justification

A.1 Circumstances Making the Collection of Information Necessary

In the United States, heart disease is the leading cause of death and stroke is the fifth leading cause of death. Cardiovascular disease (CVD), including heart disease, stroke, and other vascular diseases, accounts for about 1 in 3 deaths per year. Primary risk factors for CVD include high blood pressure (HBP) and high blood cholesterol (HBC). These risk factors can be reduced through lifestyle change and clinical care, but behavior modification improvements and health system advances are needed to reduce the need for treatment.

The mission of the Centers for Disease Control and Prevention (CDC), Division for Heart Disease and Stroke Prevention (DHDSP), is to provide public health leadership to improve cardiovascular health for all, reduce the burden, and eliminate disparities associated with heart disease and stroke. CDC provides funding, guidance, and technical assistance to health departments and other partner organizations to promote medication adherence, national guidelines for electronic health records, and better continuity of care across health care and community settings. CDC/DHDSP activities are organized in three core program areas: clinical quality measures (CQM), team-based care/medication therapy management (TBC/MTM), and community-clinical linkages (CCL).

CDC requests OMB approval to collect information needed to assess activities conducted under a new cooperative agreement, CDC-RFA-DP18-1815PPHF18: Improving the Health of Americans Through Prevention and Management of Diabetes and Heart Disease and Stroke, hereafter referred to as “1815” (Attachment 1). CDC is authorized to conduct these activities by the Public Health Service Act (42 U.S.C. 242), Section 301(a) (Attachment 2).

The 1815 cooperative agreement is a collaboration between CDC’s Division of Diabetes Translation (DDT) and CDC’s DHDSP, and is structured in two categories aligning with each Division:

• Category A: Diabetes Management and Type 2 Diabetes Prevention

• Category B: Cardiovascular Disease (CVD) Prevention and Management

Awardees are 50 state health departments and the Washington, D.C., health department (hereafter referred to as “HD recipients,” see Attachment 3a). Health department recipients are working with partner organizations to implement evidence-based strategies for preventing or controlling diabetes and cardiovascular disease in populations and communities that are disproportionately affected by these conditions (Attachment 3b). Recipients are encouraged to implement Category A and B strategies in the same high burden areas/communities, so that work on these strategies is mutually reinforcing and implemented in a coordinated fashion to accelerate progress toward goals. Activities conducted under the 1815 cooperative agreement build upon CDC’s previous work with health departments, health care providers, and community-based organizations to identify promising diabetes and CVD prevention and management practices that can be scaled and replicated. Previous work has included efforts to strengthen coordination within health care systems as well as community-clinical linkages (CDC-RFA-DP13-1305 and CDC-RFA-DP14-1422).

The current information collection request will support evaluation of activities conducted under 1815 Category B, CVD Prevention and Management. The objectives for Category B align with CDC’s three core CVD program areas: clinical quality measures (CQM), team-based care/medication therapy management (TBC/MTM), and community-clinical linkages (CCL). Based on findings from prior CVD prevention and control efforts, evidence-based strategies for meeting each objective have been defined and are currently being implemented by HD recipients. Expected short-, intermediate-, and long-term outcomes of these strategies have also been defined and suggest targets for evaluation (see logic model, Attachment 3c). The objectives for the cooperative agreement, CVD programmatic context, and corresponding strategies selected for further evaluation (B1-B7) are summarized below.

• Clinical Quality Measures (CQM) Objective: Track and monitor clinical measures shown to improve healthcare quality and identify patients with hypertension

Strategies B1 and B2 seek to increase the use of electronic health records (EHR) and health information technology (HIT) to encourage health providers to better monitor and address key risk factors for CVD, including HBP and HBC, to reduce health disparities and improve public health outcomes. Tracking and monitoring clinical measures shown to improve healthcare quality and identify patients with hypertension assists providers in screening for CVD risk factors, providing information on management and treatment protocols, and providing tailored recommendations for CVD management such as adherence to medication and engagement in self-management among patients with high blood pressure and high blood cholesterol.

• Team-Based Care /Medication Therapy Management (TBC/MTM) Objective: Implement team-based care for patients with high blood pressure and high blood cholesterol

Strategies B3 and B4 encourage multidisciplinary teams to work together to identify hypertension, educate patients on CVD and treatments options, and provide guidance to patients on their health behaviors. The collaboration between physician and non-physician practitioners has been shown to be a cost-effective strategy for increasing patients’ medication adherence and lowering blood pressure among diverse populations within a range of settings.

• Community-Clinical Linkages (CCL) Objective: Link community resources and clinical services that support systematic referrals, self-management, and lifestyle change for patients with high blood pressure and high blood cholesterol

Strategies B5, B6, and B7 help those with or at-risk for HBP and HBC by providing referrals and access to community resources and lifestyle change programs. These initiatives support the prevention and management of these conditions and increase their likelihood to follow treatment plans and to be an active participant in managing their health condition.

Table A.1-A provides additional implementation detail about each strategy. Through the 5-year cooperative agreement, each HD recipient is required to implement and evaluate at least three of the Category B strategies.

Table A.1-A Category B: Cardiovascular Disease Prevention and Management Strategies
1815 Goal / CVD Program Area	Strategy Label	Strategy Description
Clinical Quality Measures (CQM)	B1	Promote the adoption and use of electronic health records (EHRs) and health information technology (HIT) to improve provider outcomes and patient health outcomes related to identification of individuals with undiagnosed hypertension and management of adults with hypertension
	B2	Promote the adoption of evidence-based quality measurement at the provider level (e.g. use dashboard measures) to monitor healthcare disparities and implement activities to eliminate healthcare disparities
Team-Based Care / Medication Therapy Management TBC/MTM	B3	Support engagement of non-physician team members (e.g., nurses, nurse practitioners, pharmacists, nutritionists, physical therapists, social workers) in hypertension and cholesterol management in clinical settings
	B4	Promote the adoption of medication therapy management (MTM) between pharmacists and physicians for the purpose of managing high blood pressure, high blood cholesterol, and lifestyle modification
Community-Clinical Linkages (CCL)	B5	Develop a statewide infrastructure to promote sustainability for CHWs to promote management of hypertension and high blood cholesterol
	B6	Facilitate use of self-measured blood pressure monitoring (SMBP) with clinical support among adults with hypertension
	B7	Implement systems to facilitate systematic referral of adults with hypertension and/or high blood cholesterol to community programs/resources

CDC requests Office of Management and Budget (OMB) approval to gather new data to conduct a systematic and in-depth national evaluation of the 1815 Category B efforts. Each strategy (B1-B7) will be assessed in terms of Approach, Efficiency, Effectiveness (reach, health outcomes, and facilitators), Sustainability, and Impact. To improve understanding of context and interactions throughout systems of care, information will be collected from both HD recipients and public health partners.

Overview of Evaluation Components

The national evaluation consists of three components: 1) Case Studies, 2) Cost Study, and 3) Recipient-led Evaluation. Each evaluation component will seek to gather in-depth information about specific program strategies with the intent of a) identifying promising practices, particularly those reaching high-burden populations/communities; b) determining the contribution of each strategy to intended outcomes; c) determining activities that are promising to share among programs; and d) determining the most effective roles of state health departments in supporting health system/community programs for CVD prevention and management.

Case Study Evaluation Component. CDC will conduct case studies to gather in-depth information about each of the three-core heart disease program areas: CQM, TBC/MTM, and CCL. Five HD recipients will participate in each case study, and no HD will participate in more than one case study. Respondents include HD personnel and representatives of community/clinical partner organizations. Information will be collected through in-depth interviews and group discussions.

Overview of Case Studies
Case Study Group	No. Participating HDs	No. of Partner Sites per HD
CQM	5	2-3
TBC/MTM	5	2-3
CCL	5	2-3
TOTAL	N = 15	Maximum N = 45

Cost Study Evaluation Component. Respondents will be 20-25 selected HD recipients, stratified by HHS Region, and their community/clinical partners. HD recipients may participate in both the Case Study component of the evaluation and the Cost Study component of the evaluation. The Cost Study will help CDC understand the costs associated with implementing each 1815 Category B strategy, determine the variability in implementation cost across various HD recipients, and identify the factors driving cost and variability. Data will be collected using a self-administered, web-based Resource Use and Cost Inventory Tool. Separate tools will be used to capture program costs from HD recipients and partner sites.

Overview of the Cost Study
HHS Region	No. HDs per Region (state or WDC)	No. of Partner Sites per Region
1	2-3	2-3
2	2-3	2-3
3	2-3	2-3
4	2-3	2-3
5	2-3	2-3
6	2-3	2-3
7	2-3	2-3
8	2-3	2-3
9	2-3	2-3
10	2-3	2-3
TOTAL	Maximum N = 20-25	Maximum N=50

Recipient-Led Evaluation Component. All 1815 recipients are required to submit an annual performance report on their recipient-led evaluations which entails using a stepwise approach to evaluate 1815 program activities each year, so that they can describe the health impact of their program throughout the cooperative agreement. The evaluation report deliverable will be specialized for each year of 1815 funding/activity.

Overview of the Recipient-led Evaluation
No. of HD Recipients	Year	Recipient-Led Evaluation Deliverable
51	1	Implementation Brief
51	2	Efficiency/Strategy Mapping
51	3	Effectiveness Brief or Manuscript
51	4	Sustainability and Action Report
51	5	Health Impact Statement for each strategy evaluated (minimum of 3 strategies per HD Recipient)

OMB approval is requested for three years. Evaluation findings will be shared with HD recipients, HD recipients’ partner sites that participate in the evaluation, and other key partners that collaborate with the HD recipients. Evaluation findings will help DHDSP demonstrate the reach and impact of evidence-based diabetes and CVD strategies and determine the effect of CDC funding on health system and community-based CVD prevention and management efforts. This determination will include identifying which strategies may be viable for sharing with other programs (e.g., implementing in new sites or in other types of organizations, or even other states). Evaluation findings will help to identify areas where HD recipients may need additional technical support and help inform design of targeted technical assistance and guidance for HD recipients. Evaluation findings will also help HD recipients learn what types of activities are working well within other jurisdictions and what factors facilitate this success, serving as input for their own program planning processes.

CDC’s plan for assessing 1815 Category A activities is described in a separate information collection request.

A.2 Purpose and Use of Information Collection

Data collection will be conducted by the National Evaluation Team, which is comprised of the DHDSP Evaluation and Program Effectiveness Team (EPET) and the Deloitte Consulting Evaluation team.

Table A.2-A provides an overview of how each component of the Category B national evaluation effort supports the overarching evaluation questions that are specified in the 1815 funding opportunity announcement. The three evaluation components address each of the evaluation questions for the respective categories of approach, effectiveness, efficiency, sustainability, and impact. The evaluation sub-questions pertaining to effectiveness are outlined in the 1815 NOFO and will be used to guide the national evaluation efforts. The three sub-concepts: reach, health outcomes, and facilitators, each build upon each other to operationalize the overall concept of effectiveness for this evaluation. This evaluation will assist in helping to identify notable contributions of 1815 funding within health systems and community landscapes supporting CVD prevention and management. A detailed crosswalk of the evaluation components and data collection tools, evaluation questions, and respondent type can be found in Attachment 3d.

Table A.2-A. Overview of Evaluation Questions and Evaluation Components

Overarching Evaluation Questions (From NOFO)	Evaluation Components
	1. Category B Case Studies	2. Category B Cost Study	3. Recipient Led Evaluation
Approach: To what extent has the recipient’s implementation approach resulted in achieving the desired outcomes?	•		•
Effectiveness (Reach): Question 1: To what extent has the recipient increased the reach of Category A and B Strategies to prevent and control diabetes and cardiovascular disease?	•		•
Effectiveness (Health Outcomes): Question 2: To what extent has implementation of Category A and B strategies led to improved health outcomes among the identified priority population(s)?	•		•
Effectiveness (Facilitators) Question 3: What factors were associated with effective implementation of Category A and B strategies?	•	•	•
Efficiency: To what extent has the NOFO affected efficiencies with regards to infrastructure, management, partners, and financial resources?	•	•	•
Sustainability: To what extent can the strategies implemented be sustained after the NOFO ends?	•	•	•
Impact: To what extent have the strategies implemented contributed to a measurable change in health, behavior, or environment in a defined community, population, organization, or system?	•	•	•

Category B Evaluation Component 1: Case Studies

The purpose of the Category B case studies is to gather in-depth information about each of the three-core heart disease program areas: CQM, TBC/MTM, and CCL. Case studies will provide a nuanced understanding of the ways in which strategies under each of the three program areas are implemented, help identify which types of interventions work better than others and help determine how those interventions are contributing to change at the health system and community levels. Five HD recipients will be selected for each program area case study (for a total of 15 participating HD recipients). These HD recipients will then work with CDC to each identify 2-3 partner sites for inclusion in the case study (for a total of no more than 45 sites across the 15 HD recipients). Data collection will include staff interviews and group discussions. Small group discussions will be scheduled with health department and partner site staff to facilitate scheduling when possible.

HD-Level Recipient Interviews (Att. 4a, 4d, 4e, 4j): Semi-structured interviews or small group discussions to facilitate scheduling when possible will be conducted with 3 to 5 health department staff who are working on the program area selected for inclusion in the case study. Participants will be interviewed a total of three times, once in Years 3, 4, and 5 of the cooperative agreement. In Year 3 the interviews will be held virtually due to the coronavirus pandemic. In Years 4 and 5 interviews will be in-person and all participating individuals will be asked to conform to CDC and local COVID-19 public health recommendations. The same HD-Level interview guide will be used each time. The purpose of these interviews is to gain perspective on what is and is not working well in implementing the strategies under each program area, understand the practical considerations in implementing various interventions for each strategy, and understand HD recipients’ perspectives on how their efforts are influencing change within the partner sites they are working in.

HD-Level Group Discussions (Att. 4b, 4f, 4k): Within each HD recipient selected for the case study, group discussions will be held with approximately 4-8 staff members working on the program area selected for inclusion in the case study. HD recipient representatives will participate in group discussions three times, once in Years 3, 4, and 5 of the cooperative agreement. In Year 3 the discussions will be held virtually due to the coronavirus pandemic. In Years 4 and 5 discussions will be in-person and all participating individuals will be asked to conform to CDC and local COVID-19 public health recommendations. The same HD-level discussion guide will be used each time. The purpose of these discussions is to assess and validate the extent to which Category B strategies and HD recipient activities align with intended program outcomes. Group discussions will also produce conversational dynamics and direction that cannot be captured from single in-person interviews.

Partner Site-Level Interviews (Att. 4c, 4g, 4h, 4i, 4l): Semi-structured phone interviews or small group discussions to facilitate scheduling when possible will be conducted with 2-3 interviewees at each partner site participating in the case study (for a total of no more than 135 interviewees across the participating partner sites). Participants will be interviewed twice, once in Year 3 and once in Year 5 of the cooperative agreement. The same partner site-level interview guide will be used both times. These interviews will be conducted via telephone. Partner site interviews will explore the perceived benefits of the HD recipient’s support within each respective program area and determine how partner site-level implementation of 1815 Category B strategies is influencing provider, patient, health system, and/or other outcomes.

Category B Evaluation Component 2: Cost Study

Cost Study Resource Use and Cost Inventory Tool (Att. 5a, 5b): A cost study will be conducted to understand the costs associated with implementing each 1815 Category B strategy, determine the variability in implementation cost across various HD recipients, and identify the factors driving cost and variability. Findings from the cost study will be helpful in informing cost of replication and scaling up of key strategies. Results can also help determine what resources are needed for sustaining these efforts beyond the 1815 period of performance and lay the foundation for future cost-effectiveness analyses. Cost study data will be collected at two different points of time - once in Year 3 and once in Year 5 of the cooperative agreement, using the same Resource Use and Cost Inventory Tool. Data will be collected from 20-25 purposefully selected HD recipients and from 25-50 voluntary partner sites that work with participating HD recipients. Data will be collected through a web-based Cost Study Resource Use and Cost Inventory Tool. Separate tools will be used to capture program costs from HD recipients and partner sites. The Resource Use and Cost Inventory Tool will be self-administered by HD recipients and partner sites.

Category B Evaluation Component 3: Recipient-Led Evaluations

Category B Recipient-Led Evaluation Deliverable Templates (Att. 6a, 6b, 6c, 6d, 6e): All 1815 recipients are also required to report the progress of their recipient-led evaluations of Category B strategies to CDC on an annual basis. Each year, recipients will submit a different evaluation deliverable that reports findings associated with a specific core evaluation area (Year 1: Implementation Brief, Year 2: Efficiency, Year 3: Effectiveness, Year 4: Sustainability, and Year 5: Health Impact) for three Category B strategies. These deliverables are in lieu of a traditional evaluation report. Recipient-led evaluations for Category B strategies will emphasize a different evaluation area for each year of the cooperative agreement, building upon the previous year’s evaluation work. Therefore, the evaluation deliverable template is distinct for each year. The purpose of the annual evaluation deliverable is to ensure accountability, improve the program at the recipient and CDC levels, and expand practice-based evidence through sharing successful strategies. Recipients will submit the annual reports using the templates and associated guidance document that CDC provides.

A.3 Use of Improved Information Technology and Burden Reduction

Electronic Data Collection: The Category B recipient-led evaluation templates and reports will be completed using Microsoft Word. The Category B cost study inventory tool will be completed by recipients via Microsoft Excel and submitted electronically via email by the HD recipients. These templates are structured to minimize HD recipient effort required to collect and enter data, thereby reducing burden on HD recipients.

Non-Electronic Data Collection: Interview and group discussion data will be collected via in-person, video conference or phone interviews and documented in writing by the National Evaluation Team. Interviews and group discussions will also be digitally recorded, per the consent of participants and transcribed by the National Evaluation Team. In-person data will be collected during site visits to HD recipients and partner sites in Years 4 and 5. There will be no additional burden on HD recipients and their partner sites other than their participation in interviews with the National Evaluation Team.

A.4 Efforts to Identify Duplication and Use of Similar Information

This ICR is for a national level evaluation of a new, five-year, federally funded cooperative agreement issued by the CDC, which began in October 2018. As the cooperative agreement is new and data to be collected through this evaluation relates directly to HD recipients’ implementation of 1815 strategies, the information to be collected from HD recipients is not available from other sources, including other federal agencies, academic institutions, and/or NGOs. Additionally, there have been no other evaluation data collection efforts conducted to date, nor does the information to be collected exist in any existing centralized data source.

Each data collection tool submitted through this package has a distinct purpose with no overlap across other tools or data collection efforts including routine performance measurement data collection.

All 1815 recipients are required to report performance measure targets and data on an annual basis for Category B. The performance measures provide standard quantitative measures of recipient progress towards expected outcomes and are collected on a routine basis. The national evaluation will gather data to better contextualize and understand the reported measures at the state and health system level providing insight on organizational level changes and outcomes that addresses the specific questions outlined in the evaluation that is not currently captured by routine reporting (Table A.2-A & Attachment 3d). Performance measure templates have been submitted under the Generic Information Collection mechanism of the NCCDPHP OMB Clearance Center (O2C2) – – OMB ICR No. 0920-1132, Performance Progress and Monitoring Report, with the expiration date of 10/31/2022.

The 1815 National Evaluation data will be analyzed in conjunction with 1815 performance measure data as a secondary data source, which is submitted by HD recipients on an annual basis. Specifically, interviews and group discussions conducted as part of the Category B Case Studies (Att. 4a – 4l) will provide recipient and partner site-specific information that cannot be obtained through HD recipient calls with CDC project officers and/or evaluators, routine reporting documents, or other data collected through OMB ICR No. 0920-1132. Similarly, cost information collected through the Category B Cost Study tools (Att. 5a, 5b) is not captured through required cooperative agreement financial reports submitted by recipients. Recipient-led evaluation deliverables are tightly focused on HD recipient- specific implementation experiences, which will not be systematically collected or reported through any other existing mechanism.

The data collection activities included in this ICR will allow CDC to capture critical information needed to continuously improve programmatic efforts for the 1815 cooperative agreement and clearly demonstrate the use of federal funds.

A.5 Impact on Small Businesses or other Small Entities

It is possible that some partner sites which will be recruited to complete the Category B case study site level interviews and Category B resource use and cost inventory tool may be representatives of a small business, such as a small health center or community-based organization offering health education. However, CDC anticipates that this will be a rare occurrence, and participation is completely voluntary. There are no specific requirements for small businesses. Questions have been limited to the absolute minimum required for the intended use of the data/information. Outside of these partner sites, there will be no other small businesses involved in the data collection for the National Evaluation of the 1815 cooperative agreement.

A.6 Consequences of Collecting the Information Less Frequently

There are different data collection frequencies for different components of the evaluation. The frequency of data collection, along with consequences of collecting information less frequently, are detailed below.

Category B Case Studies

HD Interview Guides (Att. 4a, 4d, 4e, 4j) and HD Group Discussion Guides (Att. 4b, 4f, 4k): HD recipient representatives will respond to the proposed information collection three times, once in Year 3, 4, and 5 of the cooperative agreement. The frequency of data collection will provide insights on the nuanced changes in the adoption and uptake of the proposed CQM, TBC/MTM, or CCL strategies. Group dynamics and team members’ interactions with one another during the group conversation creates an opportunity to capture additional information around strategy implementation that may not arise during single in-person interviews, providing the rationale for including both interviews and group discussions for HD recipient representatives. If the data collection is conducted less frequently, CDC staff will not be able to track changes in program implementation and determine how HD-Level interventions are contributing to changes at the health system and community level, and therefore, they will be unable to identify promising practices that can be scaled and replicated to better improve health outcomes.

Partner Site-Level Interview Guides (Att. 4c, 4g, 4h, 4i, 4l): Stakeholders from recipient HDs’ partner sites will participate in interviews or small group discussions twice, once in Year 3 and once in Year 5 of the cooperative agreement. If the partner site interviews are conducted less frequently, CDC will not have a sense of how 1815 funded activities impact partner sites’ implementation of CQM, TBC/MTM, or CCL strategies and corresponding strategies within health systems and community-based programs.

Category B Cost Study

Cost Study Resource Use and Cost Inventory Tool (5a, 5b): Respondents will participate in the cost study data collection efforts twice, once in Year 3 and once in Year 5 of the cooperative agreement. During both data collection cycles, data will be collected retrospectively for the previous program year since there is a lag in the availability of state-level expenditure and spend down data. Collection of cost data at two points during the cooperative agreement period is required to differentiate between the start-up and ongoing implementation and maintenance costs for the 1815 cardiovascular disease prevention and management strategies.

Category B Recipient-Led Evaluations

Category B Recipient-Led Evaluation Deliverable Template (Att. 6a, 6b, 6c, 6d, 6e). All HD recipients are required to conduct an evaluation of at least three of their 1815 Category B strategies for each year of the cooperative agreement. The purpose of the annual evaluation deliverable is to ensure accountability, improve the program at the recipient and CDC levels, and expand practice-based evidence through sharing successful strategies. If the collection is conducted less frequently, CDC will not be able to track the progress of recipients’ evaluation and will be unable to assess the health impact of the program by the end of the cooperative agreement.

A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This request fully complies with the regulation 5 CFR 1320.5.

A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A. Federal Register Notice

A 60 Day Federal Register Notice was published in the Federal Register on July 5, 2019 (Attachment 7). There were no public comments in response to the Notice.

B. Other Consultations

The data collection instruments were designed collaboratively by the National Evaluation Team. In addition, a select group of no more than nine HD recipients will be asked to review the data collection tools prior to the 30-day FRN review period.

A.9 Explanation of Any Payment or Gift to Respondents

No payments or gifts will be provided to Category B respondents.

A.10 Protection of the Privacy and Confidentiality of Information Provided by Respondents

CDC’s Privacy Office has reviewed this submission and has determined that the privacy act does not apply. The data collection does not involve collection of sensitive or identifiable personal information. Although contact information is obtained for each recipient, the contact person provides information about the organization, not personal information. No system of records will be created under the Privacy Act.

A.11 Institutional Review Board (IRB) and Justification for Sensitive Questions

Respondents for the data collection efforts included in the ICR are cooperative agreement recipients and staff members from their partner sites. The data collection does not involve research with human subjects. Data to be collected is not sensitive in nature and reflects information at the organization level rather than individual level. The information collection does not require consent from individuals or IRB approval (Attachment 8a, 8b).

A.12 Estimates of Annualized Burden Hours and Costs

A.12-A. Estimated Annualized Burden Hours

CDC estimates there are approximately 253 people each year who will participate in the evaluation of Category B. The estimated burden per response is between 1 and 2 hours to complete each data collection tool, except for the annual evaluation reporting deliverable templates, which are anticipated to require up to 8 hours to complete. This is due to the complexity of the information being requested in the evaluation report. Burden time estimates have been calculated based on the National Evaluation team’s experience developing and administering surveys and interviews. Over the three-year requested approval period of this information collection request, the total estimated annualized burden for the 51 current HD recipients and corresponding partner sites implementing Category B strategies is approximately 743 hours as summarized in the tables below.

Total burden has been calculated to reflect annualized burden hours over the three-year collection period. It has been weighted appropriately to reflect the number of times (once, twice, or thrice) data is being collected over the three-year collection period. Annualized number of respondents has also been calculated for the three-year data collection period (Attachments 3e).

Table A.12-A. Annualized Burden Hours

Type of Respondent	Form Name	Number of Respondents	Number of Responses per Respondent	Average Burden per Response (in hours)	Total Burden (in hours)
Health Department (1815 Recipient)	Att. 4a: CQM Health Department Interview Guide	17	2	1.5	26
	Att. 4b: CQM Group Discussion Guide	27	2	2	54
	Att. 4d: TBC Health Department Interview Guide	9	2	1.5	14
	Att. 4e: MTM Health Department Interview Guide	8	2	1.5	12
	Att. 4f: TBC Group Discussion Guide	27	2	2	54
	Att. 4j: CCL Health Department Interview Guide	17	2	1.5	26
	Att. 4k: CCL Group Discussion Guide	27	2	2	54
	Att. 5a: Cost Study Resource Use and Cost Study Inventory Tool – Health Department	8	1	2	16
	Att. 6c-6e: Recipient-Led Evaluation Annual Report Template – Year 3 Effectiveness Brief	51	3	8	408
Partner/Site-Level	Att. 4c: CQM Partner Site-Level Interview Guide	15	1	1	15
	Att. 4g: TBC Partner Site-Level Interview Guide	8	1	1	8
	Att. 4h: MTM Partner Site-Level Interview Guide	7	1	1	7
	Att. 4l: CCL Partner Site-Level Informant Interview Guide	15	1	1	15
	Att. 5b: Cost Study Resource Use and Cost Inventory Tool – Partner/Site Level	17	1	2	34
				Total	743

A.12-B. Estimated Annualized Cost to Respondents

Total cost has been calculated to reflect annualized cost over the three-year collection period. Annualized cost has been calculated using U.S. Department of Labor Bureau (DOL) of Labor Statistics estimates using the best approximation of DOL occupation titles and wage classification for each type of respondent. The expected equivalent occupation titles and wages for target respondents’ positions were obtained from the DOL database and used to populate Table A.12-B. In some cases, individuals in different roles/positions (i.e., occupational titles) will respond to the same data collection tool. The average hourly wage is a composite and average of the identified wage classification for each type of respondent.

Table A.12-B. Annualized Cost to Respondents

Type of Respondent	Form Name	Number of Respondents	Number of Responses per Respondent	Total Burden Hours	Average Hourly Wage	Total Respondent Cost
Health Department (1815 Recipient)1	Att. 4a: CQM Health Department Interview Guide	17	2	26	$44.00	$1,144.00
	Att. 4b: CQM Group Discussion Guide	27	2	54	$44.00	$2,376.00
	Att. 4d: TBC Health Department Interview Guide	9	2	14	$44.00	$616.00
	Att. 4e: MTM Health Department Interview Guide	8	2	12	$44.00	$528.00
	Att. 4f: TBC Group Discussion Guide	27	2	54	$44.00	$2,376.00
	Att. 4j: CCL Health Department Interview Guide	17	2	26	$44.00	$1,144.00
	Att. 4k: CCL Group Discussion Guide	27	2	54	$44.00	$2,376.00
	Att. 5a: Cost Study Resource Use and Cost Study Inventory Tool – Health Department2	8	1	16	$41.00	$656.00
	Att. 6c-6e: Recipient-Led Evaluation Annual Report Template – Year 3 Effectiveness Brief3	51	3	408	$36.00	$14,688.00
Partner/Site-Level	Att. 4c: CQM Partner Site-Level Interview Guide4	15	1	15	$63.00	$945.00
	Att. 4g: TBC Partner Site-Level Interview Guide5	8	1	8	$54.00	$432.00
	Att. 4h/4i: MTM Partner Site-Level Interview Guide6	7	1	7	$80.00	$560.00
	Att. 4l: CCL Partner Site-Level Informant Interview Guide7	15	1	15	$58.00	$870.00
	Att. 5b: Cost Study Resource Use and Cost Inventory Tool – Partner/Site Level8	17	1	34	$55.00	$1,870.00
					Total	$30,581.00

A.13 Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

No capital or maintenance costs are expected. Additionally, there are no start-up, hardware or software costs.

A.14 Annualized Cost to the Government

A. Development, Implementation, and Maintenance

The average annualized cost to the Federal Government is $1,041,626.00 as summarized in Table A.14-A. Major cost factors for tool development include form design and development costs and maintenance costs.

Table A.14-A. Annualized Cost to the Federal Government – Category B

Cost Category	Total Cost Over 3-Year Period	Total Annualized Cost
CDC - DHDSP Personnel 100% GS-12@$ 78,446/year = $78, 446 50% GS-13 @ $ 93,282/year = $46,641 50% GS-13 @ $ 93,282/year = $46,641 25% GS-14 @ $ 110,231/year = $27,558 Subtotal, CDC Personnel	$597,858.00	$199,286.00
Data Collection Contractor Category B = $842,340.00 Total, Category B Contractor	$2,527,022.00	$842,340.00
Total, Category B	$3,124,880.00	$ 1,041,626.00

A.15 Explanation for Program Changes or Adjustments

This is a new collection.

A.16 Plans for Tabulation and Publication and Project Time Schedule

Tabulation

Attachment 3f is a Gantt chart for data being collected over a four-year period during Years 2 through 5 of the cooperative agreement. Recipient-submitted reporting deliverables will be due to CDC annually on September 30^th. Final reports for the last year of the cooperative agreement (July 1, 2022 – June 30, 2023) will be due no later than September 30, 2023 (90 days after the end of the funding period). OMB approval is being requested for three years with the desired data collection process to begin in March 2020. CDC plans to seek an extension of OMB approval for the final data collection for the national evaluation.

B. Publication Plan

Information collected by the HD recipients will be reported in internal CDC documents and shared with state-based programs.

Category B Case Studies: Throughout the case study period of Years 3 through 5, CDC will provide summary reports of information collected through in-person and virtual interviews, group discussions, and document reviews from site visits. Information will be shared via web, in-person, and print formats through written reports and manuscripts, oral presentations for key stakeholders such as 1815 HD recipients and their partners, and during key milestones and/or decision-making timeframes Additionally, three to five-page long case study briefs will be shared in Years 4, 5, and 6 to reflect activities from program Years 3-5. They will highlight the most actionable findings, spotlight promising practices, and translate findings into program recommendations. These case study briefs will also build upon the findings from each year, and in Year 6, a final case study report will summarize and highlight key findings and patterns that will help articulate the extent to which the CQM strategies and activities met the proposed 1815 outcomes.

Category B Cost Study: Cost study data will be collected and shared during two points in the 5-year program period: Year 3 and Year 5. In Year 3, a cost study brief will be developed to provide an overview of the initial costs of implementing the strategies. After Year 5, a cost study report highlighting the initial and overall actual costs of the strategy implementation will be produced. A final cost study report will be disseminated via web, print, and in-person formats through written reports and manuscripts and as oral presentations with key stakeholders at the end of the cooperative agreement.

Recipient-Led Evaluation Reporting Deliverables: Findings noted within the recipient-led evaluation deliverables will be combined with findings from the case studies, partner site-level rapid evaluations, and performance measures to provide a comprehensive overview of HD recipient progress and outcomes.

Qualitative Data: Qualitative data from key informant interviews, group discussions, and the contribution analysis will be imported and analyzed separately using NVivo. The National Evaluation Team will conduct both content and thematic content analysis for an examination of both manifest (i.e., the actual words used) and latent (i.e., the underlying meaning of the words) content on open-ended statements to identify key themes. The thematic analysis will be theory-driven, based on the program logic model and program operational guidance. The National Evaluation Team will construct a codebook to facilitate the thematic analysis, developing a-priori codes based on themes expected per the program logic model and program operational guidance. The team will revise the coding structure in an iterative manner to ensure that emergent themes are captured in a systematic manner. For the recipient-led EPMPs and evaluation reports, the National Evaluation Team will conduct a systematic analysis to assess the strategies that are being evaluated across the recipients, their evaluation questions, and proposed indicators. The qualitative analysis and the systematic analysis will look across the collected data for similarities and differences in barriers, facilitators, and implied impacts and recommendations related to continuous quality improvement.

Quantitative Data: Close-ended responses from HD and partner site-level interviews, surveys, and other quantitative data such as performance measures and de-identified patient reports, will be analyzed descriptively. Data may be stratified by strategy, demographic characteristics, or other factors, with the potential for using cross-tabs and other techniques to break-down data by components of interest.

For case studies, it is not expected that statistical tests will be appropriate given the sources of data, small numbers of respondents, and purposes of the case studies.

For the recipient-led evaluations, insights will be triangulated with findings from the other performance monitoring assessments to either corroborate or refute other findings, thereby strengthening the conclusions to be drawn from the assessment.

Cost Analysis (Category B): For each strategy, we will present key descriptive statistics (e.g., mean, median, minimum, maximum and variance) for costs of implementing a strategy across all participating HD recipients. Comparisons will look at cost variance across strategies within the same HD as well as cost variance for a given strategy across HD recipients. Given the small sample for the HD-level study, regression analyses will not be feasible. However, we will use performance measure data and qualitative data from the National Evaluation to contextualize the cost data and qualify potential factors for cost variance. For instance, costs may be higher due to number of participants served, inefficiencies in the system, structural bottlenecks, geographic and sociodemographic characteristics of the areas served, or due to incorrect reporting.

Provided there is an adequate sample size at the partner-level, we will conduct multivariate regression analysis to identify the potential drivers of cost for a given strategy, as well as the key factors driving cost variability across implementing sites. Independent variables such as size of the site, area served (urban vs rural), type of organization (independent practice vs FQHC vs hospital system) will be used as part of the model to understand cost drivers. Similar to the HD-level analysis, we will use data from the National Evaluation (including recipient-led evaluations) to qualitatively assess factors that could potentially affect the costs of the scale-up of interventions. This will assist in identifying possible sources of economies and diseconomies of scale, which in turn can help model non-linear cost functions.

A.17 Reason(s) Display of OMB Expiration Date is Inappropriate

The data collection tools will display the expiration date for OMB approval.

A.18 Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification statement.

1 HD Recipient Staff (Program Director; Team Lead/Manager; Evaluator; Health Scientist) = [(Medical and Health Services Manager ($54.68) + Medical Scientist ($46.36) + Epidemiologist ($36.39) + Environmental Scientists and Specialists, including health ($37.30)]/(4) = $43.68 = $44.00

2 HD Recipient Staff (Team Lead/Manager; Evaluator) = [Medical scientist ($46.36) + Epidemiologist ($36.39)]/(2) = $41.37 = $41.00

3 HD Recipient Staff (Evaluator) = Epidemiologist ($36.39) = $36.00

4 Partner Site Staff (Providers, Pharmacists, Nurses, and Administrative Staff) = [Provider ($101.43) + Pharmacist ($58.52) + Registered Nurse ($36.30) + Medical and Health Services Manager ($54.68)]/ (4) = $62.73 = $63.00

5 Partner Site Staff (Providers, Pharmacists, Nurses) and health care extenders (CHWs) = [Provider ($101.43) + Pharmacist ($58.52) + Registered Nurse ($36.30) + CHWs ($20.36)]/ (4) = $54.15 = $54.00

6 Partner Site Staff (Providers, Pharmacists) = [Provider ($101.43) + Pharmacist ($58.52)]/ (2) = $79.98 = $80.00

7 Partner Site Staff (Providers, Nurses) and health care extenders (CHWs) = [Provider ($98.02) + Medical and Health Services ($54.58) + CHWs ($20.90]/ (3) = $57.83 = $58.00

8 Site Staff = Medical and Health Services Manager = $54.68 = $55.00

21