Supporting Statement A

Supporting Statement A.docx

[NCCHPHP] Comprehensive Evaluations of theDP-23-003, DP-23-004, and DP-23-0005 Cooperative Agreement Programs: The National Cardiovascular Health Program, The Innovative Cardiovascular Health Program

OMB: 0920-1453

Document [docx]
Download: docx | pdf



Comprehensive Evaluations of the DP-23-0003, DP-23-0004, and DP-23-0005 Cooperative Agreement Programs: The National Cardiovascular Health Program, The Innovative Cardiovascular Health Program, and The Well-Integrated Screening and Evaluation of Women Across the Nation Program (WISEWOMAN) Program



[OMB No. 0920-xxxx] [OMB expiration date]


Supporting Statement A








Program Official/Contact

Julia Jordan

Health Scientist

National Center for Chronic Disease Prevention and Health Promotion

Centers for Disease Control and Prevention

P: (770) 488-1053

F: (770) 488-8151

[email protected]


8/12/2024

TABLE OF CONTENTS




ATTACHMENTS


1. Centers for Disease Control and Prevention National Center for Chronic Disease Prevention and Health Promotion Notice of Funding Opportunity

1a. The National Cardiovascular Health Program CDC-RFA-DP-23-0004

1b. The Innovative Cardiovascular Health Program CDC-RFA-DP-23-0005

1c. WISEWOMAN: Well-Integrated Screening and Evaluation of WOMen Across the Nation CDC-RFA-DP-23-0003


2. Authorizing Legislation

2a. Public Health Service Act [42 U.S.C. 247b]

2b. Patient Protection and Affordable Care Act [42 U.S.C 300u-11]

2c. Public Health Service Act [42 U.S.C. 241a] 93.426

2d. Public Health Service Act [42 U.S.C. 300n-4a]

2e. Public Health Service Act [42 U.S.C. 300l]

2f. FY 2023 Consolidated Appropriations Act (Pub. L. 117-328, Div. H).

2g. Public Health Service Act [42 USC 300k]


3. Evaluability Assessment Data Collection Instruments

3a. Evaluability Assessment Nomination Form - National/Innovative

3b. Evaluability Assessment Nomination Form - WISEWOMAN

3c. Evaluability Assessment Interview Guide - National/Innovative Learning Collaborative

3d. Evaluability Assessment Interview Guide - CQM National/Innovative Recipient-Level

3e. Evaluability Assessment Interview Guide - CQM WISEWOMAN Recipient-Level

3f. Evaluability Assessment Interview Guide - CQM National/Innovative Partner-Level

3g. Evaluability Assessment Interview Guide - CQM WISEWOMAN Partner-Level

3h. Evaluability Assessment Interview Guide - TBC National/Innovative Recipient-Level

3i. Evaluability Assessment Interview Guide - TBC WISEWOMAN Recipient-Level

3j. Evaluability Assessment Interview Guide - TBC National/Innovative Partner-Level

3k. Evaluability Assessment Interview Guide - TBC WISEWOMAN Partner-Level

3l. Evaluability Assessment Interview Guide - CCL National/Innovative Recipient-Level

3m. Evaluability Assessment Interview Guide - CCL WISEWOMAN Recipient-Level

3n. Evaluability Assessment Interview Guide - CCL National/Innovative Partner-Level

3o. Evaluability Assessment Interview Guide - CCL WISEWOMAN Partner-Level


4. Exploratory Assessment Data Collection Instruments

4a. Exploratory Assessment Interview Guide - National/Innovative Learning Collaborative

4b. Exploratory Assessment Interview Guide - CQM National/Innovative Recipient-Level

4c. Exploratory Assessment Interview Guide - CQM WISEWOMAN Recipient-Level

4d. Exploratory Assessment Interview Guide - CQM National/Innovative Partner-Level

4e. Exploratory Assessment Interview Guide - CQM WISEWOMAN Partner-Level

4f. Exploratory Assessment Interview Guide - TBC National/Innovative Recipient-Level

4g. Exploratory Assessment Interview Guide - TBC WISEWOMAN Recipient-Level

4h. Exploratory Assessment Interview Guide - TBC National/Innovative Partner-Level

4i. Exploratory Assessment Interview Guide - TBC WISEWOMAN Partner-Level

4j. Exploratory Assessment Interview Guide - CCL National/Innovative Recipient-Level

4k. Exploratory Assessment Interview Guide - CCL WISEWOMAN Recipient-Level

4l. Exploratory Assessment Interview Guide - CCL National/Innovative Partner-Level

4m. Exploratory Assessment Interview Guide - CCL WISEWOMAN Partner-Level


5. Cost Study Data Collection Instruments

5a. Resource Use and Cost Inventory Tool - Recipient-Level

5b. Resource Use and Cost Inventory Tool - Partner-Level

5c. Cost Study Interview Guide - Recipient-Level

5d. Cost Study Interview Guide - Partner-Level


6. 60-Day Federal Register Notice


7. Non-Research Determination





JUSTIFICATION SUMMARY

Goal of the project: The purpose of the project is to conduct a comprehensive implementation and outcome evaluation to assess the unique contributions of The National Cardiovascular Health Program (The National CVH Program), The Innovative Cardiovascular Health Program (The Innovative CVH Program), and Well-Integrated Screening and Evaluation of Women Across the Nation (WISEWOMAN) cooperative agreements to identify promising cardiovascular disease (CVD) prevention and management practices that can be scaled and replicated, advance health equity, and the transformation of health systems for improved CVD prevention and management. The programs work together to create efficiencies and improve outcomes in jurisdictions that receive funding for two or more of the cooperative agreements. This evaluation aims to describe program implementation, assess the extent to which short-term, intermediate, and long-term outcomes have been met, and estimate the costs involved in program implementation.


Intended use of the resulting data: The findings from the data collection will provide tailored, action-oriented, and timely recommendations for program improvement throughout the program period. Findings will contribute to the evidence base, support replication and scaling of promising program strategies, and inform future funding opportunities supported by CDC.


Methods to be used to collect data: The two methods of information collection are 1) qualitative semi-structured interviews with recipients, their partnering sites, and Learning Collaborative members and 2) cost data collection via key informant interviews and an Excel-based cost inventory tool completed by funding recipients and partnering sites. We are collecting program-level qualitative and cost implementation and outcome data. We are not collecting individual-level data. This evaluation does not request sensitive or personally identifiable information.


The subpopulation to be studied: Recipients receiving non-research funding through The National CVH Program, The Innovative CVH Program, and the WISEWOMAN cooperative agreements, recipients’ partner sites, and Learning Collaborative members in jurisdictions where funded activities are implemented.


How data will be analyzed: Thematic analysis will be used to analyze qualitative data (i.e., interviews). Interview data will be analyzed in aggregate and discussed in summary reports that do not contain any personal identifiers. Quantitative data (i.e., Cost Inventory Tool) will be analyzed using descriptive statistics, such as counts, means, and standard deviations. Qualitative and quantitative data will be triangulated to synthesize key findings across data sources.

Shape2




JUSTIFICATION

A1. Circumstances Making the Collection of Information Necessary

This is a new three-year information collection request (ICR) for a comprehensive evaluation of three five-year cooperative agreement programs that target cardiovascular disease (CVD) prevention and management with a focus on high blood pressure; 1) CDC-RFA-DP-23-0004: The National Cardiovascular Health Program (The National CVH Program), 2) CDC-RFA-DP-23-0005: The Innovative Cardiovascular Health Program (The Innovative CVH Program), and 3) CDC-RFA-DP-23-0003: Well-Integrated Screening and Evaluation of Women Across the Nation (WISEWOMAN), led by the Division of Heart Disease and Stroke Prevention (DHDSP) at the Centers for Disease Control and Prevention (CDC) (Attachment 1). These initiatives are authorized under Section 317(a) of the Public Health Service Act (42 U.S.C 247b)A; Title IV Section 4002 of the Affordable Care Act, Prevention and Public Health Fund (42 U.S.C 300u-11)A; Section 30l(a) of the Public Health Service Act (42 U.S.C. 241a) 93.426;

Title XV, Section 1509 of the Public Health Service Act (42 USC 300n-4a)B; Section 1502 of the Public Health Service Act (42 U.S.C. 300l)C; FY 2023 Consolidated Appropriations Act (Pub. L. 117-328, Div. H)C; and Section 1501 of the PHS Act (42 USC 300k)C (Attachment 2).


Extensive scientific evidence links nonmedical factors, including systemic racism and the lack of economic opportunities, with poor health outcomes and increased mortality rates, all of which are preventable.I Factors such as poverty, inadequate housing, poor health care, and other debilitating social conditions, commonly referred to as social determinants of health, contribute to long-standing disparities and health inequities.I These social conditions contribute to the increased prevalence of CVD in the US population.I CVD is the leading cause of death in the US; stroke is the 5th leading cause.II In 2020, about 1 in 5 adults who died from CVD were younger than 65 years old.III It is estimated that 1 in 9 health care dollars are spent on CVD.IV Primary risk factors for CVD include high blood pressure and high blood cholesterol which can be addressed through advancements in lifestyle change and clinical care. IV


DHDSP’s mission is to provide national leadership, public health and scientific expertise, and program support to optimize cardiovascular health for all. Underpinning DHDSP’s work is a commitment to improve health equity, focus on priority populations, and strategically engage partners. DHDSP provides funding, guidance, and technical assistance to health departments and other partner organizations to promote three evidence-based strategies in its cooperative agreement programs: 1) clinical quality measurement (CQM), 2) team-based care (TBC), and 3) community-clinical linkages (CCL). While recipients across all three programs implement the same three strategies, the programs differ on a variety of factors, including the populations of focus and sub-strategy implementation.


The National CVH Program: This five year cooperative agreement began in July 2023 and funds state health departments (recipients) to implement and evaluate evidence-based strategies to prevent and manage CVD in populations at disproportionate risk, emphasizing control of hypertension and high cholesterol among adults aged 18-85 (Attachment 1a). This cooperative agreement requires recipients to form Learning Collaboratives (LCs) comprised of public health leaders and partners to facilitate sustainable change and improvement in cardiovascular health outcomes, particularly for those at the highest risk of poorer health outcomes. This National CVH Program builds on prior initiatives (CDC-RFA-DP18-1815 and CDC-RFA-DP18-1817), to identify promising CVD prevention and management practices and promote health equity.


The Innovative CVH Program: This five year cooperative agreement began in September 2023 and funds various organizations including state and county governments, American Indian or Alaska Native tribal governments, non-government organizations, and institutions of higher education to implement innovative evidence-based strategies that identify and address healthcare disparities to improve outcomes for adults with hypertension and high cholesterol, targeting areas with a high prevalence of these conditions (Attachment 1b). This program also creates and maintains LCs and builds on the outcomes and achievements of prior initiatives (CDC-RFA-DP18-1815 and CDC-RFA-DP18-1817).


WISEWOMAN: This five year cooperative agreement began in September 2023 and funds various organizations including state and territorial governments and American Indian or Alaska Native tribal governments to improve cardiovascular health for low-income, uninsured, and underinsured women aged 35-64 (Attachment 1c). It expands services to participants of the National Breast and Cervical Cancer Early Detection Program (NBCCEDP), addressing cardiovascular health in populations affected by social determinants of health. Building on CDC-RFA-DP18-1816, WISEWOMAN emphasizes health equity, improving cardiovascular health in the target demographic while addressing social and economic factors. Approved under OMB Control No. 0920-0612, WISEWOMAN currently has a reporting system, Minimum Data Elements (MDEs) for collecting and collating data relating to screening, assessment and healthy behavior support services. Health outcome measures assessed in the MDE include, but are not limited to, systolic and diastolic blood pressure readings, total cholesterol, weight, smoking status, nutrition, physical activity, and number of healthy behavior support services attended. 


Table A.1-A provides additional details about the strategies and sub strategies implemented by cooperative agreement recipients.

Table A.1-A Cardiovascular Disease (CVD) Prevention and Management Strategies and Sub-Strategies

Strategy

Sub-Strategy

The National CVH Program

Strategy Description

The Innovative CVH Program

Strategy Description

WISEWOMAN

Strategy Description

Clinical Quality Measures (CQM)

1A

Advance the adoption and use of electronic health records (EHR) or health information technology (HIT), to identify, track, and monitor measures for clinical and social services and support needs to address health care disparities and health outcomes for patients at highest risk of CVD with a focus on hypertension (HTN) and high cholesterol (HC).

Advance the adoption and use of EHR or health information technology HIT, to identify, track, and monitor measures for clinical and social services and support needs to address health care disparities and health outcomes within approved populations of focus.

Provide CVD risk assessment to under- and uninsured participants in the priority range of 35-64 years during the baseline, follow-up, and reassessment office visits, as appropriate.

1B

Promote the use of standardized processes or tools to identify the social services and support needs of patient populations at highest risk of CVD, with a focus on HTN and HC, and monitor and assess the referral and utilization of those services, such as food assistance, transportation, housing, childcare, etc.

Promote the use of standardized processes or tools, such as GIS or other Geo-mapping tools, to identify the social services and support needs within approved populations of focus and monitor and assess the referral and utilization of those services, such as the need for transportation, housing, childcare, etc.

Use EHR and HIT data to query, monitor, and track clinical and social services and support needs data for improved identification, management, referrals, treatment, and outcomes of those at risk of CVD, particularly HTN.

1C

--

--

Use standardized procedures to identify social services and support needs of participants and monitor and assess the referral and utilization of those services, such as food assistance, transportation, housing, childcare, etc.

1D

--

--

Use metrics from program data to guide quality improvement activities, e.g., Plan Do Study Act (PDSA) cycles, participant and partner feedback, etc., to increase program enrollment, retention, and referrals to additional services.

1E

--

--

Use EHR, HIT, or program data to identify health care disparities and address health outcomes within their WISEWOMAN population.

Team-Based Care (TBC)

2A

Advance the use of health information systems that support TBC to monitor population health with a focus on health disparities, HTN, and HC.

Advance the use of health information systems that support TBC to monitor and address HTN and HC within approved populations of focus.

Engage program participants, health professionals, community health workers (CHWs), social workers, patient navigators, pharmacists, and other members of the care team in community settings outside of health care facilities to enhance participant follow-up and communication and coordination among the care team.

2B

Assemble or create multidisciplinary teams (e.g., nurses, nurse practitioners, pharmacists, nutritionists, physical therapists, social workers, and community-based workers) to identify patients' social services and support needs and to improve the management and treatment of HTN and HC.

Assemble or create multidisciplinary teams to identify social services and support needs within approved populations of focus.

Build and maintain a network of state, regional, and local social services and support based on social determinants of health (SDOH) within the recipient’s jurisdiction.

2C

Build and manage a coordinated network of multidisciplinary partnerships that address identified barriers to social services and support needs (e.g., childcare, transportation, language translation, food assistance, and housing) within populations at highest risk of CVD.

Build and manage a coordinated network of multidisciplinary partnerships that address identified barriers and needs within approved populations of focus, related to their social services and support needs (e.g., childcare, transportation, language translation, food assistance, and housing).

--

Community-Clinical Linkages (CCL)

3A

Create and enhance community-clinical links to identify SDOH (e.g., inferior housing, lack of transportation, inadequate access to care, and limited community resources) and respond to the social services and support needs of populations at highest risk of CVD with a focus on HTN and HC.

Create and enhance community-clinical links to identify SDOH (e.g., housing, transportation, access to care, and community resources) and respond to the individual social services and support needs within approved populations of focus.

Identify, enhance, or build systems that facilitate provider and community bidirectional referrals to support medical follow-up, healthy behavior support services (HBSS), and social services and support.

3B

Identify and deploy dedicated CHWs (or their equivalents) to provide a continuum of care and services which extend the benefits of clinical interventions and address social services and support needs leading to optimal health outcomes.

Identify and deploy dedicated CHWs (or their equivalents) to provide a continuum of care and services which extend the benefits of clinical interventions and address social needs leading to optimal health outcomes within approved populations of focus.

Collaborate with community groups who represent and serve the priority population, provide evidence-informed HBSS, and refer participants to those HBSS.

3C

Promote use of self-measured blood pressure monitoring (SMBP) with clinical support within populations at highest risk of HTN.

Promote the use of SMBP with clinical support within approved populations of focus.

Use evidence-based and evidence-informed strategies to ensure participants are actively engaged in HBSS.

3D

--

--

Refer participants to appropriate social services and support; track and monitor use.


CDC requests Office of Management and Budget (OMB) approval to gather new data to conduct a comprehensive evaluation of The National CVH Program, The Innovative CVH Program, and WISEWOMAN with a sample of cooperative agreement recipients and their partners. CDC has contracted with Deloitte Consulting to design and implement the comprehensive evaluation. Deloitte Consulting, together with the Division for Heart Disease and Stroke Prevention (DHDSP) are responsible for data collection and analysis activities. Deloitte and DHDSP are referred to collectively as the Comprehensive Evaluation Team. In this comprehensive evaluation, a “recipient” indicates the organization funded directly by the cooperative agreement. A “site” refers to the funded recipient organization, its implementation partners, and Learning CollaborativesD.

A2. Purpose and Use of the Information Collection

The purpose of the comprehensive evaluation is to 1) document the differences and similarities among the cooperative agreements by describing program implementation, 2) assess the extent of change in desired program outcomes, 3) estimate the value of resources recipients are investing in improving health system infrastructure and outcomes, 4) identify successful strategies for implementation, and 5) understand how overlapping strategies and unique characteristics of the programs contribute to health equity and CVD prevention and management.


Findings from the evaluation will contribute to the continuous improvement of quality and effectiveness of program strategies for CVD prevention and management, identify recipients’ programmatic and evaluative technical assistance needs, build the evidence base to support replication and scaling of program strategies, document and share program success and actionable lessons learned with collaborators and the public health field, document health outcomes among populations bearing disproportionate risk of hypertension and CVD, and inform future funding opportunities supported by CDC. The Comprehensive Evaluation Team will share evaluation findings with DHDSP leadership, The National CVH Program, The Innovative CVH Program, and WISEWOMAN recipients, partner sites that participate in the evaluation, and other key partners that collaborate with recipients. Funding recipients will learn what types of activities are working well within other jurisdictions and what factors facilitate this success, serving as input for their own program planning processes.


This implementation and outcome evaluation will consist of three components 1) Evaluability Assessments, 2) Exploratory Assessments, and a 3) Cost Study. Table A.2-A displays the evaluation questions by evaluation type and component. Unless indicated, the evaluation question applies to all three cooperative agreements.


Table A.2-A Overarching Evaluation Questions and Corresponding Evaluation Components

Evaluation Type

Evaluation Question

Evaluation Component

Implementation

1. To what extent has the recipient’s implementation approach resulted in achieving the desired outcomes? (The National CVH Program)

Evaluability Assessment

Exploratory Assessment

Cost Study

2. To what extent have recipients increased the reach of the program strategies to prevent and control cardiovascular disease? (The National CVH Program/WISEWOMAN) / To what extent have recipients increased the reach of program strategies to improve cardiovascular disease within approved populations of focus? (The Innovative CVH Program)

Evaluability Assessment

3. To what extent has program strategy implementation led to improved health outcomes among the identified population(s) at the highest risk of cardiovascular disease? (The National CVH Program/WISEWOMAN)

Evaluability Assessment

Exploratory Assessment

Cost Study

4. What factors were associated with effective implementation of the program strategies? (The National CVH Program/WISEWOMAN)/ What factors or components were associated with effective implementation of program strategies and learning collaboratives? (The Innovative CVH Program)

Evaluability Assessment

Exploratory Assessment

Cost Study

5. What factors were associated with identifying and addressing social services and support needs and social determinants of health for populations at the highest risk of cardiovascular disease? (The National CVH Program/WISEWOMAN)


What factors were associated with identifying and addressing social services and support needs and social determinants of health within approved populations of focus? (The Innovative CVH Program)

Evaluability Assessment

Exploratory Assessment

6. To what extent can the implemented program strategies be sustained after the program ends? (The National CVH Program/WISEWOMAN)

Exploratory Assessment

Cost Study

Outcome

1. To what extent have the implemented strategies and learning collaboratives contributed to a measurable change in health outcomes, health equity, and health systems in a defined community, population, organization, or system? (The National CVH Program/WISEWOMAN)


To what extent have the implemented strategies and learning collaboratives contributed to a measurable change in health outcomes within approved populations of focus (The Innovative CVH Program)?

Exploratory Assessment

2. To what extent have the program strategies contributed to reduced healthcare disparities and improved health outcomes within populations at the highest risk of cardiovascular disease? (The National CVH Program/WISEWOMAN)

Exploratory Assessment

3. What innovative strategies implemented by recipients can be replicated and/or scaled? (The Innovative CVH Program)

Exploratory Assessment

Cost Study


The data collection will focus on obtaining qualitative information and cost data about strategy implementation, facilitators and barriers, and other contextual information that affects program implementation and participant outcomes at the organizational level.


Evaluability Assessment

The purpose of the Evaluability Assessment is to assess capacity for formal evaluation and serve as a precursor to the Exploratory Assessment. The Evaluability Assessment will collect in-depth information from a sample of recipients, their partners, and Learning Collaborative membersD (referred to as “sites” collectively) to identify sites that have the capacity to participate in further evaluation via the Exploratory Assessment. Conducted in Year 2 of each cooperative agreement, the Evaluability Assessment seeks to:

  • Describe the extent of strategy implementation among selected sites.

  • Identify promising practices.

  • Assess recipients’ capacity for participation in the Exploratory Assessment.

The Comprehensive Evaluation Team will use a purposive sampling method to select 24 recipients to participate in the Evaluability Assessments for one of the three strategy areas (CQM, TBC, CCL). Criteria for selection include recipient self-nomination, project officer recommendations, geographical considerations, recipient type (e.g., university, state health department, local health department), partner types, populations of focus, implementation setting, and how long the recipient has been receiving funding from CDC related to CVD prevention and management. Table A.2-B shows the proposed sample size for the Evaluability Assessment for each program strategy.


Table A.2-B Sample Size for Evaluability Assessment


Strategy

# of Recipients per Strategy

Clinical Quality Measures (CQM)

8

Team-Based Care (TBC)

8

Community-Clinical Linkages (CCL)

8

TOTAL N

24



The Evaluability Assessment will address specific implementation evaluation questions using the following data collection methods:

  • Evaluability Assessment Nomination Form (Attachment 3a and 3b): All recipients will be invited to complete the Evaluability Assessment Nomination form. The Nomination Form will allow recipients interested in participating in the comprehensive evaluation to provide brief information about their organization, partners, and the strategy for which they would like to participate.

  • Document Review: After selecting the participating recipients and prior to conducting semi-structured interviews, the Comprehensive Evaluation Team will aggregate and review materials related to the activities implemented under the cooperative agreements such as recipient and partner websites, policy documents, and progress and evaluation reports submitted to CDC. Document reviews will help to inform the development of logic models and prepare interviewers for the key informant interviews.

  • Semi-Structured Key Informant Interviews (Attachment 3c-3o): The Comprehensive Evaluation Team will conduct up to 192 virtual interviews with key personnel from 24 recipient organizations, their partners, and Learning Collaborative members to gather insights into program goals and approaches, intended outcomes, details about program implementation and the recipients’ capacity to participate in an exploratory assessment. Key personnel include various staff members such as program managers, evaluators, data preparers, clinical and field staff, health equity specialists, and Learning Collaborative leads, and participants. We will used tailored semi-structured guides for each cooperative agreement and respondent type.


Exploratory Assessment

The purpose of the Exploratory Assessment is to follow-up with a sub-set of the recipients that participated in the Evaluability Assessment to learn more about program implementation and outcomes. Scheduled for Year 4 of each cooperative agreement, the Exploratory Assessment will focus on:

  • Assessing the effectiveness of program activities.

  • Identifying key facilitators, barriers, successes, and innovations that contribute to or hinder progress towards outcomes.

  • Establishing, maintaining, scaling, and replicating effective programs and practices.

The Comprehensive Evaluation Team will select a total of 12 recipients from the 24 recipients that participated in the Evaluability Assessment. Recipient selection will be informed by Evaluability Assessment findings and guided by prioritization criteria including time in operation, program maturity, implementation progress, data system capacity, focus on health equity, innovative strategies, evidence of effectiveness, sustainability, and organizational capacity. Table A.2-C shows the proposed sample size for the Exploratory Assessment for each program strategy.


Table A.2-C Sample Size for Exploratory Assessment

Strategy

# of Recipients per Strategy

Clinical Quality Measures (CQM)

4

Team-Based Care (TBC)

4

Community-Clinical Linkages (CCL)

4

TOTAL N

12


The Exploratory Assessment will address specific implementation and outcome evaluation questions using the following data collection methods:

  • Document Review: The Comprehensive Evaluation Team will conduct document reviews on new program documents developed after the Evaluability Assessment such as operation guides, promotional materials, aggregated and deidentified participant summary reports, CQI plans, referral materials, enrollment guides, and progress reports. Document reviews will help prepare interviewers for the key informant interview and supplement interview findings.

  • Semi-Structured Key Informant Interviews (Attachment 4): The Comprehensive Evaluation Team will conduct a total of 96 semi-structured in-depth interviews with key personnel from 12 recipient organizations, their partners, and Learning Collaborative members to learn more about facilitators and barriers to implementation, effectiveness, and impact. Key personnel will include the same types of roles as in the Evaluability Assessments and may include the same and new individuals. We will used tailored semi-structured guides for each cooperative agreement and respondent type.


Cost Study

The Cost Study is designed to estimate program implementation costs and assess the value of resources invested by The National CVH Program, The Innovative CVH Program, and WISEWOMAN recipients and their partners to improve health system infrastructure and outcomes for their communities and identified populations of focus. Scheduled for Year 3 and 4 of each cooperative agreement, the Cost Study aims to:

  • Assess program costs among a subset of program recipients and their partners, such as health systems, social service providers, and community organizations.

  • Examine the level of variability in cost estimates across all three programs, considering total costs and costs per strategy.

  • Understand the resources necessary to maintain, sustain, replicate, and scale-up strategies.

The Comprehensive Evaluation Team will use a call for interest process to select recipients to participate in the Cost Study. Recipients will not have to participate in the Evaluability Assessment or Exploratory Assessment to participate in the Cost Study and will indicate interest by simply responding to an email. In total, no more than 55 recipients across the three programs will be selected to participate. Table A.2-D shows the proposed sample size for the Evaluability Assessment for each cooperative agreement.


Table A.2-D. Sample Size for Cost Study

Cooperative Agreement

# of Recipients per Program

The National CVH Program

26

The Innovative CVH Program

11

WISEWOMAN

18

TOTAL N

55


The Cost Study will address specific implementation and outcome evaluation questions using the following data collection methods:

  • Document Review: The Comprehensive Evaluation Team will conduct a systematic review and data extraction of recipient-submitted reports including Work Plans and Budget Narratives. Extracted data will populate the Partner and Recipient Cost Inventory Tools prior to sharing the cost inventory tools with respondents.

  • Partner and Recipient Cost Inventory Tools (Attachment 5a and 5b): Recipients and up to three of their partners will validate data entered into the corresponding Cost Inventory Tool and complete any missing information. The Recipient inventory tool will collect direct and indirect costs, such as costs for personnel, consultants/subcontractors, program equipment and materials, travel, overhead costs, and in-kind contributions. The partner inventory tool is a streamlined version of the Recipient tool and focuses on a subset of cost categories.

  • Semi-Structured Cost Interviews (Attachment 5c and 5d): The Comprehensive Evaluation Team will conduct approximately 15 interviews per recipient with key personnel from recipient and partner sites who have a comprehensive understanding of implementation activities and budget and staffing structures. Key personnel will include various staff members such as program managers, program coordinators, and learning collaborative leads.


A3. Use of Improved Information Technology and Burden Reduction

Video Conference Semi-structured Interviews

Video conference interviews will collect qualitative data without the costs and respondent burden associated with traditional face-to-face site visits to reduce burden on recipients. The Comprehensive Evaluation Team will work with interview participants to find convenient times for them to complete the interviews. Once mutual availability has been established, the Comprehensive Evaluation Team will send each interviewee an electronic meeting request containing information for a secure conference line. The Comprehensive Evaluation Team will be responsible for setting up and initiating the conference line for the interviews; respondents will be able to easily join a secure conference line using a passcode provided to them in advance. Respondents will not be asked to make special preparations in advance of the call but will be provided with a list of the topics to be discussed during the interview. With the consent of the participants, interviews will be digitally recorded and transcribed.


Cost Inventory Tools

The Cost Inventory Tool is a Microsoft Excel based data collection tool that allows for detailed tracking and analysis of financial data. First, the Comprehensive Evaluation Team will populate the Cost Inventory Tool using cost information extracted from the Cost Study document review to minimize burden. Then, the Comprehensive Evaluation Team will email the pre-populated Cost Inventory Tool to each participating recipient and partner. The template includes prompts that guide data input, reducing the complexity and time required for report completion. Recipients and partners will verify the accuracy of the pre-populated information, apply corrections (if needed), and complete remaining sections of tool and submit it electronically via password protected email to the Comprehensive Evaluation Team. This tool was adapted from previously developed cost tools, leveraging development and lessons learned from the DP18-1815 Resource Use and Cost Inventory Tool and the DP19-1907 Cost Tool. The Cost Inventory Tool will be tailored to reflect specific nuances in cost categories for implementation of each program at the recipient and partner level. The Cost Inventory Tool may be pilot tested with fewer than nine recipients and their partner sites to assess its ability to provide requested data and identify approaches to minimize burden. During the pilot test, the clarity of the instrument, usability of the Excel workbook, and accuracy of the data entered will also be assessed. The tool will be streamlined to only include the most important questions to inform the relevant assessment questions; therefore, no extraneous information will be collected.

A4. Efforts to Identify Duplication and Use of Similar Information

This ICR supports the comprehensive evaluation of three federally funded cooperative agreements, which began in 2023 and run through 2028. As the cooperative agreements are new and data to be collected through this evaluation relates directly to implementation of The National CVH Program, The Innovative CVH Program, and WISEWOMAN strategies, the information to be collected from recipients is not available from other sources, including other federal agencies, academic institutions, and/or NGOs. Additionally, there have been no other evaluation data collection efforts conducted to date, nor does the information to be collected exist in any existing centralized data source. Each data collection tool submitted through this package has a distinct purpose with no overlap across other tools or data collection efforts including routine performance measurement data collection.


The comprehensive evaluation is designed to complement performance measure and evaluation reports submitted by recipients as part of the requirements for receiving cooperative agreement funding. The Evaluability Assessments and Exploratory assessments will collect in-depth information about implementation progress and program effectiveness that cannot be obtained through these reports or via recipient calls with CDC project officers and/or evaluators. The Comprehensive Evaluation Team will review these materials prior to interviews to prepare interviewers and avoid asking questions about information that has already been provided. Similarly, cost information collected through the Semi-Structured Cost Interview and Cost Inventory Tool is not fully captured through required cooperative agreement financial reports submitted by recipients. The data collection activities included in this ICR will allow CDC to capture critical information needed to continuously improve programmatic efforts for The National CVH Program, The Innovative CVH Program, and WISEWOMAN and clearly demonstrate the use of federal funds.


A5. Impact on Small Businesses or Other Small Entities

There is a possibility that some partner sites participating in the comprehensive evaluation might include small business entities, such as small health clinics or community-based organizations involved in health promotion. Despite this, the CDC anticipates that involvement of small businesses will be infrequent, and all participation from such entities will remain entirely voluntary.


No specific obligations are imposed on small businesses. The questions and data requirements have been streamlined to the essential minimum needed to achieve the evaluation's objectives. This is to ensure that the burden on small entities is minimized. Outside of these engagements, no other small businesses are expected to be directly involved in the data collection processes of this comprehensive evaluation.


A6. Consequences of Collecting the Information Less Frequently

We have designed the comprehensive evaluation to minimize data collection in any one year of the program. The frequency of data collection, along with consequences of collecting information less frequently, are detailed below.

Evaluability Assessment
The Evaluability Assessment will occur in Year 2 of each cooperative agreement. Information collected during the key informant interviews will provide important insight into how selected sites are implementing strategies (process evaluation). At this point in the cooperative agreement lifecycle, recipients have identified and formalized agreements with partners, are progressing with strategy implementation, and have started to identify challenges and facilitators. Some recipients may also start to identify early outcomes and most will be able to identify and describe their capacity to evaluate program outcomes in Year 4 of the cooperative agreement. The Evaluability Assessment data collection is necessary so that the Comprehensive Evaluation Team can begin to identify these challenges and facilitators which will allow CDC to provide any needed technical assistance, identify and share emerging, promising, and best practices with funding recipients, and support improved outcomes. The Evaluability Assessment also provides key information about funding recipients and their partners that will inform site selection for the Exploratory Assessment.

Exploratory Assessment
The Exploratory Assessment will occur in Year 4 of each cooperative agreement. Information collected during the key informant interviews will provide valuable information about the effectiveness of program activities (outcome evaluation), additional information about the factors that contribute to or hinder progress toward outcomes, and important insights into how program activities can be maintained after the cooperative agreement ends, and the potential for maintaining, replicating, and scaling successful activities. Without this information, the CDC will not be able to identify and promote best practices that have demonstrated outcomes.

Cost Study

This OMB request includes Cost Study data collection that will occur in Year 3 of each cooperative agreement. Information collected during the Cost Study will provide information about the initial start-up costs that recipients and their partners experience when implementing CVD prevention and management strategies. The Cost Inventory Tool will collect quantitative information that is not available via other sources and the Cost Semi-Structured Interview will provide CDC with additional insight into the range of inputs and resources used for each strategy. Without these cost data, CDC will not be able to assess the cost-effectiveness, cost, and health benefits associated with strategies to establish CVD prevention and management strategies.


A7. Special Circumstances Relating to the Guidelines of 5 CRF 1320.5

This request fully complies with the regulation 5 CFR 1320.5.


A8. Comments in Response to the FRN and Efforts to Consult Outside the Agency

A. Federal Register Notice

A 60 Day Federal Register Notice was published in the Federal Register May 7, 2024 in Volume 89, Number 89, pages 38151- 38153 (Attachment 6). There were no public comments in response to the Notice.


B. Other Consultations

The data collection instruments were designed collaboratively by the Comprehensive Evaluation Team. The Comprehensive Evaluation Team is convening voluntary internal and external advisory groups, or Evaluation Planning Groups (EPGs) to advise on the Comprehensive Evaluation planning, implementation, and use of findings. The internal advisory group, referred to as CDC EPG, is comprised of evaluation subject matter experts and DHDSP leadership at CDC. The external advisory groups, referred to as R-EPGs, are convened for each cooperative agreement. The R-EPGs include representatives from 1) The National CVH Program, 2) The Innovative CVH Program, and 3) WISEWOMAN recipient teams.


A9. Explanation of Any Payment or Gift to Respondents

No payments or gifts will be provided to the Comprehensive Evaluation respondents.

A10. Protection of the Privacy and Confidentiality of Information Provided by Respondent

CDC’s Privacy Office has reviewed this submission and has determined that the Privacy Act does not apply. The information collection does not involve collection of sensitive or personal information. No system of records will be created under the Privacy Act. Respondents are state and local health departments, healthcare organizations, non-profit organizations, and universities. The proposed study involves a minimum amount of information in identifiable form (IIF) and includes only contact information for each respondent (i.e., name, telephone number, email address, role at organization). The information to be obtained through the nomination form, interviews, and the cost inventory tool concern organizational activities and costs rather than personal matters and is not considered highly sensitive.


Semi-Structured Key Informant and Cost Interviews

The Comprehensive Evaluation Team will conduct interviews with representatives from recipient and partner organizations and may include program managers, evaluators, data preparers, health equity specialists, clinical and field staff, and Learning Collaborative leads, coaches, or participants.


Interview responses will be linked to respondents’ organizations and roles to ensure that findings can be linked to organizations and other existing organizational data. IIF data will be stored separately from response data. A linking file will be created and available only to the Comprehensive Evaluation Team.


The interviews will be recorded and transcribed (only first names of respondents and the recipient agencies’ identifying information will be collected); names of respondents will be redacted from interview transcripts and all information will be transmitted and stored securely on Deloitte servers. Transcriptions will be coded and uploaded into a qualitative database, using software such as MAXQDA. Key themes will be developed based on the qualitative data analysis. Such identified themes and quotes may be included in reports; specific quotes will not be attributed to any single person in any reports. Original recordings and transcriptions from the site visits will not be shared with CDC to protect key informant privacy though de-identified notes highlighting key findings may be shared with CDC.


Cost Study Inventory Tool

The cost information collected will focus on programmatic spending and will not require individuals to input any IIF.

A11. Institutional Review Board (IRB) and Justification for Sensitive Questions

Respondents for the data collection efforts included in the ICR are cooperative agreement recipients and staff members from their partner sites. We are not collecting individual-level data and data collection does not request sensitive or personally identifiable information. CDC’s Institutional Review Board determined that this project does not constitute research with human subjects as defined by the US Code of Federal Regulations (45 CFR 46.102) (Attachment 7).

A12. Estimates of Annualized Burden Hours and Costs

A.12-A. Estimated Annualized Burden Hours


OMB approval is requested for three years. In this section, we provide detailed information about the anticipated burden for each evaluation component of data collection in the Comprehensive Evaluation. Table A.12-A and A.12-B provide a summary of the annual burden hours across the three years of data collection.


Evaluability Assessment Nomination Form

All 107 recipients will be invited to complete the Evaluability Assessment nomination form once in Year 2 of the cooperative agreements. The National CVH Program and the Innovative CVH program will use the same nomination form (Attachment 3a); WISEWOMAN will use a separate nomination form because WISEWOMAN recipients are not required to engage a Learning Collaborative (Attachment 3b). The burden estimate for this data collection effort is 0.5 hours for both nomination forms.


Evaluability Assessment Semi-Structured Key Informant Interviews
The Comprehensive Evaluation Team will select 24 sites across the three cooperative agreements to participate in the Evaluability Assessment for one of the three program strategies (CQM, TBC, or CCL). Evaluability Assessments will occur once during Year 2 of the cooperative agreements.


At each selected National CVH Program and Innovative CVH Program site, the Comprehensive Evaluation Team will conduct Key Informant Interviews with up to three representatives from the recipient organization (Attachment 3d, 3h, 3l) and up to three representatives from partner organizations about the strategy of interest (Attachment 3f, 3j, 3n). The burden estimate for these interviews is 1.5 hours. The Comprehensive Evaluation Team will also conduct Key Informant Interviews with up to two Learning Collaborative representatives (Attachment 3c) from each recipient site.D The burden estimate for each Learning Collaborative Interview is 1 hour.


At each selected WISEWOMAN site, the Comprehensive Evaluation Team will conduct Key Informant Interviews with up to four representatives from the recipient organization (Attachment 3e, 3i, 3m) and up to four representatives from recipient partner organizations about the strategy of interest (Attachment 3g, 3k, 3o). The burden estimate for these interviews is 1.5 hours.


Exploratory Assessment Semi-Structured Key Informant Interviews

The Comprehensive Evaluation Team will select 12 sites across the three cooperative agreements to participate in the Exploratory Assessment for one of the three program strategies (CQM, TBC, or CCL). Exploratory Assessments will occur once during Year 4 of the cooperative agreements.


At each selected National CVH Program and Innovative CVH Program site, the Comprehensive Evaluation Team will conduct Key Informant Interviews with up to three representatives from the recipient organization (Attachment 4b, 4f, 4j) and up to three representatives from recipient partner organizations about the strategy of interest (Attachment 4d, 4h, 4l). The burden estimate for these interviews is 1.5 hours. The Comprehensive Evaluation Team will also conduct Key Informant Interviews with up to two Learning Collaborative representatives from each site (Attachment 4a).D The burden estimate for each Learning Collaborative Interview is one hour.


At each selected WISEWOMAN site, the Comprehensive Evaluation Team will conduct Key Informant Interviews with up to four representatives from the recipient organization (Attachment 4c, 4g, 4k) and up to four representatives from recipient partner organizations about the strategy of interest (Attachment 4e, 4i, 4m). The burden estimate for these interviews is 1.5 hours.


Cost Study Semi-Structured Interview and Cost Inventory Tool
The Comprehensive Evaluation Team will select up to 55 recipients to participate in the Cost Study. The Cost Study will occur during Year 3 and 4 of the cooperative agreements.


At each site, the Comprehensive Evaluation Team will conduct semi-structured interviews with up to two representatives from the recipient organization (Attachment 5c) and one representative from up to three partner organizations per recipient (Attachment 5d). The burden estimate for these interviews is one hour. One or two representatives from each recipient organization will work together to complete one recipient cost inventory tool (Attachment 5a). Up to three partner organizations per recipient will complete a partner cost inventory tool (Attachment 5b). The estimated burden to complete the cost inventory tool is 2.5 hours.


Table A.12-A. Annualized Burden Hours

Type of Respondents

Evaluation Component

Form Name

No. of Respondents

No. of Responses per Respondent

Average Burden per Response (in hours)

Total Burden Hours

Recipients 

Evaluability Assessment

Evaluability Assessment Nomination Form_NCHP_ICHP (Attachment 3a)

24

1

0.5

12

Evaluability Assessment Nomination Form_WW (Attachment 3b)

12

1

0.5

6

Eval Assessment CQM Recipient Interview Guide NCHP_ICHP (Attachment 3d)

6

1

1.5

9

Eval Assessment CQM Recipient Interview Guide WW (Attachment 3e)

3

1

1.5

5

Eval Assessment TBC Recipient Interview Guide NCHP_ICHP (Attachment 3h)

6

1

1.5

9

Eval Assessment TBC Recipient Interview Guide WW (Attachment 3i)

3

1

1.5

5

Eval Assessment CCL Recipient Interview Guide NCHP_ICHP  (Attachment 3l)

6

1

1.5

9

Eval Assessment CCL Recipient Interview Guide WW (Attachment 3m)

3

1

1.5

5

Exploratory Assessment

Ex Assessment CQM Recipient Interview Guide NCHP_ICHP (Attachment 4b)`

3

1

1.5

5

Ex Assessment CQM Recipient Interview Guide_WW (Attachment 4c)

2

1

1.5

3

Ex Assessment TBC Recipient Interview Guide NCHP_ICHP (Attachment 4f)

3

1

1.5

5

Ex Assessment TBC Recipient Interview Guide_WW (Attachment 4g)

2

1

1.5

3

Ex Assessment CCL Recipient Interview Guide NCHP_ICHP (Attachment 4j)

3

1

1.5

5

Ex Assessment CCL Recipient Interview Guide_WW (Attachment 4k)

2

1

1.5

3

Cost Study

Cost Study Interview Guide_Recipient (Attachment 5c) 

37

1

1

37

Comprehensive Evaluation Resource Use and Cost Inventory Tool_Recipient (Attachment 5a)

37

1

2.5

93

Partners 

Evaluability Assessment

Eval Assessment CQM Partner Interview Guide NCHP_ICHP (Attachment 3f)

6

1

1.5

9

Eval Assessment CQM Partner Interview Guide WW (Attachment 3g)

3

1

1.5

5

Eval Assessment TBC Partner Interview Guide NCHP_ICHP (Attachment 3j) 

6

1

1.5

9

Eval Assessment TBC Partner Interview Guide WW (Attachment 3k)

3

1

1.5

5

Eval Assessment CCL Partner Interview Guide NCHP_ICHP (Attachment 3n) 

6

1

1.5

9

Eval Assessment CCL Partner Interview Guide WW (Attachment 3o)

3

1

1.5

5

Exploratory Assessment

Ex Assessment CQM Partner Interview Guide NCHP_ICHP (Attachment 4d)

3

1

1.5

5

Ex Assessment CQM Partner Interview Guide_WW (Attachment 4e)

1

1

1.5

2

Ex Assessment TBC Partner Interview Guide NCHP_ICHP (Attachment 4h)

3

1

1.5

5

Ex Assessment TBC Partner Interview Guide_WW (Attachment 4i)

1

1

1.5

2

Ex Assessment CCL Partner Interview Guide NCHP_ICHP (Attachment 4l) 

3

1

1.5

5

Ex Assessment CCL Partner Interview Guide_WW (Attachment 4m)

1

1

1.5

2

Cost Study

Cost Study Interview Guide_Partner  (Attachment 5d)

55

1

1

55

Comprehensive Evaluation Resource Use and Cost Inventory Tool_Partner (Attachment 5b)

55

1

2.5

138

Learning Collaborative 

Evaluability Assessment

Eval Assessment LC Interview Guide_NCHP_ICHP (Attachment 3c)

12

1

1

12

Exploratory Assessment

Ex Assessment LC Interview Guide NCHP_ICHP (Attachment 4a)

6

1

1

6

Total

 

484


A.12-B. Estimated Annualized Cost to Respondents

Total cost has been calculated to reflect annualized cost over the three-year collection period. Annualized cost has been calculated using U.S. Department of Labor Bureau (DOL) of Labor Statistics estimates using the best approximation of DOL occupation titles and wage classification for each type of respondent.v The expected equivalent occupation titles and wages for target respondents’ positions were obtained from the DOL database and used to populate Table A.12-B. In some cases, individuals in different roles/positions (i.e., occupational titles) will respond to the same data collection tool. The average hourly wage is a composite and average of the identified wage classification for each type of respondent.

To calculate the average hourly wage rate, we identified all the occupations involved for each type of respondent and form name, then calculated the weighted average based on the number of respondents or the estimated distribution of respondents across these occupations. Below is a detailed breakdown of each occupation and their corresponding hourly wages.


Occupations and Hourly Wages

  • Project Manager: $50.44

  • Epidemiologist: $43.48

  • Data Entry Workers: $19.29

  • Health Specialist: $41.70

  • Environmental Scientists and Specialists, Including Health: $41.69

  • Provider: $126.85

  • Pharmacists: $64.81

  • Community Health Workers (CHWs): $25.30

  • Healthcare Social Workers: $32.42

  • Healthcare Support Occupations: $18.37

Consolidated Calculation

The following details how we consolidated the hourly wage rates for each form:

For Recipients

  • Evaluability Assessment Nomination Forms (Attachment 3a and 3b)

  • Project Manager: $50.44

  • Evaluability Assessment and Exploratory Assessment Recipient Semi-Structured Interviews (Attachment 4b, 4c, 4f, 4g, 4j, 4k)

  • Project Manager: $50.44

  • Epidemiologist: $43.48

  • Data Entry Workers: $19.29

  • Health Specialist: $41.70

Calculation: Average Hourly Wage = (50.44 + 43.48 + 19.29 + 41.70) / 4 = $38.73

  • Cost Study Recipient Semi-Structured Interviews (Attachment 5c)

  • Project Manager: $50.44

  • Recipient Cost Inventory Tool (Attachment 5a)

  • Project Manager: $50.44

  • Data Entry Workers: $19.29

Calculation: Average Hourly Wage = (50.44 + 19.29) / 2 = $34.87


For Partners

  • Evaluability Assessment and Exploratory Assessment Partner Semi-Structured Interviews (Attachment 3f, 3g, 3j, 3k, 3n, 3o, 4d, 4e, 4h, 4i, 4l, 4m)

  • Project Manager: $50.44

  • Environmental Scientists and Specialists: $41.69

  • Provider: $126.85

  • Pharmacists: $64.81

  • CHWs: $25.30

  • Healthcare Social Workers: $32.42

  • Healthcare Support Occupations: $18.37

Calculation: Average Hourly Wage = (50.44 + 41.69 + 126.85 + 64.81 + 25.30 + 32.42 + 18.37) / 7 = $51.41

  • Cost Study Recipient Semi-Structured Interviews (Attachment 5d)

  • Project Manager: $50.44

  • Partner Cost Inventory Tool (Attachment 5b)

  • Project Manager: $50.44

  • Data Entry Workers: $19.29

Calculation: Average Hourly Wage = (50.44 + 19.29) / 2 = $34.87


For Learning Collaborative

  • Evaluability Assessment and Exploratory Assessment Semi-Structured Interviews (Attachment 3c and 4a)

  • Eval Assessment LC Interviews Project Manager: $50.44

  • Environmental Scientists and Specialists: $41.69

  • Provider: $126.85

  • Pharmacists: $64.81

  • CHWs: $25.30

  • Healthcare Social Workers: $32.42

  • Healthcare Support Occupations: $18.37

Calculation: Average Hourly Wage = (50.44 + 41.69 + 126.85 + 64.81 + 25.30 + 32.42 + 18.37) / 7 = $51.41


Table A.12-B. Annualized Cost to Respondents


Type of Respondents

Form Name

Total Annual Burden Hours

Average Hourly Wage Rate

Total Respondent Labor Cost

Recipients

Evaluability Assessment Nomination Form_NCHP_ICHP (Attachment 3a)

12

$50.44

$605.28

Evaluability Assessment Nomination Form_WW (Attachment 3b)

6

$50.44

$302.64

Eval Assessment CQM Recipient Interview Guide NCHP_ICHP (Attachment 3d)

9

$38.73

$348.57

Eval Assessment CQM Recipient Interview Guide WW (Attachment 3e)

5

$38.73

$193.65

Eval Assessment TBC Recipient Interview Guide NCHP_ICHP (Attachment 3h)

9

$38.73

$348.57

Eval Assessment TBC Recipient Interview Guide WW (Attachment 3i)

5

$38.73

$193.65

Eval Assessment CCL Recipient Interview Guide NCHP_ICHP (Attachment 3l)

9

$38.73

$348.57

Eval Assessment CCL Recipient Interview Guide WW (Attachment 3m)

5

$38.73

$193.65

Ex Assessment CQM Recipient Interview Guide NCHP_ICHP (Attachment 4b)`

5

$38.73

$193.65

Ex Assessment CQM Recipient Interview Guide_WW (Attachment 4c)`

3

$38.73

$116.19

Ex Assessment TBC Recipient Partner Interview Guide NCHP_ICHP (Attachment 4f)

5

$38.73

$193.65

Ex Assessment TBC Recipient Partner Interview Guide_WW (Attachment 4g)

2

$38.73

$77.46

Ex Assessment CCL Recipient Interview Guide NCHP_ICHP  (Attachment 4j)

5

$38.73

$193.65

Ex Assessment CCL Recipient Interview Guide_WW  (Attachment 4k)

3

$38.73

$116.19

Cost Study Interview Guide_Recipient (Attachment 5c) 

37

$50.44

$1,866.28

Comprehensive Evaluation Resource Use and Cost Inventory Tool_Recipients (Attachment 5a)

93

$34.87

$3,242.91

Partners

Eval Assessment CQM Partner Interview Guide NCHP_ICHP (Attachment 3f)

9

$51.41

$462.69

Eval Assessment CQM Partner Interview Guide WW (Attachment 3g)

5

$51.41

$257.05

Eval Assessment TBC Partner Interview Guide NCHP_ICHP (Attachment 3j) 

9

$51.41

$462.69

Eval Assessment TBC Partner Interview Guide WW (Attachment 3k)

5

$51.41

$257.05

Eval Assessment CCL Partner Interview Guide NCHP_ICHP (Attachment 3n)

9

$51.41

$462.69

Eval Assessment CCL Partner Interview Guide WW (Attachment 3o)

5

$51.41

$257.05

Ex Assessment CQM Partner Interview Guide NCHP_ICHP (Attachment 4d)

5

$51.41

$257.05

Ex Assessment CQM Partner Interview Guide_WW (Attachment 4e)

2

$51.41

$102.82

Ex Assessment TBC Partner Interview Guide NCHP_ICHP (Attachment 4h)

5

$51.41

$257.05

Ex Assessment TBC Partner Interview Guide_WW (Attachment 4i)

2

$51.41

$102.82

Ex Assessment CCL Partner Interview Guide NCHP_ICHP (Attachment 4l)

5

$51.41

$257.05

Ex Assessment CCL Partner Interview Guide_WW (Attachment 4m)

2

$51.41

$102.82

Cost Study Interview Guide_Partner (Attachment 5d)

55

$50.44

$2,774.20

Comprehensive Evaluation Resource Use and Cost Inventory Tool_Partner (Attachment 5b)

138

$34.87

$4,812.06

Learning Collaborative

Eval Assessment LC Interview Guide (Attachment 3c)

12

$51.41

$616.92

Ex. Assessment LC Interview Guide NCHP_ICHP (Attachment 4a)

6

$51.41

$308.46

Total

 

484


$20,285.03


A13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

No capital or maintenance costs are expected. Additionally, there are no start-up, hardware or software costs.

A14. Annualized Cost to the Federal Government

The average annual contractor cost for this data collection is $575,000 per year for a three-year total of $1,725,000. Additional annual costs include personnel costs of federal employees involved in program management, technical assistance, and data analysis. The annual staff cost is estimated at $133,782.50 per year for three years, broken down as follows:

  • CDC DHDSP GS-12: Two staff members at an annual salary of $74,441 each, allocating 25% of their time to the data collection, resulting in an estimated annual cost of $37,220.50.

  • CDC DHDSP GS-13: Two staff members at an annual salary of $88,520 each, allocating 25% of their time to the data collection, resulting in an estimated annual cost of $44,260.00.

  • CDC DHDSP GS-14: Two staff members at an annual salary of $104,604 each, allocating 25% of their time to the data collection, resulting in an estimated annual cost of $52,302.00.

Table 14-A presents the types of labor costs to the government the program proposes to incur: (1) external contracted data collection and analyses, and (2) government personnel.


Table A14.-A. Estimated Annualized Federal Government Cost Distribution

Cost Category

Annualized Cost

CDC DHDSP GS-12 (25% FTE for 2 staff)

$37,220.50

CDC DHDSP GS-13 (25% FTE for 2 staff)

$44,260.00

CDC DHDSP GS-14 (25% FTE for 2 staff)

$52,302.00

Data Collection Contractor

$575,000.00

Total

$708,782.50

A15. Explanation for Program Changes or Adjustments

This is a new request for information collection.

A16. Plans for Tabulation and Publication and Project Time Schedule

Analysis Plan

The overarching evaluation design is a mixed-methods approach that will collect and analyze qualitative and quantitative data. Data will be analyzed 1) individually by each cooperative agreement to assess how WISEWOMAN, National, and Innovative recipients develop and implement programs activities to address the program strategies and achieve outcomes, and 2) comparatively for a cross-program analysis that differentiates and assesses the unique contribution of each program.


Evaluability Assessment

  • Evaluability Assessment Nomination Form (Attachment 3a and 3b): The Comprehensive Evaluation team will systematically review nomination forms and extract data using a standardized protocol. Extracted data will be compiled into excel-based matrix that organizes data by Evaluability Assessment selection criteria.

  • Semi-Structured Key Informant Interviews (Attachment 3c-3o): Interviews will be first analyzed individually and then compared to other interviews within a site for a site-level analysis. For each site, interview data will be entered into an excel-based matrix to compare data across interviewees per site. The data matrix will help the Comprehensive Evaluation Team identify patterns, highlight emerging themes, and conduct negative case analyses. After data matrixes have been developed, we will conduct thematic analysis using qualitative software to analyze site interview data. Interview data will be triangulated with document reviews. Findings from the Evaluability Assessment will be used to inform the site selection for Exploratory Assessments.


Exploratory Assessment

  • Semi-Structured Key Informant Interviews (Attachment 4): Thematic analysis will be used to analyze interview data. A codebook will be constructed to facilitate thematic analysis, in which a-priori codes will be developed that are based on the program logic model and Evaluability Assessment findings. The coding structure will be revised in an iterative manner to ensure that emergent themes are being captured in a systematic manner. Interview data will be triangulated with document reviews.


Cost Study

  • Semi-Structured Cost Interviews (Attachment 5c and 5d): The Comprehensive Evaluation Team will conduct content and thematic analysis to examine both the manifest (i.e., the actual words used) and latent (i.e., the underlying meaning of the words) content. Thematic analysis will be theory-driven, based on program logic models, program operational guidance, and budget guidance. The Comprehensive Evaluation Team will construct a codebook to facilitate thematic analysis, developing a-priori codes based on expected themes and cost categories.

  • Partner and Recipient Cost Inventory Tools (Attachment 5a and 5b): For each strategy, the Comprehensive Evaluation Team will present key descriptive statistics (e.g., mean, median, minimum, maximum, standard deviation) for costs of implementing a strategy across all cost study participants, including the distribution of expenditures across sub-strategy, strategy, and program. Comparisons will explore the cost variance across strategies within the same recipient and cost variance for a given strategy across recipients and programs. Given the anticipated small sample size of participating recipients and their partners, regression analysis will not be feasible. However, we will use performance measure data and qualitative data to contextualize the cost data and qualify potential factors for cost variance. Analysis will also examine the cost by budget categories and determine feasibility to allocate to specific categories/activities within each strategy.


Publication Plan

The publication and dissemination plan ensures that internal teams receive detailed, actionable insights to guide strategic decision-making, while external partners receive high-level summaries that showcase program achievements, describe best practices, provide recommendations to foster knowledge sharing across recipients and partners, and promote transparency and engagement. Findings will be disseminated to interested parties via web, print, and in-person formats through memos, reports, manuscripts, infographics, presentations, or webinars.


Internally, detailed information will be synthesized into program-specific summaries and cross-program summaries. Program-specific presentations and reports will provide in-depth insights into program successes, gaps, sustainability plans, and overall health outcomes and health equity contributions by cooperative agreement. Cross-program presentations and reports will offer a comprehensive comparison of summative and overarching findings across all cooperative agreements, focusing on program commonalities and differentiators. Externally, findings will be disseminated through concise, targeted documents aimed at recipients and partners. These documents will provide an anonymized summary of similarities and differences across recipients and will highlight best practices, identify areas for improvement, and offer recommendations for recipients to enhance their work.


Internal and external dissemination products will be developed for each evaluation component in order to share findings at each stage of data collection. Evaluability Assessment findings will be shared at the end of Year 2 and dissemination products will synthesize intervention types, characteristics of populations of focus, early successes, barriers to intervention and evaluation, and evaluation capacity. Exploratory Assessment findings will be shared at the end of Year 4, and will summarize the effectiveness of program activities, identify best practices and promising approaches for replication, assess the impact on populations of focus and health equity, evaluate strengths and gaps in partnerships, and outline sustainability plans and resources needed for scale-up. Cost Study data will be collected and shared during Year 3 and 4. Cost Study dissemination products will focus on site-level and aggregate-level findings on implementation costs, maintenance costs, and cost of scale-up and sustainability, and cost breakdowns by program strategy and reach.


Timeline

The estimated schedule for key data collection, analysis, and reporting tasks relevant to this request for OMB approval is presented in Table A.16. The evaluation timeline considers the need for evidence throughout the five-year project period and data collection over this period to ensure that information is gathered, analyzed, and published at appropriate points in time. OMB approval is being requested for three years with the desired data collection process to begin in November 2024. 


Table A.16. Estimated Time Schedule for Project Activities

Activity

Anticipated Timeline

Evaluability Assessment (Year 2)

Data Collection (n=96)

2-13 months after OMB approval

Data Analysis

8-15 months after OMB approval

Data Publication

11-18 months after OMB approval

Exploratory Assessment (Year 4)

Data Collection (n=48)

24-30 months after OMB approval

Data Analysis

25-31 months after OMB approval

Data Publication

29-33 months after OMB approval

Cost Study (Year 3 and 4)

Data Collection (n=208)

20-33 months after OMB approval

Data Analysis

21-26 months after OMB approval

Data Publication

25-29 months after OMB approval


A17. Reason(s) Display of OMB Expiration Date is Inappropriate

The data collection tools will display the expiration date for OMB approval.

A18. Exceptions to Certification for Paperwork Reduction Act Submission

There are no exceptions to the certification statement.



REFERENCES

  1. Havranek, E. P., Mujahid, M. S., Barr, D. A., Blair, I. V., Cohen, M. S., Cruz-Flores, S., Davey-Smith, G., Dennison-Himmelfarb, C. R., Lauer, M. S., Lockwood, D. W., Rosal, M., & Yancy, C. W. (2015). Social Determinants of Risk and Outcomes for Cardiovascular Disease. Circulation, 132(9), 873-898. https://doi.org/doi:10.1161/CIR.0000000000000228


  1. Centers for Disease Control and Prevention. (2021). Leading causes of death. National Center for Health Statistics. https://www.cdc.gov/nchs/fastats/leading-causes-of-death.htm


  1. Centers for Disease Control and Prevention. (2021). Underlying cause of death, 1999-2020. CDC WONDER Online Database. https://wonder.cdc.gov/ucd-icd10.html


  1. Centers for Disease Control and Prevention. (2021). Heart disease facts. https://www.cdc.gov/heartdisease/facts.htm


  1. U.S. Bureau of Labor Statistics. (n.d.). Occupational employment and wage statistics. https://www.bls.gov/oes/current/oes_stru.htm


A The National CVH Program

B The Innovative CVH Program

C WISEWOMAN Program

D Learning Collaboratives are a requirement for The National CVH Program and The Innovative CVH Program. It does not apply to WISEWOMAN.

I

File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
File TitleComprehensive Evaluations of the DP-23-0003, DP-23-0004, and DP-23-0005 Cooperative Agreement Programs: The National Cardiovascu
SubjectSupporting Statement A template
AuthorCenters for Disease Control and Prevention
File Modified0000-00-00
File Created2024-11-09
© 2025 OMB.report | Privacy Policy