SSA_1815 Category A_v09

INFORMATION COLLECTION REQUEST

NEW

National Evaluation of the DP18-1815 Cooperative Agreement Program:

CATEGORY A, DIABETES MANAGEMENT AND TYPE 2 DIABETES PREVENTION

Supporting Statement Part A

Program Official/Contact

Name: Kimberly Farris

Telephone: 770-488-0543

Email: [email protected]

National Center for Chronic Disease

Prevention and Health Promotion

Centers for Disease Control and Prevention

Atlanta, Georgia

April 22, 2020

Updated December 1, 2020

TABLE OF CONTENTS

A. JUSTIFICATION

1. Circumstances Making the Collection of Information Necessary

2. Purposes and Use of the Information Collection

3. Use of Improved Information Technology and Burden Reduction

4. Efforts to Identify Duplication and Use of Similar Information

5. Impact on Small Businesses or Other Small Entities

6. Consequences of Collecting the Information Less Frequently

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

9. Explanation of Any Payment or Gift to Respondents

10. Protection of the Privacy and Confidentiality of Information Provided by Respondents

11. Institutional Review Board (IRB) and Justification for Sensitive Questions

12. Estimates of Annualized Burden Hours and Costs

13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

14. Annualized Cost to the Federal Government

15. Explanation for Program Changes or Adjustments

16. Plans for Tabulation and Publication and Project Time Schedule

17. Reason(s) Display of OMB Expiration Date is Inappropriate

18. Exceptions to Certification for Paperwork Reduction Act Submissions

ATTACHMENTS

Attachment 1.

• Centers for Disease Control and Prevention. National Center for Chronic Disease Prevention and Health Promotion. Notice of Funding Opportunity: Improving the Health of Americans through Prevention and Management of Diabetes and Heart Disease and Stroke – Financed in part by the 2018 Prevention and Public Health Funds. CDC-RFA-DP18-1815PPHF18

Attachment 2.

• Authorizing Legislation Section 301(a) of the Public Health Services Act [42.U.S.C. 242k]

Attachment 3. 1815 Awardees and Information Collection Plan

1. 1815 List of Health Department Awardees

2. 1815 Strategies for Preventing and Controlling Diabetes and Heart Disease and Stroke

3. 1815 Logic Model

4. Category A 1815 Crosswalk of Evaluation Components and Data Collection Tools

5. Category A 1815 Summary of Annualized Respondents

6. Category A 1815 Evaluation Gantt Chart

Attachment 4. Category A Partner Site-level Rapid Evaluations

1. DSMES Partner Site-Level Rapid Evaluation Rapid Nomination Form (Word Version)

27. DSMES Partner Site-Level Rapid Evaluation Rapid Nomination Form (Screenshot Version)

2. National DPP Partner Site-Level Rapid Evaluation Nomination Form (Word Version)

27. National DPP Partner Site-Level Rapid Evaluation Nomination Form (Screenshot Version)

3. DSMES Partner Site-Level Rapid Evaluation Survey Questionnaire

4. DSMES Partner Site-Level Rapid Evaluation Program Coordinator Interview Guide

5. DSMES Partner Site-Level Rapid Evaluation Professional Team Member Interview Guide

6. DSMES Partner Site-Level Rapid Evaluation Paraprofessional Team Member Interview Guide National DPP Partner Site-Level Rapid Evaluation Survey Questionnaire

7. National DDP Partner-Site Level Rapid Evaluation Survey Questionnaire

8. National DPP Partner Site-Level Rapid Evaluation Program Coordinator Interview Guide

9. National DPP Partner Site-Level Rapid Evaluation Lifestyle Coach Interview Guide

Attachment 5. Category A Recipient-led Evaluations

1. Category A Evaluation and Performance Measurement Plan (EPMP) including evaluation questions

2. Category A Recipient-Led Evaluation Reporting Template

Attachment 6. 60-Day Federal Register Notice

Attachment 7.

1. Institutional Review Board Approval Notification or Exemption Determination Part A

b. Institutional Review Board Approval Notification or Exemption Determination Part B

Attachment 8. Introductory/Follow-Up Letters

1. Category A DSMES/National DPP Site-Level Rapid Evaluation Pre-Survey Email

2. Category A DSMES/National DPP Site-Level Rapid Evaluation Survey Invitation

3. Category A DSMES/National DPP Site-Level Rapid Evaluation Survey Reminder

4. Category A DSMES/National DPP Site-Level Rapid Evaluation Final

List of Acronyms

ADCES	Association of Diabetes Care & Education Specialists
ADA	American Diabetes Association
CDC	Centers for Disease Control and Prevention
CHWs	Community Health Workers
DDT	Division of Diabetes Translation
DPP	National Diabetes Prevention Program
DPRP	Diabetes Prevention Recognition Program
DOL	U.S. Department of Labor Bureau
DSMES	Diabetes Self-Management Education and Support
EPMP	Evaluation and Performance Measurement Plan
HD	Health Department
ICR	Information Collection Request
LCP	Lifestyle Change Program
NCCDPHP	National Center for Chronic Disease Prevention and Health Promotion
NOFO	Notice of Funding Opportunity
OMB	Office of Management and Budget
PIE	Performance Improvement and Evaluation

Overview

This is a three-year information collection request for an evaluation of a five-year

Cooperative Agreement program CDC-RFA-DP18-1815PPHF18: Improving the Health

of Americans Through Prevention and Management of Diabetes and Heart Disease and

Stroke, “1815”. 1815 is a collaboration between two Divisions at the Centers for Disease

Control and Prevention (CDC) and is structured into two categories (1) Category A:

Diabetes Management and Type 2 Diabetes Prevention, and (2) Category B:

Cardiovascular Disease Prevention and Management. This information collection request

package focuses on data collection activities for the Category A diabetes assessment.

• Goal of the assessment: The purpose of the assessment is to determine all 50 state health departments’ and the Washington, D.C. health department’s progress of utilizing CDC’s DP18-1815 cooperative agreement funds to implement evidence-based strategies and how the efforts are contributing to state- and health-system level changes in support of the prevention and management of diabetes.

• Intended use of the resulting data: The data collected from this assessment will be used to: (1) determine health departments’ progress using 1815 funds to implement evidence-based strategies; (2) identify practices that have shown promise in preventing and managing diabetes to share across programs; and (3) to determine how these efforts are contributing to state level, health system, or other organizational level changes and outcomes to support the prevention and management of diabetes.

• Methods to be used to collect data: The assessment is comprised of the following primary data collection methods: (1) virtual site visits and program observations; (2) key informant interviews and group discussions; and (3) online surveys. CDC will also review health departments’ program records and respective evaluation reports.

• The subpopulation to be studied: Recipients from all 50 state health departments and the Washington, D.C. health department receiving non-research funding from the CDC National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP) through the DP18-1815 Cooperative Agreement and the recipients’ partner sites where 1815-funded activities are implemented.

• How data will be analyzed: Qualitative data collected from primary and secondary sources will be analyzed for key and emerging themes to measure specific constructs related to the implementation of the 1815 strategies for diabetes. The data will be triangulated with quantitative data collected from surveys, which will be analyzed using descriptive statistics.

National Evaluation of the DP18-1815 Cooperative Agreement Program – Category A: Diabetes Management and Type 2 Diabetes Prevention

1. Justification

A.1 Circumstances Making the Collection of Information Necessary

Diabetes is the seventh leading cause of death in the U.S. and the number one cause of kidney failure, lower-limb amputations, and adult-onset blindness. In addition, approximately 84 million Americans - or 1 in 3 adults - have prediabetes, a health condition that increases a person’s risk of developing type 2 diabetes. The mission of the Centers for Disease Control and Prevention (CDC), Division of Diabetes Translation (DDT), is to reduce the population burden of diabetes. CDC provides funding, guidance, and technical assistance to health departments and other partner organizations to increase the availability and utilization of interventions that can help prevent or control diabetes. Priorities include: (1) increasing enrollment and retention in evidence-based diabetes prevention and management programs, by facilitating referrals and reducing barriers to participation, (2) identifying and sharing best practices for program implementation and sustainability, and (3) eliminating health disparities in under-served areas and populations.

CDC requests OMB approval to collect information needed to assess activities conducted under a new cooperative agreement, CDC-RFA-DP18-1815PPHF18: Improving the Health of Americans Through Prevention and Management of Diabetes and Heart Disease and Stroke, hereafter referred to as “1815” (Attachment 1). CDC is authorized to conduct these activities by the Public Health Service Act (42 U.S.C. 242), Section 301(a) (Attachment 2).

The 1815 cooperative agreement is a collaboration between CDC’s Division of Diabetes Translation (DDT) and CDC’s Division for Heart Disease and Stroke Prevention (DHDSP), and is structured in two categories aligning with each Division:

• Category A: Diabetes Management and Type 2 Diabetes Prevention

• Category B: Cardiovascular Disease (CVD) Prevention and Management

Awardees are 50 state health departments and the Washington, D.C., health department (hereafter referred to as “HD recipients,” see Attachment 3a). Health department recipients are working with partner organizations to implement evidence-based strategies for preventing or controlling diabetes and cardiovascular disease in populations and communities that are disproportionately affected by these conditions (Attachment 3b). Recipients are encouraged to implement Category A and B strategies in the same high burden areas/communities, so that work on these strategies is mutually reinforcing and implemented in a coordinated fashion to accelerate progress toward goals. Activities conducted under the 1815 cooperative agreement build upon CDC’s previous work with health departments, health care providers, and community-based organizations to identify promising diabetes and CVD prevention and management practices that can be scaled and replicated. Previous work has included efforts to strengthen coordination within health care systems as well as community-clinical linkages (CDC-RFA-DP13-1305 and CDC-RFA-DP14-1422).

The current information collection request focuses on seven diabetes prevention and control strategies selected for implementation and further evaluation under 1815 Category A (strategies A1-A7). CDC selected the strategies because of their potential to improve health outcomes by increasing participation in evidence-based diabetes prevention and management programs (see logic model, Attachment 3c). Specifically, the strategies are designed to:

• Improve care and management of people with diabetes by increasing access to and utilization of diabetes self-management education and support (DSMES) programs

DSMES: 1815’s Category A diabetes management strategies (A1, A2, A3) support increased access, engagement and coverage for DSMES services. DSMES is the ongoing process through which health care providers facilitate both the knowledge, skills, and abilities necessary for diabetes self-care and the activities that assist a person in implementing and sustaining the behaviors needed to manage his or her condition on an ongoing basis. A large body of evidence supports the effectiveness of DSMES in improving health outcomes (A1c, systolic blood pressure), lowering medication use, and decreasing hospitalizations and other health care costs for people with diabetes. However, DSMES utilization rates are low. Increasing access to and participation in DSMES programs in communities and strengthening coverage for DSMES services are critical goals of the 1815 cooperative agreement.

• Improve access to, participation in, and coverage for the CDC-led National Diabetes Prevention Program (National DPP) lifestyle change program for people with prediabetes, particularly in underserved areas and populations

National DPP: 1815’s Category A Type 2 diabetes prevention strategies (A4, A5, A6) support increased access to and coverage for the National DPP. In 2010, Congress authorized CDC to establish the National DPP, a partnership of public and private organizations working together to build a nationwide delivery system for a lifestyle change program (LCP) proven to prevent or delay onset of type 2 diabetes in adults with prediabetes. Organizations that implement effective type 2 diabetes prevention programs may apply for recognition through the CDC National Diabetes Prevention Recognition Program (“CDC-recognized programs”). 1815 strategies include both direct support to CDC-recognized lifestyle change programs, as well as indirect support to the National DPP by strengthening policies and processes to increase identification of individuals with prediabetes, expand referrals to the National DPP LCPs, and secure payment and reimbursement mechanisms to reduce barriers to participation.

• Increase utilization and support for community health workers (CHW) in referrals to DSMES and LCP recognized by the National DPP

Strategy A7 cuts across diabetes management and type 2 diabetes prevention and is aimed at developing a statewide infrastructure to support long-term sustainability for Community Health Workers (CHWs) as a means to establish or expand their involvement in CDC-recognized LCPs.

A summary of the strategies is provided in Table A.1-A. Through the 5-year cooperative agreement, each HD recipient is required to implement and evaluate at least three of the Category A strategies.

Table A.1-A Category A: Diabetes Management and Type 2 Diabetes Prevention Strategies

Type of Strategy	Strategy Label	Strategy Description
Diabetes Management (for Individuals with Diabetes)	A1	Improve access to and participation in (ADA) American Diabetes Association-recognized/Association of Diabetes Care & Education Specialists (ADCES) accredited DSMES programs in underserved areas
	A2	Expand or strengthen DSMES coverage policy among public or private insurers or employers, with emphasis on one or more of the following: Medicaid and employers
	A3	Increase engagement of pharmacists in the provision of medication management or DSMES for people with diabetes
Diabetes Prevention (for Individuals with Prediabetes)	A4	Assist health care organizations in implementing systems to identify people with prediabetes and refer them to CDC-recognized lifestyle change programs for type 2 diabetes prevention
	A5	Collaborate with payers and relevant public and private sector organizations within the state to expand availability of the National DPP as a covered benefit for one or more of the following groups: Medicaid beneficiaries; state/public employees; employees of private sector organizations
	A6	Implement strategies to increase enrollment in CDC-recognized lifestyle change programs
Diabetes Management and Prevention	A7	Develop a statewide infrastructure to promote long-term sustainability/ reimbursement for Community Health Workers (CHWs) as a means to establish or expand their use in: a) CDC-recognized lifestyle change programs for type 2 diabetes prevention and/or b) ADA-recognized/ADCES-accredited DSMES programs for diabetes management

CDC requests Office of Management and Budget (OMB) approval to gather new data to conduct a systematic and in-depth national evaluation of the 1815 Category A efforts (National Evaluation). The strategies selected by HD recipients will be assessed in terms of Approach, Efficiency, Effectiveness (reach, health outcomes, and facilitators), Sustainability, and Impact. To improve understanding of context and interactions throughout systems of care, information will be collected from both health departments and the partner DSMES/National DPP sites.

Each HD recipient will:

• Submit an Evaluation and Performance Measurement Plan (EPMP) that identifies the strategies (A1-A7) selected for implementation in the jurisdiction (state or District of Columbia); the indicators selected for assessing approach, efficiency, effectiveness, sustainability, and impact; and the data sources for those indicators.

• Submit an annual report template on progress and achievements.

The EPMP and the annual report comprise the “Recipient-led evaluation,” which provides a framework for the National Evaluation and includes elements that apply to all HD recipients as well as elements that are specific to each 1815 strategy. This evaluation component will allow CDC to monitor HD recipients’ progress in using 1815 funds to implement evidence-based strategies, and to determine how those efforts are contributing to changes and outcomes at the state level, health system level, or other organizational level.

In addition, each HD recipient will:

• Nominate 2 DSMES sites and 2 National DPP sites for the “Partner rapid site-level evaluation” (51 HD recipients x 4 sites/HD = total of 204 partner sites).

Each DSMES or National DPP site participating in the partner rapid site-level evaluation will

• Complete an online survey.

• Schedule selected staff for semi-structured interviews.

The site-level rapid evaluations will document the implementation processes across the full lifecycle of DSMES/National DPP sites; identify innovations and promising practices for establishing, attaining accreditation/recognition for, and maintaining a DSMES/National DPP program site; identify innovations and promising practices for increasing enrollment and retention in DSMES/National DPP, particularly among priority populations; and determine the key contributions of the HD recipient in supporting the implementation of DSMES/National DPP strategies at the site level, and opportunities for further support.

OMB approval is requested for three years. Data collection will be conducted by the National Evaluation Team, which is comprised of the DDT Performance Improvement and Evaluation (PIE) team and the Deloitte Consulting Evaluation team.

CDC’s plan for assessing 1815 Category B activities is described in a separate information collection request.

A.2 Purpose and Use of Information Collection

The overarching goal of the cooperative agreement program is to improve public health programs and systems for achieving measurable health impact. As an action-oriented process, the evaluation will serve to identify programs that have positive outcomes, identify those that may need additional technical assistance support, and highlight the specific activities that make the biggest contribution to improving diabetes prevention and management efforts, so that these strategies can be shared and replicated. Without collection of new evaluative data, CDC will not be able to capture critical information needed to continuously improve programmatic efforts and clearly demonstrate the use of federal funds to improve health outcomes.

This evaluation is comprised of two components: 1) Category A partner site-level rapid evaluations, and 2) Category A recipient-led evaluations. Each evaluation component will seek to gather in-depth information about specific program strategies with the intent of a) identifying promising practices, particularly those reaching high-burden populations/communities; b) determining the contribution of each strategy to intended outcomes; c) determining activities that are promising to share among programs; and d) determining the most effective roles of state health departments in supporting health system/community programs for diabetes prevention and management.

Table A.2-A provides an overview of how each component of the Category A national evaluation effort supports the overarching evaluation questions for the respective categories of approach, effectiveness, efficiency, sustainability, and impact. The evaluation sub-questions pertaining to effectiveness are outlined in the 1815 NOFO and will be used to guide the national evaluation efforts. The three sub-concepts: reach, health outcomes, and facilitators, each build upon each other to operationalize the overall concept of effectiveness. A detailed crosswalk of the evaluation components and data collection tools, evaluation questions, and respondent type can be found in Attachment 3d.

Table A.2-A. Overview of Evaluation Questions and Evaluation Components

Overarching Evaluation Questions (from NOFO)	Evaluation Components
	1. Category A Partner Site-Level Rapid Evaluation	2. Recipient Led Evaluation
Approach: To what extent has the recipient’s implementation approach resulted in achieving the desired outcomes?	•	•
Effectiveness (Reach): Question 1: To what extent has the recipient increased the reach of Category A and B Strategies to prevent and control diabetes and cardiovascular disease?	•	•
Effectiveness (Health Outcomes): Question 2: To what extent has implementation of Category A and B strategies led to improved health outcomes among the identified priority population(s)?	•	•
Effectiveness (Facilitators): Question 3: What factors were associated with effective implementation of Category A and B strategies?	•	•
Efficiency: To what extent has the NOFO affected efficiencies with regards to infrastructure, management, partners, and financial resources?		•
Sustainability: To what extent can the strategies implemented be sustained after the NOFO ends?		•
Impact: To what extent have the strategies implemented contributed to a measurable change in health, behavior, or environment in a defined community, population, organization, or system?		•

Category A Evaluation Component 1: Partner Site-Level Rapid Evaluations

The purpose of the Category A partner site-level rapid evaluations is to gather in-depth, contextualized information about how HD recipients are supporting the implementation of the 1815 DSMES and National Diabetes Prevention Program (National DPP) strategies within partner sites (sites); determine the key barriers, facilitators, and implementation successes at partners sites; and identify promising practices at partners sites. Findings from the partner site-level rapid evaluations will be used to demonstrate the HD recipients’ optimal role in supporting DSMES and National DPP program sites and inform CDC technical guidance provided to HD recipients on how to implement the 1815 strategies. Findings will also help to identify specific partner site-level interventions that show success and will help to develop more robust evaluation methods to gather additional evidence to scale and replicate these promising practices. Partner site-level evaluations will offer a unique view on how federal funds are supporting and contributing to diabetes prevention and management efforts in the field. Without an assessment at this level, DDT will not be able to completely track how 1815 strategies are being implemented on the ground and how they influence improved health outcomes. The Category A Partner Site-Level Rapid Evaluation consists of HD recipients nominating partner sites to participate in a survey questionnaire and partner site-level interviews. The following section describes the nomination process.

DSMES/National DPP Site Nomination Forms (Att. 4a, 4aa, 4b, 4bb): Using the site nomination forms, each HD recipient will be invited to nominate a total of 2 National DPP sites (known as CDC-recognized lifestyle change programs) and 2 DSMES sites to participate in the rapid evaluation. A total of 204 sites will be invited to participate in the partner site-level rapid evaluation. The forms will provide foundational information about the nominated sites to help determine their eligibility and viability to participate in the partner site-level interviews. The nomination forms will be reviewed by a CDC Category A Evaluation Panel to finalize the selection of sites for the partner site-level interviews. A sample of 24-28 sites (12-14 National DPP and 12-14 DSMES sites) will be selected for virtual site visits each year.

DSMES/National DPP Survey Questionnaire (Att. 4c, 4f): All 204 sites that are selected and agree to participate in the rapid evaluation will be asked to complete an online survey in Years 3 and 5 of the cooperative agreement. Separate survey questionnaires will be administered to DSMES and National DPP sites, but the same two survey questionnaires will be used in Years 3 and 5. The purpose of the survey is to capture multiple perspectives on processes and factors influencing referrals, enrollment, and retention in the respective pre-diabetes and diabetes education programs, identify opportunities to promote efficiencies, and assess the perceived value and gaps in 1815-funded support received from the HD recipient. The surveys will be administered to key staff within DSMES and National DPP programs including Program/Quality Coordinators, Lifestyle Coaches, and other team members. The National Evaluation Team will work with the sites to develop a contact database for the online survey.

DSMES/National DPP Partner Site-Level Interviews (Att. 4d, 4e, 4g, 4h, 4i): Virtual, semi-structured interviews will be conducted with representatives from up to 28 sites (12-14 DSMES sites and 12-14 National DPP sites) each year starting in Year 3 of the cooperative agreement. The virtual interviews will be used to mitigate for COVID-19 transmission. The same partner site-level interview guide will be used each year. DSMES program representatives to be interviewed from each site include the program/quality coordinator, key professional (e.g. Registered Dietitian, Nurse, Pharmacist), and paraprofessional (e.g. Medical Assistant, Pharmacy Technician, Peer Educators, Community Health Workers) team members engaged in program delivery. National DPP program representatives to be interviewed from each site include the program coordinators and lifestyle coaches. Separate interview guides will be used to facilitate interviews for each target audience within each program. The purpose of the interviews is to learn more about the operation of the program across the different program phases (start-up, recognition, recruitment, enrollment, delivery, retention), understand partnership networks and how those partnerships support program delivery, and gain a more nuanced perspective on how the HD recipients’ 1815-funded activities are supporting program delivery and where there may be a need for additional technical assistance or other types of support.

Evaluation Component 2: Category A Recipient-Led Evaluations

Category A Evaluation and Performance Measurement Plan (EPMP) Template (Att. 5a). Per the 1815 cooperative agreement, HD recipients are required to develop and submit an EPMP that demonstrates how the HD recipient will fulfill the requirements described in the CDC Evaluation and Performance Measurement and Project Description sections of the cooperative agreement. The Category A template includes a set of evaluation questions that HD recipients are required to answer as a part of their Category A recipient-led evaluation.

Category A Recipient-Led Evaluation Reporting Template (Att. 5b). All 1815 HD recipients are required to report the progress of their recipient-led evaluations of 3 strategies to CDC on an annual basis. The purpose of the Category A recipient-led annual evaluation report is to track recipient progress toward accomplishing the cooperative agreement’s goals for Category A strategies and identifying areas for evaluation technical assistance. HD recipients will submit the annual recipient-led evaluation report for Category A strategies using the Category A Recipient-Led Evaluation Reporting template and associated guidance document that CDC provides. HD recipients will provide an update of each of the three Category A strategies they have selected to evaluate. The template includes the following sections: background on the selected strategy/activities, evaluation purpose, evaluation design and data collection, evaluation data collection matrix, findings and conclusions.

Evaluation findings will be shared with HD recipients, HD recipients’ partner sites that participate in the evaluation, and other key partners that collaborate with the HD recipients. Evaluation findings will help DDT demonstrate the reach and impact of evidence-based diabetes strategies and determine the effect of CDC funding on health system and community-based diabetes prevention and management efforts. This determination will include identifying which strategies may be viable for sharing with other programs (e.g. implementing in new sites or in other types of organizations, or even other states). Evaluation findings will also help to identify areas where HD recipients may need additional technical support and help inform design of targeted technical assistance and guidance for HD recipients. Evaluation findings will help HD recipients learn what types of activities are working well within other jurisdictions and what factors facilitate this success, serving as input for their own program planning processes.

A.3 Use of Improved Information Technology and Burden Reduction

Electronic Data Collection: Data for the Category A rapid evaluation site nomination forms and survey questionnaires will be collected through the web-based survey tool, Qualtrics. Web-based data collection tools will be developed on user-friendly platforms selected with the user experience in mind and will be designed to streamline data entry using skip patterns and pre-populated fields, as appropriate. The Category A recipient-led evaluation templates and reports will be completed using Microsoft Word. These templates are structured to minimize HD recipient effort required to collect and enter data, thereby reducing burden on HD recipients.

Non-Electronic Data Collection: Interview and group discussion data will be collected via in-person or phone interviews and documented in writing by the National Evaluation Team. Interviews and group discussions will also be digitally recorded per the consent of participants and transcribed by the National Evaluation Team. These data will be collected during virtual site visits to HD recipients and partner sites. There will be no additional burden on HD recipients and their partner sites other than their participation in interviews with the National Evaluation Team.

A.4 Efforts to Identify Duplication and Use of Similar Information

This ICR is for a national level evaluation of a new, five-year, federally funded cooperative agreement issued by the CDC, which began in October 2018. As the cooperative agreement is new and data to be collected through this evaluation relates directly to HD recipients’ implementation of 1815 strategies, the information to be collected from HD recipients is not available from other sources, including other federal agencies, academic institutions, and/or NGOs. Additionally, there have been no other evaluation data collection efforts conducted to date, nor does the information to be collected exist in any existing centralized data source.

Each data collection tool submitted through this package has a distinct purpose with no overlap across other tools or data collection efforts including routine performance measurement data collection.

All 1815 recipients are required to report performance measure targets and data on an annual basis. Performance measure templates are submitted under –OMB No. 0920-1132, Performance Progress and Monitoring Report (expiration date 10/31/2022). The performance measures provide standard quantitative measures of recipient progress towards expected outcomes and will be used as a secondary data source for the National Evaluation Category A activities. However, partner sites do not submit information to CDC under this clearance.

The additional information to be collected for the National Evaluation is needed to better contextualize and understand the routine reported measures, and to provide insight on organizational level changes and outcomes. The specific information required for the 1815 National Evaluation involves additional questions for HD recipients, and additional respondents (DSMES and National DPP sites), not captured by routine reporting (Table A.2-A & Attachment 3d). Specifically, interviews and group discussions conducted as part of the Category A Rapid Evaluation (Att. 4c – 4i) will provide recipient and partner site-specific information that cannot be obtained through HD recipient calls with CDC project officers and/or evaluators, routine reporting documents, or other data collected through OMB No. 0920-1132. Recipient-led evaluation deliverables are tightly focused on HD recipient-specific implementation experiences, which will not be systematically collected or reported through any other existing mechanism.

The clinical studies underlying the National DPP, and the logic model for the 1815 cooperative agreement (Attachment 3c), predict beneficial changes in health indicators for individuals who participate in lifestyle change programs offered by the National DPP. Organizations that deliver evidence-based LCPs may apply to CDC for recognition, which is awarded upon achievement of predicted health outcomes (see OMB No. 0920-0909, CDC Diabetes Prevention Recognition Program [DPRP], exp. 2/28/2021). CDC will use DPRP data, along with additional data sources identified by HD recipients, as secondary sources of information about intermediate outcomes of the 1815 cooperative agreement (e.g., increased enrollment in CDC-recognized LCPs), and long-term outcomes (e.g., positive changes in health indicators for LCP participants). Secondary sources of data will also be used to assess changes in the reach and impact of DSMES. The proposed data collection and analysis plan will allow CDC to leverage other data sources and eliminate duplicative data collection for selected indicators of 1815 reach and impact. CDC acknowledges limitations in the methodology proposed for the 1815 National Evaluation but anticipates that findings will sufficiently identify strategies for more rigorous impact studies.

The data collection activities included in this ICR will allow CDC to capture critical information needed to continuously improve programmatic efforts for the 1815 cooperative agreement and clearly demonstrate the use of federal funds.

A.5 Impact on Small Businesses or other Small Entities

It is possible that some partner sites which will be recruited to complete the Category A partner site-level interviews may be representatives of a small business, such as a small health center or community-based organization offering health education. However, CDC anticipates that this will be a rare occurrence, and participation is completely voluntary. There are no specific requirements for small businesses. Questions have been limited to the absolute minimum required for the intended use of the data/information. Outside of these partner sites, there will be no other small businesses involved in the data collection for the National Evaluation of the 1815 cooperative agreement.

A.6 Consequences of Collecting the Information Less Frequently

There are different data collection frequencies for different components of the evaluation. The frequency of data collection, along with consequences of collecting information less frequently, are detailed below.

Category A Partner Site-Level Rapid Evaluations

DSMES/National DPP Partner Site-Level Rapid Evaluation Site Nomination Form (Att. 4a, 4aa, 4b, 4bb): HD recipients will have the option to nominate sites in Year 3 of the cooperative agreement. In subsequent years, HD recipients may be asked to nominate a new site using the site nomination form if the site previously nominated chooses not to participate in the evaluation or withdraws from the evaluation. If these nomination forms are not collected, DDT will not be able to identify the programs HD recipients are working with that have the capacity to participate in the evaluation and would not be able to initiate the partner site-level rapid evaluations.

DSMES/National DPP Survey Questionnaire (Att. 4c, 4g): Respondents from HD recipients’ partner sites will respond to proposed information collection twice, once in Year 3 and once in Year 5 of the cooperative agreement. If survey data collection is conducted less frequently, CDC will not be able to track program processes at the partner site-level, and changes in technical support needs as the programs mature and/or expand.

DSMES/National DPP Site-level Interviews (Att. 4d, 4e, 4f, 4h, 4i): Stakeholders from HD recipients’ partner sites will be asked to participate in an interview only once, either in Year 3 or 5 of the cooperative agreement. Interviews will be held with different partner sites each year so that a specific partner site is included in the information collection only once. Data collection efforts are also spread across multiple years to make it more feasible for the National Evaluation Team to gather data from a broad sample of partner sites. Reducing the number of partner sites will reduce the richness of the findings and the benefit to the partner site-level interviewees.

Category A Recipient-Led Evaluations

Category A Evaluation and Performance Measurement Plan (inc. Strategy-Specific Recipient-Led Evaluation Questions and Indicators) (Att. 5a): HD recipients are required to complete and submit the Category A EPMP once in Year 1 of the cooperative agreement. If their evaluation focus changes, HD recipients may need to submit an updated plan in Years 2, 3, or 4 of the cooperative agreement. If the EPMP is submitted less frequently, CDC will not be able to track changes in the HD recipients’ evaluation and performance measurement plans.

Category A Recipient-Led Evaluation Reporting Template (Att. 5b): HD recipients are required to submit a report on their evaluation efforts and findings annually. The purpose of the annual recipient-led evaluation report is to track HD recipients’ progress toward accomplishing the cooperative agreement’s goals and identify areas for evaluation related technical assistance. If the collection is conducted less frequently, CDC will not be able to track the progress of the recipient’s evaluation and key challenges and successes in implementing selected strategies.

A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This request fully complies with the regulation 5 CFR 1320.5.

A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A. Federal Register Notice

A 60 Day Federal Register Notice was published in the Federal Register on July 5, 2019 [Volume 84, Number 129, pages 32185-32187] (Attachment 6). There were no public comments in response to the Notice.

B. Other Consultations

The data collection instruments were designed collaboratively by the National Evaluation Team.

A.9 Explanation of Any Payment or Gift to Respondents

Category A Partner Site-Level Rapid Evaluations

The Partner site-level rapid evaluation will gather data from organizations not directly funded by CDC. As such partner sites participating in the evaluation (N=204) will receive a stipend, to offset labor hours dedicated to participation in the evaluation data collection activities. All sites will receive a $125 stipend $25 per respondent x (an estimated) 5 respondents per site, after completion of each round of site survey data collection (Years 3 and 5 of the cooperative agreement). Partner sites that are selected for and that agree to participate in a virtual site visit will receive an additional one-time stipend of $500 following completion of the virtual site visit to augment additional labor hours dedicated specifically to reviewing rapid evaluation documents, completing survey questionnaires, participating in on-site data collection activities, and any additional evaluation activities that go above and beyond routine operations.

CDC acknowledges the burden of data collection on each delivery site and the scheduled interruptions of services, staff coordinating interviews, and staff preparing secondary data for collection efforts.  These sites do not receive additional funding for these activities; thus, providing a stipend will ensure staff (mostly hourly staff) are compensated for any additional work outside the scope of their daily work activities. In addition, CDC provides considerable amounts of funds to state health departments to provide technical assistance and training to these sites. Our evaluation efforts will allow insight into the impact of CDC funding to the state health departments related to activities implemented, what can be improved, and how best to scale up or scale back the technical assistance and resources provided to these sites.

Additionally, the National Diabetes Prevention Program (National DPP) helps those with prediabetes to make lifestyle and behavioral changes to prevent type 2 diabetes. Participant incentives may help with program success, but their impact has not been systematically assessed. CDC conducted a landscape analysis of current literature to evaluate the impact of using incentives to improve health outcomes in lifestyle change or behavioral change programs, identifying 60 articles for analysis. The main incentive types were cash (55%), non-cash financial (28%), and non-financial (17%). The main behavioral/health outcomes were BMI/weight, physical activity, blood pressure, and blood glucose. An additional example is peer-reviewed research which tested the role of financial incentives to motivate engagement in diabetes prevention programs (DPPs), specifically the impact of financial incentives on DPP program completion rates. Findings show consistent and significant impacts that providing incentives improves DPP class attendance and program completion across states included in the study.

A.10 Protection of the Privacy and Confidentiality of Information Provided by Respondents

CDC’s Privacy Office has reviewed this submission and has determined that the Privacy Act does not apply. The data collection does not involve collection of sensitive or identifiable personal information. Although contact information is obtained for each recipient, the contact person provides information about the organization, not personal information. No system of records will be created under the Privacy Act.

A.11 Institutional Review Board (IRB) and Justification for Sensitive Questions

Respondents for the data collection efforts included in the ICR are cooperative agreement recipients and staff members from their partner sites. The data collection does not involve research with human subjects. Data to be collected is not sensitive in nature and reflects information at the organization level rather than individual level. The information collection does not require consent from individuals or IRB approval (Attachment 7a, 7b).

A.12 Estimates of Annualized Burden Hours and Costs

A.12-A Estimated Annualized Burden Hours

CDC estimates there are approximately 857 people each year who will participate in the evaluation of Category A. The estimated burden per response is between 0.5 and 2 hours to complete each data collection tool, except for the annual evaluation reporting deliverable templates, which are anticipated to require up to 8 hours to complete. This is due to the complexity of the information being requested in the evaluation report. Burden time estimates have been calculated based on the National Evaluation team’s experience developing and administering surveys and interviews. Over the three-year requested approval period of this information collection request, the total estimated annualized burden for the 51 current HD recipients and corresponding partner sites is 1084 hours as summarized in the table below.

Total burden has been calculated to reflect annualized burden hours over the three-year collection period (see footnotes for calculations). It has been weighted appropriately to reflect the number of times (once, twice, or thrice) data is being collected over the three-year collection period. Annualized number of respondents has also been calculated for the three-year data collection period (Attachment 3e).

Table A.12-A. Annualized Burden Hours

Type of Respondent	Form Name	Number of Respondents	Number of Responses per Respondent	Average Burden per Response (in hours)	Total Burden (in hours)
Health Department (1815 Recipient)	Att. 5a: Category A EPMP Template	17	1	8	136
	Att. 5b: DDT Recipient-Led Annual Evaluation Reporting Template	51	1	8	408
	Att. 4a. & 4aa: National DSMES Rapid Evaluation Nomination Form	17	1	0.5	9
	Att. 4b. & 4bb: National DPP Rapid Evaluation Nomination Form	17	1	0.5	9
DSMES Partner Site	Att. 4c: DSMES Rapid Evaluation Survey Questionnaire	340	1	0.5	170
	Att. 4d: DSMES Rapid Evaluation Interview Guide - Program Coordinator	14	1	2	28
	Att. 4e: DSMES Rapid Evaluation Interview Guide - Professional	28	1	2	56
	Att. 4f: DSMES Rapid Evaluation Interview Guide - Paraprofessional	28	1	2	56
National DPP Partner Site	Att. 4g: National DPP Rapid Evaluation Survey Questionnaire	340	1	0.5	170
	Att. 4h: National DPP Rapid Evaluation Interview Guide - Program Coordinator	14	1	1	14
	Att. 4i: National DPP Rapid Evaluation Interview Guide - Lifestyle Coach	28	1	1	28
				Total	1,084

A.12-B Estimated Annualized Cost to Respondents

Total cost has been calculated to reflect annualized cost over the three-year collection period. Annualized cost has been calculated using U.S. Department of Labor Bureau (DOL) of Labor Statistics estimates using the best approximation of DOL occupation titles and wage classification for each type of respondent. The expected equivalent occupation titles and wages for target respondents’ positions were obtained from the DOL database and used to populate Table A.12-B. In some cases, individuals in different roles/positions (i.e. occupational titles) will respond to the same data collection tool. The average hourly wage is a composite and average of the identified wage classification for each type of respondent.

Table A.12-B. Annualized Cost to Respondents

Type of Respondent	Form Name	Number of Respondents	Number of Responses per Respondent	Total Burden Hours	Average Hourly Wage1	Total Respondent Cost2
Health Department (1815 Recipient)3	Att. 5a: Category A EPMP Template	17	1	136	$44.00	$5,984.00
	Att. 5b: DDT Recipient-Led Annual Evaluation Reporting Template	51	1	408	$44.00	$17,952.00
	Att. 4a. & Att. 4aa: National DSMES Rapid Evaluation Nomination Form	17	1	9	$44.00	$396.00
	Att. 4b. & Att. 4bb: National DPP Rapid Evaluation Nomination Form	17	1	9	$44.00	$396.00
DSMES Partner Site	Att. 4c: DSMES Rapid Evaluation4 Survey Questionnaire	340	1	170	$29.00	$4,930.00
	Att. 4d: DSMES Rapid Evaluation Interview Guide – Program Coordinator5	14	1	28	$27.00	$756.00
	Att. 4e: DSMES Rapid Evaluation Interview Guide - Professional6	28	1	56	$41.00	$2,296.00
	Att. 4f: DSMES Rapid Evaluation Interview Guide – Paraprofessional7	28	1	56	$18.00	$1,008.00
National DPP Partner Site	Att. 4g: National DPP Rapid Evaluation Survey Questionnaire8	340	1	170	$27.00	$4,590.00
	Att. 4h: National DPP Rapid Evaluation Interview Guide – Program Coordinator9	14	1	14	$27.00	$378.00
	Att. 4i: National DPP Rapid Evaluation Interview Guide – Lifestyle Coach10	28	1	28	$21.00	$588.00
				Total		$39, 274.00

A.13 Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

No capital or maintenance costs are expected. Additionally, there are no start-up, hardware, or software costs.

A.14 Annualized Cost to the Government

The average annualized cost to the Federal Government is $1,333,819.00 as summarized in Table A.14-A. Major cost factors for tool development include form design and development costs and maintenance costs.

Table A.14-A. Annualized Cost to the Federal Government

Cost Category	Total Cost Over 3-Year Period	Total Annualized Cost
CDC - DDT Personnel 100% GS-12@$ 78,446/year = $78, 446 50% GS-13 @ $ 93,282/year = $46,641 50% GS-13 @ $ 93,282/year = $46,641 25% GS-14 @ $ 110,231/year = $27,558 Total, CDC Personnel	$597,858.00	$199,286.00
Data Collection Contractor Category A = $974,533.00 Total, Category A Contractor	$2,923,600.00	$974,533.00
Category A Stipends (204 sites participating in survey x $125 Stipend x 2 Collections over 3 Year Period)/ (3 Years) = $17, 000 (28 sites selected for virtual site visits x $500 stipend x 3 Collections over 3 Year Period)/ (3 Years) = $14,000 Total, Category A Stipends	$93,000.00	$31,000.00
Total, Category A	$3,614,458.00	$1,204,819.00

A.15 Explanation for Program Changes or Adjustments

This is a new collection.

A.16 Plans for Tabulation and Publication and Project Time Schedule

Tabulation

Attachment 3f is a Gantt chart for data being collected over a four-year period during Years 2 through 5 of the cooperative agreement. Recipient-submitted reporting deliverables will be due to CDC annually on September 30^th. Final reports for the last year of the cooperative agreement (July 1, 2022 – June 30, 2023) will be due no later than September 30, 2023 (90 days after the end of the funding period). OMB approval is being requested for three years with the desired data collection process to begin in March 2020. CDC plans to seek an extension of OMB approval for the final data collection for the national evaluation.

B. Publication Plan

Information collected by the HD recipients will be reported in internal CDC documents and shared with state-based programs.

Category A National DPP/DSMES Partner Site-Level Rapid Evaluation: Findings from each virtual site visit will be summarized and shared with the site and respective HD recipients. Cross-site findings, including virtual site visits, document review, surveys, and American Diabetes Association (ADA)/Association of Diabetes Care & Education Specialists (ADCES) or Diabetes Prevention Recognition Program (DPRP) data analysis findings, will be compiled annually and shared in written reports as well as oral presentations with key stakeholders. In addition to a comprehensive report, we will develop 1-2 cross-site briefs (3-5 pages each) on key constructs of interest that highlight the most actionable findings, spotlight promising practices, and translate findings into program recommendations. CDC will explore opportunities for co-presenting and co-publishing partner site-level rapid evaluation findings following completion of the analysis and reporting of the data.

Recipient-Led Evaluation EPMP/Reporting Deliverables: Findings noted within the recipient-led EPMPs, evaluation reports and deliverables will be combined with findings from the case studies, partner site-level rapid evaluations, and performance measures to provide a comprehensive overview of HD recipient progress and outcomes.

1. Analysis plan

Qualitative Data: Qualitative data from key informant interviews, group discussions, and the contribution analysis will be imported and analyzed separately using NVivo. The National Evaluation Team will conduct both content and thematic content analysis for an examination of both manifest (i.e., the actual words used) and latent (i.e., the underlying meaning of the words) content on open-ended statements to identify key themes. The thematic analysis will be theory-driven, based on the program logic model and program operational guidance. The National Evaluation Team will construct a codebook to facilitate the thematic analysis, developing a-priori codes based on themes expected per the program logic model and program operational guidance. The team will revise the coding structure in an iterative manner to ensure that emergent themes are captured in a systematic manner. For the recipient-led EPMPs and evaluation reports, the National Evaluation Team will conduct a systematic analysis to assess the strategies that are being evaluated across the recipients, their evaluation questions, and proposed indicators. The qualitative analysis and the systematic analysis will look across the collected data for similarities and differences in barriers, facilitators, and implied impacts and recommendations related to continuous quality improvement.

Quantitative Data: Close-ended responses from HD and partner site-level interviews, surveys, and other quantitative data such as performance measures and de-identified patient reports, will be analyzed descriptively. Data may be stratified by strategy, demographic characteristics, or other factors, with the potential for using cross-tabs and other techniques to break-down data by components of interest.

For the partner site-level rapid evaluation, we will run comparative tests (e.g. t-tests on outcome variables of interest) for 1815 HD-supported sites and non-supported sites. We will also conduct regression analyses to examine the impact of variables such as HD support, source of referral, DSMES or CDC-recognized LCP team composition on program implementation and participant outcomes. Once analyzed separately, findings will be assessed across data sources, such as interview findings compared with survey and DPRP or ADA/ADCES data. A matrix will be developed to identify patterns, overlapping themes, and outliers and to respond to each evaluation question.

For the recipient-led evaluations, insights will be triangulated with findings from the other performance monitoring assessments to either corroborate or refute other findings, thereby strengthening the conclusions to be drawn from the assessment.

Provided there is an adequate sample size at the partner-level, we will conduct multivariate regression analysis to identify the potential drivers of cost for a given strategy, as well as the key factors driving cost variability across implementing sites. Independent variables such as size of the site, area served (urban vs. rural), type of organization (independent practice vs. FQHC vs. hospital system) will be used as part of the model to understand cost drivers. Similar to the HD-level analysis, we will use data from the National Evaluation (including recipient-led evaluations) to qualitatively assess factors that could potentially affect the costs of the scale-up of interventions. This will assist in identifying possible sources of economies and diseconomies of scale, which in turn can help model non-linear cost functions.

A.17 Reason(s) Display of OMB Expiration Date is Inappropriate

The data collection tools will display the expiration date for OMB approval.

A.18 Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification statement.

1 Estimates for the average hourly wage for respondents are based on the U.S. Department of Labor Bureau (DOL) of Labor Statistics May 2018 National Occupational Employment and Wage Estimates (http://www.bls.gov/oes/current/oes_nat.htm).

2 Total Respondent Cost = (Total burden hours) x (Average hourly wage)

3 Health Department (1815 Recipient) = [(Medical and Health Services Manager ($54.68) + Medical Scientist ($46.46) + Epidemiologist ($36.39) +Environmental Scientists and Specialists (including health) ($37.30)]/(4) = $44.00

4 DSMES Partner Site Staff (Program/Quality Coordinator, DSMES Paraprofessionals, DSMES Professionals) = [Healthcare Social Workers ($27.31) + CHWs/Medical Assistants/Pharmacy Technicians ($17.77) + Pharmacists/RNs/Dietitians ($41.42)]/(4) = 28.83 = $29.00

5 DSMES Partner Site Staff (Program/Quality Coordinator) = Healthcare Social Worker = $27.31 = $27.00

6 DSMES Partner Site Staff (Professionals: Pharmacists, RNs, RDs) = [Pharmacists ($58.52) + Registered Nurse ($36.30) + Dietitian ($29.43)]/(3) = $41.42 = $41.00

7 DSMES Partner Site Staff (Paraprofessionals: CHWs, Medical Assistants, Diabetes and Pharmacy Technicians) = [CHWs ($20.90) + Medical Assistants ($16.61) + Pharmacy Technicians ($16.35)]/(3) = $17.95 = $18.00

8 National DPP Partner Site Staff (Program Coordinator and Lifestyle Coach) = (Healthcare Social Worker ($27.31) + Health Educator ($26.68))/(2) = $27.00

9 National DPP Partner Site Staff (Program Coordinator) = Healthcare Social Worker = $27.31 = $27.00

10 National DPP Partner Site Staff (Lifestyle Coach) = Community Health Worker = $20.90 = $21.00

23