Ttaeval_ssb_5.27.20

Supporting Statement: Part B

Assessment of Technical Assistance and Training Approaches to Accelerate Comprehensive Cancer Control Outcomes

May 27, 2020

Program Official/Project Officer:

Angela Moore, MPH

Lead Public Health Advisor

Division of Cancer Prevention and Control

National Center for Chronic Disease Prevention and Health Promotion

Centers for Disease Control and Prevention

4770 Buford Hwy NE, MS-F75

Atlanta, GA 30341-3717

OFFICE: 770-488-3094

FAX: 770-488-4335

Table of Contents

Part B: Collection of Information Employing Statistical Methods

B1. Respondent Universe and Sampling Methods

B2. Procedures for Collection of Information

B3. Methods to Maximize Response Rates and Deal with Nonresponse

B4. Tests of Procedures or Methods to be Undertaken

B5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Attachments

1 Authorizing Legislation

2 60-day Federal Register Notice publication

2a 60-day Federal Register Notice Response to Comments

3a Conceptual Framework for DP18-1805

3b Evaluation Matrix for DP18-1805

4a Case Study Interview Introductory Letter

4b Worksheet for Identifying Case Study Interviewees

4c Case Study Interview Introductory Email

4d Case Study Interview Guide for TTA Provider Program Directors or Managers

4e Case Study Interview Guide for TTA Provider Evaluators

4f Case Study Interview Guide for TTA Provider Partners

4g Case Study Interview Informed Consent

5a Web-based survey Pre-notification Email

5b Web-based survey Invitation Email

5c Web-based survey First and Second Reminder Emails

5d Web-based Survey (Word format)

5e Web-based Survey (screenshots)

5f Web-based survey Informed Consent

5g Web-based survey Thank You Email

6 ICF Institutional Review Board Approval Letter

Part B: Data collection procedures

B1. Respondent Universe and Sampling Methods

The assessment of the DP18-1805 cooperative agreement will consist of two data collection activities: 1) case studies of TTA providers; 2) a web-based survey with NCCCP awardees (program directors and staff), NCCCP partners, and NCCCP coalition members. Exhibit 1 displays the expected number of respondents for each data collection activity. Statistical sampling methods are not applicable to this data collection and cannot be used to conduct the assessment.

Exhibit 1. Number of Respondents by Data Collection Activity

Type of Respondents	Data Collection Activity	Frequency and Timing of Data Collection	Number of Respondents per Collection	Total Number of Responses Over 3 Years	Annualized Number of Responses
NCCCP Awardee Program Directors and Staff	Web-based survey	2x (2020, 2022)	132	264	88
NCCCP Coalition Members, and NCCCP Partners	Web-based survey	2x (2020, 2022)	132	264	88
TTA Provider Organizations)	Worksheet for Identifying Case Study Interviewees	1x (2021)	2	2	1
TTA Provider Directors or Managers	Case Study Interview Guide for TTA Provider Program Directors or Managers	1x (2021)	4	4	1
TTA Provider Evaluators	Case Study Interview Guide for TTA Provider Evaluators	1x (2021)	4	4	1
TTA Provider Partners	Case Study Interview Guide for TTA Provider Partners	1x (2021)	8	8	3
Total Responses Per Collection			280

Case Studies

The respondent universe for case studies is comprised of the 2 organizations that have been funded under the DP18-1805 cooperative agreement to provide support via training and technical assistance (TTA) to organizations funded by the National Comprehensive Cancer Control Program (NCCCP) (henceforth NCCCP awardees). As a part of the case studies, ICF (contractor) will conduct case study interviews with key staff and partners from TTA provider organizations who are involved with the cooperative agreement and in the planning, implementation, and evaluation of TTA. Using purposive non-probability sampling, a sampling approach that facilitates the selection of individuals based on specific characteristics, up to 8 individuals from each of the 2 TTA provider organizations will be invited to participate in case study interviews, for a total of 16 respondents. The contractor will work with TTA providers to identify and recruit participants in accordance within specific target populations (e.g., involved in the planning, implementation, and evaluation of the cooperative agreement and TTA) so that their responses will provide meaningful contributions to our evaluation questions (Attachments 4d-4f).

Web-based Survey

The respondent universe for the web-based survey is comprised of individuals affiliated with the NCCCP awardee programs that could have received TTA from the TTA provider organizations and can include NCCCP program directors, managers and staff, NCCCP partners, and/or NCCCP coalition members. The team will employ convenience sampling to administer the survey. CDC program consultants who work with NCCCP awardees will reach out to all 66 NCCCP awardee program directors and ask them to identify up to three additional individuals from their staff, coalition, or partners that may have received TTA from one of the TTA providers. CDC will share those contacts with the contractor. Therefore, the sample will include 66 NCCCP awardee program directors, plus up to 198 additional individuals. The survey will be administered in Year 2 and in Year 4 of the cooperative agreement. In total, we will collect information from up to 264 individuals (66 NCCCP awardee program directors + 198 NCCCP awardee program staff, NCCCP coalition members, or NCCCP partners).

B2. Procedures for Collection of Information

Case Studies

Interview guides tailored for each specific audience will be used to collect information from participants. All participants (program directors or managers, program evaluators, and partners) will be interviewed at one time point in Year 3 of the cooperative agreement. Interviews with program directors or managers, program evaluators, and partners will explore implementation of the cooperative agreement and TTA (fidelity, dose, intensity, duration); barriers, facilitators, and contextual factors that affect implementation of the cooperative agreement and TTA; and perceptions regarding the quality and effectiveness of specific TTA efforts (Attachments 4d-4f). All interviews will be conducted virtually via telephone.

Upon OMB approval, the contractor will send an Introductory Letter and Introductory Email to each TTA provider’s program director or manager to provide them with information about the case studies and confirm their interest and willingness to participate (Attachment 4a). Within one month, the contractor will schedule a conference call with each TTA provider via email and work with a program director or manager from each TTA provider program to complete the Worksheet for Identifying Case Study Interviewees to identify appropriate respondents for each interview (Attachment 4b). The program director or manager will be asked to identify up to 7 additional individuals, from their staff or partners, who may have participated in the planning, delivery, or evaluation of TTA as a part of their DP18-1805 activities.

Two project team members will lead each interview, one as the lead interviewer and the other as the primary note-taker. Each interviewer will be trained in the full project protocol, including each of the tailored interview guides, and will review pertinent program materials. At the start of each interview, the interviewer will read aloud the informed consent and ask the respondent to give verbal consent to participate and for the interview team to audio record the interview for analysis purposes (Attachment 4g). The interviewer will use the appropriate tailored interview guide to generate questions and probes for gathering information throughout the interview (see Attachments 4d-4f). Each interview will be audio recorded to serve as a back up to interview notes.

Web-based Survey

The web-based survey will be administered at two time points, once in Year 2 and again in Year 4 of the cooperative agreement. The contractor will manage the information collection process on CDC’s behalf. To generate the initial sample for the web-based survey, the contractor will work with CDC to gather contact information for all 66 NCCCP awardee program directors. CDC program consultant who works with NCCCP awardees will reach out to all 66 NCCCP awardee program directors and ask them to identify up to three additional individuals, from their staff, coalition, or partners, who may have received TTA from one of the DP18-1805 TTA providers. CDC will share those contacts with the contractor. The contractor will compile this information into a master file for purposes of administration of the web-based survey.

For the web-based survey, all potential respondents will receive the following rounds of communication: a pre-notification email informing them of the web-based survey (Attachment 5a); an invitation email with the survey link (Attachment 5b); a first reminder email sent 1 week after the invitation email (if they have not yet responded to the survey and a second reminder email sent 1 week after the first reminder (if they have not yet responded to the survey) (Attachment 5c); and a thank you email sent within 1 week of completion of the survey (Attachment 5g).

Information will be collected, stored, and maintained by the contractor and protected under data privacy policies. Both quantitative and qualitative analyses will be performed. Quantitative analyses will involve using descriptive statistics to determine frequency distributions and corresponding variances for responses to each web-based survey question and will be conducted using a statistical software package for data management and analysis. Qualitative thematic analyses will be conducted on open-ended questions. Analysis will focus on describing: (1) the reach of DP18-1805 TTA provider efforts; (2) the TTA received among respondents, including type, dosage, frequency and format; and (3) individuals’ perceptions of the effectiveness of the TTA received.

Collectively, the information collection will be used to answer the following key questions:

1. How are TTA providers building capacity among NCCCP awardee programs?

1. To what extent did TTA providers achieve the planned short-term outcomes?

2. To what extent did TTA activities contribute to NCCCP implementation and achievement in NCCCP priorities and goals?

3. What elements of TTA were most effective in building the capacity of NCCCP awardee programs?

B3. Methods to Maximize Response Rates and Deal with Nonresponse

Case studies will include the 2 TTA provider organizations currently participating in the DP18-1805 cooperative agreements. To maximize response, the contractor will send multiple communications to each DP18-1805 TTA provider program director or manager to provide them with information about the case studies and confirm their interest and willingness to participate (Attachment 4a). The contractor will also provide TTA providers with support for identifying and scheduling interviews with appropriate respondents. In the event that one or more is unable or unwilling to participate, that organization will not be substituted with an alternate organization and the total number of case studies will be decreased. Participation is voluntary across all organizations.

The web-based survey (Attachment 5d-5e) is based on a previous web-based survey that was used for the last data collection. The average time to complete the previous web-based survey, including time for reviewing instructions, gathering needed information and completing the data collection tool, was approximately 15 minutes. Therefore, the burden on respondents will be reduced from 195 hours (for the previous survey) to 132 hours (for this survey). The contractor will use several rounds of communication reminders to help ensure a high response rate.

B4. Tests of Procedures or Methods to be Undertaken

CDC staff and contractors, who comprise the study team, were involved in the development, review, and approval of data collection instruments and other supporting documents. The web-based survey was adapted from a previous survey that received OMB approval (OMB No. 0920-1193). The main difference is that this survey involves fewer questions, which should mean reduced burden for respondents. See section B3 for more details.

B5. Individuals Consulted on Statistical Aspects and Individual Collecting and/or Analyzing Data

CDC provides overall direction for all data collection planning and implementation activities, including overseeing the data collection protocol and data reporting.

The contractor, ICF, will recruit and collect all data for all data collection activities described. ICF will also analyze and report assessment results.

The principal contacts for each organization are below.

Staff Name	Contact Information
CDC
Angela Moore, MPH Lead Public Health Advisor	Phone: 770-488-3094 Email: [email protected]
Trina Pyron, MA Public Health Advisor	Phone: 770-488-0971 Email:[email protected]
Jamila Fonseka, MPH Public Health Advisor	Phone: 770.488.4296 Email: [email protected]
ICF
Isabela Lucas, PhD Project Manager	Phone: 404-434-3154 Email: [email protected]

Angela Moore, MPH

Lead Public Health Advisor

Comprehensive Cancer Control Branch

Division of Cancer Prevention and Control

National Center for Chronic Disease Prevention and Health Promotion

Centers for Disease Control and Prevention

4770 Buford Hwy NE, MS-F75

Atlanta, GA 30341-3717

OFFICE: 770-488-3094

FAX: 770-488-4335

Information will be collected and analyzed by CDC’s contractor, ICF.