Ttaeval_ssa_5.27.20

Supporting Statement: Part A

Assessment of Technical Assistance and Training Approaches to Accelerate Comprehensive Cancer Control Outcomes

May 27, 2020

Program Official/Project Officer:

Angela Moore, MPH

Lead Public Health Advisor

Division of Cancer Prevention and Control

National Center for Chronic Disease Prevention and Health Promotion

Centers for Disease Control and Prevention

4770 Buford Hwy NE, MS-F75

Atlanta, GA 30341-3717

OFFICE: 770-488-3094

FAX: 770-488-4335

Table of Contents

Part A: Justification

A1. Circumstances Making the Collection of Information Necessary

A2. Purpose and Use of the Information Collection

A3. Use of Improved Information Technology and Burden Reduction

A4. Efforts to Identify Duplication and Use of Similar Information

A5. Impact on Small Businesses or Other Small Entities

A6. Consequences of Collecting the Information Less Frequently

A7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

A8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A9. Explanation of Any Payment or Gift to Respondents

A10. Assurance of Confidentiality of Respondents

A11. Institutional Review Board (IRB) and Justification for Sensitive Questions

A12. Estimates of Annualized Burden Hours and Costs

A13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

A14. Annualized Cost to the Government

A15. Explanation for Program Changes or Adjustments

A16. Plans for Tabulation and Publication and Project Time Schedule

A17. Reason(s) Display of OMB Expiration Date is Inappropriate

A18. Exceptions to Certification for Paperwork Reduction Act Submissions

Attachments

1 Authorizing Legislation

2 60-day Federal Register Notice publication

2a 60-day Federal Register Notice Response to Comments

3a Conceptual Framework for DP18-1805

3b Evaluation Matrix for DP18-1805

4a Case Study Interview Introductory Letter

4b Worksheet for Identifying Case Study Interviewees

4c Case Study Interview Introductory Email

4d Case Study Interview Guide for TTA Provider Program Directors or Managers

4e Case Study Interview Guide for TTA Provider Evaluators

4f Case Study Interview Guide for TTA Provider Partners

4g Case Study Interview Informed Consent

5a Web-based survey Pre-notification Email

5b Web-based survey Invitation Email

5c Web-based survey First and Second Reminder Emails

5d Web-based Survey (Word format)

5e Web-based Survey (screenshots)

5f Web-based survey Informed Consent

5g Web-based survey Thank You Email

6 ICF Institutional Review Board Approval Letter

Goal of the study. Document the implementation of training and technical assistance (TTA) administered for National Comprehensive Cancer Control Program (NCCCP) awardees and explore how TTA implementation is associated with NCCCP outcomes.    

Intended use of the resulting data. Monitor TTA and inform the development of future TTA models for NCCCP awardees.

Methods to be used to collect information. Mixed-methods approach using case studies (qualitative) and a web-based survey (quantitative).

The subpopulation to be studied. Organizations administering TTA and organizations receiving TTA (NCCCP awardees, partners, and coalition members).

How data will be analyzed. Qualitative (thematic) analysis and quantitative (descriptive statistics for survey responses) analysis.

1. Justification

A1. Circumstances Making the Collection of Information Necessary

CDC requests reinstatement, with changes, of a previously approved Information Collection Request (Assessing the Impact of Targeted Training and Technical Assistance Efforts on the Implementation of Comprehensive Cancer Control Outcomes, OMB# 0920-1193; expiration date 7/31/2019). During the period of reinstatement CDC will collect information needed to monitor and inform the delivery of training and technical assistance (TTA) administered for the National Comprehensive Cancer Control Program (NCCCP). CDC is authorized to collect the information by the Public Health Service Act (Attachment 1). The 60-day Federal Register Notice was published on December 6, 2019 (Attachment 2) and is further discussed in Section A8. OMB approval is requested for three years.

Cancer is the second leading cause of death in the United States, and health care costs for cancer care are expected to rise to $158 billion by 2020.^1,2 Addressing this public health problem requires primary prevention, early detection and treatment, cancer survivorship support systems, and a reduction in health disparities. The coordination of resources and activities can position state, tribal, territorial and local entities to implement evidence-based interventions, which may impact population-level cancer outcomes and reduce the burden of cancer.

The Centers for Disease Control and Prevention’s (CDC) National Comprehensive Cancer Control Program (NCCCP) has been a primary funder for state and community-based cancer control interventions since its inception in the late 1990s. The program supports states and communities in developing a comprehensive approach to cancer prevention and control, which includes infrastructure for state, local, and population-based interventions, multi-sectoral partnerships, and coalitions. Currently, NCCCP supports 66 cancer control program awardees including programs in all 50 states, the District of Columbia, and in a number of tribes, tribal organizations, and U.S. Associated Pacific Islands/territories.

Through previous training and technical assistance (TTA) programs to NCCCP awardees (DP13-1315), CDC has supported states and communities in the implementation of local programs and contributed to the achievement of local program aims. From 2017-2019, CDC collected information from 10 organizations (awardees) that delivered TTA and from the recipients of the TTA (NCCCP and NSBT grantees, partners, and coalition members). Some recipients of CDC-funded organizations also received TTA from national networks. Published evidence, though limited, suggests that easily accessible, collaborative TTA approaches are best suited to build the capacity of TTA recipients.³

To continue improvement and enhancements of TTA for NCCCP awardees, CDC is continuing its TTA efforts with a five year cooperative agreement titled, Provision of Technical Assistance and Training Activities to Assure Comprehensive Cancer Control Outcomes, (DP18-1805) (hereafter referred to as DP18-1805). Lessons learned from previous TTA cooperative agreements were incorporated in DP18-1805, and the number of organizations funded to provide TTA has been reduced from 10 to 2. The TTA activities for DP18-1805 include: 1) conducting a needs assessment, 2) developing a framework for building CCC capacity, 3) coordinating and collaborating with existing partners, 4) developing a TTA plan, 5) implementing a TTA plan and conducting performance monitoring and continuous quality improvement; and 6) conducting a comprehensive evaluation of TTA.

The current ICR reflects programmatic changes that have led to an adjustment of the evaluation design, modifications in the case study design, and removal of in-depth interviews with TTA consumers who used both national networks and the previous cooperative agreement (DP13-1315).

To assess whether this cooperative agreement has been implemented as intended and contributed to NCCCP awardees’ achievements in program goals and outcomes, CDC is conducting a mixed-method study. This data collection will provide a better understanding of the TTA provided and help identify the extent to which core elements of TTA were administered and most effective. This data collection will also provide the information and guidance needed to develop future TTA efforts to more effectively and efficiently support NCCCP awardees.

A2. Purpose and Use of Information Collection

The purpose of this data collection is to: (1) document the nature of the TTA provided and was able to achieve planned short-term outcomes; and (2) identify the extent to which TTA efforts contributed to NCCCP awardees’ achievement in program outcomes. The team created an evaluation conceptual framework (Attachment 3a) that was used to develop 4 primary process- and outcome-related evaluation questions and an evaluation matrix to map all corresponding evaluation questions to appropriate indicators, data sources, and potential analysis (Attachment 3b). The evaluation questions are:

1. How are TTA providers building capacity among NCCCP awardee programs?

1. To what extent did TTA providers achieve the planned short-term outcomes?

2. To what extent did TTA activities contribute to NCCCP implementation and achievement in NCCCP priorities and goals?

3. What elements of TTA were most effective in building the capacity of NCCCP awardee programs?

This proposed data collection will involve two complementary efforts: 1) case studies of TTA providers; 2) a web survey with NCCCP awardees, partners, and coalition members receiving TTA.

The case studies will entail document review and interviews with staff and partners involved in administering TTA. The evaluator will work with a program administrator from each TTA provider organization to recruit, identify, select, and contact appropriate interview respondents and schedule interviews using an introductory letter (Attachment 4a), a worksheet for identifying interviewees (Attachment 4b), and a preparatory call (Attachment 4c). Interview guides are tailored for each respondent type and include questions and probes designed to gather the most pertinent information across respondent types. Some questions and probes are asked across all respondent types and others are specific to individual respondent types. All respondents will be interviewed once in Year 3 of the evaluation period (see Attachments 4d, 4e, and 4f). All participants will be read an informed consent statement prior to participating in the interviews (Attachment 4g). Information gathered via case studies will be used to describe implementation of TTA and its components, the factors that affect implementation, and perceived effectiveness of activities.

The web-survey will assess the amount and type of TTA received and the perceived effectiveness of TTA from NCCCP awardee programs, partners, and coalition member organizations. The evaluator will recruit and communicate with web-based survey participants with a pre-notification e-mail (Attachment 5a), an invitation e-mail (Attachment 5b), and reminder e-mails (Attachment 5c). The web-based survey (Attachments 5d-5e) will be administered with recruited individuals two times during the study period—once in Year 2, and once in Year 4. An informed consent statement will be included on the cover page of the survey instrument prior to the instrument questions (Attachment 5f) and a thank you e-mail will be sent if the survey is completed (Attachment 5g).

Collectively, the proposed information collection activities and tools have been designed specifically to meet the aims of this assessment and will contribute to CDC’s ability to demonstrate the results of TTA administered. This information will also help with the types of TTA activities that best enhance NCCCP awardees’ capacity to implement high-quality, evidence-based strategies for cancer prevention and control. This information can develop new or enhance existing TTA support systems for NCCCP awardees in the future.

A3. Use of Improved Information Technology and Burden Reduction

All information collection will be conducted remotely – case study interviews will be conducted via phone (Attachments 4d-4f) and the survey will be conducted via the Web (Attachment 5d-5e). Both methods use pre-existing web infrastructure and were chosen to reduce the overall burden on respondents. Conducting case study interviews by phone will allow respondents to participate from whatever location is most convenient to them, and likely requires less time and resources to coordinate in-person interviews. The web-based platform for the survey allows respondents flexibility, in that they are not required to complete the survey all at once; respondents can return to the survey at different time points that are convenient to their schedule in order to complete the survey.

A4. Efforts to Identify Duplication and Use of Similar Information

The proposed information collection is unique in that it is the only data collection effort to assess implementation and efficacy of TTA under DP18-1805. Document review was conducted on progress reports, work plans and evaluation plans to ensure information collected through the case study interviews, and the web-based survey was not duplicative.

A5. Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this data collection.

A6. Consequences of Collecting the Information Less Frequently

This information collection is critical to expanding CDC’s understanding of how TTA under DP18-1805 is being implemented and to what extent the TTA efforts contributed to achievement in NCCCP program outcomes. Without this information collection, CDC will have limited insight to plan and provide comprehensive TTA to NCCCP awardees which will limit CDC’s ability to adequately account for federal dollars spent on this public program. Less frequent data collection would compromise CDC’s ability to fully understand and document changes in the implementation of TTA over time and the potential success of the cooperative agreement.

A7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances relating to the guidelines of 5 CFR 1320.5, and the project fully complies with the regulation.

A8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

As required by 5 CFR 1320.8(d), CDC published a 60-day Notice in the Federal Register on December 6, 2019, Vol. 84, No. 235, pages 66901-66902 (see Attachment 2). One public comment was received on 12/06/2019 and posted on 12/09/2019. A formal response along with the full comment are included in Attachment 2a.

A9. Explanation of Any Payment or Gift to Respondents

Respondents will not receive any payment or gifts for their participation in this data collection effort.

A10. Assurance of Confidentiality of Respondents

This submission has been reviewed by CDC’s Information System Security Office, which determined that the Privacy Act does not apply. The proposed study involves a minimum amount of information in identifiable form (IIF).

Respondents will be recruited from TTA provider organizations (i.e., program directors or managers, evaluators, and partners), as well as NCCCP awardees (i.e., program directors, managers, coalition members, and partners). The data collection contractor, ICF, will have access to professional contact information for the program director or manager, evaluator, and key partners from each selected organization, including work telephone numbers and e-mail addresses, to recruit and schedule their participation in case study interviews.

IIF will be stored separately from response data. A linking file will be created and available only to senior project management at ICF. This information will only be used to ensure completeness of the data files. The linking file will include the role of the respondent and their organization (and will not include the individual’s name or contact information), the date of interview/survey completion, and the code assigned to the data file. This will ensure that no personally identifiable information, outside of the individual’s role and organization, is re-linkable. Prior to conducting data collection,data collectors will be trained on the project’s protocol and procedures related to security requirements and privacy.

Key individuals from the TTA provider organizations (i.e., program directors or managers, evaluators, and partners), and NCCCP awardees (i.e., program directors, staff, coalition members, and partners) will be recruited to participate in this information collection. Participation in data collection is voluntary for all participants; respondents who decline participation will not face penalty of any kind. All informed consent statements inform participants that their participation in data collection is voluntary, and they can choose not to answer individual questions, end the interview/survey at any time, or decline participation without penalty (Attachments 4g and 5f). Whether or not individuals choose to participate will not impact current or future funding. Respondents will be required to either agree to or decline participation prior to participating in data collection. During interviews, respondents may be asked to identify and describe entities and key staff members or partners who have familiarity with tha TTA provided under DP18-1805. No contact information will be collected for individuals who are discussed during the interviews. The purpose of collecting information about key staff and partners is to guide CDC in identifying the number and types of individuals that should be engaged in similar efforts across various settings. The information collected will be concentrated on planning and implementation of the cooperative agreement, not personal information about the individual in any specific role.

Although the data collection contractor will have temporary access to identifiable information for recruitment and scheduling purposes, response data will not be recorded in a manner that can be linked to respondent identifiers. The contractor will assign each interview respondent a unique identifier code, and will store and analyze interview data by identifier code. The personal contact information for respondents will not be used for analysis or reporting purposes. Survey data will not be linked with individual respondents. All data collected will be analyzed in aggregate and discussed in summary reports that do not contain any personal identifiers.

Study information and data, including contact information for respondents, linking identifiers, and interview and survey responses, will be destroyed within 3 years of the project end date. All electronic data files (e.g., interview notes, exported survey responses) will be stored at ICF on a project shared drive on ICF’s secure network servers; only project staff who have been authorized by the project manager will have access the shared drive.

A11. Institutional Review Board (IRB) and Justification for Sensitive Questions

No sensitive information is requested. CDC’s information collection contractor obtained IRB approval to conduct this assessment (Attachment 6).

A12. Estimate of Annualized Burden Hours and Costs

OMB approval is requested for three years. Information will be collected via two methods: 1) case studies and 2) a web survey. The estimate for all burden hours for the identification of case study participants and conduct of interviews is based on previous experience with similar data collection efforts by the project staff. For the case studies, the contractor will work with a program administrator from each TTA provider organization to identify appropriate case study interview respondents and schedule interviews (estimated burden per response is one hour) (Attachment 4b). The contractor will conduct interviews with up to 8 individuals from each TTA provider organization (i.e., up to 2 program directors or managers, up to 2 program evaluators, and up to 4 affiliated partners, including Comprehensive Cancer Control coalition members). The estimate for burden hours per respondent for the program director or manager interviews is 90 minutes (Attachment 4d), and the estimate of burden per respondent for all other case study interviews is 60 minutes (Attachment 4e and 4f). Interviews are tailored for each respondent type and include questions and probes designed to gather the most pertinent information across respondent types, with some questions and probes asked across all respondent types and some questions and probes specific to individual respondent types.

The estimate for burden hours for the web-based survey is guided by a pilot test of the information collection instrument by 6 public health professionals. In the pilot test, the average time to complete the instrument including time for reviewing instructions was 15 minutes. To administer the web-based survey, the team will employ convenience sampling. CDC program consultants who work with NCCCP awardees will reach out to all 66 NCCCP awardee program directors and ask them to identify up to three additional individuals from their staff, coalition, or partners that may have received TTA from one of the TTA providers. CDC will share those contacts with the contractor. Therefore, the sample will include 66 NCCCP awardee program directors, plus up to 198 additional individuals including NCCCP program staff, NCCCP coalition members, and NCCCP partners. The survey will be administered twice during the cooperative agreement. In total, we will collect information from up to 264 individuals (i.e., 66 NCCCP awardee program directors or managers, plus 198 additional respondents that the program directors may potentially identify) (Attachment 5d and 5e). The total annualized estimated burden is 51 hours.

Table A12-1. Estimated Annualized Burden Hours

Type of Respondents	Form Name	Number of Respondents	Number of Responses per Respondent	Average Burden per Response (in hrs)	Total Burden (in hrs)
NCCCP Awardee Program Directors, and Staff	Web-based survey	88	1	0.25	22
NCCCP Coalition Members, and NCCCP Partners	Web-based survey	88	1	0.25	22
TTA Provider Organizations	Worksheet for Identifying Case Study Interviewees	1	1	1	1
TTA Provider Directors or Managers	Case Study Interview Guide for TTA Provider Program Directors or Managers	1	1	1.5	2
TTA Provider Evaluators	Case Study Interview Guide for TTA Provider Evaluators	1	1	1	1
TTA Provider Partners	Case Study Interview Guide for TTA Provider Partners	3	1	1	3
Total					51

Table A12-2 presents the calculations for cost of annualized burden hours. Average hourly wage estimates were obtained from the U.S. Department of Labor, Bureau of Labor Statistics.

• The average annual salary of $59,010 for health educators and community health workers was used to calculate the hourly wage of $ $28.37 for TTA provider program administrators.

• The average annual salary of $60,700 for survey researchers was used to calculate the hourly wage of $29.18 for TTA provider program evaluators.

• The average annual salary of $70,530 for social and community service managers was used to calculate the hourly wage of $33.91 for all other TTA provider partners.

• The average annual salary of $123,460 for general and operational managers was used to calculate the hourly wage of $59.36 for NCCCP Program Directors, Staff, Coalition Members, and Partners.

The estimated annualized cost to respondents is $2,814, as summarized below in Table A.12-2.

Table A12-2. Estimated Annualized Cost to Respondents

Type of Respondents	Form Name	No. of Respondents	No. of Responses per Respondent	Average Burden per Response (in hrs)	Average Hourly Wage	Total Cost
NCCCP Awardee Program Directors and Staff	Web-based Survey	88	1	0.25	$59.36	$1,306
NCCCP Coalition Members, and NCCCP Partners	Web-based Survey	88	1	0.25	$59.36	$1,306
TTA Provider Organizations	Worksheet for Identifying Case Study Interviewees	1	1	1	$28.37	$28
TTA Provider Program Directors/ Managers	Case Study Interview Guide for TTA Provider Program Directors or Managers	1	1	1.5	$28.37	$43
TTA Provider Evaluators	Case Study Interview Guide for TTA Provider Evaluators	1	1	1	$29.18	$29
TTA Provider Partners	Case Study Interview Guide for TTA Provider Partners	3	1	1	$33.91	$102
TOTAL		182				$2,814

A13. Estimate of Other Total Annual Cost Burden to Respondents or Record Keepers

There will be no direct costs to respondents other than their time to participate in each information collection.

A14. Annualized Cost to the Government

Total operations and maintenance costs includes work performed by both the contractor and CDC personnel. Governmental costs for this project include CDC personnel costs, including an FTE GS-14 ($47.86 hourly rate) to lead the project and coordinate all related activities of each information collection as well as another FTE GS-13 ($39.31 hourly rate) to help with data management, analysis, and reporting. Two hundred eight (208) hours of staff time were estimated for each CDC FTE staff annually for this information collection. Total contractor cost for evaluation activities represents an estimated 75% ($244,911) of total contract funds ($326,762). The contractor is responsible for leading all data collection, including recruitment, scheduling of interviews, collecting and analyzing data. The estimated annualized total cost (CDC FTEs plus annualized contract costs) to the federal government is $99,768. Table A.14 shows how the annualized cost estimate was calculated.

Table A14-1. Estimated Annualized Cost to the Federal Government

Staff (FTE)	Average Hours per Collection	Average Hourly Rate	Annualized Cost
Contractor			$81,637
Public Health Advisor (GS-14) Project planning, management, OMB review, analysis of findings, and report writing	208	$47.86	$9,955
Public Health Advisor (GS13) time for project planning, management, OMB review, analysis of findings, and report writing	208	$39.31	$8,176
Estimated Annualized Cost			$99,768

A15. Explanation for Program Changes or Adjustments

This is a reinstatement with change. The title of the this information collection request has been changed from “Assessing the Impact of Targeted Training and Technical Assistance Efforts on the Implementation of Comprehensive Cancer Control” to “Assessment of Technical Assistance and Training Approaches to Accelerate Comprehensive Cancer Control Outcomes” to better align with how the program is being referred to. The program was expanded to include grantees from non-profit organizations and thus a new category was added under “Affected Public”. In the previous OMB approval period, the total estimated annualized burden was 231 hours. The burden hours for this request is 51hours, which is a decrease of 180 hours from our previous request. This decrease is due to various reasons. First, programmatic changes have led to an adjustment of the evaluation design. Funding has been moved to a separate cooperative agreement/funding line, therefore they are excluded from this reinstatement request and the in-depth interviews with TTA consumers who used both national networks and DP13-1315 have also been removed. Both actions decrease overall burden for the reinstatement.

A16. Plans for Tabulation and Publication and Project Time Schedule

Case study interviews will be completed one time in Year 3 of the evaluation study period. The web-based survey will be completed two times within the evaluation study period—once in Year 2 and once in Year 4 of the evaluation study period. Data validation, analysis, and report preparation will follow. CDC plans to disseminate the outcomes of the study within and outside the federal government in the form of scientific presentations, peer-reviewed publications, and tools and resources developed for NCCCP awardees.

• The project timeline is outlined in Table A16-1. This timeline describes the ideal schedule for data collection over a 2-year period, with data analysis completed approximately 2 years after receiving OMB approval. However, to provide flexibility in implementing the data collection plan, we are requesting OMB approval for 3 years. The considerations underlying this request include the following.OMB review and approval may be delayed due to the COVID-19 pandemic or other factors. The availability of personnel at every stage of ICR review may be impacted by additional responsibilities.

• This information collection request does not utilize in-person data collection methods involving disease risks that must be mitigated (data will be collected via telephone interviews or web-based surveys). However, COVID-19-related considerations could delay or impact project implementation in unforeseen ways.

• CDC prefers to initiate data collection in July 2020, if possible, as this will allow us to complete data collection and analysis as envisioned within the timeframe of the support contract.

• If OMB approval is not obtained by the end of June 2020, CDC’s options are to (1) compress the follow-up timeline for the second survey, which may impact response rates and the quality of the follow-up data with less time to assess changes that occur over the period of the cooperative agreement, or (2) seek a contract modification to extend the period of performance.

Therefore, to provide flexibility in project management, CDC requests OMB approval for 3 years.

A16-1 Estimated Project Schedule

Activity	Time Schedule*	DP18-1805 Budget Year
Send web-based survey pre-notification email	July 6, 2020 (5 days following OMB approval)	Year 2
Administer web-based survey – first wave	July 9, 2020 (8 following OMB approval)	Year 2
Analyze web-based survey data	July 31, 2020 (30 days after OMB approval)	Year 2
Send case study introductory letter and email	April 1, 2021 (9 months after OMB approval)	Year 3
Select case study interview participants	April 1, 2021 (9 month after OMB approval)	Year 3
Conduct case study interviews	June 1, 2021 (11 months after OMB approval)	Year 3
Analyze case study interview data	August 1, 2021 (13 months after OMB approval)	Year 3
Send web-based survey pre-notification email	April 1, 2022 (21 months following OMB approval)	Year 4
Administer web-based survey – first wave	April 15, 2022 (21 months following OMB approval)	Year 4
Analyze survey data	June 1, 2022 (23 months after OMB approval)	Year 4
Finalize data analysis, develop final report, and conduct dissemination activities on study findings	July– September 2022 (24-26 months after OMB approval)	Year 4

*Assumes start date of July 1, 2020

A17. Reason(s) Display of OMB Expiration Date is Inappropriate

The OMB expiration date will be displayed on all information collection instruments.

A18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification. These activities comply with the requirements in 5 CFR 1320.9.

1 Detailed tables for the National Vital Statistics Report Deaths: Final data for 2016. National Vital Statistics Report64(2). Retrieved January 10, 2019, from https://www.cdc.gov/nchs/data/nvsr/nvsr67/nvsr67_05.pdf

2 Mariotto, A. B., Yabroff, K. R., Shao, Y., Feuer, E. J., & Brown, M. L. (2011, January). Projection of the cost of cancer care in the U.S.: 2010–2020. Journal of the National Cancer Institute.

3 West (2012)