Supporting Statement: Part A
Development of an Evaluation Plan to Evaluate Grantee Attainment of Selected Activities of Comprehensive Cancer Priorities
Supported by:
Division of Cancer Prevention and Control
National Center for Chronic Disease Prevention and Health Promotion
Centers for Disease Control and Prevention
U.S. Public Health Service
Department of Health and Human Services
March 7, 2013
Government Project Officer:
Angela Moore, MPH
Program Evaluation and Partnership Team Lead
Comprehensive Cancer Control Branch
Division of Cancer Prevention and Control
National Center for Chronic Disease Prevention and Health Promotion
Centers for Disease Control and Prevention
4770 Buford Hwy, NE MS K57
Atlanta, GA 30341
Phone 770-488-3094
Fax 770-488-4335
Table of Contents
Part A: Justification
A1. Circumstances Making the Collection of Information Necessary
A2. Purpose and Use of Information Collection
A3. Use of Improved Information Technology and Burden Reduction
A4. Efforts to Identify Duplication and Use of Similar Information
A5. Impact on Small Businesses or Other Entities
A6. Consequences of Collecting the Information Less Frequently
A7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
A8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency
A9. Explanation of Any Payment or Gift to Respondents
A10. Assurance of Confidentiality Provided to Respondents
A11. Justification for Sensitive Questions
A12. Estimates of Annualized Burden Hours and Costs
A12-1. Estimated Annualized Burden Hours
A12-2. Cost to Respondents
A13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers
A14. Annualized Cost to the Federal Government
A15. Explanation for Program Changes or Adjustments
A16. Plans for Tabulation and Publication and Project Time Schedule
A17. Reason(s) Display of OMB Expiration Date Is Inappropriate
A18. Exceptions to Certification for Paperwork Reduction Act
Attachments
1A. Authorizing Legislation for the National Comprehensive Cancer Control Program
1B. Public Health Service Act
2A. Federal Register Notice
2B. Response to Comments on the Federal Register Notice
3A. National Comprehensive Cancer Control Program Survey
3B. National Comprehensive Cancer Control Program Survey Screen Shots
4. National Comprehensive Cancer Control Program Survey Introductory E-Mail
5. National Comprehensive Cancer Control Program Survey Invitation E-Mail
6. National Comprehensive Cancer Control Program Survey Reminder E-Mail
7. National Comprehensive Cancer Control Program Survey Thank You E-Mail
8. National Comprehensive Cancer Control Program Focus Group Moderator Guide
9. Focus Group Invitation E-Mail
10. Focus Group Confirmation E-Mail
11. Focus Group Question-and-Answer Sheet
12. Focus Group Consent Form
Development of an Evaluation
Plan to Evaluate Grantee Attainment of
Selected Activities of
Comprehensive Cancer Priorities
Justification
A1. Circumstances Making the Collection of Information Necessary
This is a new Information Collection Request (ICR) supporting an evaluation of the National Comprehensive Cancer Control Program (NCCCP), which is funded by the Centers for Disease Control and Prevention (CDC) Division of Cancer Prevention and Control (DCPC). DCPC has worked to decrease the burden of cancer by funding state, tribe, territory, and jurisdiction grantees to develop Comprehensive Cancer Control Programs (Rochester et al., 2010). The NCCCP is authorized under sections 317(k)(2) and (e) of the Public Health Service Act (42 U.S.C. section247b [e] and [k][2]), as amended (Attachment 1A). NCCCP grantees are charged to establish NCCCP coalitions, assess the burden of cancer, and develop and implement NCCCP plans.
Comprehensive Cancer Control (CCC) is a collaborative process through which a community and its partners pool resources to reduce the burden of cancer (CDC, 2011). The concept is built on the premise that effective cancer prevention and control planning should address the cancer continuum (defined as prevention, diagnosis, treatment, survivorship, and palliative care), and include: the integration of many disciplines, major cancers, all populations, all geographic areas, and a diverse group of stakeholders who must coordinate their efforts to assess and address the cancer burden in a jurisdiction. This is accomplished by the collaborative development and implementation of a jurisdiction specific cancer control plan. CDC supports cancer control programs in 50 states, the District of Columbia, seven tribes and tribal organizations, and seven U.S. Associated Pacific Islands/territories with a goal of establishing coalitions, assessing the burden of cancer, determining priorities, and developing and implementing cancer plans.
Growth of the NCCCP depends on the extent to which resources are allocated to impactful program priorities. In 2010, CDC’s Comprehensive Cancer Control Branch (CCCB) began to strategically develop priorities that would build on the success of the NCCCP. These priorities are aligned with the cancer continuum so that the strategies will resonate with the current program and its coalitions. The priorities also capitalize on the success of the NCCCP and build on strategies and example activities that are low resource and high impact.
The six NCCCP priorities are: 1) to emphasize primary prevention of cancer; 2) support early detection and treatment activities; 3) address public health needs of cancer survivors; 4) implement policies, systems, and environmental changes to guide sustainable cancer control; 5) promote health equity as it relates to cancer control; and 6) demonstrate outcomes through evaluation.
In the summer of 2010, the six priorities were shared at the 2010 Program Director Meeting (Davis, 2010). Programs were to continue their current interventions under their CCC plans, but to emphasize the priorities as they updated their cancer plans. Implementation of these priorities was to continue through future funding opportunities.
In January 2012, CDC released a new funding announcement: request for application (RFA) number DP12-1205, Cancer Prevention and Control Program for State, Territorial and Tribal Organizations. The purpose of the funding announcement was to implement cancer prevention and control programs that would reduce morbidity, mortality, and related health disparities in accordance with Healthy People 2020 goals. The announcement created a platform for coordination and integration of long-standing cancer control activities at the state, tribal, and territorial levels. In addition to two new components to achieve coordination and integration of cancer control activities, the announcement continued to fund the NCCCP for five years. However, it is now crucial for grantee activities and work plans to align with the CDC/NCCCP priorities.
CDC proposes to evaluate the implementation of the Cancer Prevention and Control Program for State, Territorial and Tribal Organizations over the five year project period of DP12-1205 Component 2 by conducting an online survey and focus groups. These data collection efforts will allow CDC to evaluate the extent to which grantees are implementing NCCCP priorities, as well as the extent to which selected CDC capacity-building tools support implementation of these priorities. The planned information collection activities are designed to address specific evaluation questions, including: What factors facilitate implementation of the NCCCP priorities? What common barriers do grantees experience in efforts to implement the NCCCP priorities? How has CDC supported grantee efforts to implement the NCCCP priorities? What additional resources are needed to support grantees’ efforts to implement the NCCCP priorities? CDC’s general authority to collect information is provided by Section 301 of the Public Health Service Act (see 42 USC 241, Attachment 1B).
CDC plans to conduct a Web survey (Attachment 3a) of all 65 NCCCP grantee program directors from the 50 states, the District of Columbia, 7 tribes and tribal organizations, and 7 US-affiliated territories. The Pacific Island Jurisdiction of the Federated States of Micronesia (FSM) has a national program that supports comprehensive cancer control (CCC) programs within each of the 4 FSM states. Due to the diversity of activities and perspectives in the FSM, surveys will also be administered to the four state-level FSM program directors. The total estimated number of respondents for the NCCCP Survey is 69. The survey will include questions that address both evaluation focus areas: (1) NCCCP priorities and (2) CCCB capacity-building tools. Specifically, program directors will also be asked to provide information about their efforts to implement the NCCCP priorities, including implementation facilitators and barriers. The program directors will also be asked to provide information about the utilization and usefulness of the CCCB Program Evaluation Toolkit (Toolkit), a capacity-building tool developed and disseminated to NCCCP grantees in 2010 to support evaluation of CCC programs (NCCCP Priority 6). The Toolkit will be the first capacity building tool assessed as part of the evaluation. Additional tools may be assessed in future years. Program directors will also be asked to provide information about their efforts to implement the NCCCP priorities. The contractor (RTI) will, on behalf of CDC, design and administer this survey and all supporting communication (Attachments 3 to 7). The survey will be administered twice over the requested two-year clearance period. If CDC determines that findings from the initial administration of the survey and the focus group discussions should be incorporated into the second administration of the survey, CDC will submit a Change Request for OMB review and approval. The survey will be conducted electronically using Survey Monkey, a Web-based data collection system. Potential respondents will initially be contacted by a CDC staff person to alert them to the pending invitation to participate in the survey. Subsequent communication with respondents will be conducted by RTI, the contractor working on behalf of CDC, via Survey Monkey. Contact information for potential respondents will be provided to RTI by CDC. We estimate that the NCCCP evaluation data collection period will last approximately one month.
Up to four focus groups will be conducted with maximum of 10 program directors, or other CCC staff or partners, selected to participate by the program director in each focus group. Focus groups will be conducted in-person or by telephone. Focus group data collection will include a minimum amount of information in identifiable form. Potential respondents will be identified based on their responses to the NCCCP Survey. Participants will be contacted by RTI, the contractor working on behalf of CDC. CDC will provide contact information for potential respondents to RTI. The focus group moderator guide (Attachment 8) will consist of questions designed to gather in-depth information about ways in which CCCB capacity-building tools can be improved to better support implementation of the NCCCP priorities. The contractor (RTI) will conduct these focus groups and manage recruitment and supporting communication (Attachments 8 to 12) on behalf of CDC.
The National Comprehensive Cancer Control Program (NCCCP) Survey will gather information about the extent to which grantees are implementing NCCCP priorities and selected CDC capacity-building tools support implementation of these priorities including facilitators, barriers, and technical assistance received and needed in relation to these efforts. To minimize burden to respondents, the Survey will be programmed with skip patterns to route the respondent only to the most relevant questions.
An online survey, housed on Survey Monkey, will be employed for this project. The survey does not have any content directed at children less than 13 years of age. Only individuals 18 years of age or older will be provided with information to access the survey. The Survey Monkey privacy policy is posted on the Web site. The Survey Monkey privacy policy is listed on the Web site (http://www.surveymonkey.com/mp/policy/privacy-policy/). Key points from the privacy policy state that 1) data is owned by the survey creators (in this case, CDC); 2) Respondent email addresses are safeguarded; and, 3) Data is securely guarded on servers located in the United States.
A2. Purpose and Use of Information Collection
The overall purpose of the NCCCP evaluation is to 1) Assess the extent to which NCCCP grantees have adopted and implemented the NCCCP Priorities, 2) determine ways in which CDC can best support grantees in their efforts to implement the priorities, and 3) identify outcomes of grantees’ efforts to implement the priorities.
This data collection effort, consisting of an online survey and focus groups, will help CDC in several ways. The online survey will allow CDC to understand the level of adoption and attainment of outcomes related to implementing the six NCCCP priorities, and to evaluate the effectiveness of existing capacity-building tools needed to support the implementation of NCCCP priorities. Collecting data through focus groups will help CDC better understand the tools and how to revise them.
The data collected will be used to inform and enhance the level of technical assistance provided to NCCCP cooperative agreement recipients, as well as, articulate the outcomes of this national program to key stakeholders.
A3. Use of Improved Information Technology and Burden Reduction
The survey will be conducted using a web-based electronic platform. Focus group discussions will be conducted in person or by telephone. By using electronic and discussion formats for our evaluation data collection, we will reduce the burden of respondents having to use a paper format and then mail their responses back to CDC.
A4. Efforts to Identify Duplication and Use of Similar Information
There are no similar data available that meet the needs of this proposed evaluation. CDC has conducted a review of recent and ongoing survey initiatives that aim to enhance knowledge about the NCCCP and has determined that none of these efforts are gathering information that duplicates the items proposed for inclusion in the NCCCP Survey. This is the first time that data will be collected on the implementation of the six NCCCP priorities; and the first time that existing CCCB evaluation capacity building tools are being evaluated to determine their utility and usability. The proposed information collection does not duplicate any information currently being collected from program directors, staff members, or coalition leaders.
A5. Impact on Small Businesses or Other Entities
No small business will be involved in this data collection.
A6. Consequences of Collecting the Information Less Frequently
Collecting this evaluation data allows CDC to effectively develop and provide technical assistance for NCCCPs. Consequences of not collecting the information would include an inability for CDC to describe NCCCP program impact, decreased ability to disseminate information on NCCCP efforts that effectively decrease cancer burden, inability to build on the success of the NCCCP program, or inability to apply program knowledge to program improvements. In sum, not collecting the information would hinder CDC’s efforts to aid CCCPs in their program implementation.
There are no legal obstacles to reducing the burden.
A7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
There are no special circumstances relating to the guidelines of 5 CFR 1320.5 and the project fully complies.
A8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency
The 60 day Federal Register Notice was published in the Federal Register on August 8, 2012, Vol. 77, No. 153, pp. 47394-47395 (Attachment 2A). During the 60-day comment period,
compiled comments were received from two groups representing the National Association of Chronic Disease Directors, members of the steering committee of the Cancer Council and the Science, Epidemiology and Evaluation Committee. Their comments can be grouped into three broad categories: general evaluation comments, comments on web-based survey methods, and comments on focus group methods.
General evaluation comments included supportive remarks around the likelihood that CDC will receive honest feedback from grantees through the web-based survey and focus groups. Phone interviews were recommended in addition to the web-based survey and focus groups, and CDC was encouraged to build on, rather than duplicate, management information system (MIS) data collection. The survey and focus groups are designed to address gaps in information collected through the MIS (“Management Information System for Comprehensive Cancer Control Programs,” OMB No 0920-0841, exp. 2/28/013). Some MIS data (e.g., partnership composition) will be abstracted and analyzed as part of the NCCCP evaluation. Data for this evaluation will primarily be gathered through the survey and focus groups. As time and funding resources allow, a limited number key informant interviews may be conducted to expand on survey and focus group data.
Comments on web-based survey methods emphasized the importance of minimizing response burden and included a recommendation to make the survey mandatory. CDC has worked to keep the instrument as short as possible while still gathering information pertinent to the evaluation. Skip patterns are embedded in the survey to help minimize the amount of time required to complete it. Additionally, word limits have been placed on open-ended response field to encourage brief answers and minimize burden. Participation in the survey is voluntary, and grantees will not be penalized should they choose not to participate. However, Comprehensive Cancer Control Branch staff will encourage participation by announcing the survey on grantee calls and websites and by providing an overview of the survey on a grantee call.
Comments on focus group methods stressed the value of using an outside facilitator and conducting the focus groups during a grantee meeting. Suggestions for organizing focus groups—for example, by experience with the comprehensive cancer control program—were also provided. Focus groups will be conducted by a contractor without CDC staff present. CDC agrees that it would be the best option to hold focus groups in person during a time when grantees are attending a prescheduled meeting. However, due to the decreasing number of opportunities for in person grantee meetings it is possible that this opportunity will not be available. If focus groups cannot be held during a prescheduled grantee meeting, phone or web-based focus group will be conducted. CDC will consider experience with the CCC program, project home, job title and geography into account when organizing focus groups. For additional information see Attachment 2B.
A9. Explanation of Any Payment or Gift to Respondents
Respondents will not receive any payment or gifts as a result of their completing the survey or participation in focus groups.
A10. Assurance of Confidentiality Provided to Respondents
Respondents to the data collection instruments will be NCCCP state grantee program directors or designated CCC staff members or partners. RTI, the contractor working on behalf of CDC, will take steps to assure that study participants’ responses are maintained in a secure manner.
A. Privacy Act Determination
CDC’s National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP) has reviewed this information collection request and determined that the Privacy Act does not apply. Respondents will participate as representatives of funded programs, and will not provide personal information to CDC or CDC’s contractor. Although the survey will not ask for the names of any respondents, no guarantees of confidentiality will be provided to respondents. Participants will complete the survey through the online site Survey Monkey. Each respondent is asked to identify their state program, and this information will be used to identify potential participants for follow-up focus groups. We are not collecting demographic information in the survey; respondents are simply providing feedback on their program’s use of the Toolkit and implementation of the CCCB Priorities, as well as any technical assistance they have received or would like to receive in the future.
Focus
Groups
The focus groups will be comprised of program directors or their designated staff members or partners who volunteer to participate. In the typed interview notes, names of respondents will be replaced by their study identification number, and any other identifying information (e.g., references to affiliated organizations or names of colleagues/co-workers) will be deleted to prevent indirect identification.
Safeguards
RTI will have direct access to the data collected through the online survey and focus groups. RTI and CDC will safeguard the responses and will not release any identifying information. All completed surveys, as well as the electronic data files containing the survey data, will be identified only by study identification number. Neither the Internet surveys nor the electronic files of the survey data will contain names, addresses or telephone numbers of respondents. All project files will be password protected and access to files will be limited to authorized study staff. Project reports and manuscripts will contain aggregated data only; results will not be associated with any individual respondent. Any data sent to CDC will not contain individual identifiers. All data collected will be compiled into reports and manuscripts that do not contain any personal identifiers. All survey responses will be considered and reported as those of NCCCP programs, not individual participants. The survey data collection system collects and uses Internet protocol (IP) addresses for system administration and record-keeping, but IP addresses will not be provided to CDC. Additional information about Survey Monkey is available at http://www.surveymonkey.com.
All handwritten notes, typed notes, and audio recordings from focus groups will be maintained in a secure manner. Hardcopies of these materials will be stored in a locking filing cabinet. The interview notes, or any other materials produced from the focus groups, that include identifiers will be handled or viewed only by RTI staff who are directly responsible for data collection and analysis. Digital audio recordings will be stored on password-protected file servers and deleted upon study completion.
Consent
All survey respondents and focus group participants will be informed of their rights before data are collected (see Attachments 3-7 and 8-12).Consent for the internet survey is located in Attachment 3A/3B. Consent for the focus group is located in Attachment 12. This information collection does not involve research with human subjects and IRB approval is not required.
Nature of Response
Respondents will be informed through the instructions and consent statements of the purpose of the study, what their participation will involve, and the steps taken to maintain their responses in a secure manner. They will be reminded that their participation is voluntary and that they may choose not to answer any question or may withdraw from the study at any time without penalty to themselves or their NCCCP-funded program.
A11. Justification for Sensitive Questions
Participants will be asked to discuss implementation barriers and technical assistance needs, which could be considered sensitive in nature. This information will help CDC to understand how they might provide improved technical assistance to programs in the future.
A12. Estimate of Annualized Burden Hours and Costs
An estimated 69 respondents will complete the National Comprehensive Cancer Control Program Survey (Attachments 3A and 3B). The estimated burden per response is 30 minutes. The survey will be administered twice over the two-year period of this clearance request.
A subset of approximately 40 survey respondents will be invited to participate in focus group discussions, which will be conducted once over the two-year period of this clearance request. The annualized number of respondents for focus groups is 20.The estimated burden response for the focus group is 1.5 hours (see Attachment 8, NCCCP Focus Group Moderator Guide). The total estimated annualized burden is 65 hours, as summarized in Table A.12-1 below.
Table A.12-1. Estimated Annualized Burden Hours
Type of Respondent |
Form Name |
Number of Respondents |
Number of Responses per Respondent |
Average Burden per Response (in Hours) |
Total Burden (in Hours) |
NCCCP State Grantee Program Director
|
National Comprehensive Cancer Control Program Survey |
69 |
1 |
0.5 |
35 |
NCCCP State Grantee Program Director or Designated CCC Staff Member
|
National Comprehensive Cancer Control Program Focus Group Moderator Guide |
20 |
1 |
1.5 |
30 |
Total |
|
65 |
|||
There are no costs to respondents other than their time.
Table A.12-2 presents the calculations for cost of annualized burden hours. The participants’ wages may vary significantly depending on respondent employment status and the state, tribe, or territory in which they reside. Therefore, the average hourly wage data below are based on the average estimated hourly wage.
Table A.12-2. Estimated Annualized Cost to Respondents
Type of Respondent |
Form Name |
Number of Respondents |
Total Burden (in Hours) |
Average Hourly Wage* |
Total Cost |
NCCCP State Grantee Program Director
|
National Comprehensive Cancer Control Program Survey |
69 |
35 |
$25 |
$875 |
NCCCP Grantee Program Director or Designated CCC Staff Member |
National Comprehensive Cancer Control Program Focus Group Moderator Guide |
20 |
30 |
$25 |
$750 |
|
|
|
|
|
$1,625 |
A13. Estimate of Other Total Annual Cost Burden to Respondents or Record Keepers
There are no other costs to respondents.
A14. Annualized Cost to the Federal Government
Government personnel – Governmental costs for this project include personnel costs for federal staff involved in planning and designing the CCC evaluation, data collection instruments and OMB materials, collecting and analyzing the data, and reporting, which includes approximately 15% of one GS-14 lead public health advisor assuming $100,000 annual salary, 10% of one GS-13 health scientist, and 10% of two GS-13 public health advisors assuming $85,000 annual salary.
Contracted data collection –The project design and data collection is being conducted under a contract with CDC’s data collection contractor, RTI. The contract for implementation of the evaluation efforts described here totals $200,000 and includes costs for planning, conducting, and analyzing data from the survey and focus groups. The entirety of this amount is dedicated to the evaluation implementation and reporting.
Table A.14-1. Estimated Annualized Cost to the Federal Government |
|
Labor: |
|
15% of one GS-14 lead public health advisor time for project planning, management, OMB review, analysis of findings, and report writing |
$15,000 |
10% of one GS-13 health scientist time for project planning, management, OMB review, analysis of findings, and report writing |
$8,500 |
10% of three GS-13 public health advisors time for project planning, management, OMB review, analysis of findings, and report writing |
$25,500 |
Contractor |
$200,000 |
Total estimated cost |
$249,000 |
A15. Explanation for Program Changes or Adjustments
This is a new data collection effort.
A16. Plans for Tabulation and Publication and Project Time Schedule
A.16-1 Survey Time Schedule |
|
Activity |
Time Schedule |
Distribute invitation |
1 month after OMB approval |
Collect survey data |
2-3 months after OMB approval |
Collect focus group data |
4-8 months after OMB approval |
Complete data analyses |
9-12 months after OMB approval |
Report on survey and focus group results |
12-15 months after OMB approval |
Disseminate results through publications |
12-18 months after OMB approval |
A17. Reason(s) Display of OMB Expiration Date Is Inappropriate
An exemption to displaying the OMB expiration date is not being requested.
A18. Exceptions to Certification for Paperwork Reduction Act Submissions
These data will be collected in a manner consistent with the certification statement identified in Item 19 “Certification for Paperwork Reduction Act Submissions” of OMB Form 83‑I. No certification exemption is being sought.
References
Centers for Disease Control and Prevention (2011). CDC What is Comprehensive Cancer Control - NCCCP. Retrieved June 21, 2012 from http://www.cdc.gov/cancer/ncccp/what_is_cccp.htm.
Davis, L., Solera, M.K., and Seeff, L. (2010, November) Comprehensive Cancer Control Priorities. Webinar provided by the Comprehensive Cancer Control Branch of the Centers for Disease Control and Prevention.
Rochester, P.W., Townsend, J.S., Given,L., Krebill, H., Balderrama, S. and Vinson, C. (2010). Comprehensive cancer control: progress and accomplishments. Cancer Causes and Control Volume 21 (12) 1967-77.
| File Type | application/msword |
| File Modified | 2013-03-07 |
| File Created | 2013-01-14 |