Communities Putting Prevention to Work:
CPPW Resource Center Feedback Assessment
OSTLTS Generic Information Collection Request
OMB No. 0920-0879
Supporting Statement – Section A
Submitted: July 24, 2012
Program Official/Project Officer:
Nadine Doyle, MPA
Public Health Analyst
Centers
for Disease Control and Prevention
National Center for Chronic
Disease Prevention and Health Promotion
Division
of Community Health
4770 Buford Hwy., MS K-56, Atlanta, GA
30341
Phone: 770-488-5190
Email: [email protected]
Section A – Justification
Circumstances Making the Collection of Information Necessary
Background
The Centers for Disease Control and Prevention (CDC) is requesting new clearance to administer the Communities Putting Prevention to Work (CPPW) Resource Center Feedback Form using the Generic Information Collection mechanism of the OSTLTS Survey Center (OSC) – OMB No. 0920-0879, under the authority of Section 301 of the Public Health Service Act (42 U.S.C. 241).
The target audience for this assessment includes local, State, District of Columbia, and Tribal health department staff professionals who manage CPPW awards funded by CDC and acting in their official capacities. This target audience aligns with requirements of the OSC.
Established in May 2010, the purpose of the CPPW Resource Center is to support CPPW awardees (State, Local, tribal, and District Health Departments) in reaching their Community Action Plan objectives by providing technical assistance consultation, training, peer learning opportunities, and online tools and materials. The Resource Center is managed through a contract with ICF International. The CPPW Resource Center consists of 28 organizations including 10 HHS national organizations, 13 subject matter expert organizations (under subcontract to ICF International), and 5 CDC-funded peer mentoring communities. These organizations serve as an extension of CDC, which offers an array of technical resources including expertise in media, communication, health equity, tobacco, nutrition, physical activity, obesity, management of community-based initiatives, and program implementation of population, systems, and environmental change.
Group training events support multiple communities that share a common need. Training events may occur in-person or via Webinars. Attendance by all communities is required at some training events, such as the Action Institutes and the annual grantee meetings. Other trainings are designed for a subgroup of communities to address needs that may only be relevant to communities pursuing particular strategies and objectives. Besides in-person events, the CPPW Resource Center offers regular Webinars through the CPPW Webinar Series. This series features Webinars focused on the CPPW strategic focus areas that are facilitated by expert organizations and that highlight community experiences.
CPPW communities have unique needs and challenges that cannot be fully addressed by group trainings. Therefore, the CPPW Resource Center also allows communities to access a national network of experts for individualized consultation. The type and intensity of consultation are tailored to the specific needs of each community. Consultations may range from brief email exchanges in response to requests for basic information to site visits or trainings provided by one or more experts, pending availability.
To capitalize on expertise that exists within the CPPW communities, the CPPW Resource Center offers two efforts to facilitate peer exchanges between communities: Peer Teams are small teams of CPPW awardee program staff, organized by content area and community size (e.g., “Physical Activity/Rural”). These teams meet by phone monthly to build relationships with fellow peers who are engaged in similar work (and face similar challenges), share their experiences and learn from each other. Discussion topics are driven by the members of each Peer Team. Peer Topic Calls are one-time small group discussions among 10-15 participants about a particular topic that has emerged as a priority need area for multiple sites. Calls are open to all CPPW program staff members (up to 15 participants per call).
Relevant tools and resources for implementing CPPW activities are currently available at www.cdc.gov/CommunitiesPuttingPreventiontoWork/resources (see Attachment A: CPPW Online Resource Center Screen Shots). Efforts are underway to update and expand the online resources which will remain available after CPPW funding has ended. Resources span the entire spectrum of strategic focus areas at various levels. More detail about the CPPW Resource Center can be found in Attachment B - CPPW Resource Center Guide.
As of March 2012, 7 in-person training events were designed and implemented, including a grantee kick-off, three regional Action Institutes, two annual program meetings, and one tobacco peer-to-peer workshop, attended by more than 1,500 attendees in total. Sixteen webinars have been provided, reaching 1,928 participants. Peer Teams were comprised of 63 active peer team members; 34 peer team calls were facilitated and 12 peer topic calls were facilitated. For the Online Resource Center, approximately 30 new products are currently in development. In total, over 28,000 hours of technical assistance have been provided to CPPW awardees.
The purpose of the CPPW Resource Center Feedback Assessment is to collect feedback from CPPW program managers about their perceptions of the quality and usefulness of services provided by the CPPW Resource Center. This is an ideal time to collect feedback on Resource Center services, now that the majority of technical assistance (TA) services have been provided, but before the majority of CPPW programs have officially closed out. The information collected from the CPPW Resource Center Feedback Form will be used for continuous quality improvement efforts to inform decisions about transferring and expanding the CPPW resource center to become the Division of Community Health resource center to support community level practitioners including those receiving Prevention and Public Health Funding from CDC.
Privacy Impact Assessment
Overview of the Data Collection System
The data collection system consists of a single web-based instrument (see Attachment C – Web Instrument (word version) and Attachment D – Web Instrument (online version)). The instrument will be administered via Survey Monkey.
No sensitive information is being collected by the instrument. The proposed data collection will have little or no effect on respondent privacy. Respondents are participating in their official capacity as health officials in state, District, or local departments of health. No personal identifying information will be collected. The data collection form will be administered by email with a link to the online form—as a result, we will not be able to trace individual responses to a computer IP address or email address. The only identifiable information which will be available through the feedback form responses will be a voluntary response to Questions 30 and 31. These optional questions request the community or State name, as well as the type of CPPW award (Obesity, Tobacco, or both). All responses will be stored in a secure database accessible only by CPPW CDC and consultant team members. Data will be analyzed and reported in aggregate form to CDC, without any personally identifiable information.
Items of Information to be Collected
The instrument consists of 32 items, some with multiple parts, of various types including dichotomous (yes/no), interval (rating scales), categorical (multiple choice), and open ended questions. Skip patterns are in place to minimize respondent burden and only require participants to read and answer questions applicable to them. Because of these skip patterns the exact number of questions can differ between participants. An effort was made to limit questions requiring narrative responses from respondents. The instrument will collect information about the following:
perceptions on technical assistance received from ICF sub-contracted TA Providers (questions 1-5);
perceptions on technical assistance received from HHS National Organizations (questions 6-10);
perceptions on technical assistance received from CDC-managed CPPW Mentoring Communities (questions 11-15);
perceptions on CPPW Peer Activities (questions 16-28);
perceptions on various CPPW TA support modalities (question 29)
identification of CPPW award type (questions 30 – 31)
The source of information will be respondent perceptions of CPPW Technical Assistance resources—their awareness and use of the resources, and feedback for improvements. Respondents will not be provided with unique links to track their individual responses; therefore respondents will need to complete the online instrument in one session. Respondents will be sent a link to access the web-based instrument, along with instructions for completion and an estimated amount of time for completion.
Identification of Website(s) and Website Content Directed at Children Under 13 Years of Age
The data collection system involves using a web-based instrument. Respondents will be sent a link directing them to the online instrument only (i.e., not a website). No website content will be directed at children.
Purpose and Use of the Information Collection
The purpose of the evaluation is to determine knowledge and utilization of, and feedback around CPPW program implementation TA resources. The assessment will be conducted with state, tribal, district, and local health department staff awardees. This evaluation will focus on awardees’ awareness, use, and incorporation of CPPW technical assistance resources and guidance at health departments across the country. This information does not currently exist.
Respondents will be asked to provide feedback on improvements that can be made to technical assistance resources. Information derived from the instrument will be used to judge the value of CPPW TA Resources, provide accountability to CDC, and inform decisions about future TA resources and networks. Without collecting this information, it would be difficult to judge the awareness, utilization, and value of CPPW TA resources.
The information collected from the CPPW Resource Center Feedback Form will be used to inform reports and articles about the quality and effectiveness of support offered by the CPPW Resource Center and to inform decisions about current and future CDC efforts to support funded public health agencies. The instrument will compare the value of elements of the CPPW Resource Center in contributing to community outcomes and will examine communities’ perceptions of technical assistance and peer activities in particular.
Privacy Impact Assessment
The information is being collected to determine the quality and usefulness of services provided by the CPPW Resource Center to CPPW awardees. The instrument will assess health professionals in state, tribal, district, and local health departments, about their knowledge and use of CPPW TA resources and their recommendations for improvement. The data and information collected will be used to assess the value of CPPW TA resources and improve future iterations of such TA resources. Since these products are targeted for use by CPPW awardees in state, tribal, district, and local health departments, it is critical to get their input as to whether they know about, use and find these resources helpful to their work.
No sensitive or personal identifying information is being collected. The proposed data collection will have little or no effect on respondent privacy. Respondents are participating in their official capacity as health officials in state, tribal, district, or local departments of health. No personal identifying information will be collected. The data collection form will be administered by email with a link to the online form—as a result, we will not be able to trace individual responses to a computer IP address or email address. The only identifiable information which will be available through the feedback form responses will be a voluntary response to Questions 30 and 31. These optional questions request the community or State name, as well as the type of CPPW award (Obesity, Tobacco, or both).
Considerations Given to Information Technology
Data will be collected via a web-based questionnaire allowing respondents to complete and submit their responses electronically. The instrument will be administered using Survey MonkeyTM software. Survey MonkeyTM is a highly customizable product with sophisticated conditional routing and data validation capabilities. It is fully compliant with Section 508 of the Rehabilitation Act, so it meets Federal Web Accessibility Standards set to ensure that electronic and information technology utilized by Federal agencies are accessible to people with disabilities. Respondents will be asked to complete the instrument via a web-based link; all responses will be stored in a secure database accessible only by CDC and consultant project team members.
Careful consideration was given to questionnaire design, length, and layout to minimize respondent burden. The instrument was designed to collect the minimum information necessary for the purposes of this project (i.e., limited to 32 questions). An online version of the web instrument (in screen shot form) can be found in Attachment D.
Duplication of Information
A new single wave of data collection is proposed. The information being collected is specific to CPPW TA Resources and has never before been previously evaluated. There is currently no information available via web or existing literature that can substitute for the desired responses.
Reducing the Burden on Small Entities
No small businesses will be involved in this data collection.
Consequences of Not Conducting Collection
If data are not collected:
there will be no systematically obtained information to support judgments about the extent to which CPPW awardees are aware of, accessing, and utilizing CPPW TA resources for program planning and implementation
CDC will not know:
the utility, effectiveness and necessary changes for future investments
whether specific TA resources are reaching their intended users,
whether these resources are relevant and well-received, and
how these TA resources can be improved.
Ineffective implementation of TA resources by awardees would inhibit the effectiveness, efficiency, and sustainability of interventions that can reduce obesity and tobacco use. Thus, it is essential for CDC to understand the level of awareness, use, and implementation of these resources among their CPPW awardee audience. Understanding these components will allow CDC to identify future actions to improve awareness, adoption and implementation of their TA resources.
Special Circumstances
There are no special circumstances with this information collection package. This request fully complies with the regulation 5 CFR 1320.5 and will be voluntary.
Consultation with Persons Outside the Agency
This data collection is proposed under the Generic Information Collection mechanism of the OSTLTS Survey Center (OSC) – OMB No. 0920-0879. A 60-day Federal Register Notice was published in the Federal Register on October 22, 2010, Vol. 75, No. 204; pp. 65353-54. Two comments were received from the Association of State and Territorial Health Officials (ASTHO), and the National Association of County and City Health Officials (NACCHO).
CDC partners with professional STLT organizations, such as the Association of State and Territorial Health Officials (ASTHO), the National Association of County and City Health Officials (NACCHO), and the National Association of Local Boards of Health (NALBOH) along with the National Center for Health Statistics (NCHS) to ensure that the collection requests under individual ICs are not in conflict with collections they have or will have in the field within the same timeframe.
Payment or Gift
CDC will not provide payments or gifts to respondents.
Confidentiality
The Privacy Act does not apply to this data collection. Employees of state, tribal, district, and local public health agencies will be voluntarily responding from their official roles (See Attachment E – Initial Email with Instructions). This data collection is not research involving human subjects and IRB review is therefore not required. Data will be stored in a secure database accessible only by ICF International CPPW consulting staff. Data will not be individually identifiable and will be analyzed and reported in aggregate only.
Sensitive Nature
No information will be collected that are of personal or sensitive nature.
Burden of Information Collection
The estimate for burden hours is based on a pilot test of the instrument by nine public health professionals. In the pilot test, the average time to complete the instrument including time for reviewing instructions and completing the instrument was approximately 19 minutes. Depending on the responses selected, some questions may be skipped or follow-up questions may be asked of participants. Therefore, it may take slightly more or less time to complete the instrument, but not by a great amount. Based on these results, the estimated time range for actual respondents to complete the instrument is 15-20 minutes. For the purposes of estimating burden hours, the upper limit of this range (i.e., 20 minutes) is used. Since there will only be one wave of data collection, only one block of 20 minutes or less is needed from each participant.
Estimates for the average hourly wage for respondents are based on the Department of Labor (DOL) National Compensation Survey estimate for management occupations – medical and health services managers in state government (http://www.bls.gov/ncs/ocs/sp/nctb1480.pdf).
Based on DOL data, an average hourly wage of $33.61 is estimated for all 73 respondents. These respondents include CPPW program managers at state, district, local, and tribal health. There are 1-2 program managers, depending on the award (Obesity and Tobacco) for each of the 73 state, local, or tribal health department awardees. Table A-1 shows estimated burden and cost information.
Table A-1: Estimated Annualized Burden Hours and Costs to Respondents–CPPW Resource Center Feedback Form
Type of Respondent |
No. of Respondents |
No. of Responses per Respondent |
Average Burden per Response (in hours) |
Total Burden Hours |
Hourly Wage Rate |
Total Respondent Costs |
CPPW Awardee Program Managers |
73 |
1 |
20/60 |
24 |
$33.89 |
$813 |
Costs to Respondents
There will be no direct costs to the respondents other than their time to participate in the assessment.
Cost to Federal Government
There are no equipment or overhead costs. Contractors, however, are being used to support development of instrument, data collection, and analysis for this assessment. The only cost to the federal government would be the salary of CDC staff, consultants, and other project team members supporting the data collection activities and associated tasks.
The CDC lead staff for this project is a lead Public Health Analyst for Program Implementation Technical Assistance. The consultant staff lead is an evaluator by training. The CDC lead staff member will provide oversight for development of the instrument and analysis plan, OMB documents, and final products. The majority of work on this project will be carried out by consultants from ICF International, including primary development of the instrument, instrument administration, data review and analysis, and reporting of findings. Lead CDC CPPW staff will also contribute to the development of the instrument, OMB documents and analysis.
Hourly rates of $37.90 for GS-12 (step 4) and $108.32 for consultant average hourly rate were used to estimate staff costs. The estimated cost to the federal government is $21,013.60. There is no fee for using Survey Monkey or data analysis software. Table A-2 describes how this cost estimate was calculated.
Table A-2: Estimated Annualized Cost to the Federal Government
Staff (FTE) |
Average Hours per Collection |
Average Hourly Rate |
Average Cost |
CDC Lead Public Health Analyst (GS 12, Step 4) Consultation with team members on OMB package preparation, instrument development, data analysis, quality control and report preparation consultation. |
40 hours |
$37.90 |
$1,516.00 |
ICF International Contractors (average loaded rate) Instrument development, pilot testing, OMB package preparation, Web-based instrument programming, data collection, data coding and entry, quality control, data analysis, report preparation |
180 hours |
$108.32 |
$19,498 |
Estimated Total Cost of Information Collection |
$21,014 |
||
Reason for Changes
This is a new data collection.
Tabulation of Results, Schedule, and Analysis Plan
Project Time Schedule
Design instrument questionnaire (COMPLETE)
Develop instrument protocol, instructions, and analysis plan (COMPLETE)
Pilot test instrument questionnaire (COMPLETE)
Prepare OMB package (COMPLETE)
Submit OMB package (COMPLETE)
OMB approval (TBD)
Administer instrument (Instrument open 2 weeks)
Collect, code, enter, quality control, and analyze data (2 weeks)
Prepare report (2 weeks)
Disseminate results/publication of findings (Date TBD)
Tabulation of Results
Following the administration of the instrument, ICF staff will produce an overall tabulation of data and analysis that will include the results for each instrument question broken down by the appropriate segments and demographics.
ICF TA Provider Feedback (questions 1-5);
HHS National Organization TA Provider Feedback (questions 6-10);
Mentoring Community/State TA Provider Feedback (questions 11-15);
Peer Activity Feedback (questions 16-28);
Comparing TA Support Modalities (question 29)
Community and Award Type Identification (questions 30 – 31)
Table A-3 depicts how instrument items align with the project’s evaluation questions (EQ).
Table A-3: Item Alignment with Project Evaluation Questions (EQ).
Item Categories |
Dem |
EQ1 |
EQ2 |
EQ3 |
EQ4 |
EQ5 |
EQ6 |
EQ7 |
EQ8 |
EQ9 |
ICF TA Provider Feedback |
|
X |
X |
X |
X |
|
|
|
|
|
HHS National Organization TA Provider Feedback |
|
X |
X |
X |
X |
|
|
|
|
|
Mentoring Community/State TA Provider Feedback |
|
X |
X |
X |
X |
|
|
|
|
|
Peer Activity Feedback |
|
|
|
|
|
X |
X |
X |
X |
|
Comparing TA Support Modalities |
|
|
X |
|
|
|
|
X |
|
X |
Community and Award Type Identification |
X |
|
|
|
|
|
|
|
|
|
x= item addresses this evaluation question
For ease of reference, Figure A-4 provides the project’s evaluation questions.
Figure A-4: Evaluation Questions
Evaluation Questions (EQ) |
EQ1: How is each individual TA provider organization perceived by CPPW communities with regard to responsiveness, content expertise, TA quality, usefulness, and overall satisfaction? |
EQ2: How has TA provided through the CPPW Resource Center contributed to CPPW communities achieving their CAP objectives? |
EQ3: What challenges have CPPW communities experienced when receiving technical assistance from the Resource Center? |
EQ4: What differences do CPPW communities perceive between ICF TA providers, HHS national organizations, and CPPW mentoring communities with regard to TA quality and usefulness? |
EQ5: To what extent did CPPW program managers participate in peer teams and peer topic calls? |
EQ6: How satisfied are CPPW program managers with the implementation of peer teams and peer topic calls? |
EQ7: To what extent did peer teams and peer topic calls contribute to implementation benefits for CPPW communities? |
EQ8: What other reflections do CPPW communities have about peer teams and/or peer topic call? |
EQ9: To what extent did different modes of support from the CPPW Resource Center contribute differentially to community achievements? |
Analysis Plan
Data analyses will consist of descriptive statistical techniques including response frequencies, measures of central tendency, measures of distribution, cross-tabulations, and themed analysis of open ended items. The sample size is not expected to be large enough to conduct statistical group comparisons or predictive analyses. Data from this instrument could potentially be combined with performance data to examine associations between support received and achievement of CAP objectives at the community level. A detailed description of planned statistical procedures are presented in Table A-5.
Table A-5: Statistical Analysis Techniques to be Performed
Topic and Instrument Questions |
Statistical Analysis Technique |
Purpose of Analysis |
|
|
|
|
|
|
|
|
Display of OMB Approval Date
We are requesting no exemption.
Exceptions to Certification for Paperwork Reduction Act Submissions
There are no exceptions to the certification. These activities comply with the requirements in 5 CFR 1320.9.
LIST OF ATTACHMENTS – Section A
Note: Attachments are included as separate files as instructed.
CPPW Online Resource Center Screen Shots
CPPW Resource Center Guide
Web Instrument (word version)
Web Instrument (online version)
Initial Email with Instructions
Follow-Up Email
Final Reminder Email
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | ICFI |
| File Modified | 0000-00-00 |
| File Created | 2021-01-31 |