Supporting Statement A -11-17-2011

Evaluation of Core Violence and Injury Prevention Program

(Core VIPP)

Supporting Statement A

17 November, 2011

Department of Health and Human Services

Centers for Disease Control and Prevention

National Center for Injury Prevention and Control

Division of Injury Response

Project Officers:

Howard Kress

Christopher Jones

4770 Buford Highway, NE, Mailstop F-64

Atlanta, GA 30341

Telephone: 770-488-1285

Fax: 770-488-3551

Electronic Mail: Howard Kress: [email protected];

Christopher Jones: [email protected]

List of Attachments

• Attachment A: TITLE 42--THE PUBLIC HEALTH AND WELFARE (authorizing legislation)

• Attachment B: Published 60 Day Federal Register Notice

• Attachment C: State of the States (SOTS) Survey

C-1 PDF Screenshots

C-2 Word Document

• Attachment D: Telephone interviews

• Attachment E: 2011 SOTS Financial Module

E-1 PDF Screenshots

E-2 Word Document

A. Justification

1. Circumstances Making the Collection of Information Necessary

Background

The overall purpose of this NEW three-year evaluation information data request is to support an evaluation of the Centers for Disease Control and Prevention (CDC)/ National Center for Injury Prevention and Control (NCIPC) Core Violence and Injury Prevention Program (Core VIPP). NCIPC supports state health department (SHD) partners to move toward the right hand side of the public health model through implementation of evidence based interventions and strategies. One tool NCIPC will use to accomplish this is CORE VIPP. This five–year program which begins funding on August 1, 2011 will fund 28 SHDs to build effective delivery systems for dissemination, implementation and evaluation of evidence based/best practice programs and policies, with a focus on:

• Enhancement of infrastructure, including injury surveillance and development of Injury Community Planning Groups (ICPG)

• Policy, communications, and evaluation for injury and violence reduction

• Establishment of Regional Network Leaders (RNL)

• Implementation of evidence based practices (programs and policies)

• Establishment of long term state injury priorities with documented health outcomes

The establishment of the Core program is due to the burden of injury (both unintentional and violence-related injuries) places on the United States. Taken together, unintentional and intentional injuries are the leading cause of death for the first four decades of life, regardless of gender, race, or socioeconomic status. More than 179,000 individuals in the U.S. die each year as a result of unintentional injuries and violence. More than 29 million others suffer non-fatal injuries and over one-third of all emergency department (ED) visits each year are due to injuries. In 2000, injuries and violence ultimately cost the United States $406 billion, with over $80 billion in medical costs and the remainder lost in productivity.¹ Most events that result in injury and/or death from injury could be prevented if evidence-based public health strategies, practices, and policies were used throughout the nation.

The primary goal of Core VIPP is to assist SHDs to build and/or maintain effective delivery systems for dissemination, implementation, and evaluation of best practice programs and policies. This includes support for general capacity building of SHDs and their local partners, as well as strategy specific capacity building for the implementation of direct bets practice interventions. In addition, this program supports SHDs in their efforts towards integration of the strategic alignment of resources for meaningful change.

The purpose of this ICR is to permit CDC to evaluate the Core VIPP program for the benefit of the Core VIPP grantees. This ICR has two overall goals: (1) to assess state injury and violence prevention plans for completeness, measurability, and effectiveness; and (2) evaluate the effectiveness of the Core VIPP cooperative agreement. Through the evaluation of the Core VIPP, CDC plans to improve state health department program and policy activities. The ultimate goal of the Core VIPP is to assist State Health Departments (SHDs) to build and/or maintain effective delivery systems for dissemination, implementation and evaluation of best practice programs and policies. This includes support for general capacity building of SHDs and their local partners, as well as strategy specific capacity building for the implementation of direct best practice interventions (older adult falls and child injury). The CDC evaluation team is committed to using the CDC Framework for Program Evaluation in Public Health (http://www.cdc.gov/mmwr/preview/mmwrhtml/rr4811a1.htm) to guide the design and implementation of this project. This program addresses the “Healthy People 2020” focus area of Injury and Violence Prevention.

This data collection is authorized under the Section 301 of the Public Health Service Act (42 U.S.C 241).

Authority for CDC’s National Center for Injury Prevention and Control to collect this data is granted by Section 301 of the Public Health Service Act (42 U.S.C. 241) (Attachment A). This act gives federal health agencies, such as CDC, broad authority to collect data and do other public health activities, including this type of study.

Privacy Impact Assessment

The respondents to this data collection are the 28 Core VIPP funded states and the remaining 22 non-funded states. No individually identifiable information is being collected.

No children under 13 years of age are included in this information collection request.

Overview of Data Collection System

Data collection will use two techniques: annual, web-based surveys (Attachments C and E), and follow-up telephone interviews (Attachment D). The primary respondents will be the SHD Injury Program directors and associated staff. Data will be collected by the CDC and the Safe States Alliance (cooperative agreement CDC RFA CE11-1106 award date: September 30, 2011). All data will be used to determine the amount of progress a state has made towards meeting its injury and violence prevention (IVP) objectives. Overall program effectiveness will be determined by the ability of states to meet and/or exceed their objectives. Data will also be used to indicate areas for programmatic improvement. Data will be kept through the end of the Core VIPP funding period (July 31, 2016) plus two additional years for analysis purposes. Thus, all data will be discarded in July, 2019. Data will be initially housed with the Safe States Alliance and shared with the CDC; however, at the end of the cooperative agreement all data will be transferred to CDC by September 30, 2016.

Items of Information to be Collected

Data will consist of questions regarding program evaluation, state health department (SHD) injury program infrastructure, injury program strategies and partners, policy strategies, injury surveillance, quality of surveillance, and regional network leaders. Specific questions can be seen in Attachments C, D, & E.

No individually identifiable information is being collected.

Identification of Website(s) and Website Content Directed at Children Under 13 Years of Age

The proposed information collection will use a web-based survey to collect program progress and performance data annually. This website will be housed on the Safe States web server. Each state will have a password protected area on the server where they can fill out an annual survey or programmatic accomplishments and progress. Additional web sites include:

• CDC’s Evaluation Framework (http://www.cdc.gov/mmwr/preview/mmwrhtml/rr4811a1.htm)

• CDC’s Evaluation Working group (http://www.cdc.gov/eval/resources.htm)

• American Evaluation Association (http://www.eval.org/)

• W.K. Kellog Foundation Evaluation Handbook (http://www.wkkf.org/knowledge-center/resources/2010/W-K-Kellogg-Foundation-Evaluation-Handbook.aspx)

2. Purpose and Use of Information Collection

The information collected under the proposed data collection will be used to:

1. To assess state injury and violence prevention plans for completeness, measurability, and effectiveness;

2. Develop a tool to support state health department planning and evaluation efforts;

3. Evaluate the effectiveness of the Core VIPP cooperative agreement, including contextual factors.

Through the evaluation of the Core VIPP, CDC plans to improve state health department program and policy activities. The proximate goal of the Core VIPP is to assist State Health Departments (SHDs) to build and/or maintain effective delivery systems for dissemination, implementation and evaluation of best practice programs and policies. This includes support for general capacity building of SHDs and their local partners, as well as strategy specific capacity building for the implementation of direct best practice interventions. Through the evaluation of this capacity building effort, CDC seeks to quantify progress towards reductions in injury related morbidity, mortality, and disparities. Through the collection of evaluation data, CDC can determine if those goals have been met over the next five years of Core VIPP funding (FY2012 –FY 2016).

The practical utility of this evaluation to the federal government is to assess the merit, worth, and significance of the Core program. Results of the evaluation will inform the states and the CDC as to which programs and policy efforts are effective in reducing injury related morbidity and mortality, and associated disparities. This information will be used to help guide the states in implementation of programs and policies. Not collecting this data could result in inappropriate programs and policies being implemented in states, resulting in a loss of tax payer resources that could have been used more effectively. Worse, people will continue to be injured and killed due to injury and violence.

Results of this program evaluation are not generalizable in the statistical sense. Instead, evaluation results can be used to modify existing practices when ineffectiveness is discovered. Results can also be used to help other states in implementation of program or policy; however, each state has its own set of contextual variables that significantly contribute to the success or failure of an intervention (e.g. political climate, state funding, demographics of the population, and historical factors). Thus, the evaluation will produce information about program success and information on program improvement at the state level. CDC will also use this information to improve the Core program, specifically examining the effectiveness of funding levels, technical assistance provided, and training.

Privacy Impact Assessment Information

1. Why Information is being Collected

The purpose of this information collection is to determine the merit, worth, and significance of the Core VIPP. This will be accomplished by collecting state-level data on a variety of topics (see Attachments C-E), including injury surveillance, program implementation, policy efforts, partnerships and coalitions for injury and violence prevention, state implemented program evaluation, and state injury program infrastructure development. The unfunded state VIPP programs will be asked to complete an annual voluntary survey at the same time as the funded states to provide a comparison group for the Core VIPP funded states. No IIF is being collected in this evaluation. The impact of this data collection on states is low since states are already required to collect much this information for grant management purposes.

2. Intended Use of the Information

The information from this evaluation will provide empirical and qualitative evidence for the effectiveness of the Core program. The data produced will be used for future funding decisions, such as the number of states to fund, the level of funding, and the type of technical assistance provided to the funded and unfunded states.

A.3. Use of Information Technology and Burden Reduction

The evaluation will use two methods for data collection. The first method is an existing web-based questionnaire, the Safe States Alliance State of the States survey (SOTS) and its associated modules. This non-federally funded survey has been deployed biennially since 2005. For this evaluation, the CDC seeks OMB clearance to use the SOTS as an assessment tool. The SOTS is normally a web-based survey via Survey Monkey, but Safe State Alliance has made hard copy submission an option, and will continue under this ICR. Attachment C _State of the States Survey is the 2009 survey which forms the basis for this 3-year ICR, while Attachment E_2011 SOTS Finance Module is a financial module which will be introduced in for the 2012 data collection.

The second method is a follow-up telephone interview (Attachment D_Telephone interviews). The SOTS produces significant information on goal obtainment, but does not provide the nuanced contextual information that is vital for understanding program results². To gather this critical context information, 90 minute phone interviews will be conducted with the Core-funded state injury program directors and associated staff. Topics will be follow-up questions to responses on the SOTS as well as more open-ended questions about program operations, barriers, facilitators and opportunities.

A.4. Efforts to Identify Duplication and Use of Similar Information

Since CDC is the only federal agency providing funding for state injury and violence prevention infrastructure building, there has been no previous data collection on the effectiveness of the Core VIPP. A previous evaluation was conducted by CDC on the breadth of the Core funding, but this evaluation only looked at CDC functions and did not collect data from the participating states.

A.5. Impact on Small Businesses or Other Small Entities

Small businesses are not a part of the respondent universe.

A.6. Consequences of Collecting the Information Less Frequently

The proposed data collection will provide both the states and the CDC with critical data on the effectiveness of state injury and violence prevention efforts. This data is needed to both enhance current state-level programs as well as CDC’s efforts to support those programs. Annual data collection is the appropriate frequency of collection due to 1) federal grantee reporting requirements, and 2) to provide states a way of making corrections to program efforts. To not conduct this evaluation would result in CDC failing to account for the effectiveness of federal dollars spent on a public project. More significantly it would mean states would not know the efficacy of their programs.

A.7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This request fully complies with the regulation 5 CFR 1320.5.

A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency

A.8.A. A 60-day notice to solicit public comments was published in the Federal Register - volume 76, No.136, pages 41798-41799 - on July 15, 2011. Attachment B contains a copy of the notice. There were no comments in response to the Federal Register Notice.

A.8.B. During the conceptualization phase of the evaluation design several members of the CDC evaluation community were contacted, as well as subject matter experts on intentional and unintentional violence. These conversations discussed the potential approaches to the evaluation, types of data to be collected, and method for data collection. From these conversations it was determined that a mixed-methods (qualitative and quantitative) approach was both practical and necessary given the nature of the Core program. The following CDC evaluation and injury and violence prevention specialists were consulted during the development phase of the evaluation (November 2010-May 2011):

• Richard Puddy, behavioral scientist ([email protected]), phone 770-488-1369

• Sue Lin Yee, health scientist ([email protected]), phone 770-488-3941

• Tom Chapel, health scientist ([email protected]), phone 404-639-2116

• Pamela Cox, evaluation scientist ([email protected]), phone 770-488-1206

• Maureen Wilce, health scientist ([email protected]), phone 770-488-3721

• Margaret Kaniewski, public health advisor ([email protected]), phone 770-488-1371

• Diane Hall, behavioral scientist ([email protected]), phone 770-488-1734

For this study, the following CDC staff has been actively involved in developing the procedures and revising the questionnaires:

• Howard Kress, behavioral scientist ([email protected]), phone 770-488-1285

• Chris Jones, health scientist ([email protected]), phone 770-488-4993

• Rebecca Greco-Kone, public health advisor ([email protected]), phone 770-488-4713

A.9. Explanation of Any Payment or Gift to Respondents.

No payment or gifts will be provided during this data collection.

A.10. Assurance of Confidentiality Provided to Respondents.

All procedures have been developed, in accordance with federal, state, and local guidelines, to ensure that the rights and privacy of SHD directors and staff will be protected and maintained. The CDC National Center for Injury Control and Prevention’s human subjects coordinator has determined that CDC will not be engaged in human subjects research: assess the implementation, performance, coverage, and/or satisfaction with an existing public health program, service, function, intervention or recommendation.

SHD Injury and Violence Prevention (IVP) program directors and staff will be notified that their responses will be treated in secure manner and will not be disclosed, unless compelled by law. Directors will be informed that this evaluation is being conducted for programmatic improvement and their responses will not be used as a means of reducing or canceling funding. IVPP management and staff identifiers will not be used in any evaluation reports.

No personally sensitive or damaging information will be collected during the evaluation.

Privacy Impact Assessment Information

A. This submission has been reviewed by ICRO, who determined that the Privacy Act does not apply.

B. Web-based survey data will be housed on Safe States Alliance’s secure server. This server has limited physical access and is password protected.

Telephone interview data will be housed on a password protected computer in the Safe States Alliance office. Access to the data will be limited to individuals who are assigned to work on the Core State evaluation.

C. No consent is necessary since this is a program evaluation project.

D. Participation in the evaluation is stipulated in the Core VIPP FOA (CDC RFA CE11-1101). As part of the funding agreement, SHDs agreed to participate in a CDC funded program evaluation.

A.11. Justification for Sensitive Questions

No sensitive or potentially damaging information is to be collected in this evaluation.

A.12. Estimates of Annualized Burden Hours and Costs

A.12.A. Burden

Table A-12 details the annualized number of respondents, the average response burden per interview, and the total response burden for the State of the States (SOTS) Survey and follow-up telephone interviews. Estimates of burden for the survey are based on previous experience with evaluation data collections conducted by the evaluation staff. The SOTS web-based survey assessment will be completed by 28 Core Funded State Health Departments (SHDs) and 22 Non-Funded SHDs and take 3 hours to complete. The SOTS Financial Module will also be completed by the 28 Core Funded and 22 Non-Funded SHD and will take 1 hour to complete. The telephone interviews will take 1.5 hours to conclude and will be completed by the 28 Core Funded States. We expect that each of the 28 Core Funded states will complete three web-based surveys and three telephone interviews during the first three years of Core funding. It is anticipated that up to 22 unfunded states will complete three web-based surveys during the first three years of Core funding (some unfunded states may not have a VIPP program due to budget and funding cuts).

Table A.12- Estimate of Annual Burden Hours.

Type of Respondent	Form Name	No. of Respondents	No. of Responses per Respondent	Response Burden (hours)	Total Burden Hours
Core VIPP Funded SHD Injury Program director	State of the States (SOTS) Survey - Attachment C-1)	28	1	3	84
Core VIPP Funded SHD Injury Program director	SOTS Financial Module (Attachment E-1)	28	1	1	28
Core VIPP Funded SHD Injury Program management and staff	Telephone Interview (Attachment D)	28	1	1.5	42
Non-Funded SHD Injury Program management and staff	SOTS Survey - Attachment C-2	22	1	3	66
Non-funded SHD Injury Program management and staff	SOTS Financial Module (Attachment E-2)	22	1	1	22
Total					242

A.12.B. Estimated Annualized Burden Cost

The hourly wage used to calculate the Respondent Cost is $32.83, which is the May 2010 average hourly wage for an epidemiologist as calculated by the Bureau of Labor Statistics (http://www.bls.gov/oes/current/oes_nat.htm, accessed June 1, 2011).

Data Collection Instrument Name	Number of Respondents	Frequency of Response	Average Burden Response (Hours)	Average Hourly Wage Rate	Total Respondent Cost
SOTS Survey (Funded) – Attachment C-1	28	1	3	$32.83	$2757.72
SOTS Financial Module (Funded) – Attachment E-1	28	1	1	$32.83	$919.24
Telephone interview (Funded) – Attachment D	28	1	1.5	$32.83	$1378.86
SOTS Survey (Non-Funded) – Attachment C-2	22	1	3	$32.83	$2166.78
SOTS Financial Module (Non-Funded) – Attachment E-2	22	1	1	$32.83	$722.26
Total					$7,944.86

A.13. Estimates of Other Total Annual Cost Burden to Respondents or Recordkeepers.

Respondents will incur no capital or maintenance costs.

A.14. Estimates of Annualized Cost to the Federal Government.

Two types of government costs will be incurred: (1) government personnel, and (2) contracted data collection.

NCIPC has assigned a Project Officer, a Public Health scientist, and a Public Health Advisor to assist with and oversee this data collection. The Project Officer will spend 50 percent of his time on this project, the Public Health Scientist 70 percent, and the Public Health Advisor 30 percent. Table A.13 provides the total annual government personnel costs

Table A.13

Position	Annual Salary	Time on Project	Total Cost
Project Officer	$85,500	50%	$42,750
Public Health Scientist	$83,884	70%	$58,719
Public Health Advisor	$74,297	30%	$22,289
Total Cost			$123,758

CDC has issued a FOA for an evaluation cooperative agreement. This cooperative agreement will fund an Safe States Alliance for five years at up to $300,000 per year.

The average annualized direct costs for this project are $431,177. This amount includes all costs for the contracted data collection, plus the personnel costs of federal employees involved in oversight and analysis.

A.15. Explanation for Program Changes or Adjustments

This is a new data collection.

A.16. Plans for Tabulation and Publication and Project Time Schedule

Data analysis for this project centers on Injury and Violence Prevention Program (IVPP) performance. The primary analytical tools are based on goal attainment. Each IVPP had to identify 4 projects that they would complete during the five year funding period. Each project has associated SMART (Systematic, Measurable, Achievable, Relevant, and Time bound) goals. Each of these SMART goals has associated measures of success. Analysis will then focus on the degree to which those goals were achieved and the context variables that contributed or hindered achievement.

Analysis of goal achievement will be based on how the individual SMART objectives are designed by the states. For example, some states will have policy objectives. Text analysis techniques will be used for policy analysis. These techniques include looking for common themes across respondents for what worked and what hindered success in meeting a policy objective.

States can also have morbidity and mortality reduction objectives. Analysis of these objectives will rely on epidemiological techniques including within group means comparisons for continuous variables, frequency comparisons for within group categorical variables, and hazard analysis for time series data. States will be responsible for conducting much of this analysis and CDC will assist when requested. A.16.A. Tabulation and Analysis Plan. In Table 1, 2011 data collection refers to activities that occurred from August 2011 through July 2012, which reflects the funding cycle for the Core VIPP cooperative agreement.

Table 1

Activity	Time Schedule
Letters sent to awardees for 2011 data collection	1 month after OMB approval
Conduct web-based surveys for 2011 activities	August, 2012
Conduct telephone interviews 2011 activities	November, 2012
Analyze web-based surveys	September 2012-October 2012
Analyze telephone survey	December 2012- February 2012
Develop CDC internal report on 2011 awardee activities	March 2013
Send letters to awardees for 2012 data collection	June, 2013
Conduct web-based surveys for 2012 activities	August, 2013
Analyze web-based surveys for 2012 activities	August-October, 2013
Conduct telephone interviews 2012 activities	November, 2013
Analyze telephone interviews	December, 2013-Juanuary, 2014
Develop CDC internal report on 2012 awardee activities	March 2014
Send letters to awardees for 2013 data collection	June, 2014
Conduct web-based surveys for 2013 activities	August, 2014
Analyze web-based surveys for 2013 activities	September-October, 2014
Conduct telephone interviews 2013 activities	November, 2014
Analyze telephone interviews	December, 2014-Juanuary, 2015
Develop CDC internal report on 2013 awardee activities	March 2015

A.16.B Publications

No publications are planned from the data collected.

17. Reason(s) Display of OMB Expiration Date is Inappropriate

No exemption is being sought.

A.18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exemptions to the certification.

1 Finkelstein EA, Corso PS, Miller TR, Associates. Incidence and Economic Burden of Injuries in the United States. New York: Oxford University Press; 2006.

2 Patton, MQ. Utilization-Focused Evaluation 4^th Edition. Los Angeles: Sage; 2008

15