Supporting Statement A

Supporting Statement A.docx

Evaluation of Core Violence and Injury Prevention Program

OMB: 0920-0916

Document [docx]
Download: docx | pdf









Evaluation of Core Violence and Injury Prevention Program

(Core VIPP)


OMB# 0920-0916


Supporting Statement A


March 12, 2014


Department of Health and Human Services

Centers for Disease Control and Prevention

National Center for Injury Prevention and Control

Division of Injury Response


Project Officers:

Sally Thigpen


4770 Buford Highway, NE, Mailstop F-64

Atlanta, GA 30341

Telephone: 770-488-1285

Fax: 770-488-3551

Electronic Mail:

Sally Thigpen: [email protected]

Brandon Nesbit: [email protected]




List of Attachments


  • Attachment A: TITLE 42--THE PUBLIC HEALTH AND WELFARE (authorizing legislation)

  • Attachment B: Published 60 Day Federal Register Notice

  • Attachment C: State of the States (SOTS) Survey

  • Attachment D: BIC Telephone interviews

  • Attachment E: SOTS Financial Module

  • Attachment F: Supplemental Questions for SOTS survey: BIC Capacity Indicator Questionnaire

  • Attachment G: RNL Telephone Interviews

  • Attachment H: RNL Network Satisfaction Survey

  • Attachment I: RNL Needs Assessment Survey

  • Attachment J: SQI Telephone Interviews

  • Attachment K: MVP Telephone interviews




A. Justification

1. Circumstances Making the Collection of Information Necessary


Background


The overall purpose of this Revision with a request for an additional 3 years (OMB#: 0920-0916) is to support an evaluation of the Centers for Disease Control and Prevention (CDC)/ National Center for Injury Prevention and Control (NCIPC) Core Violence and Injury Prevention Program (Core VIPP). NCIPC supports state health department (SHD) partners to move toward the right hand side of the public health model through implementation of evidence based interventions and strategies. One tool NCIPC will use to accomplish this is CORE VIPP. This five–year program which began funding on August 1, 2011, funded 28 SHDs (reduced to 20 funded states in 2013) to build effective delivery systems for dissemination, implementation and evaluation of evidence based/best practice programs and policies, with a focus on:

  • Enhancement of infrastructure, including injury surveillance and development of Injury Community Planning Groups (ICPG)

  • Policy, communications, and evaluation for injury and violence reduction

  • Establishment of Regional Network Leaders (RNL)

  • Implementation of evidence based practices (programs and policies)

  • Establishment of long term state injury priorities with documented health outcomes


In addition to the Base Integration Component (BIC), the Core VIPP program funds three subsets of these 20 states as Regional Network Leaders, for Surveillance Quality Improvement projects, and for expanded Motor Vehicle Safety initiatives. The purpose of the Regional Network Leaders sub-component (RNL) is to help Core VIPP funded and non-funded states share knowledge to enhance the VIP capacity of all states. These regional networks were organized to facilitate peer-to-peer sharing and technical assistance among all states within a specific region. Four states are funded as Regional Network Leaders. The purpose of the Surveillance Quality Improvement sub-component (SQI) is for the four funded SQI states to engage in state-based and group surveillance quality improvement projects that promote and advance uniform injury case definitions, improve data quality and advance surveillance methodology. Four of the 20 Core VIPP funded states receive additional funding through the Core VIPP Motor Vehicle Child Injury Prevention Policy (MVP) component. These states are funded to conduct activities that include using surveillance findings to guide motor vehicle/child injury prevention policy activities; developing an Action Plan; identifying policy interventions; identifying collaborating partners; and evaluating outcomes.


The establishment of the Core program is to address the burden that injury (both unintentional and violence-related injuries) places on the United States. Taken together, unintentional and intentional injuries are the leading cause of death for the first four decades of life, regardless of gender, race, or socioeconomic status. More than 179,000 individuals in the U.S. die each year as a result of unintentional injuries and violence. More than 29 million others suffer non-fatal injuries and over one-third of all emergency department (ED) visits each year are due to injuries. In 2000, injuries and violence ultimately cost the United States $406 billion, with over $80 billion in medical costs and the remainder lost in productivity.1 Most events that result in injury and/or death from injury could be prevented if evidence-based public health strategies, practices, and policies were used throughout the nation.


The primary goal of Core VIPP is to assist SHDs to build and/or maintain effective delivery systems for dissemination, implementation, and evaluation of best practice programs and policies. This includes support for general capacity building of SHDs and their local partners, as well as strategy specific capacity building for the implementation of direct best practice interventions. In addition, this program supports SHDs in their efforts towards integration and strategic alignment of resources for meaningful change.


The purpose of this ICR is to permit CDC to evaluate the Core VIPP program for the benefit of the Core VIPP grantees. This ICR has two overall goals: (1) to assess state injury and violence prevention plans for completeness, measurability, and effectiveness; and (2) evaluate the effectiveness of the Core VIPP cooperative agreement. Through the evaluation of the Core VIPP, CDC plans to improve state health department program and policy activities. The CDC evaluation team is committed to using the CDC Framework for Program Evaluation in Public Health (http://www.cdc.gov/mmwr/preview/mmwrhtml/rr4811a1.htm) to guide the design and implementation of this project. This program addresses the “Healthy People 2020” focus area of Injury and Violence Prevention.


This data collection is authorized under the Section 301 of the Public Health Service Act (42 U.S.C 241). (Attachment A). This act gives federal health agencies, such as CDC, broad authority to collect data and do other public health activities, including this type of study.


1.1 Privacy Impact Assessment

The respondents to this data collection are the 20 Core VIPP funded states No individually identifiable information is being collected.


No children under 13 years of age are included in this information collection request.


Overview of Data Collection System

Data collection will use the following techniques: annual, web-based surveys (Attachments C, E, and F); follow-up telephone interviews to all 20 Core VIPP Base IC funded states (Attachment D & K); two web-based surveys related to the RNL expanded component (Attachments H and I); and interviews with the states receiving supplemental funding for SQI (n=4), RNL (n=5), and MVP (n=4) (Attachment D and G). The primary respondents will be the SHD Injury Program directors and associated staff. Data will be collected by the CDC and the Safe States Alliance (cooperative agreement CDC RFA CE11-1106 award date: September 30, 2011). All data will be used to determine the amount of progress a state has made towards meeting its injury and violence prevention (IVP) objectives. Overall program effectiveness will be determined by the ability of states to meet and/or exceed their objectives. Data will also be used to indicate areas for programmatic improvement. Data will be kept through the end of the Core VIPP funding period (July 31, 2016) plus two additional years for analysis purposes. Thus, all data will be discarded in July, 2019. Data will be initially housed with the Safe States Alliance and shared with the CDC; however, at the end of the cooperative agreement all data will be transferred to CDC by September 30, 2016.


Items of Information to be Collected

Data will consist of questions regarding program evaluation, state health department (SHD) injury program infrastructure, injury program strategies and partners, policy strategies, injury surveillance, quality of surveillance, and regional network leaders. Specific questions can be seen in Attachments C, D, E, F, G, H, & I.


No individually identifiable information is being collected.



2. Purpose and Use of Information Collection

The information collected under the proposed data collection will be used to:

  1. Assess state injury and violence prevention plans for completeness, measurability, and effectiveness;

  2. Assess surveillance quality improvement projects and tools for completeness and utility for the injury prevention field;

  3. Develop a tool to support state health department planning and evaluation efforts;

  4. Evaluate the effectiveness of the Core VIPP cooperative agreement, including contextual factors.


Through the evaluation of the Core VIPP, CDC plans to improve state health department program and policy activities. This includes support for general capacity building of SHDs and their local partners, as well as strategy specific capacity building for the implementation of direct best practice interventions. Through the evaluation of this capacity building effort, CDC seeks to quantify progress towards reductions in injury related morbidity, mortality, and disparities. Through the collection of evaluation data, CDC can determine if those goals have been met over the next five years of Core VIPP funding (FY2012 –FY 2016).


The practical utility of this evaluation to the federal government is to assess the merit, worth, and significance of the Core program. Results of the evaluation will inform the states and the CDC as to which programs and policy efforts are effective in reducing injury related morbidity and mortality, and associated disparities. This information will be used to help guide the states in implementation of programs and policies. Not collecting this data could result in inappropriate programs and policies being implemented in states, resulting in a loss of tax payer resources that could have been used more effectively. Worse, people will continue to be injured and killed due to violence and unintentional injury.


Results of this program evaluation are not generalizable beyond Core VIPP. Instead, evaluation results can be used to modify existing practices when ineffectiveness is discovered. Results can also be used to help other states in implementation of program or policy; however, each state has its own set of contextual variables that significantly contribute to the success or failure of an intervention (e.g. political climate, state funding, demographics of the population, and historical factors). Thus, the evaluation will produce information about program success and information on program improvement at the state level. CDC will also use this information to improve the Core program, specifically examining the effectiveness of funding levels, technical assistance provided, and training.


2.1 Privacy Impact Assessment Information


  1. Description of how the information will be shared and for what purpose

The purpose of this information collection is to determine the merit, worth, and significance of the Core VIPP. This will be accomplished by collecting state-level data on a variety of topics (see Attachments C-E), including injury surveillance, program implementation, policy efforts, partnerships and coalitions for injury and violence prevention, state implemented program evaluation, and state injury program infrastructure development. The information from this evaluation will provide empirical and qualitative evidence for the effectiveness of the Core program. The data produced will be used for future funding decisions, such as the number of states to fund, the level of funding, and the type of technical assistance provided to the funded states.


  1. A statement detailing the impact the proposed collection will have on the respondent’s privacy

No IIF is being collected in this evaluation. The impact of this data collection on states is low since states are already required to collect much of this information for grant management purposes.


3. Use of Improved Information Technology and Burden Reduction


The evaluation will use two methods for data collection from all 20 Core VIPP funded states for BIC. The first method is an existing web-based questionnaire, the Safe States Alliance State of the States survey (SOTS) and its associated modules. This non-federally funded survey has been deployed biennially since 2005. For this evaluation, the CDC seeks OMB clearance to use the SOTS as an assessment tool. The SOTS is normally a web-based survey via Survey Monkey, but Safe State Alliance has made hard copy submission an option, and will continue under this ICR. Attachment C _State of the States Survey is the 2013 survey which forms the basis for this 3-year ICR, while Attachment E_2011 SOTS Finance Module is a financial module which will be introduced in for the 2012 data collection. Attachment F_SOTS Supplement contains new questions to be administered as a supplement to the State of the State Survey (SOTS).


The second method is a follow-up telephone interview (Attachment D_Telephone interviews). The SOTS produces significant information on goal obtainment, but does not provide the nuanced contextual information that is vital for understanding program results2. To gather this critical context information, 90 minute phone interviews will be conducted with the Core-funded state injury program directors and associated staff. Topics will be follow-up questions to responses on the SOTS as well as more open-ended questions about program operations, barriers, facilitators and opportunities.


Evaluation of the Regional Network Leaders sub-component will consist of five brief interviews (one with each funded state) (Attachment G & J) and two additional surveys, the Annual Regional Network Satisfaction Survey (Attachment H) and the Regional Network Programmatic Inventory and Needs Assessment (Attachment I), will be delivered through the Regional Network Leaders to assess the strength and effectiveness of regional networks to connect states to each other for peer-to-peer knowledge and information sharing. Similarly, connections to Injury Control Research Centers (ICRC) and other researchers will be measured to determine the increased knowledge base and implementation of programs, practices, or policies that have best available research evidence.


The Regional Network Satisfaction Survey (survey) will be administered annually. The brief survey was developed by the five funded Regional Network Leaders (RNL) and include comparison questions as well as region specific questions. The RNLs distribute the survey to their network members and ICRC partners. The survey informs RNL program improvements and other modifications in each region.


The Regional Network Programmatic Inventory and Needs Assessment (inventory) was administered in year 1 and will be repeated in year 5 of the funding period. The inventory informed the RNLs in establishing regional goals and related activities. The follow-up inventory will measure any changes in the nature of IVP programs, practice, or policies being implemented. Changes in the way research and practice are connecting regionally will also be revealed in the inventory.


Evaluation of the Surveillance Quality Improvement sub-component will consist of four brief interviews (one with each funded state) every year (Attachment G). These interviews will elicit information from the four SQI funded states on their state-based surveillance quality improvement projects, their participation in group injury surveillance consensus processes, and any resulting tools/products developed for use by state health departments to improve injury surveillance quality.


Evaluation of the Motor Vehicle sub-component will consist of additional survey questions specific to the four funded motor vehicle policy component states on the supplemental SOTS survey BIC Capacity Indicator Questionarre (Attachment F) conducted in year 2 and 4 of the cooperative agreement and additional interview questions (Attachment D) specific to the four states that will be asked during the annual safe states telephone interviews. The survey and interview questions are intended to elicit detailed information from the states on their motor vehicle related activities including developing an action plan, identifying and implementing interventions, and evaluating outcomes.

4. Efforts to Identify Duplication and Use of Similar Information


Since CDC is the only federal agency providing funding for state injury and violence prevention infrastructure building, there has been no previous data collection on the effectiveness of the Core VIPP. A previous evaluation was conducted by CDC on the breadth of the Core funding, but this evaluation only looked at CDC functions and did not collect data from the participating states.


5. Impact on Small Businesses or Other Small Entities


Small businesses will not be involved in this data collection.


6. Consequences of Collecting the Information Less Frequently


The proposed data collection will provide both the states and the CDC with critical data on the effectiveness of state injury and violence prevention efforts. This data is needed to both enhance current state-level programs as well as CDC’s efforts to support those programs. Annual data collection is the appropriate frequency of collection due to 1) federal grantee reporting requirements, and 2) to provide states a way of making corrections to program efforts. To not conduct this evaluation would result in CDC failing to account for the effectiveness of federal dollars spent on a public project. More significantly it would mean states would not know their effectiveness and not be able to make improvements to their programs.


7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5


This request fully complies with the regulation 5 CFR 1320.5.


8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency


A. A 60-day notice to solicit public comments was published in the Federal Register –volume 78, No. 154, pages 48681-48683,. Attachment B contains a copy of the notice. There were no comments in response to the Federal Register Notice.


B. During the conceptualization phase of the evaluation design several members of the CDC evaluation community were contacted, as well as subject matter experts on intentional and unintentional violence. These conversations discussed the potential approaches to the evaluation, types of data to be collected, and method for data collection. From these conversations it was determined that a mixed-methods (qualitative and quantitative) approach was both practical and necessary given the nature of the Core program. The following CDC evaluation and injury and violence prevention specialists were consulted during the development phase of the evaluation (November 2010-May 2011):


For this study, the following CDC staff has been actively involved in developing the procedures and revising the questionnaires:


9. Explanation of Any Payment or Gift to Respondents.


No payment or gifts will be provided during this data collection.


10. Assurance of Confidentiality Provided to Respondents.

This submission has been reviewed by the CIO who determination that the Privacy Act does not apply. All procedures have been developed, in accordance with federal, state, and local guidelines, to ensure that the rights and privacy of SHD directors and staff will be protected and maintained. The CDC National Center for Injury Control and Prevention’s human subjects coordinator has determined that CDC will not be engaged in human subjects research: assess the implementation, performance, coverage, and/or satisfaction with an existing public health program, service, function, intervention or recommendation.


SHD Injury and Violence Prevention (IVP) program directors and staff have been informed that data will be treated in secure manner and will not be disclosed, unless otherwise compelled by law. Directors will be informed that this evaluation is being conducted for programmatic improvement and their responses will not be used as a means of reducing or canceling funding. Injury and Violence Prevention Program management and staff identifiers will not be used in any evaluation reports.


IRB Approval is not required.


10.1 Privacy Impact Assessment Information

1. No individually identifiable information is being collected.

2. No consent is necessary since this is a program evaluation project.


3. Web-based survey data will be housed on Safe States Alliance’s secure server. This server has limited physical access and is password protected. Telephone interview data will be housed on a password protected computer in the Safe States Alliance office. Access to the data will be limited to individuals who are assigned to work on the Core VIPP evaluation.


4. This submission has been reviewed by ICRO, who determined that the Privacy Act does not apply..


Participation in the evaluation is stipulated in the Core VIPP FOA (CDC RFA CE11-1101). As part of the funding agreement, SHDs agreed to participate in a CDC funded program evaluation.


11. Justification for Sensitive Questions

No sensitive or potentially damaging information is to be collected in this evaluation.


12. Estimates of Annualized Burden Hours and Costs


A. Burden


Table A.12.A details the annualized number of respondents, the average response burden per interview, and the total response burden for the State of the States (SOTS) Survey and follow-up telephone interviews. Estimates of burden for the survey are based on previous experience with evaluation data collections conducted by the evaluation staff. The SOTS web-based survey assessment will be completed by 20 Core Funded State Health Departments (SHDs). The SOTS Financial Module will also be completed by the 20 Core Funded SHD and will take 1 hour to complete. The supplemental SOTS Survey BIC Capacity Indicator Questions will be completed by 20 Core Funded State Health Departments (SHDs) and take 1.5 hours to complete. The telephone interviews will take 1.5 hours to conclude and will be completed by the 20 Core Funded States. We expect that each of the 20 Core Funded states will complete one web-based survey and one telephone interview during each year of Core funding. Additionally, the surveys and interviews for the subcomponents (SQI, RNL, and MVP) are detailed below.


Table A.12.A - Estimate of Annual Burden Hours.


Type of Respondents

Form Name

No. of Respondents

No. of Responses per Respondent

Avg. Burden per Response (in hrs.)

Total Burden (in hrs.)

Core VIPP Funded SHD Injury Program director

State of the States Survey (SOTS) – Attachment C

20

1

3

60

Core VIPP Funded SHD Injury Program director

SOTS Financial Module - Attachment E

20

1

1

20

Core VIPP Funded SHD Injury Program management and staff

Supplemental SOTS Survey BIC Capacity Indicator Questions – Attachment F

20

1

1.5

30

Core VIPP Funded SHD Injury Program management and staff

BIC Telephone Interview – Attachment D

20

1

1.5

30

RNL awardees

RNL Telephone Interview – Attachment G

5

1

1

5

RNL awardees

RNL Network Satisfaction Survey – Attachment H

5

1

1

5

RNL Awardees

RNL Needs Assessment Survey

Attachment I

5

1

1

5

SQI awardees

SQI Telephone Interview – Attachment J

4

1

1

4

MVP awardees

MVP Telephone Interview – Attachment K

4

1

1

4


Total

163


A.12.B. Estimated Annualized Burden Cost


The hourly wage used to calculate the Respondent Cost is $34.33, which is the May 2012 average hourly wage for an epidemiologist as calculated by the Bureau of Labor Statistics (http://www.bls.gov/oes/current/oes_nat.htm, accessed November 12, 2013).

Table A.12.B: Estimated Annualized Burden Cost

Type of Respondents

Form Name

No. of Respondents

No. of Responses per Respondent

Avg. Burden per Response (in hrs.)

Total Burden (in hrs.)

Hourly Wage Rate

Total Respondent Cost

Core VIPP Funded SHD Injury Program director

State of the States Survey (SOTS) – Attachment C

20

1

3

60

$34.33

$2059.80

Core VIPP Funded SHD Injury Program director

SOTS Financial Module - Attachment E

20

1

1

20

$34.33

$686.60

Core VIPP Funded SHD Injury Program management and staff

Supplemental SOTS Survey BIC Capacity Indicator Questions – Attachment F

20

1

1.5

30

$34.33

$1029.90

Core VIPP Funded SHD Injury Program management and staff

BIC Telephone Interview – Attachment D

20

1

1.5

30

$34.33

$1029.90

RNL awardees

RNL Telephone Interview – Attachment G

5

1

1

5

$34.33

$171.65

RNL awardees

RNL Network Satisfaction Survey – Attachment H

5

1

1

5

$34.33

$171.65

RNL Awardees

RNL Needs Assessment Survey

Attachment I

5

1

1

5

$34.33

$171.65

SQI awardees

SQI Telephone Interview – Attachment J

4

1

1

4

$34.33

$137.32

MVP awardees

MVP Telephone Interview – Attachment K

4

1

1

4

$34.33

$137.32


Total



$5,595.79




13. Estimates of Other Total Annual Cost Burden to Respondents or Recordkeepers.


Respondents will incur no capital or maintenance costs.



14. Estimates of Annualized Cost to the Federal Government.


Two types of government costs will be incurred: (1) government personnel, and (2) contracted data collection.


NCIPC has assigned a two Health Scientists, and a Behavioral Scientist to assist with and oversee this data collection. The Health Scientists will serve as project officer and science officer and will spend 50 percent of their time on this project. The Behavior Scientist will spend 25 percent on this project. Table A.13 provides the total annual government personnel costs.


Table A.13

Position

Annual Salary

Time on Project

Total Cost





Health Scientists (2)

$85,500

50%

$85,500(2)

Behavioral Scientist

$85,500

25%

$21,375

Total Cost



$106,875



CDC has issued a FOA for an evaluation cooperative agreement. This cooperative agreement will fund Safe States Alliance for five years at up to $300,000 per year.


The average annualized direct costs for this project are $433,846. This amount includes all costs for the contracted data collection, plus the personnel costs of federal employees involved in oversight and analysis.


15. Explanation for Program Changes or Adjustments


Core VIPP Evaluation is a five-year program (2011-2016) funded at $300,000 annually for a total project cost of 1,500,000. In response to the findings of year-one evaluation for program improvement, this revision is submitted because CDC has developed new questions related to capacity indicators as a supplement to the State of the State Survey (SOTS) for years two through five of data collection (attachment F). These new questions are designed to assess state injury violence prevention plans for completeness, measurability, and effectiveness. These new questions will also align with the requirements of the Core VIPP FOA and will be administered only to the funded state health departments (SHDs). Additionally, the number of SHDs involved in the evaluation has been reduced due to a loss of funding in Year two. The burden has also been reduced due to a reduction in funded states from 28 to 20.

16. Plans for Tabulation and Publication and Project Time Schedule

Data analysis for this project centers on Violence and Injury Prevention Program (VIPP) performance. The primary analytical tools are based on goal attainment. Each VIPP had to identify 4 projects that they would complete during the five year funding period. Each project has associated SMART (Specific, Measurable, Achievable, Relevant, and Time bound) goals. Each of these SMART goals has associated measures of success. Analysis will then focus on the degree to which those goals were achieved and the context variables that contributed or hindered achievement.


Analysis of goal achievement will be based on how the individual SMART objectives are designed by the states. For example, some states will have policy objectives. Text analysis techniques will be used for policy analysis. These techniques include looking for common themes across respondents for what worked and what hindered success in meeting a policy objective.


States can also have morbidity and mortality reduction objectives. Analysis of these objectives will rely on epidemiological techniques including within group means comparisons for continuous variables, frequency comparisons for within group categorical variables, and hazard analysis for time series data. States will be responsible for conducting much of this analysis and CDC will assist when requested.



Table 1,

Table 1

Activity

Time Schedule

Conduct web-based surveys for 2013 activities

January, 2014 – February, 2014, 2014

Analyze web-based surveys for 2013 activities

January, 2014 – May 2014

Conduct telephone interviews 2013 activities

April, 2014 – May, 2014

Conduct telephone interviews for 2012 & 2013 SQI activities

January 2014

Analyze telephone interviews

May, 2014 – June, 2014

Develop CDC internal report on 2013 awardee activities

August, 2014

Send letters to awardees for 2014 data collection

October, 2014

Conduct web-based surveys for 2014 activities

Novermber, 2014 – December, 2014

Analyze web-based surveys for 2014 activities

December, 2014 – January, 2015

Conduct telephone interviews for 2014 SQI activities

December, 2014 – January, 2015

Conduct telephone interviews 2014 activities

April, 2015 – May, 2015

Analyze telephone interviews

May, 2016- June, 2016

Develop CDC internal report on 2014 awardee activities

July, 2015 – September, 2015



Send letters to awardees for 2015 data collection

October, 2015

Conduct web-based surveys for 2015 activities

November, 2015 – February, 2016

Analyze web-based surveys for 2015 activities

December, 2015 – March, 2016

Conduct telephone interviews 2015 activities

April, 2016 – May, 2016

Conduct telephone interviews for SQI 2015 activities

December, 2015- January, 2016

Analyze telephone interviews

May, 2016 – June, 2016

Develop CDC internal report on 2015 awardee activities

July, 2016 – September, 2016


Safe States Alliance and CDC will jointly decide on publications of aggregate findings during years three to five.


17. Reason(s) Display of OMB Expiration Date is Inappropriate

The display of the OMB expiration date is not inappropriate

A.18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exemptions to the certification.



1 Finkelstein EA, Corso PS, Miller TR, Associates. Incidence and Economic Burden of Injuries in the United States. New York: Oxford University Press; 2006.

2 Patton, MQ. Utilization-Focused Evaluation 4th Edition. Los Angeles: Sage; 2008

17


File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
AuthorHoward Kress
File Modified0000-00-00
File Created2021-01-28

© 2024 OMB.report | Privacy Policy