0920-0879 EHDI- Attachment D- Data Collection Instrument Word vers

Form approved:

OMB No. 0920-0879

Expiration date: 03/31/2014

CDC Early Hearing Detection and Intervention (EHDI) Program:

Evaluation Self-Assessment Instrument

Section I

ATTACHMENT D: Data Collection Instrument – Word version

Introduction:

Program evaluation offers a set of tools and skills for ensuring that public health programs operate as effectively as possible. As program staff, you can use the approach to ensure that evaluation findings produce information that is useful and accurate and those methods meet the standards for quality and propriety. We recognize that there are varying levels of evaluation experience among grantees, but know that many of your public health skills are also common to evaluation.

Purpose:

This assessment instrument is designed to support the evaluation capacity development of grantees by identifying the strengths and needs and by stimulating reflection, both individually and within your organization. The assessment is intended for use by EHDI grantees. Together we can identify opportunities for you to advance your evaluation skills, share them with others or recommend additional evaluation resources that could enhance your evaluation practice. Please be aware that this instrument is intended solely for the identification of your training, technical assistance and resource needs.

We have organized the assessment in two parts. Section I provides an opportunity for you to assess your skills and experience carrying out a variety of evaluation related activities. We will use the information to help us tailor the types of resources and support we provide you, including planning for future trainings and materials development. Section II has questions about your organization and asks that you assess its experience with and attitudes toward evaluation. We will use this information to understand your needs and the context in which you work. We may also be able to assist you in building evaluation capacity in your organization.

Follow Up:

Once you have completed the assessment your Project Officer will schedule a confidential discussion of your assessment. Together we can identify opportunities for you to advance your evaluation skills and share them with others. Your Project Officer can also recommend additional evaluation resources that can assist you in honing evaluation skills.

2. Background information

Please tell us a little bit about yourself.

1. For what state/territory do you serve?

2. What is your position in the EHDI program? (Check all that apply)

Data Coordinator/Manager

EHDI Coordinator/Manager

Principal Investigator

Other (please specify):

3. How long have you been working in the early hearing detection field?

< 1 year

1 to <2 years

2 to 5 years

>5 years

4. Which of the following best describes your level of experience in program evaluation?

Relatively no experience, need comprehensive training

Little experience or background, need training on most topics

Moderate experience, need refresher courses and more advanced skill training

Advanced experience, need training or information on emerging topics

4. Evaluator Competencies

For each of the items below, mark the box that best represents your personal opinion. Please note, this list represents a fairly comprehensive set of evaluation skills and knowledge. It is not expected that you will be experienced or skilled in all of them. At the end of this survey, you will be asked to identify the areas in which you have the greatest need.

	Not at all	I have only a general knowledge	I have used or could use these
a. Updated Guidelines for Evaluating Public Health Surveillance Systems			
b. Community Tool Box			
c. CDC Framework for Program Evaluation in Public Health			
d. Multiple approaches to evaluation (e.g., participatory, theory-driven, empowerment)			

5.

	Would like technical assistance on this	Have general knowledge about this	Have done or used this
a. Involving stakeholders in the evaluation planning process……………………			
b. Writing evaluation plans…………………			
c. Designing data collection protocols………			
d. Budgeting for evaluation ………..........			
e.. Contracting for evaluations…………………..			
f. Staffing evaluations………………			
g. Establishing and maintaining databases…			
h. Creating a program logic model….…			
i. Using logic models to guide the development of evaluation questions...…			
j. Designing evaluations using mixed method evaluations…….…			
k. Developing data collection instruments..…			
l. Adapting existing data collection instruments for evaluation purposes……			
m. Synthesizing evaluation findings with stakeholders….…			
n. Developing action plans based on evaluation recommendations…			
o. Writing user-friendly evaluation reports..…			

6.
	Would like technical assistance on this		Have general knowledge about this		Have done or used this
observations					
program staff					
surveys/questionnaires					

7.

	Would like technical assistance on this	Have general knowledge about this	Have done or used this
designing evaluations using mixed methods evaluations			
developing data collection instruments….……………………………...			
adapting existing data collection instruments for evaluation purposes			

8.

Please indicate your experience with using data from a variety of sources for evaluation purposes:	Would like technical assistance on this	Have general knowledge about this	Have done or used this
insurance/hospital records			
program documents (policies, meeting minutes, promotional materials)….………………………….			
surveillance systems			

9.
Please indicate your experience with tailoring evaluation findings to multiple audiences:	Would like technical assistance on this		Have general knowledge about this		Have done or used this
media					
state partners….…………………………					
program staff					
community groups					
legislators/advocacy groups/policy makers….…………………………………					

	Would like technical assistance on this	Have general knowledge about this	Have done or used this
posters			
presentations……………………………...			
reports			
newsletters			
developing action plans based on evaluation recommendations			

SECTION II

In this section, you are asked to reflect on the organization for which you work. When considering these questions, think about how open your organization is to supporting evaluation in general, to conducting evaluations, and to using evaluation findings in making decisions. Please answer the following questions in terms of the program, unit or department in which you work and that you know best, not the larger organization.

10. Organizational Practices

Please indicate to what extent you agree with the following statements about your program, unit, or department within your organization:
	Strongly Disagree	Somewhat Disagree	Neutral	Somewhat Agree	Agree
My organization has evaluator positions or position descriptions that include evaluation as an essential job function……………….........................					
Organization leaders support and value program evaluation…………….					
Organization leaders integrate evaluation findings into the strategic planning process……………………..					
Organization leaders motivate staff to integrate evaluation into their work…………………………………...					
Program evaluation routinely drives continuous improvement of programs……………………………….					
Organizational funds are routinely dedicated to evaluation……………….					
Organization leaders support training to improve evaluation skills………………………...................					
Our performance review system values evaluation and offers promotions or rewards for evaluation work…………………………………….					
My organization has systems in place to change policies and procedures based on evaluation findings and recommendations..........					
Staff with evaluation expertise are available within my organization to assist me …..					
Organizational support is available for technology systems, including software that can be used to manage and analyze evaluation data (e.g., Excel, SPSS) ……					

11. Organizational Readiness for Learning
1. Please indicate to what extent you agree with the following statements about your program, unit or department within your organization.
	Strongly Disagree		Somewhat Disagree		Neutral		Somewhat Agree		Strongly Agree
Currently available information tells us what we need to know about the effectiveness of our program…………………………………									
Currently available information tells us what we need to know about the efficiency of our processes…………………………………									
Existing systems manage and disseminate information for those who need and can use it……………………									
Employees have access to the information they need to make decisions regarding their work………………………..									
My organization welcomes new information………………………………..									
There would be support among employees if we tried to do more (some) evaluation work…………………									
Doing (more) evaluation would make it easier to convince managers of needed changes…………………………………..									

12. Technical Assistance Needs
Please identify your greatest needs for technical assistance, whether in developing your own evaluation competencies or in building support for evaluation in your organization or building evaluation capacity.

Public reporting burden of this collection of information is estimated to average 15 minutes, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number.  Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to CDC/ATSDR Reports Clearance Officer; 1600 Clifton Road NE, MS D-74,  Atlanta, Georgia 30333; ATTN:  PRA (0920-0879)