Evaluator Interview

Form Approved

OMB No. 0920-xxxx

Exp. Date xx/xx/xxxx

Evaluator Interview Guide

Public reporting burden of this collection of information is estimated to average one hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number.  Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to CDC Reports Clearance Officer; 1600 Clifton Road NE, MS D-74, Atlanta, Georgia 30333; ATTN:  OMB 0920-XXXX

Interviewer’s Note:

Remember that all interviewees must receive the consent form.

Throughout the interview, you may choose to refer to the draft program logic model as appropriate. Ask about the logic model or, particularly, the stated activities and outcomes. Does the interviewee feel it is an accurate representation of the program?

Remember also that the lead administrator should be asked questions for the program as a whole, not just for the specific location we are visiting.

I. HISTORY OF THE INITIATIVE

1) First let me ask you a little about the background of the (Initiative). How long has the initiative been in existence? Is there an expected end date?

2) What factors led to the development and implementation of the Initiative?

• Was a needs assessment or similar data-collection activity conducted prior to the start of the initiative? If so, how did the needs assessment inform the development of the initiative?

3) Were you involved in developing the initiative?

• If so, what was your role?

• Can you describe the planning involved in designing and implementing the initiative?

• What, if any, barriers were encountered in developing the initiative? How were these challenges addressed?

II. DESCRIBE THE INITIATIVE

1. Can you please describe the organizational structure for the initiative?

   • Explain how the initiative is organized.

• Is it affiliated or housed within any organizations, government agencies, etc.?

• Are there any paid staff members? Volunteers?

• Who are the main partner organizations or groups?

2. Can you please describe the initiative?

• What types of changes have been made to the built environment?

• What types of changes to the built environment have been programmed or are being planned, but may not have been implemented yet?

• What type of policy change(s) has occurred due to the initiative?

• What geographic area does the initiative cover (name of county, city, towns, neighborhood, etc.)?

3. What would you say are the goals of the initiative overall?

[DO NOT READ; ASK ONLY IF NOT MENTIONED]

• Establish policies that promote physical activity

• Provide physical and social environments that encourage and enable physical activity

• Provide a range of developmentally appropriate community programs for sports and recreation and active transportation

• Develop long-range transportation plans

• Increase use of public transit, decrease use of private motor vehicles (single occupant)

4. What would you say are the expected outcomes? What changes do you expect to see as a result of the ATI?

[DO NOT READ; ASK ONLY IF NOT MENTIONED]

• New or modified policies

• Changes to the built environment

• Increased physical activity levels of participants

5. How has your initiative influenced other environmental or policy changes in your city, county, or State?

6. What would you say are the strengths of the initiative?

• What factors do you think positively affect the functioning of the initiative?

7. If your initiative is aligned with an umbrella organization, what kind of support do they provide to you?

8. What kind of training or support did you receive to perform your role in this initiative?

9. Has the initiative been modified since its implementation? If so, why?

• How was the initiative modified?

III. STAKEHOLDERS OF THE INITIATIVE

1. Which stakeholders (community organizations, residents, government agencies, etc.) are involved with the initiative?

2. How have these stakeholders been involved in the initiative?

• How difficult has it been to involve them?

3. How well do you think the stakeholders (community organizations, residents, government, etc.) know about this initiative?

• What efforts were made to inform these stakeholders?

4. How would you describe the community or population this initiative serves?

• Describe the target population (demographic information).

• Was the initiative adapted or tailored for this community? If yes, how?

5. How receptive does this target audience appear to be to the initiative?

• How has the target audience reacted to it? What sorts of ways have you captured this information?

6. In your opinion, who else (agencies or roles, not names) needs to be involved with the initiative?

7. Over the past year, how much staff turnover has the initiative experienced?

IV. EVALUATION OF THE INITIATIVE

1. For this initiative and what it is trying to accomplish, how would you define success?

• Has the definition of success changed over the course of the initiative? How? Why?

2. Do you think the initiative has been successful?

3. How is success measured or documented? Do you measure outcomes such as number of people using the initiative, weight, BMI, physical activity levels, etc?

4. Describe the plan for measuring the implementation of the initiative.

• What parts of this plan are being carried out at this time?

• What data do you collect?

• Who is responsible for monitoring implementation?

• Are other staff members involved in data collection? How so?

5. What are the available data sources?

• What data-collection tools or instruments are used for your initiative? (Request to see and get copies of data, data-collection tools, or evaluation reports.)

• Describe the data-collection methods used.

• What is the ease of access to data, what file formats are used, etc.?

• What cost is associated with accessing data?

6. How do you analyze your data?

7. What, if any, barriers have you encountered in evaluating the initiative?

8. Who sees your evaluation reports?

• Do they appear to want to hear about evaluation results?

• Do they use any of the information you provide? In what sorts of ways?

9. Has the initiative been modified since its adoption? What changes need to be incorporated into the initiative, if any?

10. What have been some key lessons learned from your experience with the initiative?

11. What additional evaluation are you interested in, if any?

• What questions would you like to answer?

12. If someone was to work with you to conduct a rigorous evaluation, would your program have the capacity to contribute to detailed data collection?

• What kind of assistance do you think you would need?

V. FUNDING OF THE INITIATIVE

1. How is the initiative funded?

2. Is the funding sufficient to support adequate staffing and other resources to carry out the initiative activities?

• If not, what aspects of the initiative are affected by insufficient funding? How?

3. Are there any restrictions on the funding that affect the implementation of the initiative?

4. What is your sense of whether the initiative will continue to receive these resources over time?

VI. CLOSING

Interviewer’s note: If you have not already done so, ask the lead administrator about the draft logic model. Does s/he feel it is an accurate representation of the initiative?

Thank you so much, that wraps up my list of questions for you at this time. Do you have any questions for me? [ADDRESS ANY QUESTIONS]

• If no questions, “Thank you again for taking the time to speak with me. We sincerely appreciate and value your input!”