Evaluator Interview Protocol

Evaluator Interview Protocol

Personal Background

1) Please tell me about yourself
a. What is your current role? How long have you had this position?
2) When did you start working with the ITEST project (e.g., in the design phase, after it was
operational)?
a. Relationship with the project staff?

Evaluation and Outcomes
3) Tell me a little bit about the planning process for the evaluation?
a. (If involved in design phase) Was evaluation a significant concern during the planning of
the project?
b. How did you determine the focus of the evaluation?
c. What were your considerations when designing the methodology?
4) What are the purposes of the evaluation (e.g. formative, summative)?
5) Can you describe the design of your evaluation? (e.g., quasi-experimental, qualitative with
control group, etc)
a. What are your evaluation questions?
b. What types of data do you collect? Probe for:
i.
Surveys
ii.
Focus groups/interviews
iii.
Knowledge assessments
iv.
Attitude assessments
v.
Standardized test scores
vi.
Observation data
vii.
Student records (including future enrollment)
c. Have you developed your own instruments?
i.
If so, have you shared those with the ITEST communities?
ii.
If no, where did you get your instruments?
d. How often are data collected?
6) Have there been any challenges in collecting data? (e.g., if project involves external
partners, have they been cooperative with the evaluation; if evaluation involves a control
group, has it been easy or difficult to identify and collect data for them)
7) Do you plan on making (or have you already made) changes to your evaluation design?
Why? Please describe the changes.
8) At this point in the project, do you have data on the outcomes of interest? What have you
found?
9) How do you share results with project staff?
32

a. Do you share your results more widely (e.g., web site, conferences, publications)
10) Do you see any evidence that the project is using the evaluation results?
a. Give an example.

The ITEST Community
11) To what extent do you work with, share advice, or otherwise interact with other ITEST
project evaluators?
a. What have you gained, if anything, from these interactions?
12) To what extent do you as an evaluator interact with the LRC?
a. What is the nature of most of your interaction with the LRC?
b. What have you gained, if anything, from the LRC?

33