EEP Evaluation Followup Survey

ANALYTICAL AND TECHNICAL SUPPORT FOR ADVANCING EDUCATION EVALUATIONS

Experts Bringing Evidence to Practitioners (EEP)

6 Month Follow-Up Interview Protocol

INTERVIEWER OVERVIEW:

• The interview will be framed as a discussion of EEP attendees’ work and how over the 6-months since the event they have or have not used the information that was presented. The primary goal of the interview is to determine if the information presented at EEP events is perceived as relevant to attendees’ work and useful for their practice or decision making.

• EEP attendees selected for these interviews may have already participated in a shorter, close-ended online survey. The shorter survey was conducted within a few weeks of the EEP and was intended to get an immediate assessment of the perceived relevance and usefulness of the information presented at the event. The 6-month interview is designed to obtain more in-depth information such as why the EEP presentation was relevant and how those who used the information applied it to their work or decision making.

• When the appointment was made for the EEP attendee to do the interview, he/she was given background information that you can refer to as needed.

I. INTRODUCTION

A. Greeting

Hello, My name is (NAME) and I am from Mathematica Policy Research, Inc.

As we described in the information you received about this interview, we’d like to talk with you for about 20-30 minutes to learn about the work that you do and whether or not you have been able to apply to that work anything you learned at [NAME OF EVENT] you attended [DATE OF EEP] in [EEP LOCATION].

Note: The recruitment and appointment confirmation process for the interviews will include background information about the purpose of these interviews, IES, NCEE, and the ATS team. Therefore, the interviewer should not need to take time during the interview to provide this information unless asked. The interviewer will have all the contact documentation related to the interview appointment so he/she will know exactly what information the respondent has prior to the call. If the respondent completed the Time 1 questionnaire, we may also have that information to give the interview some additional background prior to the phone contact.

IF NEEDED

-- To give you a bit of background about this project, Mathematica is conducting this study on behalf of the National Center for Education Evaluation (NCEE) of the Institute of Education Sciences (IES) at the U.S. Department of Education. The goal of this study is to learn about who attends the Regional Education Lab (REL) EEP events and how they use the information presented at the event. IES is mandated by Congress to collect this information.

-- You may have completed an online questionnaire for us a few weeks after the (NAME/LOCATION OF EVENT ATTENDED) about for your immediate impressions of the event and the information presented. The purpose of this interview is to give you an opportunity to provide more in-depth information about how, or if, you have used the information from this EEP event during the last 6 months.

B. Confidentiality and Taping

All of the information collection in these interviews will be kept confidential to the extent permitted by law. Our report to IES will reflect key themes that emerged, and will not connect the names of any individuals with information from their interview. In some cases we may use verbatim comments so the report will reflect the theme in the words of those we interviewed, but no names will be connected to these comments.

Permission: I would like to have your permission to audio tape our interview so I can make sure that we have an accurate record of your comments. Is that okay with you?

IF RESPONDENT DOES NOT GIVE PERMISSION TO AUDIOTAPE:

Let’s continue with the interview without the audio tape.

II. ATTENDEE PROFILE (5-10 MINUTES)

I’d like to begin by learning more about you and the educational work you do.

A. Professional experience

1. To make sure I have the correct information, can you tell me your title, and the organization that you work for?

PROBE AS NEEDED:

a. About how large is the organization/school district where you work? What’s your best estimate of the total number of people who work for your organization?

b. And, about what percent of them are involved with or use education research?

2. On a day-to-day basis, could you describe your one or two main job responsibilities?

3. Prior to your current position, what education-related positions have you held?

4. And, about how long have you been in this field?

B. Use of education research

1. In general, how is education-related research relevant to your daily work? What kind of education information (reports, etc.) is most useful to you?

PROBE AS NEEDED: For example, in the past year is there any education research you’ve referred to that helped you to make a decision, handle a management issue, or assist with educational instructions?

2. When you need to refer to education research what one or two places are you’re most likely to go to find it? Any others?

a. How familiar are you with What Works Clearinghouse?

b. Fast Response Reports?

c. Other NCEE research or products?

3. Is there any research you need to assist you in your work that, to your knowledge, is not available? Or that you think needs to be improved?

III. EEP EVENT (10-15 minutes)

A. Overview

1. How did you find out about the EEP event you attended?

2. Could you tell me about the main reason you decided to attend this event?

B. Relevance of EEP Information

1. Overall, was the EEP event subject [NAME OF EVENT] relevant to the work you do or not?

PROBE AS NEEDED: Could you tell me more about why it [was/was not] relevant?

C. Perception and Use of EEP Information

1. Thinking about the six months that have passed since you attended the EEP event, during that time have you had the opportunity to use any of the information that was presented there?

IF NO: Please tell me more about why you have not used any of this information?

PROBE AS NEEDED:

a. Was the information difficult to understand?

b. Was the information difficult to use?

c. Is the information not easily accessible?

d. Was the format a problem?

e. Was the information not relevant to your work?

GO TO SECTION IV

IF YES: Please tell me more about when and how you used this information? Any other ways?

PROBE AS NEEDED:

Did you use this information to:

a. Inform others about new developments and research results in this field?

b. Help you better evaluate research studies and their findings?

c. Develop a new standard for best practices in your workplace?

d. Access new resources that will assist in your daily work such as the WWC website (for example, research reports and practice guides)?

2. Who did you share it with?

3. How did you share this information?

PROBE AS NEEDED:

a. Did you make a presentation? Send out notifications about the information? Distribute copies? Provide URL or website information? Any other ways?

b. Is there another format that could be used for this information that would make it easier to distribute to others?

4. Overall, about how often have you found the information useful to something you were doing?

PROBE AS NEEDED:

a. Was there anything about the presentation that made it particularly useful to your work?

b. Was there anything in particular that made the information presented less useful to your work?

5. Overall, were you satisfied or not with the information presented at the EEP event?

PROBE AS NEEDED: Could you please tell me more about why you were [satisfied/dissatisfied]?

IV. FUTURE EEP EVENT SUGGESTIONS (5-10 minutes)

1. Have you attended, or registered for, another EEP event since the one you went to six months ago?

PROBE AS NEEDED:

a. Could you tell me more about why you [did/did not] attend another EEP event?

b. Do you have any suggestions that might increase your likelihood of attending future events?

2. Have you suggested to others that they should attend EEP events?

PROBE AS NEEDED: Could you tell me more about why you [did/did not] suggest to others that they should attend EEP events?

3. What suggestions do you have to make EEP events more useful and applicable to your daily work?

PROBE AS NEEDED: How would you improve the EEP event topics, format of the presentations, or materials that are distributed?

4. Do you see an area of need in your field where future presentations would be useful?

5. Anything else you’d like to tell me about your experience with EEP events or the research and materials presented at these events?

Thank you very much for your assistance.

4