REL Peer Review: Pilot Data Collection Methods for Examining the Use of Research Evidence

APPENDIX B. FOLLOW-UP INTERVIEW PROTOCOL

This page has been left blank for double-sided copying.

OMB Number:

Expiration Date:

REL Peer Review

Follow-Up Interview Protocol

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. The valid OMB control number for this information collection is [XXXX]-[XXXX]. The time required to complete this information collection is estimated to average 30 minutes. If you have any comments concerning the accuracy of the time estimate or suggestions for improving this form, please write to: U.S. Department of Education, Washington, DC 20202. If you have comments or concerns regarding the content or the status of your individual submission of this form, write directly to: U.S. Department of Education, Institute of Education Sciences, 550 12th Street, SW, Washington, DC 20202.

Introduction

Thank you for meeting with [me/us] today. My name is [INTERVIEWER NAME], and I am joined today by [NOTETAKER NAME], and we are both from Mathematica.

The U.S. Department of Education’s Institute of Education Sciences (IES) is sponsoring a study to help the Regional Educational Laboratories (RELs) learn more about the use of research evidence among REL partners and help IES assess the validity and reliability of the survey items to be used in future partner feedback surveys.

We are speaking to a variety of REL partners who completed the online survey a few months ago to better understand how your responses to the survey map to your actual practices. This will help to confirm whether the survey questions are a valid proxy for actual behaviors and help us better understand any obstacles you may have encountered when trying to use REL evidence and resources. Today’s conversation should last about 30 minutes.

Your answers in this interview will be used only for research purposes. Your participation is voluntary; if there are any questions you do not feel comfortable answering, we can skip that question and move on to the next. The memo prepared for this study will not associate interview responses with a specific individual. The study will not disclose the names of individual participants, except as required by law. Mathematica follows the confidentiality and data protection requirements of the U.S. Department of Education’s Institute of Education Sciences (IES) (The Education Sciences Reform Act of 2002, Title I, Part E, Section 183).

With your permission, we’d like to record the conversation to help us keep accurate notes. The recording will be securely stored and available only to the study team working on the study. Do we have your permission to record? [IF YES: Great, thank you. START RECORDING] [IF NO: No problem, we understand.]

Do you have any questions before we begin? [ANSWER ANY QUESTIONS]

Note for OMB Review: Questions 2a-2d and 3 will be modified slightly before each interview, based on the respondent’s previous answers in the REL Use of Research Evidence (URE) survey. Question 4 will be modified based on the educational role of the respondent. Questions 5 and 6 are optional questions that may be asked, if time allows, to help unpack potential reasons behind respondents’ answers to previous survey questions. The series of questions we ask each respondent will take no more than 30 minutes total, and we do not expect to ask the exact same questions to more than 9 people.

Interview Questions

1. We’d like to begin with brief introductions. Could you please introduce yourself and describe your role as a REL partner on [REL Partnership]?

2. You responded in the REL Use of Research Evidence (URE) survey that [insert description from survey response].

2a. Can you give an example of how you did that in the past [X] months?

• For example: You responded in your URE survey that you often apply research evidence produced by RELs when deciding about the content or direction of a policy or program. Can you give an example of how you did that in the past 3 months?

2b. [If the study team finds out that a respondent was unable to do this in the past 3 months, we will ask the following question]. Were there any factors outside of your control that impacted your ability to do this?

• For example: Were there any factors outside of your control that impacted your ability to apply research evidence produced by RELs when making decisions about policies or programs?

2c. What types of actions might you continue to do in order to [insert description from survey response]?

• For example: What might you do to help facilitate application of research evidence produced by RELs when making decisions about policies or programs?

2d. What additional supports would be needed?

3. You responded in your URE survey that [insert description from survey response]. Can you say more about why you answered the question this way?

• For example: You responded in your URE survey that you strongly disagree that REL research evidence and services are relevant to the learners and communities you serve, but you also answered that you strongly agree that REL engagement is relevant to your work. Can you say more about why you answered these two questions this way?

4. We are interested in how you use research evidence in your ongoing work.

• Have you embedded research evidence into the tools and resources you create? If so, can you give an example? (Here we would probe to gather information on the tool/resource as well as the evidence on which it is based; examples of tools and resources could be professional development slide decks or approaches used to train on instructional strategies)

• For respondents who say they do not create tools: Is research evidence embedded into the tools or strategies you use (such as tools used with students in the classroom)? If so, can you give an example?

• Are you able to share other examples of how you use research evidence in your day-to-day work? Can you think of a specific piece of research you turn to?

5. Optional question: Do you feel that your perspective and expertise was valued by your REL partnership? Why or why not? Can you give an example?

6. Optional question: Did you perceive any power imbalances in your REL partnership? Which routines and norms of communication, if any, do you think enabled that imbalance?

Wrap-Up

That concludes our interview questions. Thank you for your time. Do you have anything else you would like to add or any questions for us?