MDL 2 Teacher Think Aloud Interview

Instrument #2

Instrument #2B

MDL Educator Think-aloud protocol and semi-structured interview

Draft: July 24, 2019

Overview: This survey is intended for the educator partners on the MDL project who co-created the content and will be testing the platform and MDL resources in their classrooms. The goal of the survey is to gather feedback on the ease of platform and product use and content value of the MDL resources for teaching and student learning.

This questionnaire will focus on usability and potential use from the educator perspective.

For this project, we will use screen-recording methods over a telephone interview call so the team can see the exact screens the user interacts with along with their commentary. Participants will be reached by email for an invitation to participate in the interview. The interview will be scheduled for a time that is requested by the educator. Participation in this study requires no additional hardware or materials other than what is normally available in a work/home setting: access to a telephone and access to a laptop or desktop computer and internet access. The interview will take approximately 60 minutes to complete. Once the interview is transcribed by a professional transcription service, the audio and video data will be destroyed.

Recruitment:

Individuals are recruited via an email.

At the pre-scheduled time, initiate Zoom and gain participant consent to record. Ask the participant to open MDL, and share their screen.

How to start the Interview

Our introductory text would begin as such:

“Hi! My name is Kate. Thanks for coming being willing to test the MDL prototype today, we really appreciate it. I’m going to record this conversation, so that we can make improvements to the MDL platform. Is that okay with you?”

Once the participant has consented, ask them to open the MDL prototype, and share their screen.

As we are initiating the conversation, it is useful to start the interviewer with some small talk as an opener. This is a valuable opportunity to begin to build a comfortable rapport, express our sincerity and appreciation for their time, and try to set the participant at ease.

Conducting the Interview

As the participant has donated their time to do the think-aloud, we do whatever it takes to make the process easy and enjoyable for them. Our primary job is to be friendly and accommodating to people. If we succeed in this, people will be more likely to spend time with us and answer our questions. We will make every effort to be natural and comfortable and tailor the interview to suit the individual and/or group. We will also try to find the appropriate language to use.

These questions are meant to serve as guidelines for the interview. After the initial interview questions, these guiding questions will be used to consistently elicit more detailed information regarding a particular participant response. The interviews are meant to be conversational - the point is to make the participant feel comfortable and collect the information we want – how we do that is unique to each interview. We know in our minds what information we need, so we will tailor each interview to flow naturally in terms of actual responses people give.

It is important to emphasize to the participant, that we are interested in testing the prototype, not testing the participant. We really want to know their thoughts to make this an interesting and workable technology! We will also emphasize that we don’t directly work on the site, so they should feel free to tell us all their comments, positive and negative.

To introduce the task, we’ll begin:

“We’re going to ask you to go through a prototype of the MDL project. We’re going to use your thoughts and feedback to improve it before we release to the public. We’d like you to do a think-aloud as you work your way through the prototype. That means we’d like you to say tell you all of your thoughts out loud as you’re working on the prototype, no matter how small or irrelevant they might seem. By having you detail your thoughts in a stream-of-consciousness, think-aloud matter we can think about the prototype with fresh eyes.” Tell them to simply say aloud and all thoughts that are in their head aloud. In addition, we’ll ask them explicitly to tell us when they would have stopped using their phone. The interviewer will model the think aloud behavior with an example, “So I’ve decided to click on this button because it looks like where I should start…”

During the course of the conversation, we will continue to probe “Tell me what you’re thinking now.” and “Do you have any thoughts about what you’re looking at?” At the end of the interview, we’ll ask them to rate overall ease of use, interest, etc.

The interview after the think-aloud will be semi-structured.

It is likely that much of the participants’ general background information will be disclosed during the interview. However, near the end of the interview, we will ask participants some demographic questions, such as the subjects and grades they are teaching, etc. At the end of the interview, we will record this necessary demographic information on the interview sheet as well as the interview number, date, time.

When we are finished reviewing the functions of the site we will thank them, and ask the following questions. All questions but the first are open-ended.

Concluding Semi-Structured Interview

1. On a scale of 1 - 7, where 1 is “not at all” and 7 is “extremely well,” how well do you think the current tool will serve your classroom’s needs?

2. In your opinion, what grade levels can these materials be used effectively at?
   
   

3. How do MDL resources unique compared to other available digital educational resources in terms of….

   1. Clarity?

   2. Ease of use?

   3. Value?

   4. Depth of information?

4. To what extent did MDL…

   1. Seem Interdisciplinary?
      If yes, did that interdisciplinary nature add value to your teaching?
      
      

   2. Align well with Curricular materials?
      
      

5. To what extent do you feel the MDL resources can encourage 21st Century skills in students?

   1. Like skills in critical thinking?

   2. Skills in creativity?

   3. Skills in collaboration?

6. Will MDL as it is now help increase your ability to teach with museum collections-based materials? How so?

7. What suggestions do you have for improvement?

8. How critical is it to make those improvements before more teachers use this system?

9. What other features would be useful?

10. The basis of the MDL platform is to provide access to museum collections, and provide context for learning with museum objects.
    
    In your opinion, does the context that MDL provided add value for teaching and learning? And if so, how?
    
    

11. Was the process of co-creation of the resources a value-added experience for you?
    
    

4