Rapid Formative Testing Protocol

Rapid Formative Testing Protocol

Materials Needed for Interview

• Interview Protocol Script

• Morae testing tool

• Consent Form

• Demographic Questionnaire

• Mobile Use Questionnaire

• Task List

• SUS Survey

• Laptop or Mobile Phone

• Incentive

• Payment Receipt

• Pens and Paper

• Extra Equipment and Charging Chords

Usability Testing Script

Thank you for your time today. My name is XX and I work with the Human Factors and Usability Group at the U.S. Census Bureau. I will be working with you today. In this lab, we evaluate how easy or difficult it is to use our online website and web survey products. What works well, we keep. When potential users, such as you, have difficulty with something, we have the opportunity to fix it before it goes to a much larger group.

Before we start, there is a form I would like you to read and sign. It explains the purpose of today’s meeting and your rights as a participant. It also informs you that we would like to take a video recording of the device as use it along with the audio from this session to get an accurate record of your feedback. Only those of us connected with the project will review the recording or any of the other data collected during the session.

[Hand consent form; give time to read and sign; sign own name and date.]

[Have the participant fill out the background and demographic questionnaire and the mobile experience questionnaire. You can leave the room and watch from the observation room when they are ready to begin. If off-site move away from where they are answering so that participants don’t feel like we are looking over their shoulder.]

Thank you.

OK great.

Today you will be helping us evaluate the design of a website tool that we are developing. You will see on the screen a series of tasks which you will use the tool to help you find and answer. If you run into any difficulties as you work with the tool, please don’t blame yourself. Any difficulties are the result of the design of the tool you are using, not your skills or abilities. We are going to use your comments and experiences as well as comments and experiences of other participants to help improve the tool you are going to use.

Think Aloud: Let’s do a practice task before we start. Please think aloud as you answer the question, how many windows are in your home?

[Start Morae recording. Explain controls for the pop up scenario window]

[Probe: if they fall silent, if the just give a number ask them to do it again but this time how they came up with the number; after they do they think aloud explain why we are asking to do this, e.g. if the Census was interested in the number of windows, which we are not, but if we were and someone was looking at their sliding glass door and wondered aloud that they didn’t know if they should include it or not, them saying those words would clue us in that we needed to fix our question, make it clear that sliding doors should be included, etc.]

Probes that will be used in the study:

• Keep talking

• Um-hum?

• What are you thinking?

• Tell me more

Great that’s what I want you to do throughout our session. I will remind you to think aloud if you get quiet.

At the end of the session, I will have some questions for you about your experience as you worked on the tasks. You will be given a quick survey at the end of each task asking you how easy or hard you felt it was to complete the task.

We may open up some of the screens and talk about them.

We will begin each task by having you read the task out loud. Once you are ready to begin the task click on “Start Task” to begin the task. As you work, remember to talk to me about what you are thinking and feeling. Click on the area where you would click to find the information you are looking for. Once you have found the information you are looking for, click on “End Task” in the scenario window. You will then be given a quick survey asking how difficult or easy it was to complete the task. You will then be ready to begin the next task.

[Administer tasks. When all tasks are finished the SUS survey will automatically open.]

Now you can fill out the SUS Questionnaire.

[Step away from the computer as participant fills out the SUS survey]

When you are done, I will ask you some questions about the tool you just used.

Debriefing on Individual Web components/screens?

Do you have any comments on this screen?

[If there were issues with it:] Tell me more about your experience with this screen.

Additional Briefing Questions

1. What is your general reaction to the site?

2. What did you like best about the site?

3. What did you like least about the site?

4. Is there anything that you feel is missing on this site?

5. If you were to describe this site to a colleague, what would you say?

6. Do you have any other final comments or questions?

Finishing up session

[Give the respondent incentive envelope and thank them for their time and effort. Collect materials and escort respondent out of the interviewing room.]

Okay, that’s all that I have today. Many thanks for your work on this project. The last thing I need you to do is sign this form that says I am giving you this incentive as a thank you for your participation.