Census Business Builder Online Tool

Usability Testing Protocol - CBB

Census Business Builder Tool

Thank you for your time today. My name is XX and I work here with the Human Factors and Usability group at the Census Bureau. I will be working with you today. Our team evaluates how easy or difficult Census products are to use. What works well, we keep. When potential users, such as you, have difficulty with something, we have an opportunity to fix it before it goes live to a much larger group.

Before we start, there is a form I would like you to read and sign. It explains the purpose of today’s session and your rights as a participant. It also informs you that we would like to audio record the session to get an accurate record of your feedback. Only those of us connected with the project will review the recording and it will be used solely for research purposes. Your name will not be associated with the recording or any of the other data collected during the session.

[Hand consent form; give time to read and sign; sign own name and date, start recording.]

Thank you.

I now have two questionnaires that I’d like you to fill out. One collects some information on your demographics and one collects information about how often you use different electronic devices.

[Have participant fill out the demographic questionnaire and the background questionnaire. Move away from where they are answering so they don’t feel like we are looking over their shoulder.]

Thank you.

Today you will be helping us evaluate the design of the Census Business Builder by using it to complete a set of task questions. This tool was created to assist current and prospective small business owners and small business advisors.

If you run into any difficulties as you work on the tasks, please don’t blame yourself. Any difficulties are the result of the design of the web tool, and not your skills or abilities. We are going to use your comments and experiences as well as comments and experiences of other participants to help improve the tool. I did not create the tool, so don’t feel like you have to hold back on your thoughts to be polite to me. We appreciate your help so we can make the online survey work well for everyone.

Think Aloud: I want you to answer the questions exactly the way you would if you were at your home, but with one major difference. I would like you to think aloud as you answer the questions. I am interested in your answers, but I am also interested in the process you go through in your mind when you use the tool. I would like you to tell me everything that you are thinking and feeling as you go about using the tool. For example, tell me how you expect the tool to act, or if you are looking for links and buttons.

Practice: Let’s do a practice task before we start: Please think aloud as you try to find a sofa on Craigslist.

[Bring up Craigslist, allow them to start practice task. Give feedback on their thinking aloud. If they just gave a number have them do it again while thinking aloud. If they still don’t get it, explain how they could have done it]

Great that’s what I want you to do throughout our session. I will remind you to think aloud if you get quiet.

At the end of the session I will have some questions for you about your experience as you worked on specific screens and how satisfied you were with the tool. We may open up some of the screens and talk about them.

[Eye tracking]

We are also going to record where you look on the screen as we work on our study. Now we will do a simple task that will allow the computer to find your eyes. I am going to have you position yourself in front of the laptop as you would normally sit.

When I click the start button, a red ball will move across the screen. Follow it with your eyes, without moving your head too much.

[Calibrate the participants’ eyes]

Thank you. We are ready to begin the main portion of the session. You will help us evaluate the design of the Census Business Builder by completing several tasks. Before you begin each task, please read the question aloud. After you have read the question, I will bring up the tool that we would like you to use to answer the task question. When you have found your answers, please tell me “My answer to question X is…(answer)”. That is how I will know that you feel you have completed the task.

After tasks, have participant answer satisfaction questions.

Now I would like you to answer these satisfaction questions. (Hand paper questionnaire to participant.)

Debriefing: Now I’ll ask you some debriefing questions that I have for you.