Various Demographic Pretesting Activities

General Introduction for ACS Usability Test.

Thank you for your time today. My name is (Test Administrator). I work here in the U.S. Census Bureau Usability Lab, and I will be working with you today. In this lab, we evaluate how easy or difficult Census products are to use. We bring in people like you who are potential users of our products to try them out while there is still time to make changes to them. What works well, we keep. When potential users such as you have difficulty with something, we have an opportunity to fix it.

Today, we will be evaluating the American Community Survey Web site by having you work on several tasks. I will give you specific task questions, and you will complete the tasks by using the ACS Web site. I may ask you more questions as we go on. The entire session should last about an hour.

Before we start, here is a form I would like you to read and sign. It explains the purpose of today’s session and your rights as a participant. It also informs you that we would like to videotape the session to get an accurate record of your feedback. Only those of us connected with the project will review the tape and it will be used solely for research purposes. Your name will not be associated with the tape or any of the other data collected during the session.

[Hand consent form; give time to read and sign; sign own name and date.]

Thank you.

Before we start, I want to tell you that you can’t make a mistake or do anything wrong here. Difficulties you may run into reflect the design of the Web site, not your skills or abilities. If you have a problem using parts of it, do not blame yourself. This product is intended for people like you. Where it works well, that’s great. Where it does not work well that is also great, because you will be able to help us identify things that can be corrected.

Your comments are very important to us. I’d like you to tell me your impressions and thoughts as you work through the tasks. So give me your open impressions, both good and bad of what you see and what you experience on the site. We call this “thinking out loud.” In other words, I’d like you to tell me what you are thinking, describe the steps that you are taking, what you are expecting to see, why you are doing what you are doing, what you are going to do, and why. Tell me why you clicked on a link or where you expect the link to take you. Tell me if you are looking for something and what it is and whether you can find it or not. I will be here to help if you get stuck.

Ok, now we will practice thinking aloud. [open www.craigslist.com and do practice question.]

Ok, that was fine. Do you have any questions about the “think-aloud” process we’ve just practiced and that I have asked you to use?

In addition to the tasks I am about to give you, we are also going to do some eye tracking to record where you are looking on the screen. In a moment we will do a very short simple task that will allow the computer to find your eyes. I am going to have you position yourself in front of the screen so that you can see your nose in the reflection at the bottom of the monitor. To calibrate your eyes, please follow the dot across the screen with your eyes.

Now that we have your eyes calibrated, we are ready to begin. Here are your task questions. When we are ready to begin, you will be working with them. Also, here is the questionnaire you will complete at the very end. I will tell you when to complete this.

[Set the task questions and questionnaire by participant.]

I am going to go around to the other room and do a sound check. While I am doing that, please take a moment to complete this computer usage and Internet experience and this Demographic questionnaire. I am going to leave, but we will still be able to communicate through a series of microphones and speakers. Do you have any questions?

[Open computer/Internet experience questionnaire on the participant’s computer and leave the paper Demographic questionnaire with them. Go into control room.]

[Start video recording. When participant has completed the two questionnaires say the following:]

We will begin each task by having you read the task question out load. As you work, remember to talk to me about what you are thinking and feeling. Click on the page where you would click if the page was functioning. Once you have found the information you are looking for please state your answer aloud. For example, say, “My answer is ---” or “This is my final answer.” After each task, I will return you to the page where you can begin the next task.

[Administer tasks.]

Now can you fill out the Satisfaction Questionnaire I left for you on the table? When you are done I will ask you some questions about the Web site.

[Go back into the participant’s testing room. Ask Debriefing questions.]

Thanks. That concludes our session.

[Give payment.]