Various Demographic Area Pretesting Activities

General Introduction for AFF Web Site

Thank you for your time today. My name is ------. I work here in the U.S. Census Bureau Usability Lab, and I will be working with you today. In this lab, we evaluate how easy or difficult Census products are to use. We bring in people like you who are potential users of our products to try them out while there is still time to make changes to them. What works well, we keep. When potential users such as you have difficulty with something, we have an opportunity to fix it.

Today, we will be evaluating the use of language on the Census web site. We will do this by having you work on several tasks. After each task we will have you answer a few satisfaction questions about the site.

There are two parts to our session. First, you will complete several tasks using the Web site. Then, at the end of the session, we’ll ask you some questions about the site and you’ll fill out a questionnaire about your experience during the session.

Before we start, there is a form I would like you to read and sign. It explains the purpose of today’s session and your rights as a participant. It also informs you that we would like to videotape the session to get an accurate record of your feedback. Only those of us connected with the project will review the tape and it will be used solely for research purposes. Your name will not be associated with the tape or any of the other data collected during the session.

[Hand consent form; give time to read and sign; sign own name and date.]

Thank you.

Any difficulties you may run into reflect the design of the Web site, not your skills or abilities. This product is intended for people like you. Where it works well, that’s great. And where it does not work well—that is important for us to learn. We are going to use your comments and data as well as comments and data from the other participants to give feedback to the developers of the site. Your comments and thoughts will help the developers as they continue to make changes to improve the site. I did not create the site, so don’t feel like you have to hold back on your thoughts to be polite. We are not evaluating you or your skills, but rather you are helping us see where the site does and does not work.

I am going to give you some tasks to work on. Your comments are very important to us. I’d like you to tell me your impressions and thoughts as you work through the tasks. So give me your open impressions, both good and bad of what you see and what you experience on the site. We call this thinking out loud. So basically, I’d like you to tell me what you are thinking, describe the steps you are taking, what you are expecting to see, why you are doing what you are doing, what you are going to do, and why. Tell me why you clicked on a link or where you expect the link to take you. Tell me if you are looking for something and what it is and whether you can find it or not.

Ok, now we will practice thinking aloud. [open www.craigslist.com and do practice question.]

Ok, that was fine. Do you have any questions about the “think-aloud” process we’ve just practiced and that I have asked you to use?

In addition to the tasks I am about to give you, we are also going to do some eye tracking to record where you are looking on the screen. In a moment we will do a very short simple task that will allow the computer to find your eyes. I am going to have you position yourself in front of the screen so that you can see your nose in the reflection at the bottom of the monitor. To calibrate your eyes, please follow the dot across the screen with your eyes.

Now that we have your eyes calibrated, we are ready to begin. Here are your task questions. When we are ready to begin, you will be working with them. Also, here is the questionnaire you will complete at the very end. I will leave it here in the corner of the desk and I will tell you when to complete this.

[Set the task questions and final questionnaire by participant.]

I am going to go around to the other room and do a sound check. While I am doing that, please take a moment to complete this computer usage and Internet experience and this Demographic questionnaire. I am going to leave, but we will still be able to communicate through a series of microphones and speakers. Do you have any questions?

[Open computer/Internet experience questionnaire on the participant’s computer and leave the paper Demographic questionnaire with them. Go into control room.]

[Start video recording. When participant has completed the two questionnaires say the following:]

We will begin each task by having you read the task question out load. As you work, remember to talk to me about what you are thinking and feeling. Click on the page where you would click if the page was functioning. Once you have found the information you are looking for please state your answer aloud. For example, say, “My answer is ---” or “This is my final answer.” After each task, I will return you to the page where you can begin the next task.

[Administer tasks. After each task open up the post-task satisfaction questions. At the end of the last task say:]

Now can you fill out the Satisfaction Questionnaire I left for you on the table? When you are done I will come around to ask you some questions about the Web site.

[Go back into the participant’s testing room. Ask Debriefing questions.]

Thank you. That concludes our session.

[Give payment.]