Various Demographic Area Pretesting Activities

General Introduction for AFF Web Site

Thank you for your time today. My name is XX and I work here with the Human Factors and Usability group. I will be working with you today. In this lab, we evaluate how easy or difficult Census products are to use. What works well, we keep. When potential users, such as you, have difficulty with something, we have an opportunity to fix it.

Before we start, there is a form I would like you to read and sign. It explains the purpose of today’s session and your rights as a participant. It also informs you that we would like to videotape the session to get an accurate record of your feedback. Only those of us connected with the project will review the tape and it will be used solely for research purposes. Your name will not be associated with the tape or any of the other data collected during the session.

[Hand consent form; give time to read and sign; sign own name and date, start recording.]

Thank you.

Today, we will be working on a few projects together:

1: First, we’ll do a word recall task. We are going to work on that now. This is a short exercise that gives us information about memory.

[Proctor the memory task—this is in testing docs folder on M drive: 2013Commitment Research Study]

Ok great. I now have a demographic questionnaire that I’d like you to fill out.

[Hand the demographic questionnaire to participant and wait in the room while they fill it out and return it to you]

Thank you. We are ready to begin the main portion of the session. You will help us evaluate some new designs of the American FactFinder Web site by having you work on several tasks. The site is undergoing some tweaks and changes to the interface, and we have been asked to evaluate and give feedback on these changes. The site is not a completely working site. Some of the screens are not working, and so in some of the places where you would click I may ask you what would you expect to happen next, or I may open another screen and have you continue on your task in that way.

If you run into any difficulties, it will be the result of the design of the Web site, not your skills or abilities. This Website is intended for people like you. If you have a problem using parts of it, please don’t blame yourself. We are going to use your comments and experiences as well as comments and experiences of other participants to help improve the site. I did not create the site, so don’t feel like you have to hold back on your thoughts to be polite to me. We appreciate your help so we can make the site work well for everyone.

While you are working, I’d like you to think aloud. In other words, I’d like you to tell me what you are thinking, describe the steps you are taking, what you are expecting to see, why you are doing what you are doing, what you are going to do, and why. Tell me why you clicked on a link or where you expect the link to take you. Tell me if you are looking for something and what it is and whether you can find it or not.

Ok, now we will practice thinking aloud. [open www.craigslist.com and do practice question. Practice can be that they have a friend who moved into a new apartment and asked for help finding a couch. Can also do the how many windows where you live task.]

Ok, that was fine. Do you have any questions about the “think-aloud” process we’ve just practiced and that I have asked you to use?

[Eye tracking]

We are also going to record where you look on the screen as we work on our study. Now we will do a simple task that will allow the computer to find your eyes. I am going to have you position yourself in front of the screen so that you can see your nose in the reflection at the bottom of the monitor. To calibrate your eyes, please follow the dot across the screen with your eyes.

[Calibrate the participants’ eyes]

Now that we have your eyes calibrated, we are ready to begin. Here are your task questions. When we are ready to begin, you will be working with them. Also, here is the questionnaire you will complete at the very end. I will tell you when to complete this.

[Set the task questions and questionnaire by participant.]

I am going to go around to the other room and do a sound check. While I am doing that, please take a moment to complete this computer usage and Internet experience questionnaire. I am going to leave, but we will still be able to communicate through a series of microphones and speakers. Do you have any questions?

[Open computer/Internet experience questionnaire on the participant’s computer. Go into control room.]

[Start video recording. When participant has completed the questionnaire say the following:]

We will begin each task by having you read the task question out load. As you work, remember to talk to me about what you are thinking and feeling. Click on the page where you would click if the page was functioning. Once you have found the information you are looking for please state your answer aloud. For example, say, “My answer is ---” or “This is my final answer.” After each task, I will return you to the page where you can begin the next task.

[Administer tasks.]

Now can you fill out the Satisfaction Questionnaire I left for you on the table? When you are done I will ask you some questions about the Web site.

[Go back into the participant’s testing room. Ask Debriefing questions. Run experiment if still running commitment research study.]

[Give payment.]