Various Demographic Area Pretesting Activities

PROTOCOL

General Introduction for 2013 Follow-up to Baseline AFF Web Site

[Researcher: Click on the Start Time link. It should take you to this screen (below) Fill this out as soon as participant enters the testing room.]

[Read to participant]

Thank you for your time today. My name is ------. I work here in the U.S. Census Bureau Usability Lab, and I will be working with you today. In this lab, we evaluate how easy or difficult Census products are to use. What works well, we keep. When potential users, such as you, have difficulty with something, we have an opportunity to fix it.

Today, we will be working on a few projects together. There are three parts to our session:

1: First, we’ll do a word recall task.

2: Second you will work on completing some tasks using our web site. We’ll ask you some satisfaction questions and some final debriefing questions

3: Third you will work on a set of questions.

Then we’ll wrap up and I or my colleague will escort you back out.

Before we start, there is a form I would like you to read and sign. It explains the purpose of today’s session and your rights as a participant. It also informs you that we would like to videotape the session to get an accurate record of your feedback. Only those of us connected with the project will review the tape and it will be used solely for research purposes. Your name will not be associated with the tape or any of the other data collected during the session.

[Hand consent form; give time to read and sign; sign own name and date.]

Today, we will be evaluating the American Factfinder Web site. If you run into any difficulties, it will be the result of the design of the Web site, not your skills or abilities. This Website is intended for people like you. If you have a problem using parts of it, please don’t blame yourself. We are going to use your comments and experiences as well as comments and experiences of other participants to help improve the site. I did not create the site, so don’t feel like you have to hold back on your thoughts to be polite to me. We appreciate your help so we can make the site work well for everyone.

While you are working, I’d like you to think aloud. In other words, I’d like you to tell me what you are thinking, describe the steps you are taking, what you are expecting to see, why you are doing what you are doing, what you are going to do, and why. Tell me why you clicked on a link or where you expect the link to take you. Tell me if you are looking for something and what it is and whether you can find it or not.

Ok, now we will practice thinking aloud.

[Open www.craigslist.com and do practice question. Practice can be that they have a friend who moved into a new apartment and asked for help finding a couch. Can also do the how many windows where you live task.]

Ok, that was fine. Do you have any questions about the “think-aloud” process we’ve just practiced and that I have asked you to use?

[For eye tracking read the following:]

In addition to the tasks I am about to give you, and thinking aloud that we just practiced, we are also going to record where you look on the screen. In a moment, we will do a simple task that will allow the computer to find your eyes. I am going to have you position yourself in front of the screen so that you can see your nose in the reflection at the bottom of the monitor. To calibrate your eyes, please follow the dot across the screen with your eyes.

[Calibrate the participants’ eyes]

Ok great. I now have a demographic questionnaire that I’d like you to fill out.

[Hand the demographic questionnaire to participant and wait in the room while they fill it out and return it to you—this way they will not be confused by the Q6 in the research experiment]

Now I would like us to work on the word recall task. This is a short exercise that gives us information about memory.¹

[Proctor the memory task—this is in testing docs folder on M drive: 2013Commitment Research Study]

Thank you. Now I am going to go around to the other room and do a sound check. While I am doing that, please take a moment to complete this Background Questionnaire. I am going to leave, but we will still be able to communicate through a series of microphones and speakers. Do you have any questions?

Here are your task questions. When we are ready to begin, you will be working with them.

[Set task questions down by participant]

[Open Background Questionnaire, location M:\USABLE\2013 Commitment Research Study\Questionnaire Links and go into control room.]

[Start video recording.]

We will begin each task by having you read the task question out load. As you work, remember to talk to me about what you are thinking and feeling. Once you have found the information you are looking for please state your answer aloud. For example, say, “My answer is ---” or “This is my final answer.” After each task, I will return you to the page where you can begin the next task.

[Administer the tasks, after each task, administer the post task questions—set of three question.]

Please answer these three short questions about your experience with the Web site during the last task.

[At conclusion of last task and last post task question, open the final Satisfaction Questionnaire and say:]

Now can you fill out the Satisfaction Questionnaire? When you are done I will come around to ask you some questions about the Web site.

[Ask general Debriefing questions.]

[After debriefing is complete-- Now proctor the research component]

I just need to step out and get something.

[Exit the room to retrieve and open the randomly selected envelope.

If envelope says “C” go back into participant’s testing room and say:]

Now it’s time for our final set of questions. Please answer these questions, I’ll be in the other room.

[Open the link FQ Questionnaire—located at (M:\USABLE\2013 Commitment Research Study\Questionnaire Links) Then leave the room.]

[After completion of 10 questions, go back into testing room and Give payment.]

If the envelope says “T” go back into the participants testing room and run treatment condition,

Now it’s time for our final set of questions. When you answer the questions that you’ll see on the following screens, it’s very important that you take your time and think carefully so that you answer each question as accurately as possible.

Are you willing to commit to answer each question as accurately as you can?

[After the respondent says yes, the test administrator will lay down the commitment paper]:

Please read this over, check this box,

[point to the box and the paper that you will be having them sign]

and sign your name if you are willing to commit to answer each question as accurately as you can.

[Once receiving the commitment and signature, Open the link FQ Questionnaire—located at (M:\USABLE\2013 Commitment Research Study\Questionnaire Links) Then leave the room.]

[After completion of 10 questions, go back into testing room and Give payment.]

1 This is part of the experimental research component and was approved my OMB September 5, 2012..