Various Demographic Pretesting Activities

Attachment B: Protocol for Testing of the Census Regional Website

Thank you for your time today. My name is XX. I work here with the Human Factors and Usability group and I will be working with you today. We will be evaluating the design of the Census Regional Website by having you complete a series of tasks. Your experience with the website is an essential part of our work. I did not create the website, so please share both your positive and negative reactions to it. We are not evaluating you or your skills, but rather you are helping us see how well the website works. The entire session should last around an hour. Your comments and feedback will be given to the developers of the website and may be used to improve it.

First, I would like to ask you to read and sign this consent form. It explains the purpose of today’s session and informs you of your rights as a participant. It also tells you that we would like to videotape the session, with your permission. Only those of us connected with the project will review the tape and any other data collected during the session. The data will be used solely for research purposes. We may also use clips from the tape to illustrate key points about the website to the Web design team. In addition, there may also be observers from the project team observing this session in another room.

Hand the participant the consent form; give time to read and sign; sign own name and date if you have not already done so.

Start the tape.

While you are completing the tasks, we will record the movements of your eyes with our eye-tracking monitor to get a record of where you are looking on the screen. [note: does not apply for participants in accessibility condition] We will record your mouse movements to see how you are interacting with the website.

I would like you to tell me your impressions and thoughts about the website as you look at it. In other words, I would like you to ``think aloud'' and talk to me about your impressions. If you expect to see some piece of information or expect something to happen, tell me whether or not it was met.

Pull up www.wtop.com in Firefox.

Before we get started, let's practice thinking aloud, since it's not something that you would normally do while working online. Pretend that you wanted to sign up for Alerts from WTOP. Walk me through your thought process as try to sign up.

Ok, that’s exactly what I would like for you to do throughout the session. If at any time during the session you get quiet, I may remind you to talk to me. This is not to interrupt your thought process, but simply to remind you to keep talking to me. Please focus on verbalizing what you are thinking as you complete the survey.

Do you have any questions about the think aloud technique that we just practiced?

Now I am going to calibrate your eyes for the eye-tracking.

After Calibration

Now that we have your eyes calibrated, we are ready to begin. At the beginning of each task we will start the eye-tracking session, and let you know when you can begin the task. Please read the task aloud, and when you believe you have found the answer, please state it out loud. Please avoid using search engines like Google throughout this study.

I am going to go around to the other room to do a sound check. While I am doing that, please take a moment to complete this questionnaire.

[Show participant the Background Questionnaire in Firefox].

I will remain in the other room for the rest of the study and I will let you know when you can begin on the tasks. We will be able to communicate through the microphone and speakers located near the computer.

Leave room. Once in control room do a sound check and Start the eye-tracking software: Tobii Studio. The mouse tracing software will start when Tobii Studio opens Internet Explorer.

Encourage R to think aloud while completing the tasks. Ask probe questions about what they are thinking if they are having trouble.

After the participants complete the tasks

[Show them the QUIS in Firefox and ask them to complete it]

[Go through debriefing questions and ask participants about any unusual behavior or navigation]

Tasks

1. You are interested in becoming a Census Bureau partner, how would you submit your information to the Boston Regional Office?

2. Suppose you lived in the Commonwealth of the Northern Mariana Islands. For security purposes, how would you find out if the Census Field Representative at your door is from the Census Bureau?

3. You are seeking a position as a Field Representative. You wish to find what qualifications are needed and the workload hours in Puerto Rico.

4. Locate the phone number for the partnership and data services staff contacts.

5. How would you request a workshop at the Philadelphia Regional Office Website?

6. Find the directions to the Charlotte Regional Office via their Website.

7. You are interested in viewing data from the 1980 Census and are wondering if you need an appointment to see the microfiche data. Where would you find this information?

8. After submitting your information to the Boston Regional Office, you wish to update your partner information. How would you do it?

9. You are interested in learning how many people live in Rockville, MD in the year 2000. How would you find this information?

10. You have a complaint and wish to speak to the director of the Los Angeles Regional Office. What is this person’s name and telephone number?

11. You wish to find out the population of the state of Kansas. How would you find it on the Website?

Debriefing Questions

On any Regional Office Website:

• Did you see the Facebook and Twitter icons?

• Did you think they were clickable?

• Do you use services like Facebook and Twitter?

  • If yes: Would you follow/friend the Regional offices to get updates?

    • How do you use Facebook/Twitter?

    • What would you expect from Facebook/Twitter interaction?

On the “Local Jobs” Web page:

• Imagine you were looking for a job as a Field Representative. Would you prefer listings by Region or by State?

  • Do you expect the jobs listed by state to be specific to that state?

  • What if the jobs listed by state required traveling out of state?

• Imagine you are looking at this webpage during the peak of the Census. During this time, a lot of job listings would be on this webpage. Would you prefer jobs to be listed by region or by state?

• [Under the California and Seattle Region Local Jobs tabs], the state pages for Los Angeles and Seattle both show California jobs.

  • How can this information be indicated to show the difference?

  • Would seeing California listed in both regional offices be confusing?

• There will be job postings that link directly to announcements on the USAjobs website. Is the related text currently listed sufficient to inform site visitors?

On the “Partner Info” Web page:

• When updating the partnership information, did you notice the similarity to the original form?

  • If yes: What do you think of the similarity?

How often do you request facts about your community?

Would images placed along with the descriptive text on the page be helpful for navigating through the Website?

What would users expect on the home page of a Census Regional Website?

• In the Upcoming Events, what would you expect to see here?

  • Where do you think these events are located?