Various Demographic Area Pretesting Activities

2013 Small-Scale Non-id Panel Test Usability Study

[Researcher: Click on the Start Time link. It should take you to this screen (below) Fill this out as soon as participant enters the testing room. Also, mark which version of the screens the respondent uses first and which post-task questionnaire the respondent will see based on random order assignment.]

Version of the study shown first :____

Version of the post-task questionnaire:

[] 4-question one

[]2-question one

Thank you for your time today. My name is XX and I work here with the Human Factors and Usability group. I will be working with you today. In this lab, we evaluate how easy or difficult Census products are to use. What works well, we keep. When potential users, such as you, have difficulty with something, we have an opportunity to fix it.

Before we start, there is a form I would like you to read and sign. It explains the purpose of today’s session and your rights as a participant. It also informs you that we would like to videotape the session to get an accurate record of your feedback. Only those of us connected with the project will review the tape and it will be used solely for research purposes. Your name will not be associated with the tape or any of the other data collected during the session.

[Hand consent form; give time to read and sign; sign own name and date, start recording.]

Thank you.

Today, we will be working on a few projects together:

1: First, we’ll do a word recall task. We are going to work on that now. This is a short exercise that gives us information about memory.

[Proctor the memory task—this is in testing docs folder on M drive: 2013Commitment Research Study]

Ok great. I now have a demographic questionnaire that I’d like you to fill out.

[Hand the demographic questionnaire to participant and wait in the room while they fill it out and return it to you]

Thank you. We are ready to begin the main portion of the session. The first thing you will do is provide us some feedback on email messages that someone might receive in their inbox notifying you of a Census study.

Hand the participant the Version A of the email message and have them read it:

Take a minute to read this.

What are your initial thoughts or concerns about this email? Would you open the message if you saw it in your inbox? Why or why not?

Would you click on the link in the email to complete it? Why or why not?

Now let’s look at another version of the email. Only the subject line has changed.

Hand the participant the Version B of the email message and have them read it:

Take a minute to read this.

What are your initial thoughts or concerns about this email? Would you open the message if you saw it in your inbox? Why or why not?

Would you click on the link in the email to complete it? Why or why not?

Often a reminder email is sent if people have not responded. How long after the first email do you think a reminder email should be sent.

Now let’s look at this reminder email.

Hand the participant the First reminder email and have them read it:

Take a minute to read this.

What are your initial thoughts or concerns about this email? Would you open the message if you saw it in your inbox? Why or why not?

Would you click on the link in the email to complete it? Why or why not?

Finally, we want you to look at a final email about the survey.

Hand the participant the Second reminder email and have them read it:

Take a minute to read this.

What are your initial thoughts or concerns about this email? Would you open the message if you saw it in your inbox? Why or why not?

Would you click on the link in the email to complete it? Why or why not?

Okay, thank you for your feedback on the email messages. Now, you will help us evaluate the design of some questions for an upcoming Census online survey by completing the questions online.

If you run into any difficulties as you work on these short surveys, please don’t blame yourself. Any difficulties are the result of the design of the Web survey, not your skills or abilities. We are going to use your comments and experiences as well as comments and experiences of other participants to help improve the survey. I did not create the online survey, so don’t feel like you have to hold back on your thoughts to be polite to me. We appreciate your help so we can make the online survey work well for everyone.

At the end of the session I will have some questions for you about your experience as you worked on specific screens and how satisfied you were with the survey. We may open up some of the screens and talk about them.

Please answer the survey questions as they apply to you in your real life. Your will complete two different surveys first. For the first survey, you will pretend this is your email (hand respondent email for first survey depending upon randomized order).

While you are completing the survey I would like you to think aloud as you answer the questions. I am interested in your answers, but I am also interested in the process you go through in your mind when you answer the questions. I would like you to tell me everything that you are thinking and feeling as you go about answering each question.

Practice: Let’s do a practice question before we start: How many windows are in your home?

[Eye tracking]

We are also going to record where you look on the screen as we work on our study. Now we will do a simple task that will allow the computer to find your eyes. I am going to have you position yourself in front of the screen so that you can see your nose in the reflection at the bottom of the monitor. To calibrate your eyes, please follow the dot across the screen with your eyes.

[Calibrate the participants’ eyes]

Now I am going to go around to the other room and do a sound check. While I am doing that, please take a moment to complete this Background Questionnaire. This gives us information about your computer use and Internet experience. I am going to leave, but we will still be able to communicate through a series of microphones and speakers. Do you have any questions?

[Open background questionnaire on computer and Internet experience. Leave room. Once in control room do a sound check and start the eye-tracking software: Tobii Studio.]

[Then, based on random assignment, participant completes Version 1 or Version 2 of the address screens]

[After complete the version, then admininster the post-task questionnaire ]

Version 1:

STREET ADDRESS

Where did you live on June 1, 2013? (Help) We are asking for research purposes only to help identify the best way to gather address from people.

Please select the type of address associated with your residence.

Address Type: ● Street Address ○ P.O. Box ○ Rural Route

Address Number: _________ Street Name:________________ Unit Designation:_______________

For example: (5007) (N Maple Ave) (“Apt. A” or “Lot 3”)

City: ___________________ State: <drop-down> ZIP:________

PO BOX

Where did you live on June 1, 2013? (Help)

Please select the type of address associated with your residence.

Address Type: ○ Street Address ● P.O. Box ○ Rural Route

P.O. Box #: _________

City: ___________________ State: <drop-down> ZIP:________

RURAL ROUTE

Where did you live on June 1, 2013? (Help)

Please select the type of address associated with your residence.

Address Type: ○ Street Address ○ P.O. Box ● Rural Route

Rural Route Descriptor: <drop-down> Rural Route #:_______ Box #:________

City: ___________________ State: <drop-down> ZIP:________

What was the physical address where you were living or staying on June 1, 2013?

For example:

● an address you would give to a shipping company if you wanted a package delivered to your home, or

● an address you would provide if you were requesting emergency services such as a 911 call, or

● a location description such as “The apartment over the Post Office on County Road 5” or “The brick house with the screened porch on the northeast corner of Main Street and First Avenue”.

Observe if participant

[ ] Enters address number and street address in address number field.

[ ] Enters P.O. Box in Street address

[ ] Enters P.O. Box and Street Address in fields

[ ] Toggles through address types

[ ] Doesn’t appear to notice address types

[ ] Something else & Notes_____________________________________________

Administer post-task version

Version 2:

What is the physical address where you were living or staying on June 1, 2013?

Address Number: _________

Street Name: ________________

Apt/Suite: _______________

City: ___________________

State: <drop-down>

ZIP: ________

If you did not live at a street address on June 1st, you may provide in the space below:

● an address you would give to a shipping company if you wanted a package delivered to your home, or

● an address you would provide if you were requesting emergency services such as a 911 call, or

● a location description such as “The apartment over the Post Office on County Road 5”, or

● a name of a park, street intersection, or shelter if you are experiencing homelessness

Observe if participant

[ ] Enters address number and street address in address number field.

[ ] Enters P.O. Box in Street address

[ ] Enters P.O. Box and Street Address in fields

[ ] Toggles through address types

[ ] Doesn’t appear to notice address types

[ ] Something else & Notes_____________________________________________

Administer post-task version

Now I would like you to complete these questions and think-aloud while you are completing them. Please tell me why you choose your answers.

[Administer attitudes and behavior questions]

After Survey: Debriefing Questions

Pull up Version 1

Did you dislike anything about this design?

Were you confused by anything in this design?

(If they only completed the street address screen)

Did you see that you could also enter your P.O. Box and Rural Route address.

Do you have a P.O. Box? (If yes) Why did you not enter that address

What does Rural Route mean in your own words?

Pull up Version 2

Did you dislike anything about this design?

Were you confused by anything in this design?

[After debriefing is complete-- Now proctor the research component]

I just need to step out and get something.

[Exit the room to retrieve and open the randomly selected envelope.

If envelope says “C” go back into participant’s testing room and say:]

Please answer these questions, I’ll be in the other room.

[Open the link FQ Questionnaire—located at (M:\USABLE\2013 Commitment Research Study\Questionnaire Links) Then leave the room.]

[After completion of 10 questions]

Thank you for completing those questions, we have one final step and for that step we need to go to a different office to look at another address data collection screen.

[Walk to CAT (Center for Applied Technology) Lab]—it is on way out of building (Bring payment. We must test Version 3 in the CAT lab because it is being developed outside of a production environment.)

We have one other address screen design that we’d like you to use. Please answer the survey questions as they apply to you in your real life. Remember to think aloud why you are answering the questions.

Version 3

After the user completes their entries on this version of address collection screens, a standardized version of the address shall be displayed for the user to confirm. A sample is shown below.

Observe if participant

[] notices the address that comes up

[] changes the address in the field

[] Something else & Notes_____________________________________________

[Admininster post-task questionnaire by paper]

Administer post-task version

After Version 3 Survey: Debriefing Questions

Did you dislike anything about this design?

Were you confused by anything in this design?

That is all for today. Thank you once again for helping us.

[Give payment and complete financial form]

Email message – Version A

From: U.S. Census Bureau

Subject line: Confidential Census Study

The U.S. Census Bureau is conducting research to develop new methods to make the next census easier, more convenient and less costly for taxpayers.

Please complete the survey online at https://respond.census.gov/study

This survey will take 5 minutes or less to complete.

We are conducting this survey under the authority of Title 13 of the United States Code, Section 193. Federal law protects your privacy and keeps your answers confidential (Title 13, U.S. Code, Sections 9 and 214).

______________________________________________________________________________

Questions? Contact Us

 

STAY CONNECTED:

SUBSCRIBER SERVICES:
Manage Preferences  |  Unsubscribe  |  Help

Email message – Version B

From: U.S. Census Bureau

Subject line: Important Census Study

The U.S. Census Bureau is conducting research to develop new methods to make the next census easier, more convenient and less costly for taxpayers.

Please complete the survey online at https://respond.census.gov/study

This survey will take 5 minutes or less to complete.

We are conducting this survey under the authority of Title 13 of the United States Code, Section 193. Federal law protects your privacy and keeps your answers confidential (Title 13, U.S. Code, Sections 9 and 214).

______________________________________________________________________________

Questions? Contact Us

 

STAY CONNECTED:

SUBSCRIBER SERVICES:
Manage Preferences  |  Unsubscribe  |  Help

First email reminder

From: U.S. Census Bureau

Subject line: Reminder about Census Study

A few days ago, you should have received an email about completing an important U.S. Census Bureau study. This survey will take 5 minutes or less to complete.

If you have not yet responded, now is the time to complete the survey online at: https://respond.census.gov/study

The U.S. Census Bureau is conducting this research to develop new methods to make the next census easier, more convenient and less costly for taxpayers. We are conducting this survey under the authority of Title 13 of the United States Code, Section 193. Federal law protects your privacy and keeps your answers confidential (Title 13, U.S. Code, Sections 9 and 214).

______________________________________________________________________________

Questions? Contact Us

 

STAY CONNECTED:

SUBSCRIBER SERVICES:
Manage Preferences  |  Unsubscribe  |  Help

Second email reminder

From: U.S. Census Bureau

Subject line: Last Chance for Census Study

About a week ago, you should have received a couple of emails about completing an important U.S. Census Bureau study.

If you have not yet responded, this is your last chance to complete this survey online at: https://respond.census.gov/study

This survey will take 5 minutes or less to complete.

The U.S. Census Bureau is conducting this research to develop new methods to make the next census easier, more convenient and less costly for taxpayers. We are conducting this survey under the authority of Title 13 of the United States Code, Section 193. Federal law protects your privacy and keeps your answers confidential (Title 13, U.S. Code, Sections 9 and 214).

______________________________________________________________________________

Questions? Contact Us

 

STAY CONNECTED:

SUBSCRIBER SERVICES:
Manage Preferences  |  Unsubscribe  |  Help

Attitudes and Behavior Questions

1. Every 10 years, the Census Bureau counts everyone living in the United States.  We do this by collecting information from each address. For the next census in 2020, how would you prefer to be contacted by the Census Bureau?

❑ Mail a form to my home

❑ Email a link to the online form

❑ Call me on my home phone (landline)

❑ Call me on my cell phone

❑ Text message me a link to the online form

❑ Send an interviewer to my home

2. For the 2020 Census, would you rather:

❑ the Census Bureau contact you to get your census information?

❑ the Census Bureau use information that you have provided to other government agencies to complete the form for you?

3. What information do you think the census typically collects every 10 years?

 Select all that apply

• Names of all adults living at your address

• Names of all children living at your address

• Social security number

• Age

• Date of birth

• Race

• Ancestry

• Income

• Relationship

• Citizenship

4. Do you have...

1. A cell phone?

❑Yes

❑No

2. A handheld device made primarily for e-book reading, such as a Nook or Kindle e-reader?

❑Yes

❑No

3. A tablet computer such as an iPad, Samsung Galaxy Tab, Google Nexus or Kindle Fire?

❑Yes

❑No

4. An iPod or other MP3 player?

❑Yes

❑No

5. A game console like an Xbox or Play Station? 

❑Yes

❑No

5. (If yes 4a) Some cell phones are called 'smartphones' because of certain features they have. Is your cell phone a smartphone or not, or are you not sure?

❑Yes

❑No

❑Not sure

6. (If yes or not sure to 5) Which of the following best describes the type of cell phone you have? Is it an iPhone, a Blackberry, an Android phone, a Windows phone, or something else?

❑iPhone

❑Blackberry

❑Android

❑Windows

❑Something else

7. Are you male or female?

❑ Male

❑ Female

8. What is your age?

❑ Less than 18 years old

❑ 18-24

❑ 25-34

❑ 35-44

❑ 45-54

❑ 55-64

❑ 65 or older

9. Are you of Hispanic, Latino or Spanish origin?

❑ Yes

❑ No

10. What is your race? You may choose one or more.

❑ White

❑Black or African American

❑American Indian or Alaska Native

❑Asian

❑Native Hawaiian or Other Pacific Islander

11. What is your highest level of education?

❑ Less than High School Diploma or GED

❑ High School Diploma or GED

❑ Some College

❑ College Graduate or Professional Degree

12. Which of the following best describes your occupation:

❑ Student/Education/Academia

❑ Government

❑ Technical/Professional

❑ Service

❑ Other

13. Would you like to receive an email with a link to the research report for this study when it is complete?

❑ Yes

❑ No

Thank you for your participation in this survey. Your responses are very helpful in our research!

12