Various Economic Area Pretesting Activities

Census Classification Form (NC-99023)

(Centurion) Usability Testing Draft Protocol

(This protocol is a guide – the questions presented here won’t necessarily be asked exactly as worded in the protocol or in this order. It is important to note that not all questions will be asked in every interview.)

Part A – Introduction

1. Introduce observers and their background

2. Purpose of visit:

Let me start by telling you a little about what we will be doing today. The Economic Census Classification Report (NC-99023) is mailed out in order to collect information on businesses to obtain proper industry classification for use in the upcoming 2012 Economic Census conducted by the U.S. Census Bureau.

In the past, respondents could only report their data on paper, but starting this year respondents will have the options to report either on paper or through the Internet. Since this is the first time we’re doing this survey electronically, we want to get your feedback in order to make revisions before it is formally released to other businesses.

3. Permission to audio-tape discussion? Have R sign consent form.

Before we get started: I'd like to audio tape this interview, so I don't have to rely on my memory later. This session is confidential. Only persons connected with this project will have access to your tape. If that's all right with you, please sign this consent form. It also tells you about the confidentiality of this session.

Part B – Background Information

1. Respondent Background

• Can you tell me a little about your company organization? What products or services does it provide?

• What is your role in the organization?

• Have you completed any other government surveys? If so, which ones?

2. Electronic Reporting Background (if reported to other government surveys)

• Did you report electronically to any of those surveys?

• What made you choose to report electronically?

• What was your overall experience? Did anything stand out as being very helpful? Was there anything that you didn’t like?

• What benefits would there be to reporting electronically to a survey like the Economic Classification Report? What drawback?

Now I’d like to show you an electronic draft of the NC-99023 form. Here is the web-site address for accessing the survey (show the respondent the URL, survey key, test username and test password).

I would like you to go through the electronic version of the survey and complete. This is not a test of your ability or Internet savvy, so just proceed in whatever way makes the most sense to you. If you need to get information from records or others within the company in order to answer a question, just let us know what you would need to do.

Don’t hesitate to tell me anything you find surprising on the site, or if you have questions about the material on the site, just ask. We’ll try our best to answer your questions.

Part C – Testing

(Note: The primary interview technique is retrospective; R is asked about each question and/or group of questions after s/he has reported. Concurrent “think-aloud” probes are used when R indicates some confusion or other problem, e.g., by frowning, re-reading a question, looking away in thought, saying “Hmm”, etc.)

3. Login Screen

• Do you think that the information given on the Login Screen was sufficient?

• What are your overall reactions to the process for logging into the system?

• Is there any other information you’d like to see on any of the screens you saw while logging in?

4. Main Menu

• Can you describe to me what you seen on this page?

• Where do you need to go in order to start completing your form?

• Was this clear? If not, what could be done to make the start of the form clearer?

• If you had questions about definitions or the survey items where would you go?

• Is there anything else that you would need at this point in order to start completing the form? Anything else that should be included on this screen?

• Did you notice the toolbar on this screen? What do you think each of the options will do?

• Are there any other types of information that would be helpful for you to have? Would you look at this information before completing the form or when you needed it?

• Let’s get started with the survey.

5. Mailing Address

• In your own words, what is the difference between the two sections shown on the screen.

• Is there anything that we could do to improve the layout of the questions on this screen?

6. Principle Business or Activity

• Did you notice the icons on the screen? What sort of information would you expect to get if you clicked on them?

• Was this information helpful? What could we do to improve the way we display this information?

• If you needed to return to the main menu, how would you do that? Please go back to the main menu.

• Now that you are on the main menu, how would you return to the question you were working on?

7. Principle Business or Activity – Continued

• In your own words, describe what you are being asked to do on this screen.

• What does the information printed in red on the screen represent?

8. Class of Customer (for retail or wholesale companies)

• After answering item A, which item did you go to? Were the instructions clear?

• Is there anything that we could do to improve the layout of this screen?

9. Detail of Sales, Shipments, Receipts, or Revenue

• What is required of you on this screen?

• What would you need to do if you had more than four lines to report? Did you notice the comment at the bottom of the screen for how to handle this information?

• (If the R hasn’t triggered an edit at this point, have them do so on purpose by entering information that doesn’t total 100%.)

• Do the messages on the screen clearly indicate to you that there is an issue? Can you explain to me what the issue(s) are?

• Is there a better way that we could notify you about the issue(s) on this screen?

• What would you need to do if you wanted to ignore the problem message you were receiving and move on to the next questions? Is it clear to you what action you need to take?

7. Remarks / Certification

• If you had a question for the Census Bureau about this report, how would you contact them? Where is that information located?

• In your own words, what are you being told in the Certification section?

• Is there anything that we could do to improve the design of this screen?

8. Review your responses

• What are your first thoughts when seeing this screen?

• Can you explain to me what is expected of you? What is this screen telling you?

• (If there are edits) How will you go about fixing this/these problem(s)? When you have finished fixing the problems, what do you need to do next? How will you submit the form?

• What do you think of the title of this screen? Is it a logical title?

• What other types of review would you be interested in performing prior to submission?

• If you weren’t ready to submit your data at this point, what would you do?

11. Completion Certificate

• Is there any other information that you would need to have for your completion?

• Would you save a copy of this information for your records?
  
  
  

12. Main Menu (after submission)

• Would you save a copy of the data for your records?

• Where would you need to go on this site in order to do that? (PDF not yet working)

Part D – Wrap Up

• Now that you have gone through this form electronically, would you prefer to do this survey on paper or electronically?

• If you were completing this form electronically, would you still want/need the paper form? If so, what purpose would the paper form serve?

• Now that you have gotten a good feel for the online form, what is your overall impression of it? Was it easy, hard?

• Did you have any issues with the navigation on the site?

• What are some of the things that you liked the most about the online form?

• What are some of the things that you liked the least about the online form?

• Are there any other options that we should provide in the online form?

• Do you have any suggestions for how to make this online form easier for you to use?

Thank participant for their time and input!

5

NC-99023 (Centurion) Usability Testing Draft Protocol

May 3, 2011