Various Economic Area Pretesting Activities

Annual Survey of Manufactures and Report of Organization (Centurion) Usability Testing Draft Protocol

(This protocol is a guide – the questions presented here won’t necessarily be asked exactly as worded in the protocol or in this order. It is important to note that not all questions will be asked in every interview.)

Part A – Testing

(Note: The primary interview technique is retrospective; R is asked about each question and/or group of questions after s/he has reported. Concurrent “think-aloud” probes are used when R indicates some confusion or other problem, e.g., by frowning, re-reading a question, looking away in thought, saying “Hmm”, etc.)

1. Login Screen

• Do you think that the information given on the Login Screen was sufficient?

• Do you think that the graphic used to show the location of the username and password on the paper form is helpful? Could we show anything else on this screen that would help you locate this information?

• Is there any other information you’d like to see on the Login Screen?

2. Main Menu

• Can you describe to me what each of the sections/boxes is showing you?

• Do you think that the labels that are used match the information that is shown within each section/box? Do you have any suggestions for more appropriate labels?

• Where do you need to go in order to begin completing your form(s)?

• Was this clear? If not, what could be done to make the start of the form clearer?

• In your own words, what are the options available within the toolbar at the top of the page? What would you get or where would you go if you clicked on them?

• If you wanted to glance at information about the survey before starting a form, where would you go?

• Is there anything else that you would need at this point in order to begin completing the form? Anything else that should be included on this screen?

3. Form-Specific section -- TBD

• Form-specific probe -- tbd

• Form-specific probe -- tbd

• Form-specific probe -- tbd

4. Form-Specific section -- TBD

• Form-specific probe -- tbd

• Form-specific probe -- tbd

• Form-specific probe -- tbd

5. Form-Specific section -- TBD

• If you needed to return to a prior section, how would you do this?

• Did you have any problems navigating back to a prior section/screen? Do you have any recommendation for improving the navigation within the site?

• Form-specific probe -- tbd

• Form-specific probe -- tbd

• Form-specific probe -- tbd

6. Form-Specific section -- TBD

• (If the R hasn’t triggered an edit at this point, have them do so on purpose)

• Is it clear to you based on the messages you are receiving on the screen that there is an issue? Can you explain to me what the problem(s) are?

• Is there a better way that we could notify you about the problem(s) on this screen?

• What would you need to do if you wanted to ignore the problem message you were receiving and move on to the next questions? Is it clear to you what action you need to take?

• What other information should we provide to you about your problems and how to correct them?

• Form-specific probe -- tbd

• Form-specific probe -- tbd

• Form-specific probe -- tbd

7. Form-Specific section -- TBD

• Form-specific probe -- tbd

• Form-specific probe -- tbd

• Form-specific probe -- tbd

8. Review your Form

• What are your first thoughts when seeing this screen?

• Can you explain to me what is expected of you? What is this screen telling you?

• (If there are edits) How will you go about fixing this/these problem(s)? When you have finished fixing the problems, what do you need to do next? How will you submit the form?

• What do you think of the title of this screen? Is it a logical title?

• What other types of review would you be interested in performing prior to submission?

• Does having the option to review a PDF of your information at this time make sense?

9.Completion Certificate

• Is there any other information that you would need to have for your completion?

• Would you save a copy of this information for your records?

10.Submission

• What do you need to do in order to submit your information?

• Was this clear? Do you have any recommendations for improving this?

10. Main Menu (after submission)

    • Would you save a copy of your data for your records?

    • Where would you need to go on this site in order to do that?

Part B – Wrap Up

• Now that you have gone through this form electronically, would you prefer to do this survey on paper or electronically?

• (If respondent previously reported in Surveyor) Now that you have gone through this form electronically, would you prefer to do this survey in the old electronic system (Surveyor) or this new electronic system (Centurion)?

• Do you think that you would complete the online form in one sitting?

• If you were completing this form electronically, would you still want/need the paper form? If so, what purpose would the paper form serve?

• Now that you have gotten a good feel for the online form, what is your overall impression of it? Was it easy, hard?

• Did you have any issues with the navigation on the site?

• What are some of the things that you liked the most about the online form?

• What are some of the things that you liked the least about the online form?

• Are there any other options that we should provide in the online form?

• Do you have any suggestions for how to make this online form easier for you to use?

Thank participant for their time and input!

3

ASM & Report of Organization (Centurion) Usability Testing Draft Protocol

April 12, 2010