Survey of Business Owners and Self-Employed Persons
(Centurion) Usability Testing Draft Protocol
(This protocol is a guide – the questions presented here will not necessarily be asked exactly as worded in the protocol or in this order. It is important to note that not all questions will be asked in every interview.)
What is your role here at this business?
Have you completed any other Census Bureau surveys? If so, which ones?
Did you report electronically to any of those surveys?
What made you choose to report electronically?
What was your overall experience? Did anything stand out as being very helpful? Was there anything that you found was not helpful?
What benefits would there be to reporting electronically to a survey like the Survey of Business Owners? Would there be any drawbacks?
Have you completed the Survey of Business Owners in the past? Did you report online? What was your experience?
Now I’d like to have you walk through the process of completing the electronic draft of the Survey of Business Owners form. I am going to hand you an envelope with some information inside, please treat it as if this correspondence arrived in the mail. (Hand the respondent an envelope with the mailout letter that contains the URL, test User ID and test password).
As you go through the letter and the electronic version of the survey and I would like you to complete it as you normally would. This is not a test of your ability or Internet savvy, so just proceed in whatever way makes the most sense to you. If you need to get information from records or others within the company in order to answer a question, just let us know what you would need to do. If you do not have the information available at this time, please feel free to insert any number into the boxes you would normally report in.
Don’t hesitate to tell me anything you find surprising on the site, or if you have questions about the material on the site, just ask. We’ll try our best to answer your questions.
(Note: The primary interview technique is retrospective; R is asked about each question and/or group of questions after s/he has reported. Concurrent “think-aloud” probes are used when R indicates some confusion or other problem, e.g., by frowning, re-reading a question, looking away in thought, saying “Hmm”, etc.)
General Observations:
Scrolling: Make careful note of how respondent scrolls through long items.
Instructions: Make careful note of how respondent treats the examples at the top of some of the items.
Navigation: Make careful note of how respondent uses the navigation buttons
Error: Make note if respondent triggers any errors, if not have them trigger one purposely
1. Mailout Letter
What are your overall reactions to this letter?
What information are you looking for?
Were are you able to find the information you were looking for?
What information is the most helpful to you?
Is there any other information you would like to see?
Where would you go to find help responding online?
What was the due date? User ID? Password?
Where would you go to start responding online?
How easy or difficult was this information to locate?
If you receive this envelope in the mail, how official looking is this envelope? Do you have any problem trusting this to be an official government mailing?
In your opinion, is there anything else we should do to make this envelope and/or letter more official looking?
2. Business Help Site (BHS) Screen
How easy or difficult was it to locate this screen?
What are your overall reactions to this screen?
What information are you looking for?
Where do you expect the “online services” button would direct you?
What would be included in the “Online Services”?
Where would you go to find assistance?
Where would you find the due date?
Where would you go next?
Where do you expect the “report online” button would direct you?
Is there any other information you would like to see on this screen?
3. Login Screen
What are your overall reactions to this screen? What do you see on this page?
What information are you looking for on this screen?
Is there any other information you would like to see while logging in?
What are your overall reactions to the process for logging into the system?
4. Main Menu
Can you describe to me what you see on this page?
Where do you need to go in order to start completing your form?
Was this clear? If not, what could be done to make the start of the form clearer?
If you had questions about definitions or the survey items, where would you go?
Is there any other information that you need on this screen before you would begin answering your survey? Why?
Did you notice the toolbar on this screen? What do you think each of the options will do?
Do you have any suggestions or recommendations for improving the layout of this screen?
Watch to see if R attempts to print .pdf here
What does “print .pdf” mean to you?
Is this something you would find useful?
Where would you expect to see this function?
Note where the R starts form
5. Number of Owners
Note any difficulties with this item.
What are your overall reactions to this page?
Are there any other types of information that would be helpful for you to have on this screen?
What information stood out for you on this page?
Which business were you referring to when you answered this item?
What are your overall reactions to the italicized text?
How would you advance to the next item?
Probe if R marked “yes” in Question 3
I see you marked “Yes” in Question 3. Could you tell me a little more about this?
Probe if R marked “no” in Question 4
I see you marked “No” in Question 4. Could you tell me a little more about why you selected this answer?
6. Percent Ownership
Note any difficulties with this item.
What are your overall reactions to this page?
Are there any other types of information that would be helpful for you to have on this screen?
What information stood out for you on this page?
What are your overall reactions to the italicized text?
In your own words, what is the “Position Title” example telling you?
Would you use this box?
Do you have any recommendations for revising this?
Is the space provided sufficient for the type of comments you would provide?
Probe if R writes parent company, trust, estate, etc. under position title to see if skip pattern was followed in questions 2 and 4.
I see you wrote, “Parent company/trust/estate/etc.” under position title. Could you tell me a little more about this?
Note R’s answer in Q2, Q3, Q4 and see if skip pattern worked correctly.
How would you advance to the next item?
7. Form-specific section
Do you have any recommendations for how we could improve the layout of the items on this screen?
Is there any information you would like to see on this screen?
Would there ever be a time where you would want to select more than one item from this list?
Form-specific probe -- tbd
Form-specific probe -- tbd
Form-specific probe – tbd
Owner 1 Questions (Q7-20)
8. Form-specific section
Do you have any recommendations for how we could improve the layout of the items on this screen?
Is there any information you would like to see on this screen?
Form-specific probe -- tbd
Form-specific probe -- tbd
Form-specific probe – tbd
9. Military Service (Part B)
What are your overall reactions to this screen?
Do you have any recommendations for how we could improve the layout of the items on this screen?
Is there any information you would like to see on this screen?
What were your reactions to the italic text?
10. Navigation
If you needed to return to a prior section, how would you do this?
Did you have any problems navigating back to a prior section/screen?
Do you have any recommendation for improving the navigation within the site?
11. Edit Check
(If the R hasn’t triggered an edit at this point, have them do so on purpose)
Is it clear to you based on the messages you are receiving on the screen that there is an issue? Can you explain to me what the problem(s) are?
Is there a better way that we could notify you about the problem(s) on this screen?
What would you need to do if you wanted to ignore the problem message you were receiving and move on to the next questions? Is it clear to you what action you need to take?
For the sake of testing, can you please ignore this message and continue without entering any information.
Owner 2 Questions (Q21-34) (If applicable)
12. Form-Specific Section
Which owner are you answering these items for?
If you had to return to modify any responses for the previous owner how would you do that?
How would you return to this section?
Do you have any recommendations for how we could improve the layout of the items on this screen?
Which owner would you be answering these items for?
Is there any information you would like to see on this screen?
Form-specific probe -- tbd
Form-specific probe -- tbd
Form-specific probe – tbd
Owner 3 (Q35-48) (If applicable)
13. Form-Specific Section
Which owner are you answering these items for?
If you had to return to modify any responses for the previous owner how would you do that?
How would you return to this section?
Do you have any recommendations for how we could improve the layout of the items on this screen?
Which owner would you be answering these items for?
Is there any information you would like to see on this screen?
Form-specific probe -- tbd
Form-specific probe -- tbd
Form-specific probe – tbd
Owner 4 Questions (Q49-61) (If applicable)
14. Form-Specific Section
Which owner are you answering these items for?
If you had to return to modify any responses for the previous owner how would you do that?
How would you return to this section?
Do you have any recommendations for how we could improve the layout of the items on this screen?
Which owner would you be answering these items for?
Is there any information you would like to see on this screen?
Form-specific probe -- tbd
Form-specific probe -- tbd
Form-specific probe – tbd
15. Form-Specific Section
Do you have any recommendations for how we could improve the layout of the items on this screen?
Is there any information you would like to see on this screen?
Form-specific probe -- tbd
Form-specific probe -- tbd
Form-specific probe – tbd
16. Capital Funding Part A
Could you tell me a little bit about what you see on this page?
What information stands out for you on this page if anything?
Do you have any recommendations for how we could improve the layout of the items on this page?
17. Capital Funding Part B
Could you tell me a little bit about what you see on this page?
What information stands out for you on this page if anything?
Could you tell me what the comment in italics is referring to?
Do you have any recommendations for how we could improve the layout of the items on this page?
18. Form-Specific Section
Do you have any recommendations for how we could improve the layout of the items on this screen?
Is there any information you would like to see on this screen?
Form-specific probe -- tbd
Form-specific probe -- tbd
Form-specific probe – tbd
19. Review
Did you expect to come to this screen when you completed the form? Why or why not?
Where would you have expected to go?
If no, what would you like to see?
Did you notice the information at the top of the screen? What was this information telling you?
How would you know if there were any items you missed?
How would you return to an item? How would you get back to the review screen? What are your thoughts on this process?
What other types of review would you be interested in performing prior to submission?
Where would you go next? How will you send the form to us?
How will you know the form is completed?
If you weren’t ready to submit your data at this point, what would you do?
Would you be able to send us the form before correcting the issues?
20. Submission Confirmation Screen
What is your initial reaction to this page?
What do you think of the overall layout of this page?
What would you do after arriving to this page?
Would you save a copy of this information for your records?
Is there anything else you would like to see on this screen?
About how long would it take to complete this form online?
How did you feel about the amount of information that was provided on each page? Was it too much information, too little information, or about right?
How did you feel about the amount of scrolling? Too much, too little, about right?
How would you prefer to receive reminders about this form? Via letter or email?
Would you need to print a paper form at any point prior to submitting this form online, or is the electronic version sufficient for your purposes?
Did you have any issues with the navigation on the site?
What are some of the things that you liked the most about the online form?
What are some of the things that you liked the least about the online form?
Are there any other options that we should provide in the online form?
Do you have any suggestions for how to make this online form easier for you to use?
July 2013
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | butle360 |
| File Modified | 0000-00-00 |
| File Created | 2021-01-28 |