2016 ECORR Usability Testing
Protocol
Research Questions
eCorrespondence Mailing:
Do respondents understand the contents of the mailing?
Do respondents understand the purpose of an authentication code and how to use it?
Do respondents understand the meaning of ‘one-time-use only?’
Are respondents able to use the instructions provided in the mailing to…
…reach the eCorr website?
…register for an account?
…link a survey using the authentication code?
eCorrespondence Registration Process (CPASS):
Do respondents try to create a personal account or corporate account?
Do respondents have any concerns about the information they must provide to create an account?
Are respondents able to complete the Captcha?
Are respondents able to confirm email addresses?
Are respondents able to use their newly created credentials to log into eCorr?
eCorrespondence Prototype:
Are respondents able to access the website?
Is the navigational path through the screens of the site logical to respondents?
Do button labels and descriptions make sense to respondents and inform respondents of the correct functions of the buttons?
Are respondents able to discern the functions of individual screens?
Are respondents able to link a survey?
How do respondents feel about the delegation system and do they understand how it functions?
What survey information to respondents need and are they able to find it on this website?
Are respondents able to submit help requests if needed?
What additional features or information do respondents want in the eCorrespondence prototype?
Materials
Census Laptop with wireless card to access test sites
eCorrespondence Mailing
Testing URL for eCorrespondence, dummy survey IDs
Printed copy of eCorrespondence screenshots (as back-up)
Printed copy of delegation invitation email
Consent forms
Procedures
Introductions/Audio recording
Have respondents review mailing contents
Have respondents create eCorr account through CPASS
Have respondents login to eCorr and link a survey ID
Review eCorr features with respondents
Wrap-up
Introduction
Reason for visit – To evaluate the prototype eCorrespondence website.
Introduction of Census employees
Introduction of company employee(s)
Audio recording
Before we get started: I'd like to audio record this interview, so I don't have to rely on my memory later. This session is confidential. Only people connected with this project will have direct access to your recording. If that's all right with you, please sign this consent form. It also tells you about the confidentiality of this session.
Respondent Information
How long have you been with the company?
What is your title/role?
What are your major responsibilities?
Are you responsible for other government surveys? If so, which ones?
eCorrespondence Mailing
This is a draft of a letter you might receive to tell you it’s time to report for one of our surveys. Please read this.
What are your initial impressions of the letter?
Do you find the information useful, or not useful?
Is there any information you would get rid of?
Is there any other information that you would like to see in this letter?
Do you recall reading the portion about the authentication code?
In your opinion, that does ‘One-time-use only’ mean?
eCorrespondence Registration
Now, let’s take a look at a new website mentioned in the letter. This website will act as a hub for all Census Bureau surveys in the future. Please open the browser of your choice and enter the URL from the letter. Please keep in mind that the website is still a work in progress, so we may come across some things that don’t work as intended.
(Note which browser and operating system the respondent uses)
When you visit the eCorrespondence website for the first time, you will need to create an account. As we go through this process, I will ask you questions about the experience.
General probes
In your own words, what would you say is the purpose of this page?
What do you notice about this screen?
How would you use this screen?
In your opinion, is the information on this page helpful or not helpful?
Is there other information you think should be included?
Watch respondent fill in their information. Make notes of any issues they may have.
Security Questions:
How do you feel about the options provided for the security questions?
Are there other options you would prefer to have?
Would you be interested in typing in your own questions?
Passwords:
How do you feel about the password requirements?
Are the requirements clear or unclear?
Email confirmation:
Please follow the instructions to confirm your email address.
How do you feel about the process of confirming your email?
Are the instructions helpful or unhelpful?
Is there other information you would like to see?
Based on the information provided in the letter and on the website itself, is there any other information you would need to successfully create an account?
When creating this account, were you thinking of it as a personal account or as a corporate account?
We intended accounts made on this site to be personal accounts, even if you are reporting for a company. How do you feel about that?
Does your company have any policies (even informal ones) about creating online accounts?
Please confirm your email address according to the directions.
(After respondent confirms his or her email address, have them log into eCorr). Please login. I’ll ask you questions as we go through the website.
Dashboard
Census Survey Dashboard:
What do you think is the purpose of this page?
How would you proceed once you got to this page?
Is there anything that stands out on this page?
The flyer describes linking a survey ID to your account.
How would expect this process to work?
Please link this survey to the account according to the directions on the website and in the letter.
Did that work the way you expected it to?
Were the instructions in the letter useful or not useful?
Were the instructions on the page helpful or not helpful?
Is there any other information you would like to see on this page?
How would you refer to the square with the survey title that appeared after you enter the code?
How would you expect to access a different survey if you received multiple survey requests from Census?
Probe if necessary:
Would you expect to create a different account?
Would you expect to have to enter another authentication code?
How would expect the dashboard to appear after adding another survey?
Delegation Function
This website allows you to delegate surveys to other people in your company, please locate that function on the website.
How would you expect the delegate feature to work?
How would you use it?
Are the instructions useful or not useful?
Is there any other information you would like to see on this page?
What would you expect to happen after you enter someone’s email address?
Here is an example of the email a delegate would receive (give sample email to respondent to read).
Do you think the instructions in the email are clear or unclear?
Thinking about whom you might delegate this survey to, do you think the instructions in the email will be helpful or not helpful?
How would you expect a delegate to proceed after receiving this email?
Is there other information you would like to see in this kind of email?
Would you expect the delegate to need the authentication code you entered earlier?
How much access do you think the delegate will have to the survey?
Besides entering data, what actions should a delegate be allowed to take on a survey?
Based on the information on the page, what other actions do you think a delegate can take on a survey?
Probe if necessary: Do you think a delegate would be able to…
Delegate to another person?
Request an extension?
Submit a survey?
After taking all of these things into consideration, is this a feature you would use?
Other Survey Options (on Dashboard)
Now let’s take a look at some of the other features of the website.
If you needed more information about a survey, where would you look?
What information about a survey would you want to see?
If you needed an extension on a survey, how would you request one?
Can you locate that feature on the website?
After respondent has located feature:
Is this where you would expect to find this function?
Do you find the information on the page to be helpful or not helpful?
Is there other information you would like to see about this function?
If you wanted to check your filing status for a survey, where would you look on the website?
After respondent has located feature:
Is this where you would expect to find this function?
Do you find the information on the page to be helpful or not helpful?
Is there other information you would like to see?
Other Information and FAQs
If you needed more information about a survey, where would you expect to find it on this website?
Did you notice the small ‘i’ on the survey card?
What would you expect to happen if you clicked it?
Please click it. Is this information useful or not useful?
Is there other information you would like to see here?
Many websites have a ‘Frequently Asked Questions’ page.
Is this a feature you would typically use if you had a question?
Please locate this page on the website.
Looking over the questions and topics presented, do you find this information to be helpful or not helpful?
In your opinion, what are the most important questions and answers on this page?
Are there other questions and answers you would like to see?
Support Requests
How would you contact the Census Bureau if you needed help with a specific issue?
This website allows you to send messages to Census Bureau staff. Please locate that feature on the website.
‘Send Message’ option on survey card:
How would you proceed once you got to this page?
How would you use the different topics/subtopics?
Are there other subtopics you would like to see?
What would you type into ‘Request Title?’
What information would you include in a request?
What kinds of attachments do you think someone might include with a help request?
Overall, are the instructions clear or unclear?
In your opinion, what is the purpose of this page?
How would you expect to use this page?
Do you find the information to be useful or not useful?
Wrap-up
Are there any other comments or suggestions you would like to make about anything we have gone over today?
At the end of eCorr testing: Please remember that this is just a test account, it will be deleted after we finish testing, and you will need to create a new account if you receive another survey invitation asking you to use this website.
Thank the respondent for their time.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Aryn Hernandez (CENSUS/ESMD FED) |
| File Modified | 0000-00-00 |
| File Created | 2021-01-23 |