Third Party Credential Authentication Usability Study Materials
Overview 1
Moderated Study Materials 3
Recruitment Email 3
Recruitment Screener 3
Confirmation Email 5
Reminder Email 5
Moderator Pre-Task Checklist 6
Consent & Recording Release Form 7
Pretest Demographic and Background Information Questionnaire 8
User Persona 10
Usability Test Script 10
Post-Task Questionnaire 13
Moderator Post-Task Checklist 15
Paperwork Reduction Act Statement 16
Receipt Form: Usability Test Honorarium 16
Un-Moderated Study 17
Link to Study Website 17
Recruitment Page 17
Screener Page 18
Ineligible Thank You Message 18
Study Invitation Page 19
Consent Page 19
Pre-Task Questionnaire 20
Task 1 22
Task 1 Questionnaire 22
Task 1 Continuation 24
Post-Task Questionnaire 24
Thank You Message 25
The purpose of this study is to assess the usability and user satisfaction of using third party credentials to access IRS services. The study will test a notional prototype of an alternative flow for IRS eAuthentication, which will include a start page and bank credential flow.
The participant’s performance with each page of the form will be recorded and evaluated. Measures such as time on task, task success, participant understanding of content, help and error text will be captured. Qualitative data on user perception and satisfaction of using third party credentials to access IRS services will also be noted. Collecting this data will provide the study with:
Behavioral observations and insights into the user experience,
Insights into design solutions to improve and strengthen the user experience,
Benchmark information on the current experience that can be used as a guide for future online developments,
Insights into how people perceive using third party credentials when accessing IRS online services.
Two testing methods will be employed using the same task and questionnaires. A moderated usability study with up to 24 participants will be used to gain in depth insights into the usability and perception of using a third party credential to access IRS services. An online, un-moderated study using software such as Loop11 or OptimalSort (software decision pending) with up to 300 participants will capture additional data to verify usability metrics, satisfaction and perception data found in the moderated sessions.
No personally identifying information will be used in this study. Participants of the moderated sessions will be provided with fake credentials (both fake personal details and bank credentials) and fake personal information created for the study. For un-moderated sessions, text input fields will be disabled.
Recruiting will use listserves or known pools of participants.
SUBJECT LINE: The IRS invites you to participate in a website study and earn $40!
Hello,
My name is [NAME], and I’m helping to run a usability study for the Internal Revenue Service (IRS). In an effort to improve services, we’re looking for people who are interested in exploring new ways to access their tax records. Afterward we will simply ask for your feedback. If you participate we will pay you $40.
What will you be doing in a usability study?
You will be asked to perform several short tasks using a website. You will also be asked questions about your experience and what you think of the website.
How long is a study session?
30 minutes.
When and where?
The study will be held [DAYS, DATES, LOCATION]. Sessions may be held online or in person.
Interested in participating?
Please reply to this email with your contact information or call me at [ADD PHONE]. I will give you a call to ask you some questions so we can decide if you qualify for the study.
If you have any questions, please contact me at [EMAIL].
Thank you for your interest,
[NAME AND TITLE]
The IRS is developing a new way for taxpayers to log in to online services. As part of their development process they would like to get reaction from the general public about the possible design. If you are selected to participate, the session will take approximately 30 minutes and you will be compensated for your time with a $40 electronic gift card.
What category describes your current age?
18 to 24 years
25 to 34 years
35 to 44 years
45 to 54 years
55 to 64 years
65 to 74 years [not eligible]
75 to 84 years [not eligible]
85 years and over [not eligible]
What is your primary computing device for personal use?
Computer / Laptop
Tablet
Smartphone
None [not eligible]
How often do you use your personal computing devices(s)?
Several times a day
Daily
Weekly
Monthly [not eligible]
Never [not eligible]
Do you currently use online banking (for example, to check your balance, transfer money, or pay bills, includes PayPal)?
Yes
No [not eligible]
[Include questions on minimum system requirements for remote testing tools if test is run remotely. Those who do not meet minimum system requirements are not eligible.]
If not eligible: At this time, it appears that your interests and experience are different from the profile we are seeking for this project. We will keep you in mind for future opportunities. Thank you for your time today.
If eligible: Thank you so much for taking the time to speak with us today. We’d like to schedule 30 minutes at a time convenient for you. [Option 1] You will need to be in a quiet space with a computer connected to the Internet. Schedule session with participant, or thank them for their time. [Option 2] You will receive a follow up email to schedule your session.
SUBJECT LINE: Confirmation: Your participation in our usability study
Dear [PARTICIPANT NAME]:
Thank you for agreeing to participate in our usability testing. As mentioned, you will be asked to perform a few tasks on a website and give us thoughts about your experience. There is nothing to do to prepare before the session.
To schedule your session go to [CALENDLY URL] and choose from available time slots.
A few key reminders:
You will be given $40 in exchange for your participation. (NOTE: Government employees and/or grantees are not eligible for the honorarium unless you are participating on your own time.) With your permission, the session will be recorded. We will only use the recording in our research to decide how to improve the website. Your name will not be used for any purpose beyond this session. Also, we have only one person scheduled at a time for these sessions so if you find that you cannot participate on your scheduled day, please contact us as soon as possible so we can reschedule your session.
Thank you again!
[NAME OF FACILITATOR AND CONTACT INFORMATION]
SUBJECT LINE: Reminder: Website study tomorrow
Dear [PARTICIPANT NAME]:
Thank you for agreeing to participate in our usability testing. As mentioned, you will be asked to perform a few tasks on a website and give us thoughts about your experience. There is nothing to do to prepare before the session.
[SESSION DATE, TIME]
A few key reminders:
You will be given $40 in exchange for your participation. (NOTE: Government employees and/or grantees are not eligible for the honorarium unless you are participating on your own time.) With your permission, the session will be recorded. We will only use the recording in our research to decide how to improve the website. Your name will not be used for any purpose beyond this session. Also, we have only one person scheduled at a time for these sessions so if you find that you cannot participate on your scheduled day, please contact us as soon as possible so we can reschedule your session.
Thank you again!
[NAME OF FACILITATOR AND CONTACT INFORMATION]
Prepare data sheets and data logging software for participant
Ensure web browser cache has been cleared, log in to IRS Labs, have a web browser window open to [PROTOTYPE URL].
Only have the eAuthentication page shown to the participants at the start of the study.
Look at and remember person’s name prior to going to meet him/her.
Introduce yourself and other team members who might interact with participant.
Ask if person would like something to drink and/or to use the restroom.
Ask person to read and sign the consent form.
Reference the OMB Control Number and PRA Statement (provide printed copy):
The Paperwork Reduction Act requires that the IRS display an OMB control number on all public information requests. The OMB Control Number for this study is 1545-2256. Also, if you have any comments regarding the time estimates associated with this study or suggestions on making this process simpler, please write to:
Internal Revenue Service
Tax Products Coordinating Committee
SE:W:CAR:MP:T:T:SP
1111 Constitution Ave. NW
Washington, DC 20224
Provide participant the User Persona for the task and the Participant Task Sheets (Note: Provide the participant task sheets one at a time, so that the participant is unable to read future tasks ahead of time).
I agree to participate in the study conducted and recorded by the Internal Revenue Service (IRS). I understand that the study will take approximately 30 minutes.
I understand and consent to the use and release of the audio, video, and/or digital recording by the IRS. I understand that the information and recording is for research purposes only and that my name and image will not be used for any other purpose. I relinquish any rights to the recording and understand the recording may be copied and used by IRS without further permission.
I understand that participation in this usability study is voluntary and I agree to immediately raise any concerns or areas of discomfort during the session with the study administrator.
The Paperwork Reduction Act requires that the IRS display an OMB control number on all public information requests. The OMB Control Number for this study is 1545-2256. Also, if you have any comments regarding the time estimates associated with this study or suggestions on making this process simpler, please write to:
Internal Revenue Service
Tax Products Coordinating Committee
SE:W:CAR:MP:T:T:SP
1111 Constitution Ave. NW
Washington, DC 20224
Please sign below to indicate that you have read and you understand the information on this form and that any questions you might have about the session have been answered.
Date |
|
Please print your name |
|
Please sign your name |
|
Thank you! We appreciate your participation.
Which of the following social media websites do you use? (Check all that apply)
YouTube
Tumblr
Google+
Other Site (Please specify)
Other Site (Please specify)
Other Site (Please specify)
Do you find it easy to keep track of your username and password for different websites?
Yes, I remember my usernames and passwords
Yes, I use a tool to store all of my usernames and passwords
No
I’m not sure
Have you used any IRS services, such as IRS.gov, Where’s My Refund or Direct Pay?
Yes
No
I don’t recall
What types of online information or services have you used with the IRS? (Check all that apply) Include a short description in each line.
IRS.gov
Get Transcript
Online Payment
Where’s My Refund
Direct Pay
Online Payment Agreement
Download a form
Free File
Find information about taxes
Set up an Identity Protection PIN
Other services (Please specify)
How often do you access IRS online services, such as IRS.gov, Where’s My Refund or Direct Pay?
Several times a day
Daily
Weekly
Monthly
Yearly
Never
Have you created a username and password to log in to services available on IRS.gov?
For example, have you logged in to download a tax return, make an online tax payment, set up a payment agreement, or set up an identity protection PIN number?
Yes
No
I’m not sure
Do you access any of your bank accounts online?
Yes
No
I’m not sure
If yes, how often do you access your bank accounts online?
Several times a day
Daily
Weekly
Monthly
Yearly
Never
Have you ever used an account such as Google, Facebook, or Amazon to log in to different website?
Include an example, example image
Yes
No
I’m not sure
Have you ever used a financial account, such as your bank or Paypal account, to log in to another website?
Include an example, example image
Yes
No
I’m not sure
YOUR PERSONAL INFORMATION
Name |
Crystal Glass |
SSN |
211-22-1236 |
Current Address (as of) |
11568 Golden Ave. Orlando, FL 40706 |
Previous Address |
213 Beach Ave. Miami, FL 40389 |
Date of Birth |
March 12, 1981 |
E-Mail Address |
|
Phone Number |
407-558-5524 |
YOUR LEGAL INFORMATION
Filing Status |
Single |
Last Year Filed Tax Return |
2013 |
YOUR FINANCIAL INFORMATION
Bank |
ACME Bank |
Username |
CrystalGlass123 |
Password |
cryst@L! |
Welcome and Purpose
Thank you so much for coming in today. I want to give you a little information about what you will be looking at and give you time to ask any questions you might have before we get started.
Today we are asking you to evaluate a prototype by completing a survey and a short task. Our goal is to find out how easy or difficult it is for you to use the prototype, and what you feel about it.
Test Facilitator’s Role
I’m here to record your reactions and comments about the prototype. [Optional: In a conference room nearby, I have a colleague helping me take notes and observe your interactions with the site as well.]
During this session, I may ask you about what you’re thinking as you work to complete the tasks. I will not be able to offer any suggestions or hints, but from time to time, I may ask you to clarify what you have said or ask you for information on what you were looking for or what you expect to have happen.
Test Participant’s Role
Today I am going to ask you to use a very early prototype and discuss your experience. There are no right or wrong answers. This is an evaluation of a prototype, not you! If you have any questions, comments or feel confused about anything while you are working, please let me know. If you ever feel that you lost or that you cannot complete the task, please let me know. I will ask you what you might do in a real-world setting and then either put you on the right track or move you on to the next step in the process.
I will provide you with a made up name, address and other credentials to use today. As you work through the tasks, please try and act as if you are at home or work, but use the fake credentials in place of your own.
Consent: Request participant read and sign consent/recording release form.
Each session is recorded. We will be recording your face, your voice and what you see on the screen. Your actual name will not be used or reported with data or findings associated with this evaluation.
I’ll ask you questions as we go along and we will have wrap up questions at the end.
Do you have any questions before we begin?
Pre-Task Questions
Ok, let’s get started.
First, please fill out this questionnaire.
Provide pretest demographic and background information questionnaire.
Probe participant on answers regarding third party credentials, online banking habits and online security concerns.
Thanks! Let’s take a look at the prototype.
Usability Tasks
Today you’ll be using the name, address and credentials of Crystal Glass. It’s important that no information about you is captured in the recording, so please use these credentials and pretend you are Crystal Glass today.
Provide persona sheet.
One other thing before we start. Today you will be using a prototype, not a real website. That means that you will not be able to click on every link, and some links may not lead to the pages you would expect. Do your best to use the prototype like a normal website.
Task A
You (Crystal Glass) have a meeting about a loan at the bank later today. In preparation for the meeting you have been asked to bring in a copy of your tax return transcript from 2013. You’ve already opened a web browser and navigated to the site, but when you get to the site you realize this is the first time that you have used this system. You’re in a rush, so you want to create an account using you username and password from your bank website, ACME bank.
Remember, when creating the account use Crystal Glass’ personal information that has been provided. For questions where you are not provided information, simply pretend you are Crystal Glass and make up a response that seems to fit.
Any questions? Please complete the task.
Thank you! Go ahead and log out.
Ensure web browser is open to main prototype page (Create and account / sign in).
Now that you have signed up for an account, please sign back in to your account in order to request your tax return transcript from 2013.
Post-Session Questions/Discussion
We’ve completed the tasks for our session. I have a few more questions for you.
Provide Post-Task Questionnaire.
Ask any follow up questions based on the participants’ experience (e.g., review any critical and non-critical errors, discuss any noted hesitations, ask for any recommendations/suggestions to enhance, etc).
Probe on opinions about third party credentials, privacy and security issues based on the Post-Task Questionnaire.
That’s it for our session! Do you have any questions or additional comments?
Facilitator: Refer to Post-Task Checklist.
Overall, how difficult or easy was it to complete the process for creating an account using the bank username and password?
Very Difficult |
|
|
|
Very Easy |
||
1 |
2 |
3 |
4 |
5 |
6 |
7 |
|
|
|
|
|
|
|
How would you prefer to log in to IRS online services?
A username and password created for my IRS online account [PATH 1]
A username and password I use for another website, like my bank [PATH 2]
I prefer not to log in to IRS services [PATH 1]
[START PATH 1]
If you chose “A username and password created for my IRS online account” or “I prefer not to log in to IRS services” select a reason why (Check all that apply):
I’m not sure how my bank and the IRS will share my information
I don’t trust my bank to keep my information private and secure
I don’t trust the IRS to keep my information private and secure
I’m not sure how the IRS or the bank will use my information
I prefer to keep my financial information separate from my tax matters
In general, I prefer to have individual accounts on each website
Other reasons
Are there actions that could be taken by the IRS or financial institutions that might increase your interest in using your bank username and password to log in to IRS online services?
Yes
No
I’m not sure
If yes, select any options that would increase your interest in using your bank username and password to log in to IRS online services (Check all that apply):
Provide better information about how my bank and the IRS will share my information
Improve the security and privacy on my online bank
Improve the security and privacy of the IRS
Explain how the IRS or the bank will use my information
Other options
What other types of personal information are you comfortable providing to create an IRS account? (Check all that apply)
Social Security Number (SSN)
Drivers License Number
Filing Status
Address
Others?
[END PATH 1]
[START PATH 2]
If you chose “A username and password I use for another website, like my bank” select a reason why (Check all that apply):
I think it’s easy
I trust my bank to keep my info private and secure
I trust the IRS will keep my information private and secure
I trust that the IRS will not misuse my information
In general, I prefer to use one account for multiple websites
Other reasons
[END PATH 2]
If you chose to log in to an IRS online service with you bank username and password, which services are you most likely to use? (Check all that apply)
Get a copy of my tax return
Make an online payment
Find out the status of my refund
Create an online payment agreement
Communicate with an IRS representative
Amend a tax return
None
If you were to encounter a problem logging in with your bank username and password, whom would you prefer to speak to?
IRS customer service
Bank customer service
I’m not sure
Neither
Are you interested in using any other website credentials to login to an IRS account? (Check all the apply)
Online Tax Preparation Software
Amazon
Paypal
Healthcare.gov
SSA.gov
None
Review the purpose of the session.
Ask participants if they have any questions.
Ensure that participant completes post-study questionnaire and discuss his/her responses with him/her. Ask if (s)he has any other comments about the site.
Have participant sign receipt form and record honorarium check number on signed form.
Enter participant ID number on notes, consent forms, receipt.
Close and save Morae recording.
Empty web browser cache.
Prepare computer, data sheets, and data logging software for next participant.
The Paperwork Reduction Act requires that the IRS display an OMB control number on all public information requests. The OMB Control Number for this study is 1545-2256. Also, if you have any comments regarding the time estimates associated with this study or suggestions on making this process simpler, please write to:
Internal Revenue Service
Tax Products Coordinating Committee
SE:W:CAR:MP:T:T:SP
1111 Constitution Ave. NW
Washington, DC 20224
Please sign below to indicate that you have received the promised compensation for your participation in testing today.
Date |
|
Amount received |
|
Please print your name |
|
Please sign your name |
|
Thank you!
We appreciate your participation.
The link “Help Improve IRS Online Services” will be placed on one or more pages on IRS.gov to recruit participants. On selecting the link, a potential participant will see the standard notification that they are leaving IRS.gov, and the study website will open in a new tab or window.
Help Improve IRS Online Services
In an effort to improve our services, we are seeking people interested in participating in online usability studies. A usability study helps us gather feedback on potential new IRS online features. We hope you will take the opportunity to influence the design and direction of IRS online services!
What will you be doing in a usability study?
You will be asked to perform several short tasks using a website. You will also be asked questions about your experience, and what you think of the website.
How long is a study session?
An online usability study typically takes 10 minutes, but can range between 5 and 30 minutes. Participation is voluntary, and you will be alerted to how long the study takes prior to starting.
All participation is on a volunteer basis. No incentive or gift is provided for participation.
Interested in participating?
Get started. Take a short survey to see if you qualify for today’s usability study.
What category describes your current age?
18 to 24 years
25 to 34 years
35 to 44 years
45 to 54 years
55 to 64 years
65 to 74 years
75 to 84 years
85 years and over
What is your primary computing device for personal use?
Computer / Laptop
Tablet
Smartphone
None [not eligible]
How often do you use your personal computing devices(s)?
Several times a day
Daily
Weekly
Monthly [not eligible]
Never [not eligible]
Do you currently use online banking (for example, to check your balance, transfer money, or pay bills, includes PayPal)?
Yes
No [not eligible]
[Include questions on minimum system requirements for testing software if they exist. Those who do not meet minimum system requirements are not eligible.]
Thank you for filling out our participation survey. At this time we do not have any usability study sessions available. Please come back and try again in the future!
Link to IRS.gov
Thank you for filling out our participation survey. There is usability study session available today.
The session takes approximately 15 minutes, and you will need to be in a quiet space with a computer connected to the Internet.
Participation is voluntary. You may exit the study at any time. No incentives are provided for participation in this study.
About the study
Thank you for taking part in the [NAME] online usability study. The purpose of this session is to evaluate a new method for taxpayers to access their IRS accounts.
The evaluation should only take about 15 minutes to complete. Your responses will be anonymous and combined with responses from others.
You will be presented with a prototype and asked to carry out a task. Complete the task as best you can and approach it the way you feel is appropriate.
Link to study and link to IRS.gov
Consent
I agree to participate in the study conducted by the Internal Revenue Service (IRS). I understand that participation in this usability study is voluntary and that the study will take approximately 15 minutes. I understand that the information collected is for research purposes.
Notice
The Paperwork Reduction Act requires that the IRS display an OMB control number on all public information requests. The OMB Control Number for this study is 1545-2256. If you have any comments regarding the time estimates associated with this study or suggestions on making this process simpler, please write to:
Internal Revenue Service
Tax Products Coordinating Committee
SE:W:CAR:MP:T:T:SP
1111 Constitution Ave. NW
Washington, DC 20224
Print Paperwork Reduction Act Statement (Link to print)
Yes I agree (link to study)
No, thanks (link to IRS.gov)
Which of the following social media websites do you use? (Check all that apply)
YouTube
Tumblr
Google+
Other Site (Please specify)
Other Site (Please specify)
Other Site (Please specify)
Do you find it easy to keep track of your username and password for different websites?
Yes, I remember my usernames and passwords
Yes, I use a tool to store all of my usernames and passwords
No
I’m not sure
Have you used any IRS services, such as IRS.gov, Where’s My Refund or Direct Pay?
Yes
No
I don’t recall
What types of online information or services have you used with the IRS? (Check all that apply)
Include a short description in each line.
IRS.gov
Get Transcript
Online Payment
Where’s My Refund
Direct Pay
Online Payment Agreement
Download a form
Free File
Find information about taxes
Set up an Identity Protection PIN
Other services (Please specify)
How often do you access IRS online services, such as IRS.gov, Where’s My Refund or Direct Pay?
Several times a day
Daily
Weekly
Monthly
Yearly
Never
Have you created a username and password to log in to services available on IRS.gov?
For example, have you logged in to download a tax return, make an online tax payment, set up a payment agreement, or set up an identity protection PIN number?
Yes
No
I’m not sure
Do you access any of your bank accounts online?
Yes
No
I’m not sure
If yes, how often do you access your bank accounts online?
Several times a day
Daily
Weekly
Monthly
Yearly
Never
Have you ever used an account such as Google, Facebook, or Amazon to log in to different website?
Include an example, example image
Yes
No
I’m not sure
Have you ever used a financial account, such as your bank or Paypal account, to log in to another website?
Include an example, example image
Yes
No
I’m not sure
You have a meeting about a loan at the bank later today. In preparation for the meeting you have been asked to bring in a copy of your tax return transcript from 2013.
This is the first time that you have used this website. You’re in a rush, so you want to create an account using you username and password from your bank website, ACME bank.
What would you do to create a login using ACME bank your username and password?
Display eAuthentication Create a Log in / Log In prototype page. On selecting the bank option, link to next page.
How would you prefer to log in to IRS online services?
A username and password created for my IRS online account [PATH 1]
A username and password I use for another website, like my bank [PATH 2]
I prefer not to log in to IRS services [PATH 1]
[START PATH 1]
If you chose “A username and password created for my IRS online account” or “I prefer not to log in to IRS services” select a reason why (Check all that apply):
I’m not sure how my bank and the IRS will share my information
I don’t trust my bank to keep my information private and secure
I don’t trust the IRS to keep my information private and secure
I’m not sure how the IRS or the bank will use my information
I prefer to keep my financial information separate from my tax matters
In general, I prefer to have individual accounts on each website
Other reasons
Are there actions that could be taken by the IRS or financial institutions that might increase your interest in using your bank username and password to log in to IRS online services?
Yes
No
I’m not sure
If yes, select any options that would increase your interest in using your bank username and password to log in to IRS online services (Check all that apply):
Provide better information about how my bank and the IRS will share my information
Improve the security and privacy on my online bank
Improve the security and privacy of the IRS
Explain how the IRS or the bank will use my information
Other options
What other types of personal information are you comfortable providing to create an IRS account? (Check all that apply)
Social Security Number (SSN)
Drivers License Number
Filing Status
Address
Others?
[END PATH 1]
[START PATH 2]
If you chose “A username and password I use for another website, like my bank” select a reason why (Check all that apply):
I think it’s easy
I trust my bank to keep my info private and secure
I trust the IRS will keep my information private and secure
I trust that the IRS will not misuse my information
In general, I prefer to use one account for multiple websites
Other reasons
[END PATH 2]
You have a meeting about a loan at the bank later today. In preparation for the meeting you have been asked to bring in a copy of your tax return transcript from 2013. You are creating a log in using you username and password from your bank website, ACME bank.
Continue to create a login with your bank username and password.
To protect your privacy, all fields are disabled. To fill out a field, simply click on it.
Display eAuthentication Select Bank prototype page. User clicks through four additional prototype pages.
Thank you! Please answer a few additional questions about your experience.
Overall, how difficult or easy was it to complete the process for creating an account using the bank username and password?
Very Difficult |
|
|
|
Very Easy |
||
1 |
2 |
3 |
4 |
5 |
6 |
7 |
|
|
|
|
|
|
|
If you chose to log in to an IRS online service with you bank username and password, which services are you most likely to use? (Check all that apply)
Get a copy of my tax return
Make an online payment
Find out the status of my refund
Create an online payment agreement
Communicate with an IRS representative
Amend a tax return
None
If you were to encounter a problem logging in with your bank username and password, whom would you prefer to speak to?
IRS customer service
Bank customer service
I’m not sure
Neither
Are you interested in using any other website credentials to login to an IRS account? (Check all the apply)
Online Tax Preparation Software
Amazon
Paypal
Healthcare.gov
SSA.gov
None
Thank you for taking the time to complete this evaluation! Your input will help us to improve IRS online products.
Link to IRS.gov
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Rebecca Scollan |
| File Modified | 0000-00-00 |
| File Created | 2021-01-28 |