eAuthentication Usability Study

Collection of Qualitative Feedback on Agency Service Delivery

eAuth Usability Test Script and Materials

eAuthentication Usability Study

OMB: 1545-2256

Document [docx]
Download: docx | pdf

eAuthentication Usability Test

Test Materials







eAuthentication Usability Test Script


Welcome and Purpose


Thank you so much for coming in today. I want to give you a little information about what you will be looking at and give you time to ask any questions you might have before we get started.


Today we are asking you to evaluate a website and complete a short task. Our goal is to find out how easy it is or how difficult it is for you to use the site


Test Facilitator’s Role


I am here to record your reactions to and your comments about the website. In a conference room nearby, I have a colleague helping me take notes and observe your interactions with the site as well.


During this session, I would like you to think aloud as you work to complete the tasks. I will not be able to offer any suggestions or hints, but from time to time, I may ask you to clarify what you have said or ask you for information on what you were looking for or what you expect to have happen.


Demonstrate how to think out loud.

Test Participant’s Role


Today I am going to ask you to find some information and then tell me how easy or difficult it was. There are no right or wrong answers. This is an evaluation of the website, not you! If you have any questions, comments or feel confused about anything while you are working, please let me know.

If you ever feel that you lost or that you cannot complete the task, please let me know. I will ask you what you might do in a real-world setting and then either put you on the right track or move you on to the next step in the process.


I will provide you with a made up name, address and other credentials to use on the website today. As you work through the tasks, please try and act as if you are at home or work, but use the fake credentials in place of your own.



Consent: Request participant read and sign consent/recording release form.



Each session is recorded. We will be recording your face, your voice and what you see on the screen. Your actual name will not be used or reported with data or findings associated with this evaluation.


I’ll ask you questions as we go along and we will have wrap up questions at the end.


Do you have any questions before we begin?


Pre-Task Questions


Ok, let’s start with a few questions.


First, please fill out this questionnaire. Provide pretest demographic and background information questionnaire.


Have you ever visited IRS.gov before? If so, can you recall the purpose of your visit?


Thank you. Let’s get started.


Usability Tasks


Today you’ll be using the name, address and credentials of [USER PERSONA], a fake persona. Please use these credentials and pretend you are [USER PERSONA] today while you use the website.


Task 1


Let’s say that [USER PERSONA] has a meeting about a loan at the bank later today. You’ve been asked to bring in a copy of your return transcript from 2013.


[Open a web browser to http://www.irs.gov/Individuals/Get-Transcript]


Use this website to get your return transcript from 2013. Remember, you are pretending to be [USER PERSONA], so where needed, use those credentials. For questions where you are not provided information, simply pretend you are [USER PERSONA] and make up a response that seems to fit.


Single Ease Question (on landing on the Get Transcript page, hand the participant SEQ form): Overall, how difficult or easy was it to complete the sign in process?


Ok, thanks. Please complete the task.


Thank you! Go ahead and log out.


Task 2


Now let’s say it is a few days later and the loan officer needs return transcripts from 2012 and 2011. Log in as [USER PERSONA] and get PDFs of the transcripts to email to them.


Single Ease Question (on landing on the Get Transcript page, hand the participant SEQ form): Overall, how difficult or easy was it to complete the sign in process?


Post-Task Questions


Thanks. We’ve completed the tasks for our session. I have a few more questions for you.


Ask any follow up questions based on the participants experience with eAuthentication.


That’s it for our session! Do you have any questions or additional comments?




Post Test Check List:

  • Incentive

  • Sign Incentive Receipt

  • Close and save Morae recording

  • ID number on notes, consent forms, receipt





Moderator Checklist



  1. Look at and remember person’s name prior to going to meet him/her.

  2. Introduce yourself and other team members who might interact with participant.

  3. Ask if person would like something to drink and/or to use the restroom.

  4. Ask person to read and sign the consent form.

  5. Reference the OMB Control Number and PRA Statement (provide printed copy):


The Paperwork Reduction Act requires that the IRS display an OMB control number on all public information requests. The OMB Control Number for this study is 1545-2256. Also, if you have any comments regarding the time estimates associated with this study or suggestions on making this process simpler, please write to:


Internal Revenue Service

Tax Products Coordinating Committee

SE:W:CAR:MP:T:T:SP

1111 Constitution Ave. NW

Washington, DC 20224


  1. Review the purpose of the session.

  2. Ask participants if they have any questions.

  3. Ensure that participant completes post-study questionnaire and discuss his/her responses with him/her. Ask if (s)he has any other comments about the site.

  4. Have participant sign receipt form and record honorarium check number on signed form.


Paperwork Reduction Act Statement




The Paperwork Reduction Act requires that the IRS display an OMB control number on all public information requests. The OMB Control Number for this study is 1545-2256. Also, if you have any comments regarding the time estimates associated with this study or suggestions on making this process simpler, please write to:



Internal Revenue Service

Tax Products Coordinating Committee

SE:W:CAR:MP:T:T:SP

1111 Constitution Ave. NW

Washington, DC 20224


Consent & Recording Release Form


I agree to participate in the study conducted and recorded by the Internal Revenue Service (IRS). I understand that the study will take approximately 40 minutes.


I understand and consent to the use and release of the audio, video, and/or digital recording by the IRS. I understand that the information and recording is for research purposes only and that my name and image will not be used for any other purpose. I relinquish any rights to the recording and understand the recording may be copied and used by IRS without further permission.


I understand that participation in this usability study is voluntary and I agree to immediately raise any concerns or areas of discomfort during the session with the study administrator.


The Paperwork Reduction Act requires that the IRS display an OMB control number on all public information requests. The OMB Control Number for this study is 1545-2256. Also, if you have any comments regarding the time estimates associated with this study or suggestions on making this process simpler, please write to:


Internal Revenue Service

Tax Products Coordinating Committee

SE:W:CAR:MP:T:T:SP

1111 Constitution Ave. NW

Washington, DC 20224



Please sign below to indicate that you have read and you understand the information on this form and that any questions you might have about the session have been answered.




Date


Please print your name


Please sign your name





Thank you! We appreciate your participation.



Pretest Demographic and Background Information Questionnaire

­­­­­­­­­­­­­­­­­­­­


1. What is the primary language spoken at home? (Please select one)


English Korean

Spanish Russian

Chinese Other (please specify):________

Vietnamese


2. What payment option do you use the MOST to pay your bills?


Mail in payments

In person

Over the phone

Through the Internet via scheduled electronic bill pay

Through the Internet via same day bill pay


3. Do you have a computer at home?


Yes

No


4. Do you own and use a mobile phone?


Yes, regular mobile phone

Yes, Smartphone (mobile phone with applications and internet access)

No


5. Which of the following social media sources do you use? (Check all that apply)


Facebook Tumblr

Twitter Other (specify)______________

YouTube None

LinkedIn




Single Ease Question



Task One


Overall, how difficult or easy was it to complete the login process?



Very Difficult






Very

Easy

1

2

3

4

5

6

7




Task Two


Overall, how difficult or easy was it to complete the login process?



Very Difficult






Very

Easy

1

2

3

4

5

6

7





Recruitment Email

When possible, we will recruit using listserves or known pools of participants.


SUBJECT LINE: The IRS invites you to participate in a website study and earn $40!

Hello,


My name is [NAME], and I’m helping to run a usability study for the Internal Revenue Service (IRS). In an effort to improve services, we’re looking for people who are interested in exploring new ways to access their tax records. Afterward we will simply ask for your feedback. If you participate we will pay you $40.


What will you be doing in a usability study?

You will be asked to perform several short tasks using a website. You will also be asked questions about your experience and what you think of the website.


How long is a study session?

40 minutes.


When and where?

The study will be held [DAYS, DATES, LOCATION]. You will need to participate in person.


Interested in participating?

Please reply to this email with your contact information or call me at [ADD PHONE]. I will give you a call to ask you some questions so we can decide if you qualify for the study.


If you have any questions, please contact me at [EMAIL].


Thank you for your interest,


[NAME AND TITLE]



Confirmation Email

NOTE: This will be sent as an email or in an Outlook meeting notice.


SUBJECT LINE: Confirmation: Your participation in our usability study


Dear [PARTICIPANT NAME]:


Thank you for agreeing to participate to test the Internal Revenue Service website. As I mentioned, you will be asked to perform a few tasks on the website and give us thoughts about your experience. There is nothing to prepare before the session.


You are scheduled to participate as follows:


DATE: [DAY, DATE]

TIME: [TIME]

PLACE: [ADDRESS, LINK TO MAP]

A few key reminders:


  • You will be given $40 in exchange for your participation. (NOTE: Government employees and/or grantees are not eligible for the honorarium unless you are participating on your own time.)

  • During the study, we will ask you to complete some tasks using the website. You’ll talk out loud as you work so the facilitator can follow along.

  • With your permission, the session will be recorded. We will only use the recording in our research to decide how to improve the website. Your name will not be used for any purpose beyond this session.


Also, we have only one person scheduled at a time for these sessions so if you find that you cannot participate on your scheduled day, please contact me as soon as possible so we can reschedule your session.


Thank you again!


[NAME OF FACILITATOR AND CONTACT INFORMATION]



Reminder Email


SUBJECT LINE: Reminder: Website study tomorrow


Dear [PARTICIPANT NAME]:


Thanks again for agreeing to help us out with testing the Internal Revenue Service website. We are looking forward to talking with you.

You are scheduled to participate as follows:


DATE: [DAY, DATE]

TIME: [TIME]

PLACE: [ADDRESS, LINK TO MAP]


A few key reminders:


  • You will be given $40 in exchange for your participation. (NOTE: Government employees and/or grantees are not eligible for the honorarium unless you are participating on your own time.)

  • During the study, we will ask you to complete some tasks using the website. You’ll talk out loud as you work so the facilitator can follow along.

  • With your permission, the session will be recorded. We will only use the recording to decide how to improve the website. Your name will not be used for any purpose beyond this session.


Also, we have only one person scheduled at a time for these sessions so if you find that you cannot participate on your scheduled day, please contact me as soon as possible so I can reschedule your session.


Thanks again!


[NAME OF FACILITATOR AND CONTACT INFORMATION]



Receipt Form: Usability Test Compensation


Please sign below to indicate that you have received the promised compensation for your participation in testing today.




Date


Amount received


Please print your name


Please sign your name






Thank you!


We appreciate your participation.



Recruiting Screener


Hello, I’m [NAME] from [COMPANY] calling on behalf of the Internal Revenue Service. The IRS wants to hear from you about how we are doing. Have you heard of our website IRS DOT GOV? What we would like to do is ask you a few questions to see if you qualify to participate in a 40 minute study we are conducting of our web site. If you do qualify, we’d like to schedule you as a participant during the week of [DATE]. We are very interested in getting taxpayers opinions on the design of our website. These evaluations will be used for research purposes only and any personal information you provide will not be used for anything other than this particular study. We will pay you to come to a testing facility and give us feedback on how we are doing. Are you be willing to answer a few questions to see if you qualify?


IF NO, Can I schedule a time to call you back at a more convenient time? IF NO, terminate. IF YES, schedule a time to call back.


IF YES, CONTINUE.


[Reference the OMB Control Number and PRA Statement as noted below before continuing with the interview]


The Paperwork Reduction Act requires that the IRS display an OMB control number on all public information requests. The OMB Control Number for this study is 1545-2256. Also, if you have any comments regarding the time estimates associated with this study or suggestions on making this process simpler, please write to us at:


Internal Revenue Service

Tax Products Coordinating Committee

SE:W:CAR:MP:T:T:SP

1111 Constitution Ave. NW

Washington, DC 20224


Because we are seeking a wide variety of people, let me start by asking you some questions about yourself.


  1. Have you ever taken part in a market research interview or usability study, either one on one or in a focus group type setting? IF NO SKIP TO Q4


  1. When was the last time you participated in a market research interview or usability study? TERMINATE IF LESS THAN 6 MONTHS


  1. Do you or any member of your immediate family currently work for or did they ever work for any of the following?



Marketing, Market Research Firm


Advertising


Web Site Design Company


IF YES TO ANY, TERMINATE


For a city, state or federal government agency? If so, which one?


IF IRS, TERMINATE




  1. Do you have access to the internet where you can use it for your own personal use?
    YES

    NO TERMINATE


  1. On a scale of 1 to 5 where 1 means ‘not at all comfortable’ and 5 means ‘very comfortable’, how comfortable are you with… TERMINATE IF 1 OR 2


using a computer mouse

1

2

3

4

5

using a computer keyboard

1

2

3

4

5

using the Internet

1

2

3

4

5

using a scroll bar

1

2

3

4

5








  1. Do you use the Internet to do any of the following?


e-commerce (shop online)

YES

NO

Surf the web

YES

NO

pay bills online

YES

NO

work related reasons

YES

NO

research

YES

NO

online banking

YES

NO

pay taxes online/software

YES

NO

TERMINATE IF REPLY NO TO ALL


  1. Are you employed? IF NOT EMPLOYED SKIP TO 8A


If employed:


What is your occupation?


What industry are you in?



GET A GOOD MIX OF DIFFERENT OCCUPATIONS AND INDUSTRIES


8a. If Not Employed:


Retired


Student


Homemaker


In between jobs

GET AT LEAST 1 NOT EMPLOYED.





  1. What is the highest level of education you have completed?



DO NOT READ ANSWER CHOICES - GET A MIX



Some high school or less


High school graduate


Some college, vocational or trade school


College graduate


Graduate school


  1. Which age group from the following list do you belong?

GET A MIX - NO MORE THAN THREE PARTICIPANTS PER AGE GROUP


Under 18

TERMINATE

18-29


30-39


40-49


50-59


60-65


66+




  1. Have you ever visited the website IRS DOT GOV?


YES

NO SKIP TO Q17


  1. What was the nature of your visit to the website?



to find forms


to review tax regulations


to find phone numbers


file downloads


online tax tools


other


  1. Do you remember if the visit to the site was:



less than 15 minutes


15 minutes to 30 minutes


30 minutes to 1 hour


longer than 1 hour


Don’t remember



  1. Approximately how many times have you visited the website?



  1. What ways have you ever contacted the IRS?



visited a local branch


phone call


spoken with a specific representative


other


Thank you, now, the following questions are for classification purposes only.


  1. What is your current marital status? RECRUIT A MIX



Single


Married/Living with partner


Widowed


Divorced/Separated



RECRUIT TO OBTAIN 30 INDIVIDUALS


RECRUIT A MIX OF MEN AND WOMEN

Thank you for answering my questions. Based on your responses, you appear to qualify for participation in the study. Are you interested in helping by participating in a paid research project?


No POLITELY DISCONTINUE

Yes CONTINUE


The IRS will pay you $40, to test a portion of our web site and share your opinion with us in a 40 minute evaluation.

This is how the process works: As part of our research, we are conducting study sessions with a number of people from your area. If you choose to participate we will ask you to attempt to do specific tasks on a web site and then discuss your experience. We are evaluating our web site so we can better serve taxpayers. We will simply observe your experience and ask your opinions. The IRS will not use any personal information for anything other than this particular study. These evaluation sessions will be held on [DATE, TIME AT LOCATION]. They will last approximately 40 minutes. Would you be available for that particular date and time?


No POLITELY DISCONTINUE

Yes CONTINUE


Do you have any questions up to this point or special needs for coming to our facility?


[Answer inquiries appropriately and thoroughly.]


Again, let me thank you for your decision to participate. This research is very important to us, and your help is highly appreciated.


Now, for the rest of the steps in the process:


After I have gathered your information, we will send you a confirmation email with a map to the testing facility. You will also receive a reminder email prior to the evaluation session. In a moment, I will need your name and email address. I’d also like to get a phone number where we can reach you.


We will send you a reminder email prior to your scheduled appointment on [DATE, TIME]. We schedule one person at a time for these sessions so if you find that you cannot participate on your scheduled day, please give notice to [NAME, EMAIL ADDRESS, PHONE NUMBER] as soon as possible so we can reschedule your session.


Thank you. If you have any questions, please email [NAME] at [EMAIL].




User Personas



Crystal Glass


YOUR PERSONAL INFORMATION

Name:

Crystal Glass

Social Security No.

211-22-1236

Address:

11568 Golden Ave.

Orlando, FL 40706

Date of Birth:

March 12, 1981

e-Mail Address:


Phone Number:

407-558-5524


YOUR LEGAL INFORMATION

Tax Return Data:

Filing Status – Single



Blue Berry


YOUR PERSONAL INFO

Name:

Blue Berry

Social Security No.

211-22-1237

Address:

55684 Boston Terrace

Tampa, FL 44598

Date of Birth:

May 12, 1981

e-Mail Address:


Phone Number:

407-788-5524


YOUR LEGAL INFO

Tax Return Data:

Filing Status – Qualifying Widow



IRS eAuthentication Usability Study | OMB Control Number 1545-2256

File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
AuthorRebecca Scollan
File Modified0000-00-00
File Created2021-01-28

© 2024 OMB.report | Privacy Policy