Responsive Payments Formative Usability Study Plan
Page 1 of 25
Table of Contents
Executive Summary....................................................................................................................................... 2
RESEARCH PLAN ............................................................................................................................................ 2
Background and Study Objectives/Goals ............................................................................................... 2
Investigators or Sponsors ....................................................................................................................... 2
Study Participants................................................................................................................................... 2
Study Methodology ................................................................................................................................ 3
Study Schedule ....................................................................................................................................... 5
RESEARCH REPORT........................................................................................................................................ 6
Execution ................................................................................................................................................ 6
Plan Deviations ....................................................................................................................................... 6
Results .................................................................................................................................................... 6
Discussion ............................................................................................................................................... 7
Appendix A – Consent Form ......................................................................................................................... 8
Appendix B – Introduction Script .................................................................................................................. 9
Appendix C – Data Collection Sheets .......................................................................................................... 11
Appendix D – Study Materials..................................................................................................................... 21
Appendix E – Recruitment Email................................................................................................................. 22
Appendix F – Reminder Email (In Person Study) ........................................................................................ 23
Appendix G – Confirmation Email (In Person Study) .................................................................................. 24
Appendix H – Usability/User Research Brief............................................................................................... 25
Responsive Payments Formative Usability Study Plan
Page 2 of 25
Executive Summary
This section will be completed after the study is conducted.
RESEARCH PLAN
This section of the document, along with the appendices, contains the study plan content.
Background and Study Objectives/Goals
This is a formative usability study to determine whether the mobile layout and content of the IRS
Payments pages serve the user’s needs and present any usability issues. The specific objectives
for this study are as follows:
1.
2.
3.
4.
Assess the page layout and usability of the responsive pages vs. non-responsive.
Assess usability of the mobile menu.
Observe and record participant behaviors while completing tasks
Obtain comments and feedback about the product, which can be fed into the design of future
versions of the product.
Investigators or Sponsors
IRS UX Team Lead: Andrew Hughey ([email protected])
IRS UX Team Member: William Woodard ([email protected])
IRS UX Team Support: Michael Weiss ([email protected])
Study Recruitment, Facilitation, and Data Analysis: Daniel Vickers ([email protected])
Study Participants
General description – Individuals who are taxpayers will be recruited for participation. The
recruited participants will represent a mix of demographics including age, gender, amount of
internet usage, and mobile device platforms and operating system experience.
Geographical location – Testing will take place in Miami, Florida.
Sample size - A total of 16 participants will be recruited with a goal of segmenting the sampling
into 8 iOS users and 8 Android users. Due to limitations presented with recruiting this
segmenting may not be exact. The minimum sample size is 10.
Inclusion Criteria - Participants must be U.S. taxpayers. Participants can be government
employees, but must participate in the study on their own time to receive the incentive. Federal
government participants cannot be paid incentives.
Compensation - Participants will be compensated with a $40 Visa gift card for their
participation.
Responsive Payments Formative Usability Study Plan
Page 3 of 25
Study Methodology
Design
Method - Testing will be performed using an in-person moderated study method and will consist
of participants using the prototype and live version of IRS.gov to complete tasks. Participants
will also be asked questions to learn more about their thoughts on the products. The study will be
performed using an A|B format. Each participant will perform tasks using the responsive
IRS.gov prototype and the live IRS.gov website. Participants will complete each task using test
data. None of their personal information will be used during the study session. The test data will
be added in a future version of this study plan.
Measures - Quantitative data will be gathered pertaining to time on task, number of clicks to
complete each task, and task success rate. Qualitative data will be collected by capturing
participants’ thoughts and asking participants open ended questions while completing tasks, as
well as a 7 point ease of use scale scored by the participant after each task.
Equipment – Testing will take place on either an Apple iPhone or an Android smartphone based
upon the participant’s own experience. A laptop will be set up with Morae Recorder software
running. The computer’s onboard camera will be used to record the participant’s reactions, while
a lamp cam will be placed above the device to record activity on the device.
Experimenter - The facilitator of the testing will be Daniel Vickers of Flux Forward.
Tasks
Participants will use the responsive IRS.gov prototype and the live IRS.gov website to complete
four tasks:
1.
2.
3.
4.
Make a payment using a bank account
Make a payment using a credit card
Learn more about scheduling a future payment
Find a local IRS office to contact for help
During each task it should be noted how participants interact with the software (i.e., what menus
are used, what options/information on the screen is noticed, etc.).
Participants will complete each task using test data, so that none of their personal information
will be used during the study session. The test data will be added in a future version of this study
plan.
Task Order - Tasks will be completed in procedural order, but the order in which the participants
interact with the responsive IRS.gov prototype and the live IRS.gov site will be counterbalanced.
This will help to ensure that the order in which the participants interact with products do not
affect the data findings. See the table below for the order in which participants will interact with
the products.
Responsive Payments Formative Usability Study Plan
Participant
#
Responsive
IRS.gov
Prototype
Live
IRS.gov
website
Page 4 of 25
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
2
1
2
1
2
1
2
1
2
1
2
1
2
1
2
1
Task Success Criteria and Usability Metrics Task Success/Failure
Each task will require, or request, that the participant obtains or inputs specific data that would
be used in course of a typical task. The scenario is completed when the participant indicates the
scenario's goal has been obtained (whether successfully or unsuccessfully) or the participant
requests and receives sufficient guidance as to warrant scoring the scenario as a critical error.
Critical Errors
Critical errors are deviations at completion from the targets of the task. Obtaining or otherwise
reporting of the wrong data value due to participant workflow is a critical error. Participants may
or may not be aware that the task goal is incorrect or incomplete.
Independent completion of the scenario is a universal goal; help obtained from the other usability
test roles is cause to score the scenario a critical error. Critical errors can also be assigned when
the participant initiates an action that will result in the goal state becoming unobtainable. In
general, critical errors are unresolved errors during the process of completing the task or errors
that produce an incorrect outcome.
Non-critical Errors
Non-critical errors are errors that are recovered from by the participant or, if not detected, do not
result in processing problems or unexpected results. Although non-critical errors can be
undetected by the participant, when they are detected they are generally frustrating to the
participant.
These errors may be procedural, in which the participant does not complete a scenario in the
most optimal means (e.g., excessive steps and keystrokes). These errors may also be errors of
confusion (ex., initially selecting the wrong function, using a user-interface control incorrectly
such as attempting to edit an un-editable field).
Noncritical errors can always be recovered from during the process of completing the scenario.
Exploratory behavior, such as opening the wrong menu while searching for a function, selecting
help text links, selecting the “Contact Us” links, and footer links will be coded as a non-critical
error.
Subjective Evaluations
Responsive Payments Formative Usability Study Plan
Page 5 of 25
Subjective evaluations regarding ease of use and satisfaction will be collected via questionnaires,
and during debriefing at the conclusion of the session. The questionnaires will utilize free-form
responses and rating scales. Combined with the interview/debriefing session, these data are used
to assess attitudes of the participants.
Task Completion Time (time on task)
The time taken to complete the task (including each page the participant uses during a task), not
including subjective evaluation durations, will be recorded. This data will be captured using time
stamps from recorded session videos.
Task Error Severity - To prioritize recommendations, a method of problem severity classification
will be used in the analysis of the data collected during evaluation activities. The approach treats
problem severity as a combination of two factors - the impact of the problem and the frequency
of users experiencing the problem during the evaluation.
Impact
Impact is the evaluation of the severity of an error and how much it effects the participant’s
ability to complete a task. There are three levels of impact:
High - prevents the participant from completing the task (critical error)
Moderate - causes participant difficulty but the task can be completed (non-critical error)
Low - minor problems that do not significantly affect the task completion (non-critical error)
Frequency
Frequency is the evaluation of how often an error occurs. It is measured by the percentage of
participants who experience the error.
High: 30% or more of the participants experience the problem
Moderate: 11% - 29% of participants experience the problem
Low: 10% or fewer of the participants experience the problem
[For studies with less than ten participants in a group, the percentages may need to be adjusted.
For example, for a study with 8 participants the low frequency should be 12.5% (1/8 = .1250]
Procedure
The study will begin with the participant executing permission to record the session. The
facilitator will begin the recording and then ask the introductory and demographic questions
included in the test plan. The participant will then be asked to perform a series of tasks. After
each task they will be asked to score the ease of use on a scale of 1-7 with 1 being most difficult
and 7 being easiest.
Study Schedule
The objectives addressed by this study are needed to support the decision to develop a responsive
design for the IRS.gov Payments page. Completion of data collection, analysis, and reporting is
planned for September 4th, 2015.
Responsive Payments Formative Usability Study Plan
Page 6 of 25
RESEARCH REPORT
For Plan release: Remove all information below and provide the following statement: “This section will
be completed after the study is conducted. This section will provide the evaluation of the usability metrics
collected and provide formal recommendations.” For Report release: Use the statement below.
This section of the document contains the study report content. Because it was created after the
study was completed, it is written in the past tense. Transcribed raw data can be found in
, and all audio/video files can be in found .
Execution
For Plan release: Remove section. For Report release: Provide a paragraph explaining when the study
occurred, who conducted the session, where the sessions were conducted, and who performed the
analysis.
The study was conducted between and by
in .
Plan Deviations
For Plan release: Remove section. For Report release: Provide a paragraph that indicates whether the
study was conducted as planned or whether deviations in objectives, participants, methods, deliverables,
or data collection tools occurred. If deviations were made, list all deviations and provide rationale for
why the changes occurred.
Results
For Plan release: remove section. For Report release: Start with a presentation of demographic data for
the participant sample. Then describe the outcomes for each task. When describing the outcome for each
task state the exact number of participants that performed each reported action. Subsections, headings,
may be added, removed or retitled as needed.
Participant Background Information
Task 1
TASK DESCRIPTION
List the task details/description
SUCCESSFUL
WITH NO
ERRORS
SUCCESSFUL
WITH NONCRITICAL
ERRORS
FAILURE/CRITI
CAL ERROR
AVERAGE
TIME ON
TASK
(SECONDS)
xxx% (xxx)
xxx% (xxx)
xxx% (xxx)
xx
List task events –
Post-Session Questions
Discussion
For Plan release: remove section. For Report release: Provide a discussion of the results with expert
interpretation of why particular outcomes occurred (e.g., design issues, study method, limitations of
prototype, product issues). This is where you qualify, justify, and/or interpret outcomes. Include
information about how reported deviations affected the study results.
RECOMMENDATIONS
For Plan release: remove section. For Report release, complete the table below. Recommendations
should be solution free statements where possible so the implementation team has room to work out a
feasible approach. List the priority of each action. Consider the severity of the error(s) that led to the
recommendation when determining a recommended action’s priority.
NO.
ACTION
FROM
PRIORIT
Y
1
Design product to user’s need to accurately understand how to make a
tax payment
Task 1
High
2
Add instructional information to help users find the appropriate form.
Task 1, 2, and 3
High
CONCLUSION
For Plan release: remove section. For Report release: This section should provide a clear statement of
whether or not your objectives were met. If objectives were not met, you should discuss whether
redesign and/or retesting is needed. Provide any recommended actions or next steps for the product
development team.
Responsive Payments Formative Usability Study Plan
Page 8 of 25
Appendix A – Consent Form
I agree to participate in the study conducted and recorded by the Internal Revenue Service (IRS).
I understand that the study will take approximately .
I understand and consent to the use and release of the audio, video, and/or digital recording by
the IRS. I understand that the information and recording is for research purposes only and that
my name and image will not be used for any other purpose. I relinquish any rights to the
recording and understand the recording may be copied and used by IRS without further
permission.
I understand that participation in this usability study is voluntary and I agree to immediately raise
any concerns or areas of discomfort during the session with the study administrator.
The Paperwork Reduction Act requires that the IRS display an OMB control number on all
public information requests. The OMB Control Number for this study is 1545-2256. Also, if you
have any comments regarding the time estimates associated with this study or suggestions on
making this process simpler, please write to:
Internal Revenue Service
Tax Products Coordinating Committee
SE:W:CAR:MP:T:T:SP
1111 Constitution Ave. NW
Washington, DC 20224
Please sign below to indicate that you have read and you understand the information on this form
and that any questions you might have about the session have been answered.
Date
Please print your name
Please sign your name
Thank you! We appreciate your participation.
Responsive Payments Formative Usability Study Plan
Page 9 of 25
Appendix B – Introduction Script
Welcome and Purpose
Thank you so much for coming in today. I want to give you a little information about what you will be
looking at and give you time to ask any questions you might have before we get started.
Today we will be asking you to complete a set of tasks. These tasks were derived from actual business
scenarios. Our goal is to use the results of this study to provide us with insights that will help us improve
the design of our online services. The overall goal of our testing is to provide better services to the
American public.
Test Facilitator’s Role
Our role is to record your reactions and comments about the tasks we will present. My colleague(s) will
be helping me to take notes and observe your interactions and responses.
We will not be able to offer any suggestions or hints, but from time to time, we may ask you to clarify
what you have said or ask you for information on what you were looking for or what you expect to have
happened.
Please remember, as you perform the tasks, there are no right or wrong responses. You are helping us
test our programs and designs. If you have a problem it is our problem and we appreciate you letting us
know.
Test Participant’s Role
• Your role today is to perform a series of tasks that we give you.
• There is no right or wrong answer. We are not testing you, we are testing our services.
• If you have any questions, comments or areas of confusion while performing a task, please let us
know. Any problem you have points to a problem we have with our services.
• As you perform a task, please do so as you would at home or your office. I ask that you work through
the tasks based on what you see on screen. If you reach a point where you are not sure where to go or
how to proceed, please feel free to ask one of us.
• We will be recording this session. We will record your expressions, your voice and what you see on the
screen. Your identity will not be associated or reported with any data or findings from this evaluation.
• I may ask questions as you work and then we will finish up with questions at the end.
• Remember: Your decision to participate is completely voluntary. You are free to stop participating at
any time.
Do you have any questions before we begin?
Okay, one last thing before we continue. I need to have you to read the following:
Paperwork Reduction Act Statement Notice
The Paperwork Reduction Act requires that the IRS display an OMB control number on all public
information requests. The OMB Control Number for this study is 1545-2256. Also, if you have
any comments regarding the time estimates associated with this study or suggestions on
making this process simpler, please write to:
Responsive Payments Formative Usability Study Plan
Internal Revenue Service
Tax Products Coordinating Committee
SE:W:CAR:MP:T:T:SP
1111 Constitution Ave. NW
Washington, DC 20224
Page 10 of 25
Responsive Payments Formative Usability Study Plan
Appendix C – Data Collection Sheets
1. Which option best describes your race?
Asian
Black or African American
Hispanic or Latino
Native American or American Indian
Native Hawaiian or Other Pacific Islander
White
2. What is your gender?
Female
Male
3. What category describes your current age?
18 to 24 years
25 to 34 years
35 to 44 years
45 to 54 years
55 to 64 years
65 to 74 years
75 to 84 years
85 years and over
4. What is the highest level of education you have completed?
Less than 9th grade
9th grade to 12th grade, no diploma
High school graduate (or GED)
Some technical or vocational school
Technical or vocational school graduate
Some college, no degree
Associate Degree
Bachelor’s Degree
Master’s Degree
Post-Master’s Degree
5. What category best describes your annual household income?
Less than $15,000
$15,000 but less than $25,000
$25,000 but less than $35,000
$35,000 but less than $50,000
$50,000 but less than $75,000
$75,000 but less than $100,000
$100,000 or more
6. What is the primary language spoken at home? (Please select one)
English
Spanish
Chinese
Vietnamese
Korean
Russian
Other (please specify):__________
7. Have you ever made a payment online to the IRS?
Page 11 of 25
Responsive Payments Formative Usability Study Plan
Yes
No
8. If yes, what methods have you use?
Pay by card
Electronic Funds Withdrawal
Same Day Wire
Check or Money Order
Other
9. Have you ever used IRS.gov?
Yes
No
10. If yes, how often do you use IRS.gov?
Seldom
Sometimes
Often
Page 12 of 25
Responsive Payments Formative Usability Study Plan
Page 13 of 25
Responsive IRS.gov Prototype
Task 1 – Make a payment from bank account
You have an outstanding balance to the IRS and have made a payment arrangement. You need to
schedule your first payment to be withdrawn from your bank account. Using the site show me
how you would go about doing this. Please use the provided bank account information when
setting up your bank payments.
SUCCESS
(CIRCLE ONE)
PATHWAY(S)
☐ Select the Direct Pay
button on the payments
landing page
☐ Select the make a
payment option
☐ Complete the form
ERROR INFORMATION
0
Failure/Critical
Error
Possible critical error:
Participant is unable to make a payment using a
bank account.
1
Completed with
non-critical error
Possible non-critical error:
The participant has difficulty finding the finding
the button to make a payment.
2
Completed with
no errors
NOTES/OBSERVATIONS
(Note: why was the participant successful or not successful, e.g., wrong pathways, confusing page
layout, navigation issues, terminology, record number of times an incorrect/unexpected button or link is
pressed)
1. Overall, how difficult or easy was it to complete that process?
Very
Difficult
Very
Easy
1
2
3
4
5
6
7
Responsive Payments Formative Usability Study Plan
Page 14 of 25
Task 2 – Make a payment by credit card
You’ve decided that you would rather make your payment by credit card. Using the provided
credit card payment information, how would you go about doing this?
SUCCESS
(CIRCLE ONE)
PATHWAY(S)
☐ Select the Pay by
card button on the
payments landing page
☐ Select the make a
payment option
☐ Complete the form
0
Failure/Critical
Error
1
Completed with
non-critical error
ERROR INFORMATION
Possible critical error:
Participant is unable to make a payment using a
bank account.
Possible non-critical error:
2
Completed with
no errors
The participant has difficulty finding the
finding the button to make a payment.
Participant doesn’t understand various
payment processor options.
NOTES/OBSERVATIONS
(Note: why was the participant successful or not successful, e.g., wrong pathways, confusing page
layout, navigation issues, terminology, record number of times an incorrect/unexpected button or link is
pressed)
1. Overall, how difficult or easy was it to complete that process?
Very
Easy
Very Difficult
1
2
3
4
5
6
7
Responsive Payments Formative Usability Study Plan
Page 15 of 25
Task 3 – Learn more about scheduling a future payment
Now that you have completed all your payments you are curious if it is possible to use the Direct
Pay process to schedule future payments. Using the software please find additional information
that discusses if you are able to use the systems to setup future payments.
SUCCESS
(CIRCLE ONE)
PATHWAY(S)
☐ Navigation menu to
get to payments home
screen
☐ Select IRS Direct
Pay button
☐ Select Additional
Information button
☐ Find and select the
“Can I schedule a future
payment” accordion
menu option
0
Failure/Critical
Error
ERROR INFORMATION
Possible critical error:
1
Completed with
non-critical error
2
Completed with
no errors
The participant is unable to get to the
appropriate webpage
The participant is unable find if a future
payment can be scheduled.
Possible non-critical error:
The participant has difficulty finding the
appropriate webpage
The participant has difficulty finding if a
future payment can be scheduled.
NOTES/OBSERVATIONS
(Note: why was the participant successful or not successful, e.g., wrong pathways, confusing page
layout, navigation issues, terminology, record number of times an incorrect/unexpected button or link is
pressed)
1. Overall, how difficult or easy was it to complete that process?
Very
Easy
Very Difficult
1
2
3
4
5
6
7
Responsive Payments Formative Usability Study Plan
Page 16 of 25
Task 4 – Find contact information
You have questions about your taxes and would like to talk to someone in person. Using the website
find the closest place to you to accomplish this. Please use the provided personal information when
determining the closest IRS office location.
SUCCESS
(CIRCLE ONE)
PATHWAY(S)
☐ Footer
☐ Contact Your Local
Office
☐ Choose your state
0
Failure/Critical
Error
ERROR INFORMATION
Possible critical errors:
1
Completed with
non-critical error
2
Completed with
no errors
The participant is unable to find the contact
link in the footer
The participant is unable to find the correct
local office to contact
Possible non-critical error:
The participant has difficulty finding the link
in the footer
The participant has difficulty selecting the
correct local office to contact
NOTES/OBSERVATIONS
(Note: why was the participant successful or not successful, e.g., wrong pathways, confusing page
layout, navigation issues, terminology, record number of times an incorrect/unexpected button or link is
pressed)
1. Overall, how difficult or easy was it to complete that process?
Very
Easy
Very Difficult
1
2
3
4
5
6
7
Responsive Payments Formative Usability Study Plan
Page 17 of 25
Live IRS.gov Website
Task 1 – Make a payment from bank account
You have an outstanding balance to the IRS and have made a payment arrangement. You need to
schedule your first payment to be withdrawn from your bank account. Using the site show me
how you would go about doing this. Please use the provided bank account information when
setting up your bank payments.
SUCCESS
(CIRCLE ONE)
PATHWAY(S)
☐ Homepage
☐ Navigation menu
☐ Payments landing
page
☐ Select the pay by
bank account option
ERROR INFORMATION
0
Failure/Critical
Error
Possible critical error:
Participant is unable to make a payment using a
bank account.
1
Completed with
non-critical error
Possible non-critical error:
The participant has difficulty finding the finding
the button to make a payment.
2
Completed with
no errors
NOTES/OBSERVATIONS
(Note: why was the participant successful or not successful, e.g., wrong pathways, confusing page
layout, navigation issues, terminology, record number of times an incorrect/unexpected button or link is
pressed)
1. Overall, how difficult or easy was it to complete that process?
Very
Difficult
Very
Easy
1
2
3
4
5
6
7
Responsive Payments Formative Usability Study Plan
Page 18 of 25
Task 2 – Make a payment by credit card
You’ve decided that you would rather make your payment by credit card. Using the provided
credit card payment information, how would you go about doing this?
SUCCESS
(CIRCLE ONE)
PATHWAY(S)
☐ Homepage
☐ Navigation menu
☐ Payments landing
page
☐ Select the pay by
credit card option
0
Failure/Critical
Error
1
Completed with
non-critical error
ERROR INFORMATION
Possible critical error:
Participant is unable to make a payment using a
bank account.
Possible non-critical error:
2
Completed with
no errors
The participant has difficulty finding the
finding the button to make a payment.
Participant doesn’t understand various
payment processor options.
NOTES/OBSERVATIONS
(Note: why was the participant successful or not successful, e.g., wrong pathways, confusing page
layout, navigation issues, terminology, record number of times an incorrect/unexpected button or link is
pressed)
1. Overall, how difficult or easy was it to complete that process?
Very
Easy
Very Difficult
1
2
3
4
5
6
7
Responsive Payments Formative Usability Study Plan
Page 19 of 25
Task 3 – Learn more about scheduling a future payment
Now that you have completed all your payments you are curious if it is possible to use the Direct
Pay process to schedule future payments. Using the software please find additional information
that discusses if you are able to use the systems to setup future payments.
SUCCESS
(CIRCLE ONE)
PATHWAY(S)
☐ Navigation menu to
get to payments home
screen
☐ Select IRS Direct
Pay button
☐ Select Additional
Information button
☐ Find and select the
“Can I schedule a future
payment” accordion
menu option
0
Failure/Critical
Error
ERROR INFORMATION
Possible critical error:
1
Completed with
non-critical error
2
Completed with
no errors
The participant is unable to get to the
appropriate webpage
The participant is unable find if a future
payment can be scheduled.
Possible non-critical error:
The participant has difficulty finding the
appropriate webpage
The participant has difficulty finding if a
future payment can be scheduled.
NOTES/OBSERVATIONS
(Note: why was the participant successful or not successful, e.g., wrong pathways, confusing page
layout, navigation issues, terminology, record number of times an incorrect/unexpected button or link is
pressed)
1. Overall, how difficult or easy was it to complete that process?
Very
Easy
Very Difficult
1
2
3
4
5
6
7
Responsive Payments Formative Usability Study Plan
Page 20 of 25
Task 4 – Find contact information
You have questions about your taxes and would like to talk to someone in person. Using the website
find the closest place to you to accomplish this. Please use the provided personal information when
determining the closest IRS office location.
SUCCESS
(CIRCLE ONE)
PATHWAY(S)
☐ Footer
☐ Contact Your Local
Office
☐ Choose your state
0
Failure/Critical
Error
ERROR INFORMATION
Possible critical errors:
1
Completed with
non-critical error
2
Completed with
no errors
The participant is unable to find the contact
link in the footer
The participant is unable to find the correct
local office to contact
Possible non-critical error:
The participant has difficulty finding the link
in the footer
The participant has difficulty selecting the
correct local office to contact
NOTES/OBSERVATIONS
(Note: why was the participant successful or not successful, e.g., wrong pathways, confusing page
layout, navigation issues, terminology, record number of times an incorrect/unexpected button or link is
pressed)
1. Overall, how difficult or easy was it to complete that process?
Very
Easy
Very Difficult
1
2
3
4
5
6
7
Responsive Payments Formative Usability Study Plan
Page 21 of 25
Appendix D – Study Materials
TBD – Paper printouts of tasks, and screenshots of the screens that users will interact with will
be added in a future version of the study plan.
Responsive Payments Formative Usability Study Plan
Page 22 of 25
Appendix E – Recruitment Email
SUBJECT LINE: Internet Study: Let the IRS pay to serve you better!
Hello,
My name is Daniel Vickers. I do research for the IRS and we plan to conduct a usability study. The goal
of this study is to improve our services by asking people to try out our prototype website and give us
feedback. If you qualify, you will receive $40 to particípate.
What will you do in the usability study?
You will be asked to do several short tasks to evaluate a prototype website. We may also ask you
questions about your thoughts and perceptions of the website.
How long is a session?
The testing will take about 30 minutes.
When and where?
The study will be held August 16th and 17th 2015. You will be asked to participate at 350 Lincoln Rd.,
Miami Beach.
Interested in participating?
Please reply to this email with your contact information. When we talk I will ask you a short list of
questions to help decide if you qualify for the study.
If you have questions, please contact me at [email protected].
Thank you for interest,
Daniel Vickers
Flux Forward
Responsive Payments Formative Usability Study Plan
Page 23 of 25
Appendix F – Reminder Email (In Person Study)
SUBJECT LINE: Reminder: Website study tomorrow
Dear :
Thanks again for agreeing to help test a prototype website. We are looking forward to talking
with you.
You are scheduled to participate as follows:
DATE:
TIME: