OMB Package
Usability Study for Short Term eTranscripts
Introduction
The purpose of the short term eTranscripts project is to take the phone capability and provide it on the Web. The application will allow taxpayers to go onto IRS.gov and order a transcript via the registered user portal.
Background
Currently taxpayers are required to call the 1-800 phone number to order a transcripts of the account or their return. Phase 1 of the project will create a phone line specific for handling transcript requests. The other option is for the taxpayer to complete Form 4506-T and mail it into the IRS. This can be a long and cumbersome process. This application will allow taxpayers the ability to order a transcript on the IRS.gov Web Site via the Registered User Portal.
The taxpayer will be authenticated prior to being allowed to order the transcript. The transcript will be sent to the address of record with the IRS.
Objectives of Research
The purpose of usability testing the short term eTranscripts is to determine if it is meeting user needs and if users are able to clearly navigate through the application without confusion from content and navigation flow. Throughout usability testing we will gather information based on the following…
The following are examples of objectives that address these areas.
Content
Content needs to be clear and concise
Writing for the web practices need to be used
Bullets, instead of long paragraphs
Examples if necessary
Help Content is easily understood
Information displayed regarding user’s tax information is easily understood
Navigation
Can user’s move through the application without feeling lost
Error recovery rate
Do user’s understand where they are within the application
Methodology
We will be utilizing a prototype of the application where users will be completing a variety of scenarios using dummy data.
We will be recording these sessions including, the screens in order to correctly identify problems with the usability of the application. The user will be recorded so that the usability research team can evaluate body language in addition to what the user may state as positive and negative feedback.
Data will be collected to evaluate the following areas;
Time spent to complete a task
Navigation paths used
User satisfaction ratings
Umber of times user required help
Task success rate
Error Recovery
Sample Design
There will be a total of 10 participants. The test will have a user complete a variety of tasks using the eTranscripts application. The participant will be asked to talk-aloud as they complete the tasks.
Testing will be based on scenarios created by the Wage and Investment business operating division in the IRS. Scenarios are short statements that include a task to be completed by using MIRSA application.
The moderator will complete an introduction, and provide an overview of the test process and ask the participant to sign a consent form. The form will explain what the participant is going to be doing and how the IRS plans to use the information gathered from testing.
No personally identifiable information will be shared during testing.
After the introduction each participant will work each scenario individually on a computer. Participants will be required to complete each task without assistance from the moderator. They will be asked to “think aloud”, so that the usability team can capture as much data as possible.
When the participant has completed the scenarios there will be a debriefing session that will consist of open discussion and an exit questionnaire. We will be awarding $85 for participants.
Participants will use a PC running Windows XP with a 17-inch monitor. The screen resolution of the monitor will be set to 800 x 600 with Windows display properties set to “Small Fonts”. Participants will access the product by using Microsoft Internet Explorer 6.0.
The testing room has a desk with a computer that the participant will be using to complete the test. The room also has a two way mirror. Behind the mirror is the rest of the usability testing team who will be observing the testing.
Data Collection Date
Testing will occur September 21-24, 2010.
Data to be collected
This test will gather usability information regarding the eTranscripts application. The testing team will gather information metrics for the following
Time spent to complete a task
Navigation paths used
Search vs. navigation
User satisfaction ratings (Post Test Questionnaire)
Number of times user required help
Task success rate
Error recovery
The participant will be monitored by having a moderator sitting with them. Testing sessions are expected to last approximately 60 minutes per participant.
How data will be used
Data will be used to evaluate taxpayers’ satisfaction and identify areas for improvement for the eTranscripts application. All information collected is strictly for research, and will not be used to personally identify participants nor will it be shared for commercial purposes.
How data will be analyzed
After data collection the testing team will evaluate the key metrics. In addition the scenarios in which the users were asked to complete will be evaluated. Was the user able to complete the task?
Analysis of the usability testing data will be conducted by Wage & Investment (W&I), Electronic Tax Administration’s (ETA) Portal Business Management Division (PBM. Observations are going to be used to asses the performance of the application.
Who is conducting research?
The Portal Business Management Division will be conducting the research using the Ogden Usability Testing Lab. The test team includes 3 PBM team members, IT developers and the business unit.
The following roles are required before and during the period over which usability testing takes place:
Planner and coordinator – Usability Administrator ensures that everything is in place to allow the usability test sessions to take place as planned and without problems.
Participant recruiter – recruits participants and schedules them for available time slots; provides test facilitator with honorarium checks to give to each participant at the end of their test session.
The following roles are required in order to conduct a usability test session:
Test facilitator –Usability Consultant oversees the test itself, introduces and explains the test session to the participant, observes participant’s actions and obtains participant feedback at completion of test session, conducts issues assessment session. The facilitator is also available to answer questions and ensures that the observer stakeholders stay on task and have a productive and successful experience.
Test data logger – creates a written record of the participant’s actions and comments as they use the site using data logging software, a word processor, or a spreadsheet.
Test Team Lead – responsible for coordination meetings and set up for testing.
Observer-team member – observes test to identify functionality and performance, compares notes with other team members, and actively participates in Issues Assessment at end of each test session.
Cost of Study
The one-time cost for this usability test is $ 15,000.
Location
Testing will occur in Ogden, UT at the IRS’ Ogden Usability Lab.
Stipend
An $85 per participant stipend will be paid to each participant. A stipend is required in order to get participants for the usability testing.
Recruitment Efforts
Consumer Opinion Services, an external market research firm, will recruit 10 participants from one user group: Individuals. Participants will be recruited the week prior to usability testing, with the usability study being occurring between September 21, 2010 and September 23, 2010.
Participant Criteria
External Customers |
Target # |
Individuals – Individual taxpayers to include new/first time taxpayers, homemakers, retired citizens, employed individuals. |
10 |
General knowledge of the internet with frequent use of the internet
General knowledge of tax filing and processing
Knowledge of / responsible for individual tax filing
Burden Hours
Individual Participants:
Total Screened: 30 @ 5 minute screening interview = 2.5 hours
Total Recruited: 10 testers
Travel Time estimated 60 minutes per participant to travel to testing facility and return = 6 hours
Total testing time averaged at 1 hour/participant = 10 hours
Total
estimated burden is 18.5 hours
Efforts to not duplicate research
Usability testing has not been conducted on the eTranscripts application previously.
Privacy, Security, Disclosures
The IRS will ensure compliance with the Taxpayers Bill of Rights II. All participants will be treated fairly and appropriately.
During the testing sessions, participants will be identified to IRS personal by their first names only. In addition, no participant names will be mentioned in the usability study final report. Participants will be advised that observers from the IRS may be observing the session and via video monitoring equipment and that their comments may be recorded to ensure accurate data capture.
Attachments
Telephone/ Participant Screeners Questionnaire
Recruiting Service Cover Letter
Moderator Checklist
Consent Form
Scenarios
Exit Questionnaires
Confirmation Phone Call Script
| File Type | application/msword |
| File Title | OMB Package |
| Author | 401fb |
| Last Modified By | qhrfb |
| File Modified | 2010-07-15 |
| File Created | 2010-07-15 |