SOI-323 -Supporting Statement

OMB Package

A Customer Satisfaction Study for Taxpayers, Tax Practitioners and Businesses using

the IRS.gov Website – Usability Study

Introduction

During the past 5 years, the IRS.Gov website has doubled in traffic, seen a double digit increase in user satisfaction scores and won numerous awards. With a 2006 average of 3.7 million pages viewed per day, peak usage of over 24 million page views on April 17, 2006, and a yearly volume of 1.3 billion, www.irs.gov is the most heavily used federal government website. Based upon this success, the IRS would like to enhance the website to include an even better user experience.

The Website Updates effort is an implementation of various design changes to support the display of the website in a new default resolution layout, usability recommendations, eGovernment mandates, Operations & Maintenance (O&M) Change Requests, and Security Change Requests. The website is also implementing a Rotating Spotlight feature. The Rotating Spotlight is an area of the user interface that updates with article content on a set interval basis. This feature requires usability testing to be conducted in order to determine if an optimal user experience has been achieved.

The Search portion of the website has also received design modifications. These modifications are collectively called Enhanced Search. Usability testing will be conducted on the Enhanced Search feature to determine if it is improving the user’s experience.

The website also contains an application that allows users to select tax-related forms from a list of static files. This application is called Picklists. The Picklists have received some design changes, and usability testing will therefore be conducted on them.

Background

In response to a Congressional directive, the IRS conducted a comprehensive review of its current portfolio of taxpayer services to create a Taxpayer Assistance Blueprint (TAB). The TAB team identified the need to improve website self-service applications. Usability testing conducted in 2006 found problems with some of the various applications on the IRS.gov web site. Due to this, the website underwent changes with the intention of creating an improved user experience. These changes are collectively referred to as the IRS.gov Website Updates.

Objectives of Research

The purpose of this test is to determine if the Website Updates are logical and easy to understand.

The Usability testing is intended to gather user data on the overall navigation and user interface design of the site.

We will determine the user’s ability to recognize and navigate to some of the various locations on the web site.

Methodology

The test is designed to take approximately 90 minutes for each participant to complete. Test participants will consist of individuals, tax practitioners, and business owners that prepare or have knowledge about tax filing. Testing will occur at the Ogden Usability Lab in Ogden, UT. Testing will occur in two labs simultaneously. Consumer Opinion Services will recruit the test participants using a series of screening questions approved by the Website Updates usability team.

Sample Design

The goal for each user group (individuals, tax practitioners, and businesses) is 9 participants for a total of 27 participants. The test will have a user complete a variety of tasks using the IRS.gov web site. The participant will be asked to talk-aloud as they complete the tasks.

Testing will be based on scenarios created by Accenture and the IRS. Scenarios are short statements that include a task to be completed by using the IRS.gov website. The scenarios will be unique for each user group (i.e. individuals, tax practitioners, and business owners).

The moderator will complete an introduction, and provide an overview of the test process and ask the participant to sign a consent form. The form will explain what the participant is going to be doing and how the IRS plans to use the information gathered from testing.

No personally identifiable information will be shared outside of the test environment. The test team will destroy all background information upon completion of the test.

After the introduction, each participant will work each scenario individually on a computer. Participants will be required to complete each task without assistance from the moderator. They will be asked to “think aloud”, so that the usability team can capture as much data as possible.

When the participant has completed the scenarios there will be a comprehension question about the design and layout of the website in general. After that there will be a debriefing session that will consist of open discussion and an exit questionnaire. We are rewarding $75 to each participant.

Participants will use a PC running Windows XP with a 17-inch monitor. The screen resolution of the monitor will be set to 1024 x 768 with Windows display properties set to “Small Fonts”. Participants will access the product by using Microsoft Internet Explorer 6.0.

Data Collection Date

Research will be conducted the week of November 17 – 21, 2008. Testing will occur in Ogden, UT at the Ogden Usability Lab.

Data to be collected

This test will gather usability information regarding the IRS.gov web site. The testing team will gather information metrics for the following

• Time spent to complete a task

• Navigation paths used

• User satisfaction ratings (Post Test Questionnaire)

• Number of times user required help

• Task success rate

• Error recovery

The participant will be monitored by having a test facilitator get them started on the test. In addition, they will be monitored by the Website Updates usability team members through 2-way mirrors and video for data collection. Testing sessions are expected to last approximately 90 minutes per participant.

How data will be used

Data will be used to evaluate taxpayers’ satisfaction and identify areas for improvement for the IRS.gov web site. All information collected is strictly for research, and will not be used to personally identify participants nor will it be shared for commercial purposes.

How data will be analyzed

Analysis of the usability testing data will be conducted by Wage & Investment (W&I), and the IRS.gov contractor, Accenture. Observations are going to be used to asses the performance (design) of the website as the result of the Website Updates design effort.

Who is conducting research?

The Website Updates usability test team includes two IRS.gov usability specialists, a management analyst assigned to the Public Portal Branch of ETARC, and a contractor from Accenture.

Role	Name(s)	Business Operating Division	Organization	Job Title / Role	Contact Information
Planner and coordinator	Shawn Tafoya	MITS	AD	Usability Administrator	[email protected] (801) 620-4060
Participant recruiter	Consumer Opinion Services
Test facilitator – Lab 1	Jana Roberts	MITS	AD	Usability Administrator	[email protected] (801) 620-4283
Test facilitator – Lab 2	TBD
Test data logger – Lab1	Shawn Tafoya	MITS	AD	Usability Administrator	[email protected] (801) 620-4060
Test data logger – Lab2	TBD
Test team lead	Micah Morris		Accenture	Test Team Lead	[email protected]
Observer-stakeholder	Liz Baker	W&I	ETARC:PPB	Project Lead	[email protected]
Observer-team member – Lab1	Beth Krappweis	W&I	ETARC:PPB	IT Specialist	[email protected] 202-283-4844
Observer-team member – Lab 2	Mark Del Grande		Accenture		[email protected]
Observer-team member	Cindy May	W&I	ETARC:PPB	Project Lead	[email protected]
Observer – team member	TBD		Accenture

The following roles are required before and during the period over which usability testing takes place:

• Planner and coordinator – Usability Administrator ensures that everything is in place to allow the usability test sessions to take place as planned and without problems.

• Participant recruiter – recruits participants and schedules them for available time slots; provides test facilitator with honorarium checks to give to each participant at the end of their test session.

The following roles are required in order to conduct a usability test session:

• Test facilitator –Usability Consultant oversees the test itself, introduces and explains the test session to the participant, observes participant’s actions and obtains participant feedback at completion of test session, conducts issues assessment session. The facilitator is also available to answer questions and ensures that the observer stakeholders stay on task and have a productive and successful experience.

• Test data logger – creates a written record of the participant’s actions and comments as they use the site using data logging software, a word processor, or a spreadsheet.

• Test Team Lead – responsible for coordination meetings and set up for testing.

• Observer-team member – observes test to identify functionality and performance, compares notes with other team members, and actively participates in Issues Assessment at end of each test session.

Cost of Study

$2,000

Location

Testing will occur in Ogden, UT, at the Ogden Usability Test Lab.

Stipend

A $75.00 per participant stipend will be paid to each participant. This is necessary in order to recruit a sufficient number of study participants.

Recruitment Efforts

Consumer Opinion Services, an external market research firm, will recruit 21 - 27 participants from three user groups: Individuals Tax Professionals, and Business owners. Participants will be recruited the week prior to usability testing, with the usability study being completed by November 21, 2008.

Participant Criteria

External Customers	Target #
Individuals – Individual tax payers (9 participants). Students (newer / first time tax payers), homemakers, retired citizens, average taxpayers (employed individuals).	7 – 9
Tax Practitioners – (9 participants). Tax practitioners that prepare taxes for individuals, have used one of the pick lists on the website in the past.	7 – 9
Businesses – (9 participants). Small or large business owners that are required to file taxes.	7 – 9

• General knowledge of the internet with frequent use of the internet

• General knowledge of tax filing and processing

• Knowledge of / responsible for individual tax filing

Burden Hours

Individual Participants

Total Screened: 36 @ 10 minute screening interview = 6 hours

Total Recruited: 9 people (25% of screened individuals qualify)

Travel Time estimated 20 minutes per participant to travel to testing facility and return = 3 hours

Total testing time averaged at 1.5 hours/participant = 13.5 hours

Total estimated burden is 22.5 hours

Tax Professional Participants

Total Screened: 36 @ 10 minute screening interview = 6 hours

Total Recruited: 9 people (25% of screened individuals qualify)

Travel time estimated 20 minutes per participant to travel to testing facility and return = 3 hours

Total testing time averaged at 1.5 hours/participant = 13.5 hours

Total estimated burden is 22.5 hours

Business Owners Participants

Total Screened: 36 @ 10 minute screening interview = 6 hours

Total Recruited: 9 people (25% of screened individuals qualify)

Travel Time estimated 20 minutes per participant to travel to testing facility and return = 3 hours

Total testing time averaged at 1.5 hours/participant = 13.5 hours

Total estimated burden is 22.5 hours

Efforts to not duplicate research

Usability testing has been conducted on the IRS.gov web site previously. Some tasks may be repeated to gather metrics on improvement.

Privacy, Security, Disclosures, Confidentiality

The IRS will ensure compliance with the Taxpayers Bill of Rights II. All participants will be treated fairly and appropriately.

Confidentiality will be safeguarded. During the testing sessions, participants will be identified to IRS personal by their first names only. In addition, no participant names will be mentioned in the usability study final report. Participants will be advised that observers from the IRS may be observing the session via video monitoring equipment and that their comments may be recorded to ensure accurate data capture.

Attachments

1. Telephone/ Participant Screeners Questionnaire

2. Recruiting Service Cover Letter

3. Moderator Checklist

4. Consent Form

5. Scenarios

6. Exit Questionnaires

7. Confirmation Phone Call Script

7