SOI-320 - Supporting Statement

OMB Package

A Customer Satisfaction Study for Taxpayers, Tax Practitioners and Businesses using

the various IRS.gov Pick Lists – Usability Study

Introduction

The enhanced pick lists will enhance the user experience by making the various pick lists on IRS.gov easier to use. There will be a sorting and searching option. The lists will be viewed in an easier to read more organized format.

Usability testing is necessary to make sure that the changes we made were the right ones and are going to help all IRS.gov users find what they are looking for.

Taxpayers will have the opportunity to provide their feedback regarding the pick lists and the content associated with them.

The new release will be available on the website by the start of the filing season (Jan. 2009)

Background

In response to a Congressional directive, the IRS conducted a comprehensive review of its current portfolio of taxpayer services to create a Taxpayer Assistance Blueprint (TAB). The TAB team identified the need to improve website self-service applications. Usability testing conducted in 2006 found problems with the various pick lists on the IRS.gov web site. These pick lists include those owned by Media and Publications, Communications and Liaison, and Counsel.

The Pick List Redesign will enhance the user experience by allowing users to search and filter through the different pick lists, allowing them to find their information easier.

Objectives of Research

The purpose of this test is to determine if the pick lists portion of the web site is logical and easy to understand.

The Usability testing is intended to gather user data on the overall navigation, organization, searchability and user interface design of the site.

We will determine the user’s ability to navigate to the various pick lists on the web site.

We will determine the user’s ability to use the search features to search through the appropriate pick lists.

We will determine whether the column headings and instructions are easily understood by users.

Methodology

The test is designed to take approximately 60 minutes for each participant to complete. Test participants will consist of individuals, tax practitioners, and business owners that prepare or have knowledge about tax filing. Testing will occur at the Accenture building in Reston, VA. Testing will occur in one lab. Consumer Opinion Services will recruit the test participants using a series of screening questions approved by the Pick List redesign usability team.

Sample Design

The goal for each user group (individuals and tax practitioners) is between 4 and 6 participants for a total of 9 – 15 participants. The test will have a user complete a variety of tasks using the IRS.gov web site. The participant will be asked to talk-aloud as they complete the tasks.

Testing will be based on scenarios created by Accenture and the IRS. Scenarios are short statements that include a task to be completed by using the pick lists portion of IRS.gov. The scenarios will be unique for each user group (i.e. individuals, tax practitioners, and business owners).

The moderator will complete an introduction, and provide an overview of the test process and ask the participant to sign a consent form. The form will explain what the participant is going to be doing and how the IRS plans to use the information gathered from testing.

No personally identifiable information will be shared outside of the test environment. The IRS team will destroy all background information upon completion of the test.

After the introduction each participant will work each scenario individually on a computer. Participants will be required to complete each task without assistance from the moderator. They will be asked to “think aloud”, so that the usability team can capture as much data as possible.

When the participant has completed the scenarios there will be a comprehension question about the design and layout of the pick lists page in general. After that there will be a debriefing session that will consist of open discussion and an exit questionnaire. We are rewarding $125 to each participant.

Participants will use a PC running Windows XP with a 17-inch monitor. The screen resolution of the monitor will be set to 1024 x 768 with Windows display properties set to “Small Fonts”. Participants will access the product by using Microsoft Internet Explorer 6.0.

Data Collection Date

Research will be conducted the week of November 3 – 7, 2008. Testing will occur in Reston, VA at the Accenture Building.

Data to be collected

This test will gather usability information regarding the various pick lists on the IRS.gov web site. The testing team will gather information metrics for the following

• Time spent to complete a task

• Navigation paths used

• Search terms / Search by

• User satisfaction ratings (Post Test Questionnaire)

• Number of times user required help

• Task success rate

• Error recovery

The participant will be monitored by having a test facilitator get them started on the test. In addition, they will be monitored by the pick list redesign usability team members through video. Testing sessions are expected to last approximately 60 minutes per participant.

How data will be used

Data will be used to evaluate taxpayers’ satisfaction and identify areas for improvement for the pick list enhancements on the IRS.gov web site. All information collected is strictly for research, and will not be used to personally identify participants nor will it be shared for commercial purposes.

How data will be analyzed

Analysis of the usability testing data will be conducted by Wage & Investment (W&I), and the IRS.gov contractor, Accenture. Observations are going to be used to asses the performance (design) of the pick list pages.

Who is conducting research?

The pick list usability test team includes two IRS.gov usability specialists, a management analyst assigned to the Public Portal Branch of ETARC, and a contractor from Accenture.

Role	Name(s)	Business Operating Division	Organization	Job Title / Role	Contact Information
Planner and coordinator	Beth Krappweis / Micah Morris	W&I/ Accenture	ETARC:PPB		[email protected] 202-283-4844
Participant recruiter	Consumer Opinion Services
Test facilitator	Beth Krappweis / Liz Baker	W&I	ETARC:PPB		[email protected] 202-283-4844 [email protected] 202-283-7847
Test data logger	Beth Krappwei / Liz Baker	W&I	ETARC:PPB		[email protected] 202-283-4844 [email protected] 202-283-7847
Test team lead	Micah Morris Beth Krappweis	Accenture/ W&I	ETARC: PPB	IT Specialist	[email protected] 202-283-4844
Observer-stakeholder	Cindy May	W&I	ETARC:PPB	Project Lead
Observer-team member	Accenture

The following roles are required before and during the period over which usability testing takes place:

• Planner and coordinator – Usability Administrator ensures that everything is in place to allow the usability test sessions to take place as planned and without problems.

• Participant recruiter – recruits participants and schedules them for available time slots; provides test facilitator with honorarium checks to give to each participant at the end of their test session.

The following roles are required in order to conduct a usability test session:

• Test facilitator –Usability Consultant oversees the test itself, introduces and explains the test session to the participant, observes participant’s actions and obtains participant feedback at completion of test session, conducts issues assessment session. The facilitator is also available to answer questions and ensures that the observer stakeholders stay on task and have a productive and successful experience.

• Test data logger – creates a written record of the participant’s actions and comments as they use the site using data logging software, a word processor, or a spreadsheet.

• Test Team Lead – responsible for coordination meetings and set up for testing.

• Observer-team member – observes test to identify functionality and performance, compares notes with other team members, and actively participates in Issues Assessment at end of each test session.

Cost of Study

$5,150

Location

Testing will occur in Reston, VA, at the Accenture Building.

Stipend

A $125.00 per participant stipend will be paid to each participant. This is necessary in order to recruit a sufficient number of study participants.

Recruitment Efforts

Consumer Opinion Services, an external market research firm, will recruit 9 - 15 participants from three user groups: Individuals Tax Professionals, and Business owners. Participants will be recruited the week prior to usability testing, with the usability study being completed by November 3 - 7, 2008.

Participant Criteria

External Customers	Target #
Individuals – Individual tax payers (5 participants). Students (newer / first time tax payers), homemakers, retired citizens, average taxpayers (employed individuals).	3 - 5
Tax Practitioners – (5 participants). Tax practitioners that prepare taxes for individuals, have used one of the pick lists on the website in the past.	3 - 5
Businesses – (5 participants). Small or large business owners that are required to file taxes.	3 – 5

• General knowledge of the internet with frequent use of the internet

• General knowledge of tax filing and processing

• Knowledge of / responsible for individual tax filing

Burden Hours

Individual Participants

Total Screened: 20 @ 10 minute screening interview = 3.2 hours

Total Recruited: 5 people (25% of screened individuals qualify)

Travel Time estimated 20 minutes per participant to travel to testing facility and return = 2 hours

Total testing time averaged at 1 hour/participant = 5 hours

Total estimated burden is 10.2 hours

Tax Professional Participants

Total Screened: 20 @ 10 minute screening interview = 3.2 hours

Total Recruited: 5 people (25% of screened individuals qualify)

Travel time estimated 20 minutes per participant to travel to testing facility and return = 2 hours

Total testing time averaged at 1 hour/participant = 5 hours

Total estimated burden is 10.2 hours

Business Owners Participants

Total Screened: 20 @ 10 minute screening interview = 3.2 hours

Total Recruited: 5 people (25% of screened individuals qualify)

Travel Time estimated 20 minutes per participant to travel to testing facility and return = 2 hours

Total testing time averaged at 1 hour/participant = 5 hours

Total estimated burden is 10.2 hours

Efforts to not duplicate research

Usability testing has been conducted on the pick lists portion of the IRS.gov web site previously. Some tasks may be repeated to gather metrics on improvement.

Privacy, Security, Disclosures, Confidentiality

The IRS will ensure compliance with the Taxpayers Bill of Rights II. All participants will be treated fairly and appropriately.

Confidentially will be safeguarded. During the testing sessions, participants will be identified to IRS personal by their first names only. In addition, no participant names will be mentioned in the usability study final report. Participants will be advised that observers from the IRS may be observing the session and via video monitoring equipment and that their comments may be recorded to ensure accurate data capture.

Attachments

1. Telephone/ Participant Screeners Questionnaire

2. Recruiting Service Cover Letter

3. Moderator Checklist

4. Consent Form

5. Scenarios

6. Exit Questionnaires

7. Confirmation Phone Call Script

7