SOI-362 - Supporting Statement

OMB Package

A Customer Satisfaction Study for Taxpayers and Tax Practitioners using

IRS.gov Customer Online Decisions Support (COLDS) Release 2 – Usability Study

Introduction

Customer On-Line Decision Support (COLDS) Release 2 will replace the Wage and Investment’s (W&I) Tax Law Categories (TLCs) content found in Tax Trails currently available on the IRS.gov web site. COLDS R2 will modify the existing Content Management Application (CMA), Search Engine, and WebTrends components to improve TLCs questions and responses. The Interactive Tax Assistant (ITA) tool is the solution that will enable taxpayers to conduct interactive probes on Tax Law topics and receive specific tax law guidance based on the responses. Taxpayers will have the opportunity to provide feedback on the usefulness of the content, thereby enabling TLC content authors to improve content effectiveness. COLDS R2 will also provide TLC content authors the capability to easily update the TLC content to the web site.

Release 2 will be implemented as a pilot in February, 2010 for the 2010 Filing Season. Subsequent modifications will be made with a final implementation in June, 2010.

Background

In response to a Congressional directive, the IRS conducted a comprehensive review of its current portfolio of taxpayer services to create a Taxpayer Assistance Blueprint (TAB). The TAB team identified the need to improve website self-service applications. The W&I Strategy and Program Plan addressed the need to provide solutions to ensure that Taxpayers, Practitioners and other IRS partners have access to accurate and timely responses to tax law issues and inquiries at all hours during the day. This included the need to increase availability of self-assist applications through multiple channels, including the Internet.

COLDS will provide an efficient electronic alternative on IRS.gov for taxpayers seeking assistance with their federal tax questions who would otherwise require assistance from a Call Site or a Tax Assistance Center (TAC). COLDS will also support the TAB initiatives as well as the W&I Strategy and Program plan.

An interactive on-line “probe and response” solution for customer-specific tax law questions will mitigate the burden on telephone assistance centers during peak traffic times, freeing the assistors to handle more complex questions. This project will move the IRS customers away from the more expensive channels to the less expensive internet customer self-service channel.

Objectives of Research

The purpose of this test is to determine if the W&I’s Interactive Tax Assistant tool on the irs.gov website is logical and easy to understand.

The Usability testing is intended to gather user data on the overall content, navigation, search, and user interface design of the site.

• We will determine the user’s ability to navigate through the interactive tool on the web site.

• We will determine the user’s ability to use the interactive probe and response tool and search features to search for the appropriate tax law guidance to their specific tax law questions.

• We will determine whether the probe and response options are easily understood by users.

Methodology

The test is designed to take approximately 90 minutes for each participant to complete. Test participants will consist of individuals, tax practitioners, and visually impaired individuals that prepare tax returns or have knowledge about tax filing. Testing will occur at the Ogden Usability Lab in Ogden, UT. Consumer Opinion Services will recruit the test participants using a series of screening questions approved by the COLSD R2 Integrated Project Team (IPT) usability team members.

Sample Design

The goal for each user group (individuals, tax practitioners and visually impaired participants) is between 3 and 5 participants for a total of 9 – 15 participants. The test will have a user complete a variety of tasks using the IRS.gov web site, specifically the ITA tool. The participants will be asked to talk-aloud as they complete the tasks.

Testing will be based on scenarios created by Accenture and the IRS. Scenarios are short statements that include a task to be completed by using the ITA tool on IRS.gov. The scenarios will be unique for each user group (i.e. individuals and tax practitioners).

The moderator will complete an introduction, and provide an overview of the test process and ask the participants to sign a consent form. The form will explain what the participant is going to be doing and how the IRS plans to use the information gathered from testing.

No personally identifiable information will be shared outside of the test environment. The IRS team will destroy all background information upon completion of the test. After the introduction each participant will work each scenario individually on a computer. Participants will be required to complete each task without assistance from the moderator. They will be asked to “think aloud”, so that the usability team can capture as much data as possible.

When the participant has completed the scenarios, there will be a comprehension question about the ITA tool in general. After which there will be a debriefing session, consisting of an open discussion and an exit questionnaire. We are rewarding $75 to each participant.

Participants will use a PC running Windows XP with a 17-inch monitor. The screen resolution of the monitor will be set to 800 x 600 with Windows display properties set to “Small Fonts”. Participants will access the product by using Microsoft Internet Explorer 6.0.

Data Collection Date

Research will be conducted the week of November 16-20, 2009. Testing will occur in Ogden, Utah at the Ogden Usability Lab.

Data to be collected

This test will gather usability information regarding the ITA tool on the IRS.gov web site. The testing team will gather information metrics for the following:

• Time spent to complete a task

• Navigation paths used

• Search vs. navigation

• User satisfaction ratings (Post Test Questionnaire)

• Number of times user required help

• Task success rate

• Error recovery

The test facilitator will assist each participant at the start of the test. The participants will be monitored by the COLDS R2 IPT usability team members through a two way mirror and video. Testing sessions are expected to last approximately 90 minutes per participant.

How data will be used

Data will be used to evaluate taxpayers’ satisfaction and identify areas for improvement for the Tax Trails redesign on the IRS.gov web site. All information collected is strictly for research, and will not be used to personally identify participants nor will it be shared for commercial purposes.

How data will be analyzed

Analysis of the usability testing data will be conducted by Wage & Investment (W&I), Customer Account Services (CAS), Joint Operations Center (JOC), and the IRS.gov contractor, Accenture. Observations will be used to asses the performance (design) of the ITA pages.

Who is conducting research?

The COLDS R2 test team includes two IRS.gov usability specialists , a management analyst assigned to the Public Portal Branch of ETARC, two internal usability specialists from the IRS – Ogden Usability Lab, Bill Moses (COLDS R2 project manager), and at least one contractor from Accenture and Oracle. Additional observers may include other IPT members.

Role	Name(s)	Bus Op Div	Organization	Job Title / Role	Contact Information
Planner and coordinator	Shawn Tafoya	MITS	AD	Usability Administrator	[email protected] (801)620-4060
Participant recruiter	Consumer Opinion Services
Test facilitator	Jana Roberts	MITS	AD	Usability Administrator	[email protected] 801-620-4283
Test data logger	Beth Krappweis	W&I	ETARC:PPB	IT Specialist	[email protected] 202-283-4844
Test team lead	Jon Eisenmann PPB	W&I	Accenture ETARC: PPB	IRS.gov Dev IT Specialist	[email protected] 703-947-2385
Observer-stakeholder	Bill Moses	W&I	CAS	Project Manager	[email protected] 678-530-5238
Observer-team member	Business SME Business SME (R) Evelyn Ruffin (R)	W&I W&I MITS	CAS:AM CARE:FA PPMO	IPT – SME IPT – SME IPT
Observer-team member	Accenture Rep Oracle Rep Mike Short		Accenture Oracle MITRE	IRS.gov Dev ITA Dev IPT	[email protected] 703-983-5481

The following roles are required before and during the period over which usability testing takes place:

• Planner and coordinator – Usability Administrator ensures that everything is in place to allow the usability test sessions to take place as planned and without problems.

• Participant recruiter – recruits participants and schedules them for available time slots; provides test facilitator with honorarium checks to give to each participant at the end of their test session.

The following roles are required in order to conduct a usability test session:

• Test facilitator –Usability Consultant oversees the test itself, introduces and explains the test session to the participant, observes participant’s actions and obtains participant feedback at completion of test session, conducts issues assessment session. The facilitator is also available to answer questions and ensures that the observer stakeholders stay on task and have a productive and successful experience.

• Test data logger – creates a written record of the participant’s actions and comments as they use the site using data logging software, a word processor, or a spreadsheet.

• Test Team Lead – responsible for coordination meetings and set up for testing.

• Observer-team member – observes test to identify functionality and performance, compares notes with other team members, and actively participates in Issues Assessment at end of each test session.

Cost of Study

The one-time cost for use of the usability test lab is $2,000.00. The costs for recruitment services range from $5,000 - $7,000. IRS travel cost will also be included. Total cost of the study is as follows:

Usability Lab	$ 2,000.00
Recruiter Services (includes stipend)	7,000.00
Test data logger Travel (Beth Krappweis)	2,000.00
Test Team Lead Travel (PPB)	2,000.00
Observer – Stakeholder Travel (Bill Moses)	2,000.00
Observer – Team Members Travel (Bus SME)	2,000.00
Total	$17,000.00

Location

Testing will occur in Ogden, Utah, at the IRS’ Ogden Usability Lab.

Stipend

A $75.00 per participant stipend will be paid to each participant. This is necessary in order to recruit a sufficient number of study participants.

Recruitment Efforts

Consumer Opinion Services, an external market research firm, will recruit 9 to 15 participants from three user groups: Individuals, Tax Professionals, and visually impaired individuals. Participants will be recruited the week prior to usability testing, with the usability study being occurring between November 16, 2009 and November 20, 2009.

Participant Criteria

External Customers	Target #
Individuals – Individual taxpayers to include new/first time taxpayers, homemakers, retired citizens, employed individuals.	3-5
Tax Practitioners – Tax practitioners who prepare tax returns for individuals and who use the individual’s portion of the website on a regular basis.	3-5
Visually Impaired person - individual taxpayers to include new/first time taxpayers, homemakers, retired citizens, employed individuals who have a seeing disability that cannot be corrected by glasses	3-5

• General knowledge of the internet with frequent use of the internet

• General knowledge of tax filing and processing

• Knowledge of / responsible for individual tax filing

Burden Hours

• Individual Participants

Total Screened: 20 @ 10 minute screening interview = 3.2 hours

Total Recruited: 5 people (25% of screened individuals qualify)

Travel Time estimated 20 minutes per participant to travel to testing facility and return = 1.66 hours

Total testing time averaged at 1.5 hour/participant = 7.5 hours

Total estimated burden is 12.36 hours

• Tax Professional Participants

Total Screened: 20 @ 10 minute screening interview = 3.2 hours

Total Recruited: 5 people (25% of screened individuals qualify)

Travel Time estimated 20 minutes per participant to travel to testing facility and return = 1.66 hours

Total testing time averaged at 1.5 hour/participant = 7.5 hours

Total estimated burden is 12.36 hours

• Visually Impaired Participants

Total Screened: 20 @ 10 minute screening interview = 3.2 hours

Total Recruited: 5 people (25% of screened individuals qualify)

Travel Time estimated 20 minutes per participant to travel to testing facility and return = 1.66 hours

Total testing time averaged at 1.5 hour/participant = 7.5 hours

Total estimated burden is 12.36 hours

Efforts to not duplicate research

Usability testing has been conducted on the IRS.gov web site previously. Some tasks may be repeated to gather metrics on improvement.

Privacy, Security, Disclosures, Anonymity

The IRS will ensure compliance with the Taxpayers Bill of Rights II. All participants will be treated fairly and appropriately.

Anonymity will be safeguarded. During the testing sessions, participants will be identified to IRS personnel by their first names only. In addition, no participant names will be mentioned in the usability study final report. Participants will be advised that observers from the IRS may be observing the session and via video monitoring equipment and that their comments may be recorded to ensure accurate data capture.

Attachments

1. Telephone/ Participant Screeners Questionnaire

2. Recruiting Service Cover Letter

3. Moderator Checklist

4. Consent Form

5. Scenarios

6. Exit Questionnaires

7. Confirmation Phone Call Script

6