SOI-333 - Supporting Statement

D iscussion Guides for IRS Contact Center Customer Satisfaction Focus Groups

T

Prepared For Office of Management and Budget

December 10, 2008

Submission for OMB Clearance

Discussion Guides for Internal Revenue Service (IRS) Contact Center Customer Satisfaction Focus Groups

OMB Number: 1545-1432

able Of Contents

I. Introduction 4

Background 4

Objectives for Data Collection 4

II. Methodology 6

Who Is Conducting Research 6

Sample Design 6

Data to Be Collected 6

How Data Will Be Used 6

How Data Will Be Analyzed 6

Data Collection Date 7

Location 7

Cost of Study 7

Stipend 7

Recruitment Efforts 7

Test Structure/Design 7

III. Participant Criteria 10

IV. Privacy, Security, Disclosure, ......................... 12

V. Burden Hours 13

Attachment A: Taxpayer Discussion Guide 14

Attachment B: Tax Preparer Discussion Guide 18

Attachment C, Part 1: Taxpayer Recruitment Screener 24

Attachment C, Part 2: Tax Preparer Recruitment Script 32

Attachment C, Part 3: Confirmation Letter to Focus Group Participants 40

Attachment C, Part 4: Participant Reminder Call Script 41

Attachment D: Privacy Statement and Consent Form 42

© 2008, Forrester Research, Inc. All rights reserved. Forrester, Forrester Wave, RoleView, Technographics, TechRadar, and Total Economic Impact are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective companies. Forrester clients may make one attributed copy or slide of each figure contained herein. Additional reproduction is strictly prohibited. For additional reproduction rights and usage information, go to www.forrester.com. Information is based on best available resources. Opinions reflect judgment at the time and are subject to change.

I. Introduction

Background

The Joint Operations Center (JOC), established within the Customer Account Services (CAS) Division, supports 25 contact center locations by providing customer support for these important capabilities as well as the technology infrastructure to support them. In the last year, the JOC has identified a dramatic increase in contact center call volumes. The IRS is looking to better understand the relationship between the systemic data related to call volumes, the management data related to support of resources to address call volumes, and the customer experience. Accordingly, the IRS is seeking to understand how it compares to commercial industry best practices associated with contact center metrics as well as how the components of system measures, business measures, and customer experience measures are captured, reported, and understood in relationship to one another.

In order to understand these relationships, the IRS has determined that it must assess customer satisfaction with its contact centers. The IRS has contracted with Forrester Research (under subcontract to Lockheed Martin, MSD, Inc.) to execute a set of four, two-hour Focus Group Interviews (FGI) composed of eight to ten respondents per group who have called the IRS Customer Service line in the past three years. Two of the FGIs will consist of consumers, and two will consist of Tax Preparation Professionals.

In these focus groups, Forrester Research will explore the call center service experience – both directly with regard to the IRS as well as with other industries – from the customer point of view in order to identify best practices that are and/or should be applied to the IRS call center environment. Given the sometimes stressful nature of interactions with the IRS, the research will also seek to understand the impact of exogenous circumstances on perceptions of satisfaction, and will recommend compensatory measures intended to neutralize these effects.

Objectives for Data Collection

The specific objectives of the focus groups are to determine the following:

• Understand how the target audience (taxpayers and tax preparers) thinks about customer service experiences – specifically, experiences with contact centers and contact center agents;

• Identify pain points associated with use of contact centers – generally, and specifically with regard to the IRS;

• Elicit characteristics of a “good” customer service experience – generally, and specifically with regard to the IRS;

• Identify companies/industries considered “best in class” for contact center experiences

• Evaluate specific past experiences of respondents with the IRS contact center, including:

  • Whether other channels for contact with the IRS may be considered or used;

  • The types of issues or situations that precipitated the contact;

  • How feelings about the context of the contact with the IRS affect perceptions of the customer experience;

  • How the IRS performed on characteristics identified as important to a “good” customer experience; and

• Elicit suggestions for improving the IRS customer service experience and neutralizing negative exogenous circumstances that directly impact perceptions of satisfaction.

II. Methodology

Who Is Conducting Research

The IRS has contracted with Forrester Research under subcontract to Lockheed Martin, MSD Inc. to conduct focus groups as qualitative research in understanding contact center customer satisfaction.

Sample Design

Forrester Research/Lockheed Martin, MSD Inc. will conduct four, two-hour focus groups with eight to ten participants each (for a total of 32 to 40 focus group participants) at one site.

Two of the focus groups will consist of taxpayers, and two of the focus groups will consist of tax preparer professionals. All participants will have had contact with an IRS call center within the past three years.

The taxpayer groups will include a mix of business filing (BMF) and individual filing (IMF) taxpayers, as well as a mix of filing methods (electronic, paper) and filing status (single, married filing separately or jointly, head of household). These groups will also reflect a representative mix of demographics, such as gender, age, ethnicity, education, income, etc.

The tax professional groups will include a mix of types of tax professionals (e.g., CPAs, financial planners, tax attorneys, enrolled agents, and unlicensed tax preparers). These groups will also include a mix of company sizes, length of time employed in the industry, types of returns filed for clients, participation in the IRS’ e-filing program, etc.

All participants in the research will be recruited from a database maintained by the focus group facility / recruiter, and offered a cash incentive as a token of appreciation for their time.

Data to Be Collected

Data from the focus groups will be qualitative in nature. The focus groups will consist of a series of discussion questions designed to meet the objectives stated above and elicit customer expectations and best practices for customer service from the IRS contact center. The specific questions and exercises to be used in the groups are provided in the Moderator’s Guidelines (see Attachments A & B).

How Data Will Be Used

Data will be used to elicit the “voice of the customer” about customer service delivered via contact center. Ultimately, findings from the focus groups will be used to inform the IRS about best practices that exist for contact center operations (across industries) and to develop recommendations for process improvement.

How Data Will Be Analyzed

Since the data will be qualitative in nature, the moderator of the focus groups will synthesize impressions and key learnings gained from the focus groups. Transcripts of each focus group session will be prepared and analyzed by the moderator in order to provide the IRS with accurate verbatim feedback from participants in order to facilitate understanding of the voice of the customer regarding the call center experience.

Data Collection Date

The Focus Group Interviews will be scheduled approximately two-to-three weeks after OMB approval is received, to allow time for participants to be recruited. Ideally, the focus groups will take place in February-March, 2009.

Location

The Focus Group Interviews will be conducted in Chicago, IL, as this location will allow us to recruit respondents from the central city as well as a variety of surrounding areas in order to obtain a participant group which represents a diverse demographic and socio-economic population, and that is likely to provide a good mix of relevant recruitment criteria (see Section III for summary of recruitment criteria).

The location of the actual focus groups will be a professional focus group facility that includes a respondent discussion room equipped with a one-way mirror, an observation room, and audio- and video-taping capabilities.

Cost of Study

The approximate cost of this study is $87,500. This number includes site costs, travel costs, labor costs, and other direct costs.

Stipend

Each individual participant will receive a cash incentive as a token of appreciation for their time participating in the research discussion. Taxpayers will receive $100 each for attending; Tax Preparer Professionals will receive $200 each.

Recruitment Efforts

Attachment C, Parts 1-4 include the participant recruiting package. All participants will be recruited via phone by the professional focus group facility, and qualified for participation using a short screening survey instrument. One screening instrument will be used for taxpayers (see Attachment C, Part 1) and one will be used for tax professionals (see Attachment C, Part 2).

Twelve participants will be recruited for each group, to ensure a minimum number of eight-to-ten participants per group.

Recruitment will begin as soon as possible after receiving OMB approval. The recruiting facility will keep track of recruitment burden hours. Participants will receive a confirmation letter in advance of the focus group discussion date, and will be reminded via telephone the day prior to the focus group session in order to ensure participation. Attachment C, Parts 3-4 includes the text of the confirmation letter and a script for the reminder call.

Test Structure/Design

Each focus group will take no more than two hours (120 minutes). Prior to the testing session, participants will be asked to read and sign a Privacy statement (Attachment D).

An outline of the areas of discussion for both taxpayer and tax preparer focus groups is below. The full moderator’s guides are available in Attachments A & B.

Section 1: Introduction

• Moderator introduces self, topic of discussion, and ground rules (audio/video taping, presence of viewers, Privacy, etc.)

• Respondents introduce themselves

Section 2: General Discussion of Customer Service Experience

• Define customer service

• Discuss customer service experiences in general; positives and negatives of various channels (in-person, internet, mail); narrow discussion down to focus on call-center experiences

• Identify attributes of a good call-center customer experience

  • For tax professionals only: identify attributes applicable to a professional service experience

• Identify best-in-class examples of good call-center customer experience

  • For tax professionals only: identify professional best-in-class examples

Section 3: Discussion of IRS call-center experience

• Describe most recent IRS call-center experience

  • Identify any steps taken prior to call

  • Identify feelings associated with call

  • Evaluate overall outcome of and satisfaction with call

• Evaluate specific elements of call (IVR, access hours, hold process, knowledge and responsiveness of rep, resolution)

• Evaluate how well specific elements deliver on attributes of good customer experience

Section 4: Design a Better IRS call-center experience

• Group exercise: divide participants into small groups, each group creates list of steps to improve IRS call-center experience

• Small groups present their solutions to the entire group

• Discuss which attributes of a good customer service experience were touched upon in their solutions

• Evaluate the extent to which each solution will improve the customer experience for specific circumstances

Section 5: Closing

• Elicit final suggestions / recommendations from participants

• Address any additional questions from observers

III. Participant Criteria

Participant Criteria – Taxpayer Groups
Must have filed a federal tax return for 2007	100%
Must have called the IRS within the past 3 years	100%; minimum of 50% must have called within the past 12 months
Reasons for calling the IRS / type of call (assistance filing a tax return, current information on tax law, general tax information, filed 1040 or 1040A return without an accompanying schedule or payment, check status of economic stimulus check or refund)	Mix; maximum of 25% per group who have called regarding status of economic stimulus or refund check. Disqualify those who have called about an audit notice or to arrange a payment
Gender	Mix
Race/Ethnicity	Mix
Age	Mix; minimum age 18, maximum age 64
Income	Mix
Education	Mix
Type of filer (IMF, BMF)	Mix
File own tax return	Mix; minimum of 50% per group file their own return
Filing status (Single, Married filing separately or jointly, Head of Household)	Mix
Filing method (electronic, paper)	Mix
Bias screening	Must not work in tax preparation, market research, or management consulting
Past Focus Group participation	Must not have participated in a focus group within the past 3 months

Participant Criteria – Tax Professional Groups
Must have filed a tax return for a client for 2007	100%
Must work as a tax preparer	100%
Type of tax professional (CPA / Accountant, Financial Planner / Certified Financial Planner, Enrolled Agent, Tax Attorney, Unlicensed tax preparer)	Mix
Must have called the IRS within the past 3 years	100%; minimum of 50% per group must have called within the past 12 months
Reasons for calling the IRS / type of call (assistance filing a client’s tax return, current information on tax law, general tax information, filed a client’s 1040 or 1040A return without an accompanying schedule or payment, check status of client’s economic stimulus check or refund)	Mix; maximum of 25% per group who have called regarding status of economic stimulus or refund check. Disqualify those who have called about an audit notice or to arrange a payment
Types of returns prepared/filed for clients (IMF, BMF)	Mix
Length of time working in industry	Mix; maximum 50% per group with less than two years experience
Size of firm (# of employees)	Mix
Participate in IRS’ e-filing program	Mix
Type of firm (tax preparation only, legal services, general financial services)	Mix
Type of employment (full time/part time, year-round/)	Mix; maximum of 25% per group part-time year round; maximum of 25% per group part-time tax season only
Bias screening	Must not work in market research or management consulting
Past Focus Group participation	Must not have participated in a focus group within the past 3 months

IV. Privacy, Security, and Disclosure

We will protect the privacy of participants in the focus group sessions by not using names in our reports or transcripts. Any respondent profile information made available to observers during the focus group sessions will include participant first names only; additionally, no contact information will be included. We will also control official access to the information and will not allow public access to the information. The screening questionnaires and any data collected during the focus groups (audio and video recordings, transcripts, and any moderator notes) will be stored in a secure location for one year, and then destroyed.

All participants will be informed of the use of audio and video taping at the beginning of the session. They will also be assured of their privacy and – that their identity will not be connected to any of their comments. They will sign a Privacy agreement as part of their participation in the research (see Attachment D).

V. Burden Hours

The estimated time to complete the participant screening is approximately 10 minutes per person screened. The estimated time to complete each focus group is two hours (120 minutes).

We anticipate that approximately 30% of taxpayers contacted for this study will qualify, and that 50% of tax preparers will qualify. With these percentages, we will need to screen approximately 84 taxpayers to get 24 taxpayer recruits, and 48 tax preparers to get 24 tax preparer recruits.

Total number of potential participants screened: 132

Estimated time to complete screening: 10 minutes

Estimated participant screening burden: 22 hours

Estimated number of focus group participants: 40

Focus group participation time: 2 hours

Estimated participant travel time to and from site: 30 minutes

Estimated participation burden: 100 hours

Total Burden (screening and study participation): 122 hours

- 12 -