Supporting Statement

© 2008, Forrester Research, Inc. All rights reserved. Forrester, Forrester Wave, RoleView, Technographics, TechRadar, and Total Economic Impact are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective companies. Forrester clients may make one attributed copy or slide of each figure contained herein. Additional reproduction is strictly prohibited. For additional reproduction rights and usage information, go to www.forrester.com. Information is based on best available resources. Opinions reflect judgment at the time and are subject to change.

I. Introduction

Background

The Joint Operations Center (JOC), established within the Customer Account Services (CAS) Division, supports 25 contact center locations by providing customer support for these important capabilities as well as the technology infrastructure to support them. In the last year, the JOC has identified a dramatic increase in contact center call volumes. The IRS would like to better understand the relationship between:

the systemic data related to call volumes,
the management data related to support of resources to address call volumes, and
the customer experience.

Accordingly, the IRS is seeking to understand how it compares to the best practices associated with contact center metrics in commercial industry, as well as how the components of system measures, business measures, and customer experience measures are captured, reported, and understood in relationship to one another.

In order to understand these relationships, the IRS has determined that it must assess customer satisfaction with its contact centers. The IRS has contracted with Forrester Research (under subcontract to Lockheed Martin, MSD, Inc.) to execute quantitative research with taxpayers and tax professionals who have called, or considered calling, the IRS Customer Service line within the past three years in order to understand what constitutes a positive customer experience and identify best practices that are and/or should be applied to the IRS call center environment.

Objectives for Data Collection

Forrester Research will explore the call center service experience – both directly with regard to the IRS, as well as with other industries – from the customer point of view. Given the sometimes stressful nature of interactions with the IRS, the research will also seek to understand the impact of exogenous circumstances on perceptions of satisfaction, and will recommend compensatory measures intended to neutralize these effects.

The specific objectives of the survey are as follows:

Identify predisposition to use various channels for delivery of customer service generally (not specific to the IRS);
Assess the importance of specific attributes associated with call centers, in order to understand the standards against which the IRS’ performance is measured;
Understand the reason behind the most recent contact with the IRS, as well as the total number of contacts required, methods of contact, and other sources consulted to reach issue resolution;
Uncover pre-contact expectations about interactions with the IRS;
Assess the performance of the IRS call centers against the standards established in the importance battery;
Measure expectations for customer service from the IRS against other industries and other governmental entities;
Explore attitudes and psychological dispositions that may act as drivers associated with expectations for and satisfaction with customer service from the IRS;
Elicit open-ended responses about most and least satisfying elements of the IRS experience in order to detect any factors that have been previously unidentified; and
Characterize any additional demographic factors that might affect willingness to use the IRS phone line for customer service.

II. Methodology

Who Is Conducting Research

The IRS has contracted with Forrester Research under subcontract to Lockheed Martin, MSD Inc. to conduct the research. Forrester Research is partnered with Momentum Market Intelligence to provide design and analysis of custom primary research. Momentum Market Intelligence will subcontract the data collection tasks of the research to an online panel vendor, eRewards, and to a telephone field vendor, Resolution Research.

Sample Design

There are two primary target audiences for this research: taxpayers and tax professionals, who use the IRS call centers. There are approximately 130 million taxpayers and 400,000 tax professionals in the United States. From this universe, the study will target (N=) 400 taxpayers and (N=) 100 professional tax preparers. The proposed sample sizes are necessary to accurately represent both the taxpayers and tax professionals who have contacted the IRS, for any one of a number of reasons. These sample sizes will also allow for analysis on a number of dimensions, such as reason for contacting the IRS or method of filing.

This study will use a combination of an online panel and a random digit dialing (RDD) telephone approach.

Taxpayers

The random digit dial (RDD) telephone approach will be used as a mode of data collection for the taxpayer sample. Requirements for this sample population are that it must be representative of the US taxpayer population in general, and of taxpayers who have contacted the IRS via phone in particular. A telephone sampling methodology is likely to be both broadly representative and less biased against those who use the telephone for customer service than would an online survey environment.

The taxpayer sample (total N=400) will include representation from 3 sub-populations:

English speakers (N=300) who have called the IRS within the past 3 years;
Spanish speakers (N=50) who have called the IRS within the past 3 years, and;
English speakers (n=50) who had cause to call / considered calling the IRS but did not do so.

Resolution Research, Inc. will administer the telephone portion of the study. For the phone portion, there will be no specific sample source. Resolution Research will randomly dial US households and apply screening criteria to identify eligible survey respondents (see survey instrument in Attachment A for specific screening questions).

Tax professionals

Among the more limited universe of the tax professional population, an online mode of data collection is likely to be more successful. Unlike the taxpayers, many of whom may not be sufficiently online-savvy or comfortable to be well represented in an online survey environment, the professional tax preparers appear to be a viable target for an online data collection methodology, and at a significantly lower cost than a phone-based approach.

The professional tax preparer will not be broken out into subpopulations, although several demographics and firmographics will be assessed (e.g., size of firm, nature of clientele) to contextualize and explicate the professional findings.

eRewards will administer the online panel portion of the study. The sample will be drawn from their Business panel of approximately 1.6 million participants. The source of this sample is a “closed” or “by-invitation-only” online panel recruitment method, exclusively inviting pre-validated individuals or individuals who share known characteristics to enroll into market research panels. By partnering with a targeted, large, and diverse set of ubiquitous brands, eRewards is able to invite everyday consumers and business decision-makers into its panels. Sampling can be tailored by targeting specific segments, using title, industry, and other demographic and firmographic variables. More information about eRewards can be found by contacting them at [email protected] or 1-888-203-6245.

Panel members will receive an invitation to participate in the research via email. (See attachment C for example email invitation). They will be asked to answer a series of screening questions to identify eligible survey respondents (see survey instrument in Attachment B for specific screening questions).

Data to Be Collected

Please see the attached questionnaires (see Attachments A & B) for the details of information to be collected. Data from the surveys will be quantitative in nature. The surveys will consist of a series of closed-ended and open-ended questions designed to meet the objectives stated above and elicit customer expectations and best practices for customer service from the IRS contact center. The data to be collected is virtually identical for the online and phone surveys, with the exception of a battery for the tax professionals to assess burden associated with specific types of issues across all of their clients. There are also slight differences in question phrasing associated with whether the respondent is answering about their own issues, or those of a client.

How Data Will Be Used

The surveys will focus on testing and expanding upon the hypotheses and issues about customer service delivered via contact center identified as meaningful and impactful in previously conducted focus group research. Ultimately, findings from the surveys will be used to inform the IRS about best practices that exist for contact center operations (across industries) and to develop recommendations for process improvement.

How Data Will Be Analyzed

The surveys will assess drivers for call center activity and customer satisfaction. Expected analytic techniques include frequency distributions, cross-tabulations, mean scores, T-tests, analysis of variance (ANOVA) to determine significant differences between groups, and correlation analysis. Regression, factor analysis and structural equation modeling may also be used for driver analysis, and to isolate potential areas for metric development.

Data Collection Date

The surveys will be fielded approximately two-three weeks after OMB approval is received, to enable programming and testing of the surveys prior to launch. Data will be collected over a 30-day period. Ideally, survey fielding will take place between June and July, 2009.

Location

Data will be collected over the phone and online, so there is no specific location for testing.

Cost of Study

The approximate cost of this study is $125,000. This number includes recruitment costs, labor costs, participant stipend, and other direct costs.

Stipend

Each individual participant will receive an incentive as a token of appreciation for their time participating in the voluntary research. Each taxpayer will receive a cash incentive of $10. Tax Preparer Professionals will receive the typical incentive provided as part of their participation in the eRewards panel (incentive costs are built into overall interview costs; these are not provided separately).

Each eRewards’ panel member earns eRewards currency or points (depending on the panel) for the time he/she spends answering market research surveys. Members can redeem their currency/points for a variety of valuable rewards that are of interest to them. Incentives are based on the value of the sample group (B2B, B2C, or Specialty Group) and estimates of the time and effort required to fully complete a survey.

Recruitment Efforts

This data gathering effort will be carried out telephonically through the use of random digit dialing and over the internet utilizing eRewards’ online panel.

All taxpayer participants will be recruited via phone by Resolution Research, and qualified for participation using a short screening section of the overall survey instrument. All tax preparer participants will be recruited online from the e-Rewards panel using a short screening section of the overall survey instrument.

Expected Response Rate

The expected response rate, for both phone and online, is 25%. This means that 5,332 taxpayers will need to be contacted in order to screen 1,333 taxpayers for qualification, which will ultimately result in 400 completed interviews. And 800 tax professionals will need to be contacted in order to screen 200 tax professionals for qualification, which will ultimately result in 100 completed interviews.

Justification for Response rate less than 50%

The response rate is expected at 25% because of the random digit dial methodology being used to reach, qualify, and get cooperation from potential taxpayer respondents, and due to the inability to target tax professionals with greater granularity than “financial services industry.” The majority of respondents in the survey will be drawn from the subset of the taxpayer population that has contacted the IRS call center within the past 3 years. Since this sub-population represents less than 50% of the general taxpaying population, we anticipate an overall response rate of less than 50% as well.

Methods to Maximize Response Rate

Forrester/MMI will strive to maximize response and cooperation rates among those selected in the sample. For the phone method, we will make up to five callbacks (at least one in the evening and one on Saturday), if necessary, to reach a respondent. Once the telephone is answered, we will administer screening questions to identify the person who has contacted the IRS call center or considered doing so. Interviewers will go through study-related training; part of this training will be teaching them refusal avoidance and refusal conversion techniques.

For the online method, potential respondents will receive up to 3 reminder email invitations to participate in the research. Respondents who partially complete surveys will receive extra reminder emails in order to maximize responses.

Efforts to Not Duplicate Research

The goal of the research is to build upon the foundation already laid by previous research efforts. While some similar questions and answer option lists may be used in the current research in order to ensure continuity of data across studies, the research design team has reviewed previous research conducted in the past by the IRS on call center satisfaction, call center metrics, taxpayer burden, and other topics associated with the objectives of this research in order to avoid duplication of the research.

Test Structure/Design

Data collection for taxpayers will occur via a 20 minute phone survey.

Data collection for tax professionals will occur via an online survey of approximately 20 minutes in length.

Sections of the survey instruments include:

Introduction and Screening
Invitation/Termination Scripts
General customer service behavior
Importance evaluation of customer service attributes
Behavioral battery on experience with IRS customer service
Performance evaluation of IRS customer service
Comparative perceptual battery
Psychographic / attitudinal batteries
Open-ended reflections on IRS customer service
Demographics / Firmographics

Please see the attached survey instruments for details about the specific questions that will be asked (Attachments A & B).

III. Participant Criteria

The target population for this test is the universe of people who may call an IRS call center for customer service. Please refer to the attached survey instruments for more specific qualification criteria. A summary is below:

Taxpayer Qualification Criteria Summary

Must not work in tax preparation, market research, or management consulting
Must be between 18 years old and 64 years old
Must have filed a tax return in 2009 (for the 2008 year) – either electronically, or via mail
Must have called the IRS for customer service at least once in the past 3 years (minimum 50% must have called in past 12 months), or have considered calling, regarding any of the following issues (WILL SOFT QUOTA FOR MIX OF ISSUES):
- Get a form or publication
- Check the status of economic stimulus check or tax refund (maximum 17%)
- Tax law questions
- Obtain tax ID number
- Return preparation assistance
- Payment information
- Obtain prior year tax return
- Inquiries regarding IRS notices

Soft quotas for mix (based on IRS tax filing population data) of:

Method of filing – electronic, mail
Preparation of return – use tax preparer or not
Type of schedule filed

Tax Professional Qualification Criteria Summary

Must work as a tax preparer
Must not work in market research or management consulting
Must be between 18 years old and 64 years old
Must have filed at least one tax return for a client in 2009 (for the 2008 year) – either electronically, or via mail
Must have called the IRS for customer service at least once in the past 3 years (minimum 50% must have called in past 12 months), or have considered calling, regarding any of the following issues (WILL SOFT QUOTA FOR MIX OF ISSUES):

Get a form or publication
Check the status of economic stimulus check or tax refund (maximum 17%)
Tax law questions
Obtain tax ID number
Return preparation assistance
Payment information
Obtain prior year tax return
Inquiries regarding IRS notices

Soft quotas for mix (based on IRS tax filing population data) of:

IRS call center line used (PPS, public)
Method of filing – electronic, mail
Type of returns/schedules filed
Size of firm
Type of tax professional (CPA, tax attorney, enrolled agent, unlicensed preparer, etc.)

IV. Privacy, Security, Disclosure, Anonymousity

We will protect the privacy of participants by not connecting respondent names or other personal identifying information to the data used in analysis and reporting. All respondents will be issued an ID number as the only identifier in the data. We will also control official access to the information and will not allow public access to the information. The questionnaires and any data collected during the survey will be stored in a secure location for three years, and then destroyed.

Forrester Research, Momentum Market Intelligence, and data collection subcontractors adhere to the CASRO (Council of American Survey Research Organizations) standards for privacy and anonymousity. These standards can be found in Attachment D.

The IRS will ensure compliance with Taxpayer Bill of Rights II. All participants will be treated fairly and appropriately.

The security of the data used in this project and the privacy of participants will be carefully safeguarded at all times. Security requirements are based on the Computer Security Act of 1987 and Office of Management and Budget Circular A-130, Appendices A7B. Physical security measures include a locked, secure office. Data security at the appropriate levels have been accomplished. Systems are password protected, users profiled for authorized use, and individual audit trails generated and reviewed periodically.

The IRS will apply and meet fair information and record-keeping practices to ensure privacy protection of all participants. This includes criterion for disclosure—laid out in the Privacy Act of 1974, the Freedom of Information Act, and Section 6103 of the Internal Revenue Code—all of which provide for the protection of taxpayer information as well as its release to authorized recipients.

Anonymousity will be safeguarded. During the data collection process, participants will not be identified to IRS personnel. In addition, no participant names will be mentioned in the reports or data files. Participants will be advised that comments will be audiotaped. Anonymousity is assured by virtue of agency policy. The terms of IRS’s contract with the data collection requires that the anonymousity of any data be maintained.

V. Burden Hours

The estimated time to complete each survey, from screening to completion, is 20 minutes.

We anticipate a 25% response rate. The time to refuse to take the survey is estimated at 2 minutes, both on the phone and online. We further anticipate that approximately 30% of taxpayers contacted for this study will qualify, and that 50% of tax preparers will qualify. The screening portion of the survey is estimated to be 5 minutes.

Given these percentages, we will need to contact 5,332 taxpayers in order to screen 1333 individuals for qualification, ultimately resulting in 400 taxpayer completed interviews. We will need to contact 800 tax professionals in order to screen 200 qualification, ultimately resulting in 100 completed interviews with tax professionals.

5332 phone non-participants contacted at 2 minutes each = 10,664 minutes = 178 hours
800 online non-participants contacted at 2 minutes each = 1,600 minutes = 27 hours

1333 phone participants screened at 5 minutes each = 6,665 minutes = 111 hours
200 online participants screened at 5 minutes each = 1,000 minutes = 17 hours

400 phone participants completed at 15 minutes each = 6,000 minutes = 100 hours
100 online participants completed at 15 minutes each = 1,500 minutes = 25 hours

Total burden hour estimate = 458 Hours

The OMB number for this study is 1545-1432.
If you have any comments regarding this study, please write to:

Internal Revenue Service, Tax Products Coordinating Committee, SE:W:CAR:MP:T:T:SP, 1111 Constitution Ave. NW, Washington, DC 20224

- 12 -

File Type	application/msword
Author	Kaitlin McDevitt
Last Modified By	qhrfb
File Modified	2009-05-26
File Created	2009-05-26