CS-08-096 - Supporting Statement

Taxpayer Advocate Service – Customer Satisfaction Survey

OMB Clearance Package

1. Introduction

Background/Overview

Requirement to perform customer satisfaction surveys.

Executive Order 12862 requires all government agencies to survey their customers, and incorporate customer preferences in their process improvement efforts.¹

Internal Revenue Bulletin 2005-45 (November 7, 2005)² prescribes that: “The customer satisfaction goals and accomplishments of operating units within the IRS will be determined on the basis of information gathered through various methods. For example, questionnaires, surveys and other types of information gathering mechanisms may be employed to gather data regarding customer satisfaction. Information to measure customer satisfaction for a particular work unit will be gathered from a statistically valid sample of the customers served by that operating unit and will be used to measure, among other things, whether those customers believe that they received courteous, timely, and professional treatment by the IRS personnel with whom they dealt. Customers will be permitted to provide information requested for these purposes under conditions that guarantee them anonymity. For purposes of this section, customers may include individual taxpayers, organizational units, or employees within the IRS and external groups affected by the services performed by the IRS operating unit.“

This Bulletin reflects the objectives of Treasury Decision 801.4 (September 7, 1999), which established a balanced performance measurement system, to include the requirement to measure customer satisfaction. Additionally, 26 CFR Part 801 and section 1201 of the Internal Revenue Service Restructuring and Reform Act of 1998, Public Law 105-206, 112 Stat. 685, 713 (1998), requires the IRS to establish a performance management system for those employees covered by 5 U.S.C. 4302 that, among other things, establishes “goals or objectives for individual, group, or organizational performance (or any combination thereof), consistent with the IRS’ performance planning procedures, including those established under the Government Performance and Results Act of 1993, division E of the Clinger-Cohen Act of 1966 . . ., Revenue Procedure 64-22 . . ., and taxpayer service surveys.”

Treasury Strategic Objective F4A, “Collect Federal Revenue When Due, Through a Fair and Uniform Application of the Law,” references as a measure of its success: “fairness of the tax system administration as measured by scores on the IRS’s customer satisfaction survey.”³

History of customer satisfaction surveys within the Taxpayer Advocate Service:

The Taxpayer Advocate Service organization (TAS) of the Internal Revenue Service (IRS) has been conducting customer satisfaction surveys since calendar year 2001. These surveys gauge the quality and timeliness of services that are delivered to taxpayers and their representatives who have recently sought TAS assistance. These surveys are a component of TAS’ organizational performance measurement system, or “balanced performance measures,” which also include employee engagement and business measures.

Objectives of data collection

It is through this process that the Taxpayer Advocate Service is able to establish and maintain an awareness of customer expectations, to identify gaps between customer expectations and organizational performance, and to better calibrate its services to meet customer expectations. Data collected will help TAS to identify the underlying causes of customer dissatisfaction in different types of case work and in specific organizational locations, and provide the basis for implementing appropriate organizational changes. TAS provides remedial services to approximately 260,000 taxpayers a year; obtaining customer feedback about the timeliness and quality of TAS’ services from a sample of this group will enable TAS to improve its operations for the whole population.

2. Methodology

Sample design

The sample will be randomly selected by an independent vendor from weekly case data extracts provided by the Government Task Manager ⁴. The vendor, in collaboration with TAS, will design the sample to ensure that each TAS office is provided with a statistically valid report of its customer base on an annual basis.

The vendor will administer the survey, analyze the data, provide sanitized copies of the data each quarter, and produce reports on the results at both the office and global level. The vendor will stratify the sample to produce survey estimates with at least a 95% confidence interval with a precision level of <1% at the national level and 5% at the local office/campus office level. A sampling plan that is currently in use for FY 2008 covering each office is attached. SEE Attachment C.

Methods to maximize response rate:

TAS will provide the vendor with multiple phone numbers for customers when available and will indicate the best times to call as indicated from customer records. Calls will be made to cell phones when consent has been provided. The vendor will conduct project-focused training for interviewers and will send pre-notification letters to prospective interviewees. Additionally, for each customer selected for the sample, up to seven callback attempts will be made and the vendor will use refusal conversion protocols. In addition, TAS is working to improve the availability of usable customer phone numbers for its vendor by addressing case worker inputting instructions.

Expected response rate:

Given recent experience with TAS’s recent telephone surveys, a predictive response rate of approximately 35% is expected using AAPOR RR1 calculations. This response rate is similar to other response rates for customer surveys at the IRS. The vendor will analyze non-response in two ways. First, the vendor will compare the demographics or other information about survey respondents to the sampling frame. This will determine face validity. Second, the vendor will analyze the results of late responders (e.g., individuals who responded after 4 callbacks), who are often viewed as a proxy for non-respondents, to early respondents.

In regard to the low response rate, the IRS will assume that all data collected from this survey is qualitative in nature, and that no critical decisions will be made by TAS solely from the analysis of data from this survey.  The results from this survey are simply one piece of a larger set of information needed to assess practitioner/taxpayer needs related to services provided by the IRS.

Efforts to not duplicate research:

No other studies are being conducted within IRS to acquire the information needed from this customer base. Taxpayers contacted in connection with the TAS survey in the past twelve months will not be subject to survey during the current cycle.

Data collection dates

Data will be collected throughout fiscal year 2009 using records of closed TAS cases for the period from October 1, 2008 through September 30, 2009. The applicable task order has been awarded to an independent research organization under an IRS Blanket Purchase Agreement.

Data to be collected

The survey will collect data on satisfaction with the TAS employee, process, and general customer experience (along with limited demographics). The survey consists of approximately 28 questions, including up to four open-ended questions. SEE ATTACHMENT A.

How data will be analyzed

Data sets are distributed on a quarterly basis to trained analysts who produce ‘drill-down’ analysis for each TAS office to facilitate understanding of customer dissatisfaction issues pertaining to TAS’ work categories and customer segments.

How data will be used

The customer satisfaction data and associated reports provided by the vendor will be used by TAS to identify opportunities for service improvement at both local and global levels and to take actions that are responsive to customer expectations. Each of TAS’ offices is given an annual performance goal for improving their customer satisfaction score and against which they are evaluated.

Who is conducting research?

• Macro International

Headquarters:

11785 Beltsville Drive

Calverton, MD 20705

Facilities:

Macro plans to conduct all data collection activities for the Customer Satisfaction Survey from its telephone call center in Burlington, Vermont.

Stipend:

TAS’ task order with Macro covering basic survey activities for the fiscal year 2009 option is in the amount of $629,785⁵. Participants in the survey will not receive a stipend.

Recruitment efforts: Participants will be recruited using the aforementioned protocols using a pre-notification letter and a CATI system.

Test structure/design: The survey procedures include interviewer training, in which experienced interviewers and supervisors will provide preliminary feedback on the instrument after practice sessions. Next, the survey will begin with a highly monitored pre-test of at least 20 real participants to determine and finalize changes to the CATI script or survey content. If the changes are not substantial, the survey contractor will keep these interviews as complete.

3. Participants Criteria

With the following exceptions, this survey will apply to all taxpayers or their powers of attorney whose cases were completed by TAS during fiscal year 2009.

Due to operational and Privacy Act considerations, the following classes of customer cases are presently excluded from this survey:

1. Criminal Investigations. To avoid the possibility of inadvertently revealing to the taxpayer that they might be the subject of an investigation, e.g., involving a fraudulent claim for a tax refund, Criminal Investigation cases are not a part of this survey.

2. Congressional case referrals. TAS has recently conducted a survey of Congressional offices to gauge their opinions about the quality and timeliness of TAS services.

3. Immediate Interventions. TAS may dedicate case advocate resources to working issues that impact a large group of taxpayers, some of whom may never actually have contact with a TAS employee. The TAMIS database records impacting these taxpayers are not provided to the vendor and they are not included in the customer satisfaction study, which seeks to measure only the impact of services provided to taxpayers who have had interactions with TAS.

4. Taxpayers surveyed by TAS during the past 12 months.

5. Taxpayer representatives who have withdrawn their representation on the case subject to the sample.

4. Privacy, Security, Disclosure, Confidentiality

The survey vendor will explicitly promise all participants that their identities will be held confidential by the survey vendor and that their responses will be provided to the IRS without any personally identifying information. In addition, the survey vendor will not provide the IRS with results for cells with fewer than ten respondents. As part of the security clearance package, all interviewers and project staff who have access to sensitive information sign and return a non-disclosure agreement. Data provided to the vendor by TAS are used for the purpose of selecting a sample and conducting the survey. Specific identifying characteristics of taxpayers or their powers of attorney are used only for the purpose of establishing contact for the survey and for ensuring vendor accountability in the event that a taxpayer initiates an inquiry about the conduct of the survey. The vendor is responsible for limiting access to the data based on a need to know basis. Vendor employees who conduct interviews have access only to enough information needed to initiate contact with prospective subjects of the survey (e.g., name and phone number). Survey respondents are called by a computer’s queuing system. Access to all case-related data are maintained by the vendor on a secure computer system to which access is restricted to the vendor’s project manager and/or the vendor’s employees (statistician/programmer) who are responsible for selecting the targeted survey population and establishing the queuing system.

All data containing specific identifying characteristics of either the taxpayer or his/her Power of Attorney (POA) responding on behalf of the taxpayer (e.g., name, address, phone number, case number, etc.) are removed by the vendor after the survey is completed and the data are returned to IRS. The vendor is responsible for sanitizing the data provided to IRS, viz., removing all taxpayer-specific identifying characteristics and for ensuring the subsequent validity, accuracy and completeness of the data set that is returned to IRS. The data available to IRS therefore is restricted to: (a) responses to the survey questions; and (b) case characteristics.

5. Burden Hours

The survey instrument has been timed by the vendor at ten minutes. Assuming a sample size of approximately 15,309 during fiscal year 2009, it is estimated that this survey will involve approximately 3,025.35 hours of taxpayer burden.

Burden Hour Calculations:

The survey will take approximately 10 (last clocked time was 9 minutes 52 seconds) minutes to complete across 15,309⁶ total respondents.

Based on the in-going goal being n=15,309 ending sample size and a response rate as high as 35%, a total of 43,740 potential respondents will have to be called in order to reach the 15,309 ending sample 43,740 minus 15,309 leaves 28,431 non-participants.

The contact time to determine non-participation could be up to 1 minute, with the resulting burden for non-participants being 28,431 x 1 = 28,431 minutes / 60 minutes = 473.85 burden hours.

Then, for those who do participate, the length of the survey is 10 minutes, with the time burden for that group being 15,309 x 10 minutes = 153,090 total minutes / 60 minutes = 2551.50 burden hours.

Thus, the total burden hours for the study would be (473.85 + 2551.50) = 3025.35 hours.

6. Cover letter/correspondence.

Individuals/businesses selected for the sample will be sent a ‘pre-notification letter’ by the vendor which alerts them to their selection for an interview, provides the underlying reason for the survey, and affords them the opportunity to ‘opt out’ of the survey if they do not wish to participate. Individuals selected for the sample will be contacted by telephone within approximately one week of having received the pre-notification letter. The pre-notification letter is provided in Attachment B.

Scenarios: Not applicable.

7. Paperwork Reduction Act (PRA) Statement:

The Paperwork Reduction Act requires that the IRS display an OMB control number on all public information requests. The OMB Control Number for this study is 1545-1342. Also, if you have any comments regarding the time estimates associated with this study or suggestions on making this process simpler, please write to the:

Internal Revenue Service

    Tax Products Coordinating Committee

    SE: W: CAR: MP: T: T: SP

    1111 Constitution Ave. NW, Washington, DC  20224

1 Reference: http://govinfo.library.unt.edu/npr/library/direct/orders/2222.html

2 Reference: http://www.irs.gov/irb/2005-45_IRB/ar12.html

3 Reference: http://www.treas.gov/offices/management/budget/planningdocs/treasury-strategic-plan.pdf

4 Case data is derived from the Taxpayer Advocate Management Information System, (TAMIS)

5 Option for FY 2009 has not been exercised as of June 19, 2008, costs are for basic survey administration costs, does not include optional consulting tasks. The $629,785 is the administrative costs directly and indirectly involved in our vendor performing the tasks of data gathering including sampling, interviewing the taxpayers, accumulating data, performing analysis and providing datasets and reports. 

6 Expected sample of 15,289 plus 20 pre-test.

Taxpayer Advocate Service Fiscal Year 2009 Customer Satisfaction Survey

- 7 -