CS-09-143 - Supporting Statement

OMB SUPPORTING STATEMENT

STUDY TO MEASURE CUSTOMER SATISFACTION

IRS SB/SE FIELD EXAMINATION TAX CUSTOMERS

February 2009

A. Introduction

Background/Overview

IRS has replaced the traditional measures of accomplishment with a balanced measurement system consisting of business results, customer satisfaction, and employee satisfaction. As an important customer interface for the Small Business/Self-Employed Division, Field Examination will need feedback from customers to continuously improve its operations. This initiative is part of the service-wide effort to establish a system of balanced organizational performance measures mandated by the IRS Restructuring and Reform Act of 1998. This is also a result of Executive Order 12862, which requires all government agencies to survey their customers and incorporate customer preferences in their process improvement efforts.

Objectives of Data Collection

The primary goals of the survey are to survey Field Examination external customers on an ongoing basis regarding their expectations, track customer satisfaction progress over time nationwide, and identify operational improvements.

B. Methodology

Sample Design

The Field Examination Program will draw a census of closed case records each month and provide this file to the sampling/analysis vendor, Macro International Inc. (Calverton, MD). The sampling is a random sample, stratified by the 7 sites and 2 outcomes (unagreed and all other respondents). Macro has designed the sample size each month that will provide a 95% confidence level with +/-4.2% precision for annual results at the Area level and 95% confidence level with +/-3.9% precision for quarterly results at the National level.

Macro will replace the taxpayer identification number with an artificial tracking code and then will provide a file with the names and addresses to the administration vendor, DRC, for fielding to customers by mail. Macro’s subcontractor, ScanTron, will receive and manually enter the survey responses and will provide a quarterly file to Macro for analysis and reporting.

Data to be Collected

The Field Examination Program will draw a sample from approximately 334,000 closed cases (not including exclusions) over the course of the year, broken down by 7 Area sites (approximately 447,700 audits closed per year at each of the 7 Areas). Macro will select a monthly sample of 88 per month (264 per quarter) from each site. Due to a historically low response rate among unagreed respondents, Macro will oversample this segment of the population each month. Within each site Macro will select a monthly sample of 33 unagreed respondents and a random sample of 55 respondents within each site. This will result in a total of 1,848 surveys mailed each quarter and 7,392 surveys each year.

We estimate a 60% response rate based on previous data collection using the same instrument.

SITE	SURVEYS SENT		NUMBER OF COMPLETES		SAMPLING ERROR*
	Annual	Quarter	Annual	Quarter	Annual	Quarter
Site 1	1,056**	264	634	159	2.5%	N/A
Site 2	1,056**	264	634	159	2.5%	N/A
Site 3	1,056**	264	634	159	2.5%	N/A
Site 4	1,056**	264	634	159	2.5%	N/A
Site 5	1,056**	264	634	159	2.5%	N/A
Site 6	1,056**	264	634	159	2.5%	N/A
Site 7	1,056**	264	634	159	2.5%	N/A
TOTAL	7,392	1,848	4,438	1,113	0.9%	1.9%
* Error margins are based on a 95% confidence interval (sampling error will be lower—1.6%—if 80% confidence is used at the site level). This is the maximum error based on a 50% rating and does not include minor design effects based on the stratification. ** Macro will select 388 unagreed customers annually (or 97 per quarter) per site.

How Data is Collected and Used

DRC will administer the survey by mail on a monthly basis. Macro has a subcontract with ScanTron who will collect the data and send Macro databases on a monthly basis. Macro will, on a quarterly basis, summarize the quantitative ratings and produce a national report showing customer satisfaction scores on all Field Examination survey items. The Fourth quarter report will include area-specific results and will serve as the FY 2009 “year-end” report. Macro will include any relevant database variables in the analysis and will weight the survey responses as necessary to accurately reflect the entire customer base.

For the quarterly reports, Macro will use basic and advanced statistical techniques including:

• Survey counts and overall response rates

• The overall level of customer satisfaction with services provided

• The frequencies and mean for all survey items

• The difference in satisfaction ratings and attitudes across customer segments

• Analysis of the relationship between survey responses

• Analysis of the one open-ended question for improvement suggestions

Macro will hold the identities of respondents private; therefore Macro will not provide the IRS with data or status updates that are linked to individual respondents. Upon completion of fielding, Macro will provide anonymous survey data to the IRS, without any individually identifying information such as name, address, or taxpayer identification number.

Dates Collection Begin and End

The survey will be administered to a sample of the population of Field Examination closed cases in April 1, 2009 through March 31, 2010. Data collection will begin in May 2009 using extracts of the April 2009 closed cases. March 2010 will be the last month of closed cases for this study.

Who is Conducting the Research and Where

Macro will be responsible for pulling the sample and conducting data analysis. Separate vendors will be responsible for printing and administering the survey via mail, and then providing the dataset to Macro.

Cost of Study

Macro contract: $ 90,608

There will be no stipends paid to participants.

Recruitment Efforts

Sample will be provided by the IRS

Location-Region/City and Facility

Data extract: IRS AIMS Database, New Carrollton, MD

Sampling and analysis: Macro Headquarters, Calverton, MD

Survey Processing and Data Collection: ScanTron, Eagan, MN

Sample Printing and Mailing: DRC, Eagan, MN

Expected Response Rate

The expected response rate is 60%.

Methods to Maximize Response Rate

Standard procedures will be used in order to obtain the highest response rate possible for the mail survey. These will include:

1) Pre-notification letter on IRS letterhead about the survey

2) [First package] Cover letter (1), questionnaire, and return postage paid envelope (RPPE)

3) Postcard reminder

4) [Second package] Cover letter (2), a copy of questionnaire, and RPPE to non-respondents.

Testing and Survey Structure/Design

The Field Examination customer satisfaction survey is attached and has been tested previously. Based on the IRS’s experience with the instrument and the results of the focus groups and IRS stakeholder interviews Macro plans to conduct this Spring 2009, we anticipate that some survey items will be modified. However, the length of the survey will not be affected; therefore the burden on the taxpayers will remain the same.

The survey will continue to ask respondents to evaluate various aspects of their experience and to provide an overall summary evaluation. The results should facilitate more effective management of SB/SE Field Examination by providing insight from the customer’s perspective about possible improvements and providing useful input for program evaluation.

The survey includes several rating questions evaluating service delivery during the examination process as well as one demographic question. In addition, ample space will be provided for suggestions for improvement. Survey scoring for this contract will be based on the Customer Satisfaction Survey Score response average to the key question—“How would you rate your overall satisfaction with the way your case was handled?” Questions will utilize a 5-point rating scale, with 1 being very dissatisfied and 5 being very satisfied.

Efforts Not to Duplicate Research

This is the only study conducted by the IRS on this specific population. Macro will ensure that customers only receive one questionnaire within a 12-month period.

Participants Criteria

The population frame consists of all individuals with closed cases each month using files provided by the IRS. All listed participants will be eligible except:

• Individuals who did not respond to the IRS audit appointment letter and subsequent correspondence

• Cases identified as having a bad address

• Cases with international addresses

• Taxpayers who have been surveyed in the previous 12 months

C. Privacy, Disclosure, and Security Issues

All participants will be subject to the provisions of the Taxpayer Bill of Rights II during this study and the vendor will ensure that all participants are treated fairly and appropriately.

The security of the data used in this project and the privacy of taxpayers will be carefully safeguarded at all times.  Security requirements are based on the Computer Security Act of 1987 and Office of Management and Budget Circular A-130, Appendices A & B.  Physical security measures include a locked, secure office.  Notes are stored in locked cabinets or shredded.  Data security at the C-2 level is accomplished via the Windows NT operating system.  Systems are password protected, users profiled for authorized use, and individual audit trails generated and reviewed periodically. 

The IRS will apply and meet fair information and record-keeping practices to ensure privacy protection of all taxpayers.  This includes criteria for disclosure—laid out in the Privacy Act of 1974, the Freedom of Information Act, and Section 6103 of the Internal Revenue Code—all of which provide for the protection of taxpayer information as well as its release to authorized recipients.

The survey will not contain tax return or taxpayer information. Survey participants will not be identified in any of the documents or files used for this project.  We will limit and control the amount of information we collect to those items that are necessary to accomplish the research questions.  We will carefully safeguard the security of data utilized as well as the privacy of the survey respondents.  We will apply the fair information and record-keeping practices to ensure protection of all survey respondents. The criterion for disclosure laid out in the Privacy Act, the Freedom of Information Act, and section 6103 of the Internal Revenue Code provides for the protection of information as well as its releases to authorized recipients.

The survey data returned to IRS will have no identifying information relating specific records to individual taxpayers. Public and official access to the information will be tightly controlled. The computer files containing this tabulated information will remain password protected at all times. Data security approaching level C-2 will be accomplished using the Windows XP operating system.

PRA Statement and OMB Control Number

For the Field Examination Survey, the OMB Control Number and required information will be provided on the survey itself.

D. Burden Hours

The Field Examination Survey has been designed to minimize burden on the taxpayer. The redesigned survey is not expected to add to the burden hours of the respondents. The time that a respondent takes to complete the survey has been carefully considered and only the most important areas are being surveyed. The average time of survey completion is expected to be 3 minutes, plus 2 minutes to read the pre-notification letter. This is based on the questionnaire consisting of 18 satisfaction questions and one demographic question, plus one open-ended question on suggestions for improvement. The questions are generally one sentence in structure and on an elementary concept level.

Using the response rate of 60%, then the total burden in hours is estimated to be:

Respondents

4,438 pre-notification letters x 2 minutes = 147.9 hours

4,438 completed surveys x 3 minutes = 221.9 hours

Subtotal = 369.8 hours

Non-respondents

2,954 pre-notification letters x 2 minutes = 98.5 hours

Subtotal = 98.5 hours

TOTAL BURDEN = 468.3 hours

E. Statistical Contact

For questions regarding the study or questionnaire design or statistical methodology, contact:

Mr. Larry Luskin

Director of Customer and Employee Research

Macro International Inc.

11785 Beltsville Drive, Suite 300

Calverton, MD 20705

Telephone: 301-572-0334

E-Mail: [email protected]

F. Attachments

Survey Materials

Field Examination Survey

Field Examination Survey Pre-Notification Letter

Field Examination Survey- Cover Letter (1)

Field Examination Reminder Postcard

Field Examination Survey- Cover Letter (2)