CS-09-144 - Supporting Statement

Office of Management and Budget

Clearance Package

Customer Satisfaction Survey for External Customers of Media and Publications Division

Internal Revenue Service

Tax Forms and Publications Division

February 25, 2009

OMB SUPPORTING STATEMENT

STUDY TO MEASURE CUSTOMER SATISFACTION

IRS MEDIA & PUBLICATIONS

February 2009

A. Introduction

Background/Overview

The Media and Publications (M&P) Division of the IRS is conducting a customer survey to measure external customer satisfaction with publications, forms, and other documents it produces. Feedback from customers is important in order to assess perceptions of M&P products and service to help the various program areas develop service improvement actions.

There will be five questionnaire instruments used to assess customer satisfaction levels–one for each of the following five audiences: individual taxpayers, business taxpayers, tax preparers, Volunteer Stakeholder, Partnership, Education & Communication (SPEC) partners, and forms distributors.

Objectives of Data Collection

The primary goals of the survey are to survey external customers who use the documents and services provided by M&P (i.e., individual taxpayers, business taxpayers, tax preparers, volunteer (SPEC) partners, and forms distributors). More specifically, the survey will:

• Assess external customer satisfaction with Media and Publications (M&P) products and services

• Solicit suggestions for product and service improvements

• Identify actionable results that M&P program areas can use to improve specific products and business processes

B. Methodology

Sample Design

The vendor, Macro International Inc. (Calverton, MD), will draw the survey sample from the 5 populations. Macro has designed a sampling plan that will provide results with at least +/-5.0% sampling error at a 95% confidence level for all but Volunteer (SPEC) Partners. For SPEC Partners, this plan provides the same level of statistical precision as the 2008 study (8.0%).

Segment	List Source	Mode of Administration
Individual Taxpayers	Household RDD	CATI Telephone
Business Taxpayers	Purchased infoGROUP List	Mail/Web
Tax Preparers	Purchased infoGROUP List	Mail/Web
SPEC Partners	Provided by IRS SPEC	Web
Forms Distributors	Provided by IRS (TFOP, CBOP)	Web

Macro will replace identifying information with an artificial tracking code and then will provide a file with the names and addresses to Macro’s survey adminstration center in Burlington, VT for the web and CATI fielding and to the mail administration vendor for the mail fielding. Completed surveys will be returned directly to Macro for processing, analysis and reporting.

The business taxpayer and tax preparer respondents will be given the option through the mail survey to complete the survey via the web. Controls will be in place so that a person cannot submit multiple surveys via both methods.

Data to be Collected

Macro will draw a sample from each population. Based on the 2008 survey and our experience within the IRS for the 5 populations and 3 survey fielding methods (mail, telephone, and web), we estimate the following response rates, which vary by population.

Segment	Sample Size	Estimated Response Rate	Completed Surveys
Individual Taxpayers	6,000	10%	600
Business Taxpayers	4,813	8%	385
Tax Preparers	1,750	22%	385
SPEC Partners	487	31%	151
Forms Distributors	2,353	34%	800

For each population, we will attempt to achieve the estimated response rate in the 2009 survey. We have enhanced protocols to maximize the response rate. The business taxpayer and tax preparer survey questionnaires will be administered via mail with the option to complete the survey via web. To achieve the maximum possible response rates, Macro and the mail administration vendor will send pre-notification letters and e-mails in addition to two reminder letters and e-mails where appropriate.

How Data is Collected and Used

Macro and the mail administration vendor will administer the survey by mail, web, and telephone. The individual taxpayer survey will be administered via CATI telephone. The business taxpayer and tax preparer surveys will be administered using a joint mail/web methodology. The mail administration vendor will send a mail survey using a protocol that includes a pre-notification letter, initial survey mailing, reminder postcard, and follow-up survey mailing for non-respondents. Each mailing will include the option for the respondent to complete the survey by mail or by accessing the web version using a web link and unique password. The volunteer (SPEC) partner and forms distributor surveys will be administered via web. Macro will e-mail an invitation to each individual selected in the sample in addition to two reminder e-mails to non-respondents, as appropriate.

Macro will summarize the quantitative ratings and produce a national report showing customer satisfaction scores on all M&P Customer Satisfaction survey items. Macro will include any relevant database variables in the analysis and will weight the survey responses as necessary to accurately reflect the entire customer base.

For this report, Macro will use basic and advanced statistical techniques including:

• Survey counts and overall response rates

• The overall level of customer satisfaction with services provided

• The frequencies for all survey items

• The difference in satisfaction ratings and attitudes across customer segments

• Analysis of the relationship between survey responses

• Analysis of the open-ended questions

Macro will hold the identities of respondents anonymous. Macro will not provide the IRS with data or status updates that are linked to individual respondents. Upon completion of fielding, Macro will provide anonymous survey data to the IRS, without any individually identifying information such as name, address, or taxpayer identification number.

Dates Collection Begin and End

The survey will be administered to a sample of the population of M&P external customers. Data collection will begin April 20, 2009 and end June 26, 2009.

Who is Conducting the Research and Where

Macro (Calverton, MD) will be responsible for pulling the sample and conducting data analysis and reporting. Macro’s Burlington, VT office will be responsible for administering the web and telephone surveys, and also for receiving and processing the completed mail surveys. A separate GPO vendor will be responsible for printing and distributing the survey via mail.

Cost of Study

Macro contract: $130,058

There will be no stipends paid to participants.

Recruitment Efforts

Sample will be provided by the IRS for volunteer (SPEC) partners and forms distributors. Macro will purchase lists for business taxpayers and tax preparers and will use random digit dial (RDD) for individual taxpayers.

Location-Region/City and Facility

Data extract:

• SPEC Partners—IRS Database, Washington, D.C.

• Forms distributors—IRS Database, Washington, D.C.

• Business taxpayers, tax preparers, and individual taxpayers—Macro, Burlington, VT

Sampling and analysis: Macro Headquarters, Calverton, MD

Mail, Web and Telephone Data Collection: Macro, Burlington, VT

Expected Response Rate

The expected response rate for each population is:

• Individual taxpayers: 10%

• Business taxpayers: 8%

• Tax preparers: 22%

• SPEC Partners: 31%

• Forms distributors: 34%

With regard to the low response rate, the IRS will assume that all data collected from this survey is qualitative in nature, and that no critical decisions will be made by this office solely from the analysis of data from this survey.  The results from this survey are simply one piece of a larger set of information needed to assess the needs related to services provided by the IRS.

Methods to Maximize Response Rate

Standard procedures will be used in order to obtain the highest response rate possible for the mail surveys. These will include:

1) Pre-notification letter on IRS letterhead about the survey

2) [First package] Cover letter, questionnaire, and return postage paid envelope (RPPE)

3) Postcard reminder to non-respondents

4) [Second package] Cover letter, a copy of questionnaire, and RPPE to non-respondents

Additionally, the mail surveys have a web option to increase the response rate from individuals who prefer this option to a mail survey.

For the web only surveys, similar procedures to the mail surveys will be used to obtain the highest response rate. These will include:

1) Pre-notification e-mail about the survey

2) [First survey e-mail] E-mail invitation to complete the survey with a link to the survey, embedded with the potential respondent’s unique password

3) [Second survey e-mail] Reminder e-mail to non-respondents to complete the survey with a link to the survey, embedded with the potential respondent’s unique password

For the CATI telephone survey, Macro’s interviewers will apply specific techniques to gain selected respondent cooperation to complete the interview. Interviewers will make up to 5 call back attempts. Additionally, our interviewers will be specifically trained on gaining cooperation and avoiding refusals from a potential respondent.

Testing and Survey Structure/Design

The five M&P satisfaction surveys are attached. The surveys ask respondents to evaluate various aspects of their experience with content, usefulness, format, graphics, delivery, mode, availability, and other aspects of the production and delievery of information to enable taxpayers to understand and meet their obligations under Federal tax law. The results should facilitate more effective products and services of M&P by providing insight from the customer’s perspective about possible improvements.

The surveys include several rating questions evaluating service and document use as well as several demographic questions. In addition, ample space will be provided for suggestions for improvement. Satisfaction questions will utilize a 5-point rating scale, with 1 being very dissatisfied and 5 being very satisfied.

Efforts Not to Duplicate Research

This is the only study conducted by the IRS to measure customer satisfaction with IRS tax forms, instructions, publications and other documents.

Participants Criteria

The population frame consists of M&P external customers who use IRS publications and other documents to meet their tax obligations under Federal tax law.

C. Privacy, Disclosure, Security Issues

All participants will be subject to the provisions of the Taxpayer Bill of Rights II during this study and Macro will ensure that all participants are treated fairly and appropriately.

The security of the data used in this project and the privacy of taxpayers will be carefully safeguarded at all times.  Security requirements are based on the Computer Security Act of 1987 and Office of Management and Budget Circular A-130, Appendices A & B.  Physical security measures include a locked, secure office.  Notes are stored in locked cabinets or shredded.  Data security at the C-2 level is accomplished via the Windows XP operating system.  Systems are password protected, users profiled for authorized use, and individual audit trails generated and reviewed periodically. 

The IRS will apply and meet fair information and record-keeping practices to ensure privacy protection of all taxpayers.  This includes criteria for disclosure—laid out in the Privacy Act of 1974, the Freedom of Information Act, and Section 6103 of the Internal Revenue Code—all of which provide for the protection of taxpayer information as well as its release to authorized recipients.

The survey will not contain tax return or taxpayer information. Survey participants will not be identified in any of the documents or files used for this project.  We will limit and control the amount of information we collect to those items that are necessary to accomplish the research questions.  We will carefully safeguard the security of data utilized as well as the privacy of the survey respondents.  We will apply the fair information and record-keeping practices to ensure protection of all survey respondents. The criterion for disclosure laid out in the Privacy Act, the Freedom of Information Act, and section 6103 of the Internal Revenue Code provides for the protection of information as well as its releases to authorized recipients.

PRA Statement and OMB Control Number

For the M&P Customer Satisfaction Survey, the OMB Control Number and required information will be provided on the survey itself.

D. Burden Hours

The M&P Customer Satisfaction Survey has been designed to minimize burden. The time that a respondent takes to complete the survey has been carefully considered and only the most important areas are being surveyed. The surveys contain demographic questions to determine what kind of taxpayers are responding to the survey.

The average time of survey completion for the individual taxpayer survey is expected to be 16 minutes. This is based on the individual taxpayer questionnaire consisting of 61 questions asking about use, and satisfaction with services and products. Ten open-ended questions and 7 demographic questions are included. The questions are generally one sentence in structure and on an elementary concept level.

The average time of survey completion for the business taxpayer survey is expected to be 25 minutes, plus 2 minutes to read the pre-notification letter.This is based on the business questionnaire consisting of 34 questions asking about use, and satisfaction with services and products. Four open-ended questions and 2 demographic questions are included. The questions are generally one sentence in structure and on an elementary concept level.

The average time of survey completion for the tax preparer survey is expected to be 22 minutes, plus 2 minutes to read the pre-notification letter.This is based on the tax preparer questionnaire consisting of 18 questions asking about use, and satisfaction with services and products. Three open-ended questions and 2 demographic questions are included. The questions are generally one sentence in structure and on an elementary concept level.

The average time of survey completion for the Volunteer (SPEC) Partner survey is expected to be 26 minutes, plus 2 minutes to read the pre-notification e-mail.This is based on the SPEC Partner questionnaire consisting of 25 questions asking about use, and satisfaction with services and products. Three open-ended questions and 3 demographic questions are included. The questions are generally one sentence in structure and on an elementary concept level.

The average time of survey completion for the forms distributors’ survey is expected to be 10 minutes, plus 2 minutes to read the pre-notification e-mail.This is based on the forms distributors’ questionnaire consisting of 34 questions asking about use, and satisfaction with services and products. Ten open-ended questions and 2 demographic questions are included. The questions are generally one sentence in structure and on an elementary concept level.

We have made every attempt in designing this survey to maximize response rate. Burden estimates are based on a 10% response rate for individual taxpayers, 8% for business taxpayers, 22% for tax preparers, 31% for SPEC Partners and 34% for forms distributors.

Individual Taxpayer Survey

Survey: 600 x 16 min = 160.00 hrs

Subtotal: 160.00 hrs

Business Taxpayer Survey

Prenotification Letter: 4,813 x 2 min = 160.4 hrs

Survey: 385 x 22 min = 141.2 hrs

Subtotal: 301.6 hrs

Tax Preparer Survey

Prenotification Letter: 1,750 x 2 min = 58.3 hrs

Survey: 385 x 20 min = 128.3 hrs

Subtotal: 186.6 hrs

SPEC Partner Survey

Prenotification E-mail: 487 x 2 min = 16.2 hrs

Survey: 151 x 24 min = 60.4 hrs

Subtotal: 76.6 hrs

Forms Distributors Survey

Prenotification E-mail: 2,353 x 2 min = 78.4 hrs

Survey: 800 x 10 min = 133.3 hrs

Subtotal: 211.7 hrs

TOTAL BURDEN = 936.5 hours

E. Statistical Contact

For questions regarding the study or questionnaire design or statistical methodology, contact:

Mr. John Hurley

Project Director

Macro International Inc.

11785 Beltsville Drive, Suite 300

Calverton, MD 20705

Telephone: 301-572-0482

E-Mail: [email protected]

F. Attachments

5 Surveys

1 Example of Survey Communication Pieces (Business Taxpayers)

Survey Pre-Notification Letter

Survey- Cover Letter [first package]

Reminder Postcard

Survey- Cover Letter [second package]

10