OMB STATEMENT FOR THE
TOLL-FREE CUSTOMER SATISFACTION SURVEY – OPTION YEAR 1 CY08
WAGE & INVESTMENT DIVISION
January 2008 - Ongoing
TIRNO-05-Z-00014
IRS has replaced the traditional measures of accomplishment with a balanced measurement system consisting of business results, customer satisfaction, and employee satisfaction. The Toll-free function within the Compliance organization of Wage and Investment (W&I) Division is responsible for responding to customer technical and account inquiries, resolving customer account issues, providing account settlement (payment options), and working related issues. As an important customer interface for W&I, Toll-free needs feedback from customers to continuously improve its operations. This initiative is part of the Service-wide system of balanced organizational performance measures mandated by the IRS Restructuring and Reform Act of 1998. This is also a result of Executive Order 12862, which requires all government agencies to survey their customers and incorporate customer preferences in their process improvement efforts.
The key goals of the Toll-free Customer Satisfaction Survey are to investigate what specific actions IRS can take to improve customer satisfaction with the current Toll-free W&I service and to gauge the effectiveness of changes to the Toll-free W&I service. The survey is used to detect changes in customer satisfaction before and after specific changes made to the Toll-free W&I services such as adding new scripts and features to Toll-free W&I. This project is expected to continue the customer satisfaction research conducted via the IRS’s internal automated voice response system.
The phone survey (attached) is conducted via an automated computer system. Completion of the survey takes approximately ten minutes. The questionnaire covers the following areas: satisfaction with the automated answering system, wait time, IRS representative, issue resolution, and various demographics. The results help to facilitate more effective management of W&I by:
Providing insight from the customer’s perspective about possible improvements.
Providing useful input for program evaluation and execution at the programmatic and area office level of service delivery.
Providing additional information on important “drill down” areas.
Survey scoring for this contract is based on the Customer Satisfaction Survey Score response average to the keystone question – “How would you rate your overall satisfaction with the service you received during this call?” Questions utilize a 5-point rating scale, with 1 being very dissatisfied and 5 being very satisfied. All survey
responses generated are kept confidential. The vendor ensures that taxpayers responding to the survey are guaranteed anonymity.
The Toll-free Customer Satisfaction Survey has already been developed and is currently in use. The survey is conducted via an automated computer system. Completion of the survey takes approximately ten minutes. It contains a standard set of questions, but allows certain “drill down” questions that vary as needed. To maintain the ability to compare results from quarter to quarter, changes that impact the meaning of questions will occur minimally. For costing purposes, changes may occur every six months.
The survey instrument remains the same as the previous year, which consists of closed-ended and open-ended questions. However, the drill-down open-ended questions are subject to change. Five questions were added to the survey in 2006 in order to obtain demographic and preference information.
The contractor will administer the survey by phone and IVR on a continuous basis January 1, 2008 - December 31, 2008. Standard procedures will be used in order to obtain the highest response rate possible.
Design and Methodology
The automated telephone survey is based upon a sample of 9,750 taxpayers per quarter, yielding 39,000 attempted interviews per year. The sampling is stratified by 25 W&I Toll-Free sites for 390 attempts per site per quarter. This sample size allows the vender to focus on targeted customer groups based on specific survey responses. For instance, the IRS might want to concentrate on dissatisfied customers. This sample size allows the vender to get valid samples for these small populations.
The contractor must plan to make 390 attempts with the goal to complete approximately 200 interviews per site per quarter, or to offer the survey to approximately 39,000 taxpayers per year, stratified by each of the 25 W&I Toll-free sites. This requires making 6 attempts per day, five days per week for 52 weeks at each site to yield approximately 9750 attempts per quarter. A precision margin of 5% and confidence interval of 95% is expected.
The contractor will provide the sampling specifications to W&I to apply to the sampling databases. This is the sampling methodology currently in use for the Toll-free Customer Satisfaction Survey. Each site will report its sampling efforts to the vendor on a daily basis. The contractor will monitor the sampling process to ensure the procedures are providing the desired number of respondents and will coordinate with appropriate W&I personnel on sampling and related matters. Sampling will be performed continuously by IRS quality control monitors in the Toll-free sites according to the procedures developed by the contractor.
Sampling is performed continuously by IRS quality control monitors in the Toll-free sites according to the procedures developed by the vendor. Each site reports its sampling efforts to the vendor on a daily basis. The vendor monitors the sampling process to ensure the procedures are providing the desired numbers of respondents and coordinate with appropriate W&I personnel on sampling and related matters. The vendor provides the sampling specifications to W&I to apply to the sampling databases. This is the same sampling methodology now in use for the Toll-free Customer Satisfaction Survey.
We expect to receive a 68% response rate that will yield 26,520 completed surveys per year (approximately 1,060 per site).
The vendor, on a quarterly basis, produces a summary report that consists of an overview of the focus groups and a summary of the quarterly targeted questions. At the end of the last reporting period in a given year, the vendor will summarize the quantitative ratings and produce a national report showing customer satisfaction scores for all Toll-free service survey items and overall improvement priorities for the function. This report will show overall results for each of the 25 W&I Toll-free sites. The project vendor will weight the survey responses as necessary to reflect accurately the entire customer base.
The survey responses will be scored on a 5-point rating scale, with 1 being very dissatisfied and 5 being very satisfied. All survey responses will be confidential. The contractor shall ensure that taxpayers responding to the survey are guaranteed anonymity. The overall satisfaction item is, “Everything considered, rate your overall satisfaction with the service you received during this call.”
The contractor will use basic and advanced statistical techniques including, but not limited to, analysis of variance and the prioritization of improvement initiatives, to provide:
Survey counts and overall response rates for the 25 W&I sites.
Overall level of customer satisfaction for W&I Toll-free.
Averages and frequencies for all ratings questions.
Differences in satisfaction ratings across customer segments.
Areas of service, in priority order, where W&I Toll-free should focus efforts to improve overall satisfaction.
Cross tabulations of variables.
Analysis of open-ended comments
The contractor will include any relevant database variables in the analysis. The contractor and W&I will agree on the exact specifications and contents of reports. Separate open-ended comments will be provided within 30 days after the end of each quarter.
The national report will show the calculation of a weighted rolling cumulative score (12-month period) for Customer Satisfaction and Dissatisfaction. Raw data will be provided to a distribution list of IRS personnel and made available through Pinpoint within 5 business days of the end of each month.
The contractor will produce reports that include:
1. Leverage Analysis
2. Satisfaction mean scores of rating questions
3. Distribution of dissatisfied, neutral, and satisfied % of rating questions
4. Quarterly comparisons with prior year’s cumulative data
5. Summary
At the end of each fiscal year, the contractor will summarize the quantitative ratings and produce one annual national report showing overall satisfaction scores for all Toll-free service items and improvement priorities. This report will provide results for each of the 25 W&I Toll-free sites. The contractor will weight the survey responses to reflect accurately the entire customer base.
Evaluation
The vendor will use basic and advanced statistical techniques including, but not limited to, analysis of variance and the prioritization of improvement priorities using the vendor’s established technique:
Survey counts and overall response rates for the 25 sites.
The overall level of customer satisfaction with services provided by W&I Toll-free.
The averages and frequencies for all ratings questions.
The differences in satisfaction ratings and attitudes across customer segments.
Which areas of service, in priority order, W&I should focus its resources to improve overall satisfaction.
Cross tabulations of variables.
Analysis of the relationship between survey responses.
The vendor’s Net Impression analysis, which prioritizes improvement areas.
Estimates of the Burden of Data Collection
The survey interview has been designed to minimize burden on the taxpayer. The time that a respondent takes to complete the phone survey has been carefully considered and only the most important areas are being surveyed. The average time of survey completion is expected to be 10 minutes. The questions are generally one sentence in structure and on an elementary concept level.
We have made every attempt in designing this survey to maximize response rate. Using the response rate of 68%, the total burden in hours is estimated to be 4,862 hours (26,520 surveys x 11 minutes/60 minutes). Note that 11 minutes reflects the time to get into the automated survey (1 minute) as well as the time to complete the survey (10 minutes maximum). For non-respondents, the total number of burden hours is estimated to be 208 hours (12,480 surveys x 1 minute/60 minutes).
The grand total is estimated to be 5,070 hours.
Estimated Cost
The estimated cost for this survey is $115,773.00.
Other Information
The following information will be provided to Human Capital Office within 60 days after the close of the survey data collection operations:
Findings: A brief summary of significant findings that were evidenced in the results.
Actions taken or lessons learned: A brief summary of any action taken or lessons learned as a result of the findings.
Taxpayer participation (number of requests for taxpayer participation and number of interviews completed).
Dates on which the data collection began and ended.
Cost (excluding regular salaries of IRS employees).
Statistical Contact
For questions regarding the study, questionnaire design, or the statistical methodology, contact:
Elaine Lowitz
Pacific Consulting Group
399 Sherman Avenue, Suite 1
Palo Alto, California 94306
(650) 327-8108
| File Type | application/msword |
| File Title | STATEMENT OF WORK |
| Author | Jeffery McConnell |
| Last Modified By | qhrfb |
| File Modified | 2007-10-16 |
| File Created | 2007-10-16 |