2009 e-Services Customer Satisfaction Survey
Background/Overview
Research is an integral part of the IRS e-file marketing cycle and monitoring of e-product development. Research is performed at the beginning of the cycle to identify customers and their likes and dislikes prior to developing a product. In addition, research is performed once the product is offered to determine its acceptance and how to improve it. Research results provide ETA and the other Business Operating Divisions (BODS) with usage volume, penetration rates, and taxpayer profiles as to who does and does not use each product. It also provides BODS with detailed information to guide the next season’s marketing campaign and/or and how to direct outreach and marketing efforts. Finally, it provides the feedback necessary to determine if funding was spent wisely and achieved the expected results.
The 2009 e-Services Customer Satisfaction Survey is one of the surveys designed to provide IRS with continuing feedback and satisfaction data, all of which allow IRS to determine product or program performance over time and how to improve the product/program and increase its usage. The 2009 Survey follows a similar 2008 Survey conducted by IRS (via Russell), which benchmarked User satisfaction with, attitudes toward, and concerns about e-Services (with reference to similar results for Heavy Users from a 2007 survey), as well as Non-User reasons for non-usage and interest in e-Services.
Objectives of data collection
The overall objective of the 2009 e-Services Customer Satisfaction Study would be to assess current levels of satisfaction with the e-Services Program among Users while also assessing awareness and reasons for non-usage among Non-Users.
Methodology
Sample design
As noted above, there has been a prior wave of this research. However, there are changes being made to the sample design for Non-Users in this wave, with that segment re-configured to better target the actual potential users of e-Services who are not using it now. There are also changes/additions being made to the areas of investigation, with “awareness” of e-Services now an important measure – especially among Non-Users. Other questions have been eliminated from the survey, leaving the total burden on respondents to be very similar to that of the past survey.
Other than the change in Non-User sample configuration and additions and deletions to specific question areas, the study will replicate the methodology (telephone interviewing) and sample design and size of the 2008 Benchmark study of the same audience. The total sample size for the study would be 1,900, including: 500 Eligible Non-Users and 1400 Total Users (allocated as 500 Heavy Users of e-Services, 500 Medium-Level Users, and 400 Low-Level/Light Users. Based on the study conducted last year, reaching an ending sample of 500 Low-Level/Light Users is not possible due to the limited sample universe and the fact that Low-Level Users are less likely to visit the e-Services Registration page that will have a message alerting eServices users to the upcoming survey conducted by Russell Research. Note: as in the past, whenever the 3 User sub-groups are combined into “Total Users”, they must be weighted to reflect their real-world proportions – which must be calculated in the new study based on current segment sizes.
Data collection date
The 1,900 interviews will be conducted after OMB authorization, ideally in October through December, 2009 – prior to the January-April tax season.
Data to be collected
Data to be collected are detailed in the attached survey instrument/questionnaire.
How data will be used
Results will be analyzed by Users (including Heavy, Medium, and Light Users) and Non-Users.
Who is conducting research
The research is being conducted by the same vendor as in 2008 – Russell Research, Inc.
Location – region/city and facilities
The research will be conducted from the national telephone interviewing facilities of Russell Research in East Rutherford, New Jersey.
Stipend
The survey consists of approximately 15 minutes of interviewing. Therefore, no stipend is needed. (Note that the attached questionnaire appears to be longer than 15 minutes in length, but it is not because any one respondent answers only one of the detailed sections).
Recruitment efforts
All respondents will be recruited from lists to be provided by IRS. Russell will provide lookup of telephone numbers of potential respondents on the lists provided by IRS.
All interviews will be conducted by telephone from the Russell field center in New Jersey, using the IRS lists – with respondents selected for interviewing on an every “nth” basis. Telephone interviewing will be manually conducted in order to avoid an electronic trail of interviewing records, with all paper versions of the completed surveys destroyed after completion of the survey – this to comply with Safeguards agreements between Russell and the Government.
Methods to maximize response rate
The survey will employ the following methods to maximize response: 1) at least 3 and up to 5 interviewing attempts on each respondent; and 2) assigning refusals on initial attempts to senior interviewing personnel trained in refusal conversion.
Expected Response Rate
The response rate for the study is expected to be approximately 60% (similar to the prior survey).
Test structure/design
A total of approximately 1,900 interviews will be conducted, yielding readable sample sizes for 1,400 Total Users (including 500 Heavy Users, 500 Medium Users, and 400 Light Users) and for 500 Non-Users. The confidence intervals for each analytical group in the study are as follows:
For the total User sample of 1,400, the confidence interval will be +/- 2.6% at the 50% level and +/-1.6% at the 10% and 90% levels, using a 95% confidence level.
For the 500 sized sub-samples of Heavy and Medium Users and the Non-Users: +/- 4.4% at the 50% level and +/- 2.6% at the 10% and 90% levels, using a 95% confidence level.
And for the User sub-group of 400 Light Users: +/- 4.9% at the 50% level and +/- 2.9% at the 10% and 90% levels, using a 95% confidence level.
Efforts to not duplicate research
No other satisfaction surveys are currently being conducted for e-Services, thus there is no duplication with other research.
Participants Criteria
To qualify for this survey, respondents would need only to be included on IRS lists being drawn by the IRS in consultation with Russell.
Privacy, Security, Disclosure
The information in the IRS lists will be protected from disclosure by Russell’s strict non-disclosure safeguards (see below). Note: These lists will not contain tax return or taxpayer information. In addition, survey participants will not be identified in any of the documents or files used for this project. Nonetheless, as with all IRS studies, Russell will limit and control the amount of information collected to those items that are necessary to accomplish the research questions. Russell will carefully safeguard the security of data utilized as well as the privacy of the survey respondents (as detailed below). Russell will also apply fair information and record-keeping practices to ensure protection of all survey respondents. The criterion for disclosure laid out in the Privacy Act, the Freedom of Information Act, and section 6103 of the Internal Revenue Code provides for the protection of information as well as its releases to authorized recipients.
Russell’s Non-Disclosure Safeguards
Having undergone a formal security audit in the past associated with IRS research and aware of the penalties that apply to disclosure, Russell has a government-approved system in place for safeguarding IRS lists and assuring non-disclosure of any information from any IRS study. This system assures that Russell will comply with the following requirements:
All work shall be performed under the supervision of Russell’s responsible employees.
Any Taxpayer or Tax Professional information made available in IRS lists shall be used only for the purpose of carrying out the provisions of this contract. Information contained in such material shall be treated as Anonymous and shall not be divulged or made known in any manner to any person except as may be necessary in the performance of the contract. Russell understands that inspection by or disclosure to anyone other than an officer or employee of Russell requires prior written approval of the Internal Revenue Service – this would not be issue, as no others would have access the data.
C. Russell understands that should a person (contractor or subcontractor) or any employee make any unauthorized inspection(s) or disclosure(s) of Anonymous tax information, the terms of the Default clause (FAR 52.2499), incorporated herein by reference, may be invoked, and the person (contractor or subcontractor) will be considered to be in breach of this contract.
C1. Taxpayer and Tax Professional names and addresses will be accounted for upon receipt and properly stored before, during, and after processing. In addition, all related output shall be given the same level of protection as required for the source material. Russell agrees to abide by its contract requirements that:
The following safeguards will be provided by Russell for protecting the lists:
Names and addresses will be provided in and worked from hard copy only.
Since the files may have to be printed and divided into sub-samples for use by different researchers in our national telephone facility, the contractors would also impose strict hard copy controls (with responsible senior supervisors in control of the data at all times and dispensing data to interviewers on a sign-in, sign-out basis.
2. With scheduling issues, it is not possible to name (this far in advance of the interviewing) the interviewers who will be working on the project. The Supervisors names will also be provided from Russell’s national telephone center.
Russell’s Non-Disclosure Safeguards (Cont’d.)
3. The manager of the Russell facility will have control of one or more hard copy pieces of the sample.
4. The manager will personally supervise each interviewer’s use of the sample throughout the screening process, will not allow samples to leave the interviewing room, and will keep the samples locked in his/her personal files when not in use (for other security reasons).
5. The manager’s personal files in the telephone facility will be locked when not in use (for other security reasons).
6. The facility itself is always locked at the end of each interviewing day/evening.
7. Russell’s facility employs cameras throughout its interviewing facility to monitor the movements of interviewers, lists, etc.
8. After completion of all interviews and validation, the hard copy samples would be shredded by Russell (within approximately 8 weeks of the conclusion of the survey).
9. Russell will provide the names of the people that will have access to the hard copy data including the interviewers under the direct control and supervision of the manager at the telephone facility.
10. Russell certifies that the data processed during the performance of this contract shall be completely purged from all data storage components of our facility and that no output will be retained by the contractors at the time the IRS work is completed. In the case that immediate purging of all data storage is not possible, Russell certifies that any IRS data remaining in any storage component will be safeguarded to prevent unauthorized disclosures.
Any spoilage or any intermediate hard copy printout, which may result during the processing of IRS data, shall be given to the IRS Contracting Officer or his/her designee. When this is not possible, Russell will be responsible for the destruction of the spoilage or any intermediate hard copy printouts, and will provide the IRS Contracting Officer or his/her designee with a statement containing the date of destruction, description of material destroyed, and the method used.
As it has in the past, the Internal Revenue Service shall have the right to send its officers and employees into the offices and plants of Russell for inspection of the facilities and operations provided for the performance of any work under this contract. On the basis of such inspection, the Contracting Officer may require specific measures in any cases where Russell is found to be non-compliant with contract safeguards.
Burden Hours & Costs
While it is anticipated that all of the potential respondents contacted for this study will qualify, the response rate will likely be similar to that of the past – or about 60%. Assuming it is about the same, a total of 3,167 Tax Professionals will have to be called in order to reach the 1,900 who agree to participate in the study, leaving 1,267 not participating.
The contact time to determine non-participation could be up to 2 minutes, with the resulting burden for non-participants being 1,267 x 2 = 2,534 minutes / 60 minutes = 42.23 burden hours.
Then, for those Tax Professionals who do participate, the length of the survey is 15 minutes, with the time burden for that group being 1,900 x 15 minutes = 28,500 total minutes / 60 minutes = 475.0 burden hours.
Thus, the total burden hours for the study would be (42.23 + 475 = ) 517.23 hours.
The total cost of the 2009 e-Services Customer Satisfaction Survey is $95,900.
Attachments
Questionnaire – Attached
Not applicable here
Cover letter/correspondence
Not applicable here
Telephone screeners
Incorporated into the Questionnaire that’s attached.
Not applicable here
| File Type | application/msword |
| File Title | Word Document |
| Author | IRS-Russell Collaboration |
| Last Modified By | mdsloa00 |
| File Modified | 2009-09-11 |
| File Created | 2009-09-11 |