2009 Free File Cognitive & Behavioral Research –
A Product Attitude, Usage & Satisfaction Survey
Background/overview
Research is an integral part of the IRS e-file marketing cycle and monitoring e-product development. Research is performed at the beginning of the cycle for the purpose of identifying the customers and their likes and dislikes prior to developing a product. In addition, it is performed once the product is offered in order to determine its acceptance and how to improve it. Results from the research provide ETA and the other Business Operating Divisions (BODS) with usage volume, penetration rates, and taxpayer profiles as to who does and does not use each product. It also provides BODS with detailed information to guide the next season’s marketing campaign and/or and how to direct outreach and marketing efforts. Finally, it provides the feedback necessary to determine if the funding was spent wisely and achieved the expected results. Various internal and external marketing research activities are performed each year that provide the BODs with taxpayer profiling data in the form of demographics, attitudinal or psychographic information and customer satisfaction.
Objectives of data collection
The specific objectives of the proposed Free File Cognitive & Behavioral Research would be to determine, among Taxpayers using Free File in 2009 (divided by those using Traditional Free File and those using Free File Fillable Forms), how they were introduced to Free File (how they learned of it), their reasons for choosing this electronic product, how they used it, and how they perceived the product in terms of its ease of use, use of specific product features, and satisfaction with the usage experience. Another objective is to provide results that can be used to assist the IRS with making policy decisions related to expanding the use of e-file, thus the study must provide a level of detail needed to indicate appropriate targeted marketing strategies. Note: this is the fourth Free File Cognitive & Behavioral Survey, the first having been conducted during the same time period in 2006, the second in the same period in 2007, and the third in the same period of 2008. So, an additional objective of this study will be to see how Free File attitudes, usage, and satisfaction have changed from year to year.
Methodology
Sample design
The study will replicate in all respects the methodology of the past, but the sample size will differ: instead of allocating all 1,800 interviews to Free File Users (generally, as in the past), the 1,800 interviews will be distributed this year as 1,000 to Users of Traditional Free File and 800 to Users of Free File Fillable Forms – so that the IRS can analyze Total Free File Users (weighted to real-world proportions) as well as differences between Users of the Traditional product and Users of the new Fillable Forms product, all while staying within sample size and budget limits of the past.
Data collection date
The 1,800 interviews will be conducted in early-September through mid-October, 2009, or after OMB authorization.
Data to be collected
Data to be collected are detailed in the attached survey instrument/questionnaire.
How data will be used
Results from the Total Sample of 1,800 will be used to analyze Total Free File Users and each of the two main User groups (Users of Traditional Free File vs. Users of Free File Fillable Forms).
Who is conducting research
The research is being conducted by the same vendor as in 2006—2008 – Russell Research, Inc.
Location – region/city and facilities
The research will be conducted from the national telephone interviewing facilities of Russell Research in East Rutherford (the Meadowlands), New Jersey.
Stipend
The survey consists of 15 minutes of interviewing. Therefore, no stipend is needed.
Recruitment efforts
All respondents will be recruited from lists to be provided by IRS of random samples of each User group, drawn from the total base of Free File Users in 2009 (i.e., IRS and Russell will draw an nth sample from the total database, then use this sub-sample as the basis for the random sample). Russell will provide lookup of telephone numbers for Taxpayers on the list.
All interviews will be conducted by telephone from the Russell-Meadowlands field center in New Jersey, using the above lists – with respondents selected for interviewing on an every “nth” basis from each list. Telephone interviewing will be manually conducted in order to avoid having any electronic trail of interviewing records, with all paper versions of the completed surveys destroyed after completion of the survey – this to comply with the Safeguards agreement between Russell and the Government.
Methods to maximize response rate
The survey will employ the following methods to maximize response: 1) at least 3 and up to 5 interviewing attempts on each respondent; and 2) assigning refusals on initial attempts to senior interviewing personnel trained in refusal conversion.
Expected Response Rate
The response rate is expected to be about 60% (or similar to the levels of the past 3 years).
Test structure/design
A total of 1,800 interviews will be conducted, yielding analysis of 1,800 Total Free File Users (weighted to real-world proportions), 1,000 Users of Traditional Free File, and 800 Users of Free File Fillable Forms. The confidence intervals for the total and sub-samples are as follows:
For the Total Sample of 1,800, the confidence interval will be +/- 2.3% at the 50% level and +/-1.4% at the 10% and 90% levels, using a 95% confidence level.
The confidence interval for the sub-sample of 1,000 Traditional Free File Users would be +/- 3.1% at the 50% level and +/- 1.9% at the 10% and 90% levels, using a 95% confidence level.
The confidence interval for the sub-sample of 800 Free File Fillable Forms Users would be +/- 3.5% at the 50% level and +/- 2.1% at the 10% and 90% levels, using a 95% confidence level.
Efforts to not duplicate research
There is the only Free File cognitive/behavioral survey being conducted, thus there is no duplication.
Participants Criteria
To qualify for this survey, respondents would need only to be Taxpayers on the list of Free File Users drawn by the IRS and Russell.
Privacy, Security, and Disclosure
The information in the IRS lists will be protected from disclosure by Russell’s strict non-disclosure safeguards (see below). Note: These lists will not contain tax return or taxpayer information. In addition, survey participants will not be identified in any of the documents or files used for this project. Nonetheless, as with all IRS studies, Russell will limit and control the amount of information collected to those items that are necessary to accomplish the research questions. Russell will carefully safeguard the security of data utilized as well as the privacy of the survey respondents (as detailed below). Russell will also apply fair information and record-keeping practices to ensure protection of all survey respondents. The criterion for disclosure laid out in the Privacy Act, the Freedom of Information Act, and section 6103 of the Internal Revenue Code provides for the protection of information as well as its releases to authorized recipients.
Russell’s Non-Disclosure Safeguards
Having undergone a formal security audit in the past associated with IRS research and aware of the penalties that apply to disclosure, Russell has a government-approved system in place for safeguarding IRS lists and assuring non-disclosure of any information from any IRS study. This system assures that Russell will comply with the following requirements:
All work shall be performed under the supervision of Russell’s responsible employees.
Any taxpayer or return information made available shall be used only for the purpose of carrying out the provisions of this contract. Information contained in such material shall be treated as confidential and shall not be divulged or made known in any manner to any person except as may be necessary in the performance of the contract. Russell understands that inspection by or disclosure to anyone other than an officer or employee of Russell requires prior written approval of the Internal Revenue Service – this would not be issue, as no others would have access the data.
C. Russell understands that should a person (contractor or subcontractor) or any employee make any unauthorized inspection(s) or disclosure(s) of confidential tax information, the terms of the Default clause (FAR 52.2499), incorporated herein by reference, may be invoked, and the person (contractor or subcontractor) will be considered to be in breach of this contract.
C1. Taxpayer and return preparer names and addresses will be accounted for upon receipt and properly stored before, during, and after processing. In addition, all related output shall be given the same level of protection as required for the source material. Russell agrees to abide by the Statement of Work’s requirements:
The following safeguards will be provided by Russell for protecting the lists:
Names and addresses will be provided in and worked from hard copy only.
Since the files may have to be printed and divided into sub-samples for use by different researchers in our New Jersey national telephone facility, the contractors would also impose strict hard copy controls (with responsible senior supervisors in control of the data at all times and dispensing data to interviewers on a sign-in, sign-out basis.
2. With scheduling issues, it is not possible to name (this far in advance of the interviewing) the interviewers who will be working on the project. The Supervisors names will also be provided from Russell’s national telephone center.
Russell’s Non-Disclosure Safeguards (Cont’d.)
3. The manager of the Russell facility will have control of one or more hard copy pieces of the sample.
4. The manager will personally supervise each interviewer’s use of the sample throughout the screening process, will not allow samples to leave the interviewing room, and will keep the samples locked in his/her personal files when not in use (for other security reasons).
5. The manager’s personal office in the telephone facility will be locked when not in use (for other security reasons).
6. The facility itself is always locked at the end of each interviewing day/evening.
7. Russell’s facility employs cameras throughout its interviewing facility to monitor the movements of interviewers, lists, etc.
8. After completion of all interviews and validation, the hard copy samples would be shredded by Russell (within approximately 8 weeks of the conclusion of the survey).
9. Russell will provide the names of the people that will have access to the hard copy data including the interviewers under the direct control and supervision of the manager at the telephone facility.
10. Russell certifies that the data processed during the performance of this contract shall be completely purged from all data storage components of our facility and that no output will be retained by the contractors at the time the IRS work is completed. In the case that immediate purging of all data storage is not possible, Russell certifies that any IRS data remaining in any storage component will be safeguarded to prevent unauthorized disclosures.
Any spoilage or any intermediate hard copy printout, which may result during the processing of IRS data, shall be given to the IRS Contracting Officer or his/her designee. When this is not possible, Russell will be responsible for the destruction of the spoilage or any intermediate hard copy printouts, and will provide the IRS Contracting Officer or his/her designee with a statement containing the date of destruction, description of material destroyed, and the method used.
As it has in the past, the Internal Revenue Service shall have the right to send its officers and employees into the offices and plants of Russell for inspection of the facilities and operations provided for the performance of any work under this contract. On the basis of such inspection, the Contracting Officer may require specific measures in any cases where Russell is found to be non-compliant with contract safeguards.
Burden Hours
While it is anticipated that all of the Taxpayers contacted for this study will qualify, the response rate will likely be similar to that of the past – or in the range of 58.8% to 60.4%. Assuming it is the same and using the high end of the range, a total of 2,980 Taxpayers will have to be called in order to reach the 1,800 who agree to participate in the study, leaving 1,180 not participating.
The contact time to determine non-participation could be up to 1 minute, with the resulting burden for non-participants being 1,180 x 1 = 1,180 minutes / 60 minutes = 19.67 burden hours.
Then, those Taxpayers who do participate, the length of the survey is 15 minutes, with the time burden for that group being 1,800 x 15 minutes = 27,000 total minutes / 60 minutes = 450.0 burden hours.
Thus, the total burden hours for the study would be (19.67 + 450 = ) 469.67 hours.
The total cost of the survey is $94,417.
Attachments
Questionnaire – Attached
Not applicable here
Cover letter/correspondence
Not applicable here
Telephone screeners
Incorporated into the Questionnaire that’s attached.
Not applicable here
| File Type | application/msword |
| File Title | Word Document |
| Author | IRS-Russell Collaboration |
| Last Modified By | mdsloa00 |
| File Modified | 2009-08-04 |
| File Created | 2009-08-04 |