OMB Clearance Package
Protocol Testing of IRS Form 1040X
Introduction
The Tax Forms and Publications (TF&P) Office of the Internal Revenue Service (IRS), Wage and Investment division, creates tax forms, instructions, and related publications to enable taxpayers to understand and fulfill their filing and reporting obligations under the tax laws of the United States. Many of the provisions of the tax laws are complex. The forms, instructions, and publications developed by TF&P necessarily reflect this complexity and may be lengthy, complex, and difficult for taxpayers to understand.
TF&P has used focus groups, protocol testing, laboratory testing, and other methods in the past to determine the usability by taxpayers of various forms, instructions, and publications. Testing has resulted in the adoption of various specific suggestions for improvement to specific products. TF&P requires the continued use of such tools to improve the materials it produces.
TF&P plans to perform protocol testing. A protocol test is a one-on-one test between a moderator and a taxpayer. It is one of the most useful ways to determine the comprehension and process a taxpayer uses to complete a worksheet and respond to a notice.
This type of test allows the Service to gain information regarding taxpayers’ ability to understand forms and instructions. Sessions with taxpayers will examine which portions of the form and instructions that may lead to misunderstanding and errors, and determine which sections are most useful or need improvement. TF&P expects that taxpayers will provide specific suggestions to improve these products.
Objectives of Research
The objective of the protocol testing is to gather input from taxpayers on the revised Form 1040X to ensure that the form and instructions are clear and user friendly. Feedback from the participants will be used to improve this form.
The specific business questions to be explored by this project are—
Participants’ overall perceptions of Form 1040X and it’s accompanying instructions.
Common errors participants make when completing Form 1040X.
Parts of the instructions and form that participants find most and least helpful.
Suggestions participants have for improving Form 1040X and instructions.
This information will assist the division in developing clear tax products that assist taxpayers in voluntarily meeting their tax obligations. To achieve these objectives, the service has developed test scenarios and plans to conduct protocol tests with in 4 separate. Each protocol test will consist of 7 participants. Upon arrival, participants will be re-screened to confirm their eligibility.
TF&P also expects to gain knowledge of participants’ overall reaction to Form 1040X, discover what parts of the documents lead to error or misunderstanding, and to determine what in the Form taxpayers find most useful. TF&P also expects that taxpayers will provide specific suggestions for improvement to Form 1040X and its instructions.
Methodology
Sample design
Protocol Test
The population for the protocol test is taxpayers who have not filed a Form 1040X and are familiar with or prepare their own tax return. All participants must also have completed their own tax return last year. Although taxpayers that are paper filers are most desirable, the 100% percent self-prepared recruitment requirement can be met with a mixture of both paper and software self-preparers. The maximum number of participants per protocol test will be 7. Screener guides have been prepared for professional recruiters to use to recruit participants for each of the protocol tests. (See Attachment 1)
The protocol tests will be held in cities with high concentrations of Form 1040X filers. The cities are Chicago, Denver, Austin, Texas and Virginia Beach, Virginia.
Data collection date
Data will be collected in May 2009.
Data to be collected
Data gathered will be taxpayers’ reactions to Form 1040X. Data will be gathered on the following:
Ease of use
Readability
Overall content
Usefulness
Clarity of language
Layout and organization
Ease of navigation
Type size and font
Order of presentation
Ease of locating information.
How the data will be used
The goal of protocol testing is to solicit general opinions about a topic. An important aspect of protocol tests is that they give rise to insights and solutions that may not be derived from other methods.
Experienced moderators from Kleimann Communications Group (KCG) will conduct the protocol tests using the moderator guides developed (see Attachment 2).
Audiotapes and scribes’ notes will be used to assist with feedback documentation during the protocol tests.
For the protocol testing, the following data will be collected:
Behavioral Data
Completion of Form1040X
Observations of participant during form completion
Comprehension of the Form 1040X
Correctness of completing Form 1040X
Debrief Data - Qualitative Responses
Processing questions regarding Form1040X calculations
Process used to complete Form 1040X
Use and understanding of information contained in the Form 1040X instruction booklet
Improvement recommendations for form and instructions
Ratings of difficulty
Location—region/city and facilities
KCG will conduct and 4 protocol test (7 per city) in the following cities:
Austin, TX
Chicago, IL
Denver, CO
Virginia Beach, VA.
Facilities to conduct the protocol tests will be provided by the vendor.
Stipend
The stipend will be $100 to $150 per participant for a 2-hour session. The stipend is a reflection of the length of time required for participation in this study.
Recruitment efforts
Using the screener guide, a vendor will recruit participants for Form 1040X.
Efforts to not duplicate research
Direct interaction with taxpayers through protocol tests has been a research design used by the IRS for a number of years. The IRS has not tested the revised Form 1040X in the past.
Test structure/design
The protocol test will utilize an experimental design in which participants will complete the following tasks. Participants will complete Form 1040X, using information contained in a scenario and supporting documents provided to them. Participants will be asked questions specific to the completion of Form 1040X. These include: the process used to complete these form, sections of the instruction booklet used, and information they felt could have helped.
The screener guides will be used to qualify and select focus protocol test participants. KCG will gather qualitative data only, which will not be, nor presented to be, representative of the population.
To participate in the study, participants must have the following characteristics:
Protocol Test interviewees will not have completed Form 1040X
All participants will have self-prepared individual tax returns
50% discovered errors on their own
50% informed of errors
Diverse in terms of age, gender, race/ethnicity and education
Diverse filing statuses
The data returned to IRS TF&P will have no identifying information relating specific records to individual taxpayers. Nonetheless, TF&P will ensure that privacy and security of the aggregated results will receive the utmost attention. Public and official access to the information will be tightly controlled. The computer files containing this tabulated information will remain password protected at all times. Data security approaching level C-2 will be accomplished using the Windows XP operating system.
Audiotapes used to record testing sessions will be destroyed when the project is completed and there is no further need for the data.
We will apply fair information and record-keeping practices to ensure protection of all taxpayers. The criterion for disclosure laid out in the Privacy Act, the Freedom of Information Act, and section 6103 of the Internal Revenue Code, provides for the protection of taxpayer information as well as its release to authorized recipients.
The estimated time to complete the participant screening is 5 minutes. The estimated time for each reminder call is 1 minute, with each testing session lasting 2 hours. We estimate that 10 percent of the taxpayers contacted for the study will qualify and be willing to participate. Using this percentage, a total of 300 individuals will need to be screened to recruit the needed 28. We estimate that 4 people will be sent home because we will only need a total of 28 for protocol testing.
Total number of potential participants screened 300
Estimated time to complete screening (5 minutes/call) 1500 minutes
Reminder phone calls (1 minute/call) 32 minutes
Estimated participant screening burden 25.5 hours
(300 x 5 + 32 minutes (28 participants plus 4 back-ups)
Number of participants 28
Time to conduct study 2 hours (2x 28) 56 hours
Travel time 32 hours
(32 x 1 hour)
Estimated study burden 88 hours (56 + 16)
Estimated total project burden hours 113.5 hours
KCG will manage every aspect of recruitment. We will use professional recruitment firms in Austin, TX; Chicago, lL; Denver, CO; and Virginia Beach, VA. All protocol testing will be held in their respective facilities. Using the screener guides KCG provides, KCG will recruit 10 participants for the protocol testing with a goal of 7 for each protocol test.
Each participant receives directions to the testing facilities.
Each participant is re-screened upon arrival at the facility to confirm eligibility.
All materials are ready for use in the protocol sessions, including tables and chairs, newsprint, colored markers, note pads, calculators, pencils, and audiotapes.
Complimentary refreshments appropriate to the time of day are available for the participants.
Audio equipment is operating properly.
Honoraria to participants are distributed upon the completion of the session.
Last names or other identifying information are NOT available to IRS personnel.
The total estimated cost associated with this study is $120,000.
Within 60 days after the completion of the protocol testing, KCG will provide the following information:
Findings
Actions taken or lessons learned
Number of requests or attempts for taxpayer participation
Number of protocol test participants
Date the data collection began
Date the data collection ended
Actual burden hours
Cost
Transcripts and audiotapes.
Attachment 1 Screener Guide and Reminder Call Script – Protocol Testing
Attachment 2 Moderator’s Guide Protocol Testing
Attachment 3 Informed Consent
| File Type | application/msword |
| File Title | TELEPHONE SCREENER |
| Author | Maria Alvarado |
| Last Modified By | qhrfb |
| File Modified | 2009-05-12 |
| File Created | 2009-04-02 |