OMB Clearance Package
Focus Group and Protocol Testing of IRS Form 1040X
Introduction
The Tax Forms and Publications (TF&P) Office of the Internal Revenue Service (IRS), Wage and Investment division, creates tax forms, instructions, and related publications to enable taxpayers to understand and fulfill their filing and reporting obligations under the tax laws of the United States. Many of the provisions of the tax laws are complex. The forms, instructions, and publications developed by TF&P necessarily reflect this complexity and may be lengthy, complex, and difficult for taxpayers to understand.
TF&P has used focus groups, laboratory testing, and other methods in the past to determine the usability by taxpayers of various forms, instructions, and publications. Testing has resulted in the adoption of various specific suggestions for improvement to specific products. TF&P requires the continued use of such tools to improve the materials it produces.
In addition to focus group testing, TF&P plans to perform protocol testing. A protocol test is a one-on-one test between a moderator and a taxpayer. It is one of the most useful ways to determine the comprehension and process a taxpayer uses to complete a worksheet and respond to a notice.
This type of test allows the Service to gain information regarding taxpayers’ ability to understand forms and instructions. Sessions with taxpayers will examine which portions of the form and instructions that may lead to misunderstanding and errors, and determine which sections are most useful or need improvement. TF&P expects that taxpayers will provide specific suggestions to improve these products.
Objectives of Research
The objective of the focus groups is to gather input from taxpayers on Form 1040X to ensure that the form and instructions are clear and user friendly. Feedback from the focus group participants will be used to improve this form.
The specific business questions to be explored by this project are—
Participants’ overall perceptions of Form 1040X and it’s accompanying instructions.
Common errors participants make when completing Form 1040X.
Parts of the instructions and form that participants find most and least helpful.
Suggestions participants have for improving Form 1040X and instructions.
This information will assist the division in developing clear tax products that assist taxpayers in voluntarily meeting their tax obligations. To achieve these objectives, the service has developed test scenarios and plans to conduct a total of 8 focus groups, (2 per city) in 4 separate cities and one protocol test in each city. Each focus group will consist of at least 8 participants, with 10 being the ideal number of participants. Each protocol test will consist of 5 participants. Upon arrival, participants will be re-screened to confirm their eligibility.
TF&P also expects to gain knowledge of participants’ overall reaction to Form 1040X, discover what parts of the documents lead to error or misunderstanding, and to determine what in the Form taxpayers find most useful. TF&P also expects that taxpayers will provide specific suggestions for improvement to Form 1040X and its instructions.
Methodology
Sample design
Focus Groups
The population for this study consists of taxpayers who file Form 1040X and are familiar with or prepare their own tax return. All participants must also have completed their own tax return last year. Although taxpayers that are paper filers are most desirable, the 100% percent self-prepared recruitment requirement can be met with a mixture of both paper and software self-preparers. The maximum number of participants per focus group will be 10. Screener guides have been prepared for professional recruiters to use to recruit participants for each of the focus groups (see Attachment 1).
Protocol Test
The population for the protocol test is taxpayers who have not filed a Form 1040X and are familiar with or prepare their own tax return. All participants must also have completed their own tax return last year. Although taxpayers that are paper filers are most desirable, the 100% percent self-prepared recruitment requirement can be met with a mixture of both paper and software self-preparers. The maximum number of participants per protocol test will be 5. Screener guides have been prepared for professional recruiters to use to recruit participants for each of the protocol tests. (See Attachment 2)
The focus groups and protocol tests will be held in cities with high concentrations of Form 1040X filers. The cities are Chicago, Denver, El Paso, Texas and Virginia Beach, Virginia.
Data collection date
Data will be collected in January and February 2009.
Data to be collected
Data gathered will be taxpayers’ reactions to Form 1040X. Data will be gathered on the following:
Ease of use
Readability
Overall content
Usefulness
Clarity of language
Layout and organization
Ease of navigation
Type size and font
Order of presentation
Ease of locating information.
How the data will be used
The goal of a focus group is to solicit general opinions about a topic. An important aspect of focus groups is that they give rise to insights and solutions that may not be derived from other methods. Focus group interviews are qualitative research, meaning no statistical difference between groups can be determined. Specifically, they are a directed discussion with small groups of 6 to 12 people on a specific topic.
Experienced moderators from Kleimann Communications Group (KCG) will conduct the focus groups using the moderator guides developed (see Attachment 2).
Audiotapes and scribes’ notes will be used to assist with feedback documentation during the focus groups.
For the protocol testing, the following data will be collected:
Behavioral Data
Completion of Form1040X
Observations of participant during form completion
Comprehension of the Form 1040X
Correctness of completing Form 1040X
Debrief Data - Qualitative Responses
Processing questions regarding Form1040X calculations
Process used to complete Form 1040X
Use and understanding of information contained in the Form 1040X instruction booklet
Improvement recommendations for form and instructions
Ratings of difficulty
Who is conducting the research?
KCG will conduct the research for TF&P.
Location—region/city and facilities
KCG will conduct 8 focus groups (two per city) and 4 protocol test (1 per city) in the following cities:
Chicago, IL
Denver, CO
El Paso, TX
Virginia Beach, VA.
Facilities to conduct the focus groups will be provided by the vendor.
Stipend
The stipend will be $100 to $150 per participant for a 2-hour session. The stipend is a reflection of the length of time required for participation in this study.
Recruitment efforts
Using the screener guide, a vendor will recruit participants for Form 1040X.
Efforts to not duplicate research
Direct interaction with taxpayers through focus groups has been a research design used by the IRS for a number of years. The IRS has not tested the Form 1040X in the past,
Test structure/design
Participants will look at the current Form 1040X, including the layout and the instructions. Participants will be asked questions specific to the content, layout, and appearance. Following this portion of the testing session, participants will be presented with one or two scenarios and will fill out Form 1040X using these scenarios1. Participants will be asked a series of questions on how useful the Forms and instructions were in completing the scenarios.
The protocol test will utilize an experimental design in which participants will complete the following tasks. Participants will complete Form 1040X, using information contained in a scenario and supporting documents provided to them. Participants will be asked questions specific to the completion of Form 1040X. These include: the process used to complete these form, sections of the instruction booklet used, and information they felt could have helped.
Each focus group will have 10 participants. Research has demonstrated that this number of participants helps create the best group dynamics: the groups are sufficiently large to stimulate relatively spontaneous interaction, but small enough to give everyone sufficient opportunity to speak.
The screener guides will be used to qualify and select focus group participants and protocol test participants. KCG will gather qualitative data only, which will not be, nor presented to be, representative of the population.
To participate in the study, participants must have the following characteristics:
Focus Group participants will have completed Form 1040X
Protocol Test interviewees will not have completed Form 1040X
All participants will have self-prepared individual tax returns
50% discovered errors on their own
50% informed of errors
Diverse in terms of age, gender, race/ethnicity and education
Diverse filing statuses
The data returned to IRS TF&P will have no identifying information relating specific records to individual taxpayers. Nonetheless, TF&P will ensure that privacy and security of the aggregated results will receive the utmost attention. Public and official access to the information will be tightly controlled. The computer files containing this tabulated information will remain password protected at all times. Data security approaching level C-2 will be accomplished using the Windows XP operating system.
Audiotapes used to record testing sessions will be destroyed when the project is completed and there is no further need for the data.
We will apply fair information and record-keeping practices to ensure protection of all taxpayers. The criterion for disclosure laid out in the Privacy Act, the Freedom of Information Act, and section 6103 of the Internal Revenue Code, provides for the protection of taxpayer information as well as its release to authorized recipients.
The estimated time to complete the participant screening is 5 minutes. The estimated time for each reminder call is 1 minute, with each testing session lasting 2 hours. We estimate that 10 percent of the taxpayers contacted for the study will qualify and be willing to participate. Using this percentage, a total of 1,240 individuals will need to be screened to recruit the needed 124 participants (12 participants per focus group and 7 participants for the protocol testing). We estimate that 24 people will be sent home because we will only need a total of 100 to stay for the focus groups and protocol testing. Of those being sent home, we estimate 30 minutes of travel time.
Total number of potential participants screened 1,240
Estimated time to complete screening (5 minutes/call) 6,200 minutes
Reminder phone calls (1 minute/call) 124 minutes
Estimated participant screening burden 105.4 hours (1240 x 5 + 124 minutes)
Number of participants 100
(10 x 8 focus groups + 5 x 4 protocol testing)
Time to conduct study 2.0 hours
Travel time 50 hours
(100 x .5 hours)
Estimated study burden 250 hours (100 x 2 + 50)
Estimated total project burden hours 355.4 hours
KCG will manage every aspect of recruitment. We will use professional recruitment firms in Chicago, lL; Denver, CO; El Paso, TX; and Virginia Beach, VA. All focus groups will be held in their respective facilities. Using the screener guides KCG provides, each firm will recruit 12 participants for each focus group with a goal of having 10 participants for each group. KCG will recruit 7 participants for the protocol testing with a goal of 5 for each protocol test.
Each participant receives directions to the testing facilities.
Each participant is re-screened upon arrival at the facility to confirm eligibility.
All materials are ready for use in the focus group and protocol session, including tables and chairs, newsprint, colored markers, note pads, calculators, pencils, and audiotapes.
Complimentary refreshments appropriate to the time of day are available for the participants.
Audio equipment is operating properly.
Honoraria to participants are distributed upon the completion of the session.
Last names or other identifying information are NOT available to IRS personnel.
The total estimated cost associated with this study is $182,288.
Within 60 days after the completion of the focus groups, KCG will provide the following information:
Findings
Actions taken or lessons learned
Number of requests or attempts for taxpayer participation
Number of focus group participants
Number of protocol test participants
Date the data collection began
Date the data collection ended
Actual burden hours
Cost
Transcripts and audiotapes.
Attachment 1 Screener Guide and Reminder Call Script – Focus Group
Attachment 1 Screener Guide and Reminder Call Script – Protocol Testing
Attachment 2 Moderator Guide Focus Group
Attachment 3 Moderator’s Guide Protocol Testing
Attachment 4 Informed Consent
1 Note: Focus group participants testing the revised 941X will not complete current Form 941c.
| File Type | application/msword |
| File Title | TELEPHONE SCREENER |
| Author | Maria Alvarado |
| Last Modified By | qhrfb |
| File Modified | 2008-12-08 |
| File Created | 2008-12-08 |