OMB Clearance Package
Focus Group Testing of IRS.gov
Introduction
The Tax Forms and Publications (TF&P) Office of the Internal Revenue Service (IRS), Wage and Investment division, creates tax forms, instructions, and related publications to enable taxpayers to understand and fulfill their filing and reporting obligations under the tax laws of the United States. This information is posted on the Internal Revenue Service website, IRS.gov to assist with the demand for timely, accurate service, and reducing taxpayer burden. TFP has specific responsibility for certain sections of the IRS.gov site, and works with TF&P’s parent organization, Media & Publications (M&P), and the Public Portal Branch to maintain the forms and publications section of IRS.gov.
Background
In order to address the increasing demand for electronic products and services, TF&P seeks reaction and suggestions for improvement from individual and business taxpayers, software developers, tax professionals, and other users of IRS.gov. TF&P would like to use focus groups to determine the usability of content on the forms and publication pages, and receive specific suggestions for improvements to specific pages. TF&P would like to use these suggestions to improve the web usability, content, and services it provides.
Objectives of Research
The objective of these focus groups is to gather input from end users—taxpayers, software developers, and tax professionals—to determine their reaction to specific web content. The specific business questions to be explored by this project are to:
Determine participants’ reactions to specific web pages on IRS.gov
Obtain information about the usability of content on the forms and publications pages of IRS.gov
Discover what content leads to misunderstanding and errors
Determine what content these specific populations find most useful
Obtain suggestions for improvement to specific IRS.gov web pages
This information will assist the division as they update specific web pages in the forms and publication section of IRS.gov. To achieve these objectives, the division plans to conduct a total of 9 focus groups. One group with software developers will be completed via an online focus group. This option will enable TF&P to obtain feedback from software developers located across the country. Eight traditional focus group (2 per city) will be conducted in 4 separate cities. Each focus group will consist of at least 8 participants, with 10 being the ideal number of participants. Upon arrival, participants will be re-screened to confirm their eligibility.
TF&P also expects to gain knowledge of participants’ reactions to newly updated web pages and discover what content leads to error or misunderstanding, and determine what content is most useful to each of the target populations. TF&P also expects that participants will provide specific suggestions for improvement to specific web pages. Upon completion of this project, Macro International Inc.’s (Macro’s) moderator will submit a report of findings to TF&P to assist in determining any changes or modifications that should be made to these web pages.
Methodology
Sample design
The population for this study consists of software developers, tax professionals, and taxpayers who are fluent in English and have accessed www.irs.gov to locate information about tax forms, instructions, or publications in the past 12 months. The maximum number of participants per focus group will be 10. A screener guide has been prepared for professional recruiters to use to recruit participants for each of the traditional focus groups (see Attachment 1). TF&P will recruit software developers to participate in the online focus group.
The in-person focus groups will be held in geographically dispersed cities in the country, with two groups in each city. The cities are Philadelphia, PA; Kansas City, MO; Phoenix, AZ; and Portland, OR.
Data collection date
The online group will be conducted February-April 2009, and the in-person focus groups in February and March 2009.
Data to be collected
Data gathered will be software developers’ taxpayers’ and tax professionals’ reactions to specific web pages in the forms and publications area of IRS.gov. Data will be gathered on the following:
Ease of use
Readability
Overall content
Usefulness
Clarity of language
Layout and organization
Navigation
Ease of locating information.
How the data will be used
The goal of a focus group is to solicit general opinions about a topic. An important aspect of focus groups is that they give rise to insights and solutions that may not be derived from other methods. Focus group interviews are qualitative research, meaning no statistical difference between groups can be determined. Specifically, they are a directed discussion with small groups of 8 to 12 people on a specific topic.
Experienced moderators from Macro will conduct the focus groups using a moderator guide that has been developed (see Attachment 2).
Audiotapes and scribes’ notes will be used to assist with feedback documentation during the focus groups.
Who is conducting the research?
Macro will conduct the research for TF&P.
Location—region/city and facilities
Macro will conduct 9 focus groups in the following cities:
The online group will be hosted from the contractor’s (Macro) office in Calverton, MD
Philadelphia, PA
Kansas City, MO
Phoenix, AZ
Portland, OR.
Facilities to conduct the in-person focus groups will be provided by the vendor.
Stipend
The purpose of the stipend is to encourage participation, and to thank participants for sharing their time and contributions to the discussion. The amount of the stipend varies by location of the focus group and target audience(s). The stipend varies depending on the location of the focus group and the target audience. Software developers will receive $150 for their participation. Taxpayers in Kansas City will receive $75 and taxpayers in Philadelphia, Phoenix, and Portland will each receive $100 for a 2-hour session. Tax professionals in Kansas City and Portland will receive $100 for a 2 hour session. Tax professionals in Philadelphia and Phoenix will each received $125 for a 2 hour session. These stipends are typical for the locations of the focus groups, type of individuals recruited, and are in line with the industry standard.
Recruitment efforts
Using the screener guide, a vendor in each city will recruit 10 participants for the focus groups (see Attachment 1).
Efforts to not duplicate research
Direct interaction with taxpayers through focus groups has been a research design used by the IRS for a number of years. The IRS has not tested the redesign of forms and publications web pages on IRS.gov and therefore there is no duplication of research.
Test structure/design
In-person focus groups
In order to view and comment on different web pages, each focus group participant will have a computer with access to the Internet. The moderator, using a computer and LCD projector, will project the web pages on a large screen in the focus group room. After some general introductory questions, participants will review and comment on the forms and publications homepage of IRS.gov. From this page, participants will be directed to review other subpages and asked to comment on these pages as well.
Online focus group
Using Go-To-Meeting (GTM) software, participants will be instructed to logon 15 minutes prior to the focus group to view a PowerPoint presentation that contains instructions and introductory materials to be shown running on their screens as participants gather. It will include illustrated instructions on using the GTM interface and save the time otherwise needed for answering questions about it. Participants will be provided a telephone number to call if they need any technical assistance. A trained online focus group moderator will direct participants to specific web pages and lead the discussion in the same manner that the in-person groups are conducted.
Each focus group will have 8 to 10 participants. Research has demonstrated that this number of participants helps create the best group dynamics: the groups are sufficiently large to stimulate relatively spontaneous interaction, but small enough to give everyone sufficient opportunity to speak.
The screener guide will be used to qualify and select focus group participants. Macro will gather qualitative data only, which will not be, nor presented to be, representative of the population.
To participate in the study, participants must have the following characteristics:
Fluent in reading, writing, and speaking English
In the last 12 months, accessed IRS.gov to locate information about tax forms, instructions, and/or publications
At least 18 years old
Diverse in terms of age, gender, race/ethnicity, and education.
For tax professionals only –prepared at least 25 Federal tax returns each year
For software developers only—persons who develop tax products.
The data returned to IRS TF&P will have no identifying information relating specific records to individual taxpayers. Nonetheless, TF&P will ensure that privacy and security of the aggregated results will receive the utmost attention. Public and official access to the information will be tightly controlled. The computer files containing this tabulated information will remain password protected at all times. Data security approaching level C-2 will be accomplished using the Windows XP operating system.
Audiotapes used to record testing sessions will be destroyed when the project is completed and there is no further need for the data.
We will apply fair information and record-keeping practices to ensure protection of all taxpayers. The criterion for disclosure laid out in the Privacy Act, the Freedom of Information Act, and section 6103 of the Internal Revenue Code, provides for the protection of taxpayer information as well as its release to authorized recipients.
The estimated time to complete the participant screening is 5 minutes. The estimated time for each reminder call is 1 minute, with each testing session lasting 2 hours. We estimate that 10 percent of the taxpayers contacted for the study will qualify and be willing to participate. Using this percentage, a total of 800 individuals will need to be screened to recruit the needed 8 participants (8 study participants and 2 back-up participants) for each focus group. We estimate that 16 people will be sent home because we will only need 64 to stay for the focus groups. Of those being sent home, we estimate 30 minutes of travel time for in-person focus group participants.
Total number of potential participants screened 800
Estimated time to complete screening 5 minutes
Reminder phone calls 80 minutes
(80 x 1 minute/call)
Estimated participant screening burden 68 hours
(800 potential recruits x 5 + 80 [participants] minutes)
Number of participants 64
(64)
Time to conduct study
(including 30 minutes for travel) 2.5 hours
Travel time 8 hours
(16 x .5 hours)
Estimated study burden (in-person groups) 168 hours (64 x 2.5 + 8)
Estimated total project burden hours 236 hours
The National Association of Computerized Tax Processors (NACTP) will recruit software developers for this focus group. The estimated time to complete the participant screening is 5 minutes. We estimate that reminder e-mails will take 1 minute each, with each testing session lasting 2 hours. We estimate that 10 percent of the taxpayers contacted for the study will qualify and be willing to participate. Using this percentage, a total of 100 individuals will need to be screened to recruit the needed 8 participants. Please note that all 10 individuals “show,” all will be asked to participate in the online focus group. Travel time is not built into the estimate, as these individuals will participate from their offices or homes.
Total number of participants screened 100
Estimated time to complete screening 3 minutes
Reminder e-mail 5 minutes
Estimated participant screening burden 5.8 hours
(100 potential recruits x 3 + 10 [participants] x 5 minutes)
Number of participants 8
Time to conduct study 2 hours
Estimated study burden (in-person groups) 16 hours (8 x 2)
Estimated total project burden hours 21.8 hours
Estimated total project burden hours (9 focus groups) 257.8 hours
Macro will manage every aspect of recruitment. They will use professional recruitment firms in Philadelphia, PA; Kansas City, MO; Phoenix, AZ; and Portland, OR. Focus groups will be held in the respective facilities in each city. The IRS, with the assistance of NACTP, will recruit software developers for the online focus group. Using the screener guides Macro provides, each firm will recruit 10 participants for each focus group with a goal of having 8 participants for each group.
For the in-person focus groups, Macro will ensure that:
Each participant receives directions to the focus group room.
Each participant is re-screened upon arrival at the facility to confirm eligibility.
Each participant signs an informed consent before participating in the focus group (see Attachment 3)
All materials are ready for use in the focus group session, including tables and chairs, computer with Internet access, colored markers, note pads, pencils, and audiotapes.
Complimentary refreshments appropriate to the time of day are available for the focus group participants.
Audio equipment is operating properly.
Honoraria to participants are distributed upon the completion of the session.
A technical person is available onsite should any computer-related issues emerge.
Last names or other identifying information are NOT available to IRS personnel.
Individual computer equipment is available for each participant at each focus group location.
For the online focus group, Macro will ensure that:
Each participant is able to participate in the GTM online group and can access a phone conference call feature simultaneously.
Each participant signs and submits electronically the informed consent form (see Attachment 3).
Each participant receives, completes, and submits a W-9 so that they can receive their stipend.
Participants are provided with a number to call should they experience any problems logging on to GTM or at any time during the online session.
GTM audio is operating properly.
Two technical staff members will participate to handle any technical problems.
Participant will receive by mail a honoraria for their participation.
The total estimated cost associated with this study is $101,434.
Within 60 days after the completion of the focus groups, Macro will provide the following information:
Findings
Actions taken or lessons learned
Number of requests or attempts for taxpayer participation
Number of focus group participants
Date the data collection began
Date the data collection ended
Actual burden hours
Cost
Transcripts and audiotapes.
Attachment 1 Screener Guide, Confirmation Letter, and Reminder Phone Script
Attachment 2 Moderator’s Guide
Attachment 3 Informed Consent
| File Type | application/msword |
| File Title | TELEPHONE SCREENER |
| Author | Maria Alvarado |
| Last Modified By | qhrfb |
| File Modified | 2009-02-03 |
| File Created | 2009-01-27 |