SOI-467 ACA Toll Free

Supporting Statement

Approval Request to Conduct Cognitive and
Psychological (or Customer Satisfaction) Research

(OMB #1545-1349)

Examining the Usability of the Affordable Care Act (ACA) Toll-Free Line with Focus Groups and Comprehension Testing

1. JUSTIFICATION

1. Circumstances Making the Collection of Information Necessary

As outlined in the IRS Strategic Plan under the goal of improving service to make voluntary compliance easier, it is noted that the IRS is dedicated to “taking proactive steps to better understand issues from the taxpayer’s perspective” through techniques such as focus groups and surveys.

Implementation of ACA is predicted to increase service inquiries for IRS’

three major service channels:

• IRS.gov;

• Call Center; and

• Field Assistance

In FY-2015, the IRS will need to support tax-related questions from individuals and businesses arising from:

• Open enrollment for 2015 coverage

• New Forms and Requirements including Premium Tax Credits exemptions

• Execute ACA related account handling and inquiries during the 2015
  Filing Season

• Address post-filing account issues

• Report volumes to support planning and oversight from internal and external stakeholders

2. Purpose and Use of the Information Collection

The findings from this research will allow the IRS to pre-emptively plan and allocate resources based on the taxpayer experiences rather than relying on lessons learned during implementation. This research will not only improve level of service, but also facilitate voluntary compliance and reduce taxpayer burden by incorporating taxpayer perspectives and improving all Service interactions through:

1. Ensuring operational decisions are made only after considering the views of affected taxpayers.

2. Using focus groups, comprehension testing, usability testing, and other research methods before rolling out new services and processes.

3. Implementing phased rollouts wherever possible, including beta-testing periods.

4. Facilitating voluntary compliance with the tax system for all segments of the public.

The project will conduct exploratory research to better understand the taxpayer experience of calling into the IRS while seeking information on the ACA. To achieve these objectives, WIRA plans to conduct a total of 12 focus groups, (three per city) in four separate cities. The goal of a focus group is to solicit general opinions about a topic. An important aspect of focus groups is that they give rise to insights and solutions that may not be derived from other methods. Specifically, they are directed discussion groups with approximately 10 people, with the discussion focused on a specific topic. Focus group interviews are qualitative research, meaning no statistical difference between groups can be determined. Each focus group will consist of at least 10 participants. In each session participants will complete the ACA knowledge pre-test, seek answers to task specific questions using the ACA toll-free lines and answer a series of comprehension questions. Lastly, the taxpayers will participate in a focus group to discuss recommendations for improvement as well as discuss the overall user experience. Upon arrival to the facility, participants will be re-screened to confirm their eligibility.

The data collected will be used to develop a better understanding of the taxpayer decision making process and usability levels when contacting the IRS about ACA related information. It will assist the IRS in developing an understanding of the service needs and preferences of these taxpayers.

3. Consideration Given to Information Technology

No online tools will be used for this study. Participants will be using a phone to call the ACA information lines.

4. Duplication of Information

Direct interaction with taxpayers through focus groups has been a research design used by the IRS for a number of years. Our research questions have not been studied previously and therefore there is no duplication of research.

5. Reducing the Burden on Small Entities

NA

6. Consequences of Not Conducting Collection

Without these types of feedback, the IRS will not have timely information to adjust its services to meet customer needs.

7. Special Circumstances

There are no special circumstances. The information collected will be voluntary and will not be used for statistical purposes.

8. Consultations with Persons Outside the SOI

Not applicable.

9. Payment or Gift

Participants will be paid $75.00 for their participation in this 90 minute study. The purpose of the stipend is to encourage participation, and to thank participants for sharing their time and contributions to the discussion. These stipends are typical for the locations of the groups, time spent in the session, the type of individuals recruited, and in line with the industry standard.

10. Confidentiality

No PII will be collected during the focus group. Moderators will indicate that no names will be used in the final report. Also, the data returned to the IRS will have no identifying information relating specific records to individual taxpayers. Nonetheless, the IRS will ensure that privacy and security of the aggregated results will receive the utmost attention. Public and official access to the information will be tightly controlled. The computer files containing this tabulated information will remain password protected at all times. Data security approaching level C-2 will be accomplished using the Windows 7 operating system. Audiotapes used to record testing sessions will be destroyed when the project is completed and there is no further need for the data. We will apply fair information and record-keeping practices to ensure protection of all taxpayers. The criterion for disclosure laid out in the Privacy Act, the Freedom of Information Act, and section 6103 of the Internal Revenue Code, provides for the protection of taxpayer information as well as its release to authorized recipients.

11. Sensitive Nature

No questions will be asked that are of a personal or sensitive nature.

12. Burden of Information Collection

The estimated time to complete the participant screening is five minutes. The estimated time for each reminder call is one minute, with each testing session lasting 90 minutes. We estimate that 10 percent of the taxpayers contacted for the study will qualify and be willing to participate. Using this percentage, a total of 1680 individuals will need to be screened to recruit the needed 10 participants for each focus group. We estimate that 48 people will be sent home because we will only need 120 to stay for the focus groups .To achieve these objectives, WIRA plans to conduct a total of 12 focus groups, (three per city) in four separate cities.

Estimated Study Burden
Type of Collection	No. of Respondents	Annual Frequency Per Response	Minutes Per Response	Total Hours
Participant Screening	1680	1	5	140
Recruited Participant Reminder Calls	168	1	1	2.8
Completing Informed Consent	168	1	1	2.8
Study Participation	120	1	90	180

13. Costs to Respondents

No costs are anticipated.

14. Costs to Federal Government

The anticipated cost to the Federal Government is a one-time cost of approximately $90,000. This includes payments to the contractor (project planning, recruiting participants, communication with WIRA, transcripts, video, etc.) and a participant stipend ($75).

15. Reason for Change

Not applicable.

16. Tabulation of Results, Schedule, Analysis Plans

There will be two types of data collected during each focus group.

Participants will first complete the ACA knowledge pre-test. This information is strictly qualitative and will be used to establish a baseline of prior knowledge for the taxpayers. Secondly, participants will answer a series of questions using the ACA toll-free lines. The questions will be directly derived from the content/flow of the existing toll-free lines and qualitative data about their experiences will be collected. Responses to the questions will be quantitative in nature and will be scored for accuracy. These scores will be used to measure how well the participants comprehended the information presented on the toll-free lines.

Data from the discussion portion of the focus group will be qualitative in nature. The Contractor will provide transcripts and audio/videotape of each focus group session to WIRA. WIRA will analyze the verbatim feedback from participants to identify key themes in order to facilitate understanding of the taxpayer decision making process when contacting the IRS when seeking ACA related information.

Information from this will be useful, but will not yield data that can be generalized to the overall population. As such, the findings will be used for general service improvement, but are not for publication or other public release. Although WIRA does not publish its findings, information will be shared (when appropriate) with other organizations within the IRS, and will include specific discussion of the limitation of the data as discussed above.

17. Display of OMB Approval Date

We are requesting no exemption.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

These activities comply with the requirements in 5 CFR 1320.9.

19. Dates collection will begin and end

Data collection will begin on June 1, 2014 and end on June 30, 2014.

B. STATISTICAL METHODS

The primary purpose of these collections will be for internal management purposes; there are no plans to publish or otherwise release this information.

1. Universe and Respondent Selection

To participate in this study, participants must have the following characteristics:

• Taxpayer or spouse earned wages in calendar year 2013 or received a pension or annuity in 2013 for services performed as an employee of the U.S. Government or any U.S. state or local government

• At least 18 years of age

• Diverse in terms of age, gender, race/ethnicity, household income, education, filing status, and tax return preparation method.

Each focus group will have 10 participants. Research has demonstrated that this number of participants helps create the best group dynamics: the groups are sufficiently large to stimulate relatively spontaneous interaction, but small enough to give everyone sufficient opportunity to speak.

The screener guide will be used to qualify and select focus group participants. Qualitative and quantitative data will be gathered, which will not be, nor presented to be, representative of the population.

2. Procedures for Collecting Information

Each focus group will take no more than 90 minutes. Prior to the focus group, participants will be asked to read and sign a Consent Form. An outline of the areas of discussion is below.

Introduction

• Moderator introduces self, plan for pre-test and comprehension testing, topic of discussion, and ground rules (audio/video taping, presence of observers, privacy, etc.)

• Respondents introduce themselves by first name

Section 1: Pre-Test

• Participant responds to a short questionnaire about their knowledge and prior experiences in reference to ACA.

Section 2: Task Specific Questions with scenarios

• The taxpayers will be asked to answer questions using the information they can gather from calling the automated tool

• Taxpayer will be asked to take notes on their experiences

• Taxpayers will be observed as they complete the task, as well as recorded.

• Taxpayers will respond to a short questionnaire about their experiences and will provide the answers to the questions based on the ACA toll-free line content.

Section 3: Discussion

• Taxpayers will be asked to provide comments, suggestions and feedback on their experiences with the toll-free line content, transfer options and also provide general feedback

Section 4: Closing

• Elicit final suggestions/recommendations from participants

• Address any additional questions from observers

WIRA team members will moderate the focus groups. The moderator’s guide was developed by WIRA.

3. Methods to Maximize Response

Information collected will not yield generalizable quantitative findings; it can provide useful customer input, but it does not yield data about taxpayer opinions that can be generalized.

4. Testing of Procedures

Pretesting of the ACA information pre-test, the task specific questions, and moderators guide will be tested with internal IRS staff.

5. Contacts for Statistical Aspects and Data Collection

The statistical expertise of Social Scientists within WIRA will be sufficient for the needs of the data collected from this project.

9