W&I Taxpayer Experience Survey 2023
Cognitive Interview Questions
12/22/2022
INTERVIEW INTRODUCTION
Thank you very much for agreeing to participate in this interview. I want to start out by introducing myself: my name is _______________ and I work for a research company called ICF, which is headquartered near Washington, D.C. I am not an employee of the IRS.
My company has been hired by the IRS to conduct interviews with taxpayers such as yourself. We are interested in finding out how people understand and interpret questions in an IRS customer satisfaction survey. We want to know if the way the questions are worded and the answer choices provided in the survey make sense to you.
I am going to ask you some questions based on the survey questions and invite you to give your interpretation, thoughts, and opinions. There are no wrong answers. Also, while we are interested in your survey responses, we are actually more interested in your reactions to and interpretations of the questions.
IF APPLICABLE: We also have additional team members observing this interview. They have all signed confidentiality agreements in order to observe.
[SLIDE 2]
Before we begin, did you review the informed consent that was sent in your invitation to this interview? I am now sharing it on the screen.
If YES: Obtain verbal consent to record interview and ask if they have any questions.
If NO: I’d like to go over the informed consent. [Obtain verbal consent to record interview and ask if they have any questions].
INTERVIEWER: VERBAL CONSENT TO PARTICIPATE MUST BE RECORDED.
[SLIDE 4]
Also I also am required to let you know that for the IRS to speak with the public, we are required to have approval from the Office of Management and Budget. The approval number for this project is 1545-2274. If you have any comments regarding this study, please write to: IRS, Special Services Committee, SE:W:CAR:MP:T:M:S – Room 6129, 1111 Constitution Avenue, NW, Washington, DC 20224.
Do you have any questions?
Yes
No
INTERVIEWER: IF RESPONDENT ANSWERS NO, PROCEED TO START THE INTERVIEW.
The questions you will see are from a national IRS survey that will be administered online and over the phone. During this discussion I am going to guide you through the survey question through the Teams screenshare. As we go through, I’d like you to read each survey question and answer choice out loud before letting me know which answer choices to select. As you answer the questions, please also think out loud. For example, if you are surprised by an answer choice, not sure what the choice means, or unsure of what we are asking, please let me know. After you respond to each question, I will ask you a few questions about how you arrived at your answer. I might ask you if anything was unclear or confusing about the question or what certain words/phrases mean to you. Additionally, please let me know if you have thoughts or questions about something even if I don’t ask you about it.
INTERVIEWER QUESTIONS APPEAR IN RED.
INTERVIEWER NOTE: Prioritize probes about meaning, such as “What do you think this question is asking you?” and “What does XX mean to you?” General probes about confusion and difficulty to answer are lesser priorities and can be skipped if necessary to complete the interview within 1 hour.
[SLIDE 6, etc.]
W&I Taxpayer Experience Survey 2022
W1.
What devices do you use to access the internet for personal use, such as to read online articles, access social media, schedule personal appointments, etc.?
Select all that apply
Smartphone (e.g., iPhone, Android phone)
Laptop computer
Desktop computer
Tablet or eReader (e.g., iPad, Kindle)
Smart TV (e.g., Apple TV, Roku TV)
Game console (e.g., Xbox, PlayStation)
Handheld gaming device (e.g., Nintendo Switch)
Smartwatch (e.g., Apple watch, Galaxy watch)
Smart speaker (e.g., Alexa, Echo)
Something else
What do you think of the term “devices”? Is that the best term to use in the question?
What does “access the internet for personal use” mean to you?
To what extent is this phrase confusing?
To what extent do the example help you to understand the phrase?
What changes, if any, would you make to the examples in the question?
Is it confusing to think about whether or not you access the internet as you use the devices?
Please consider the list of devices.
Is there anything unclear in this list of devices?
Is there anything missing in this list of devices?
Is there anything that should be removed from this list of devices?
To what extent do the examples help you to understand the devices?
What changes, if any, would you make to the examples of the devices?
Is it difficult to answer for any of the devices?
W2.
How often do you use each of these devices to access the internet for personal use, such as to read online articles, access social media, schedule personal appointments, etc.?
Smartphone (e.g., iPhone, Android phone)
Laptop computer
Desktop computer
Tablet or eReader (e.g., iPad, Kindle)
Smart TV (e.g., Apple TV, Roku TV)
Game console (e.g., Xbox, PlayStation)
Handheld gaming device (e.g., Nintendo Switch)
Smartwatch (e.g., Apple watch, Galaxy watch)
Smart speaker (e.g., Alexa, Echo)
RESPONSE OPTIONS
Several times a day or more
Once a day
Several times a week
Once a week
A few times a month
In the previous question, you said you used [fill in one or two devices noted above]. How easy or difficult would it be for you to answer this question about those devices?
To what extent do these response options fit how often you actually use your devices? Would you recommend any changes to the response options?
D1_2
Which of the following social media do you use regularly?
Select all that apply
YouTube
Snapchat
Other (please specify): ______[SMALL TEXTBOX]________
None
What does “social media” mean to you?
Do you use any other social media that you think should be included in this list?
What, if anything, would you suggest we take off this list?
Q1SOCMED1
Is social media a good way for the IRS to communicate brief, general tax information to someone like you?
1 Definitely
2 Probably
3 Maybe
4 Not really
5 Definitely not
What do you consider to be “brief, general tax information” that would be shared on social media? What does that mean to you?
What would a response of “Definitely” mean to you? What about “Maybe”? What about “Definitely not”?
Q1a_1
Which of the following IRS.gov (online) services/tools have you heard of?
Interactive Tax Assistant (ITA) (asks you a series of questions that guide you to a response tailored to your situation)
IP PIN (Identity Protection Personal Identification Number) Application (a unique number from the IRS used for identity theft protection)
Tax Withholding Estimator (helps you determine whether you have the right amount of tax withheld from your paycheck)
Online Employer Identification Number (EIN) Request (online application to receive an EIN)
IRS Online Account (secure tool allowing taxpayers to view their IRS account balance, payment history, tax record information, and more)
Online Payment Agreement (OPA) (online application for an installment agreement)
Online Chat with an Automated Bot
Online Chat with a Live Representative
RESPONSE OPTIONS
1 Yes, I’ve heard of this
2 No, I haven’t heard of this
To what extent is this question confusing?
In your own words, what is an “Online Chat with Automated Bot?”
In your own words, what is an “Online Chat with Live Assistor?”
How easy or difficult is it to respond about these services/tools?
Q1a_2
Which of the following payment options have you heard of?
Pay by bank account with Electronic Federal Tax Payment System (EFTPS) (allows taxpayers to transmit regular tax payments electronically)
Pay by bank account with Direct Pay (pay money you owe the IRS securely directly from your bank account for free without registration)
Pay by bank account through your IRS Online Account
Pay by bank account with your e-filed return
Pay by credit/debit card (card payment with a fee)
Pay by digital wallet (e.g., PayPal, Venmo)
Pay with cash via a retail partner
Offer in compromise
Short-term payment plans
Installment agreements
RESPONSE OPTIONS
1 Yes, I’ve heard of this
2 No, I haven’t heard of this
To what extent is this question confusing?
Which if any of the options are unclear or confusing?
In your own words, what is a “digital wallet”? To what extent to the examples help you to understand what this is?
How easy or difficult is it to respond about these options?
Q1c_V1
IRS notices are letters about issues such as balances due, overpayments, underpayments, information about tax credits or requests for more information.
Did you receive a notice in the mail from the IRS...?
Within the past 12 months
1-2 years ago
3-5 years ago
More than 5 years ago
RESPONSE OPTIONS
1. Yes
2. No
77. Not sure
To what extent is the description of IRS notices clear?
What do you include as n “IRS notice”. For example, did you think about the Economic Impact Payments, or stimulus payments, you may have received related to the coronavirus pandemic?
How easy or difficult is it/would it be to remember when you received a notice from the IRS?
Q1c_V2
IRS notices are letters about issues such as balances due, overpayments, underpayments, information about tax credits or requests for more information. When thinking about notices, please do not include letters you may have received about Economic Impact Payments related to the coronavirus.
Did you receive a notice in the mail from the IRS...?
Within the past 12 months
1-2 years ago
3-5 years ago
More than 5 years ago
RESPONSE OPTIONS
1. Yes
2. No
77. Not sure
This version of the question includes mention of the Economic Impact Payments
To what extent is the mention of the Economic Impact Payments clear?
To what extent is the mention of the Economic Impact Payments helpful in understanding what to think about when responding to the question?
Which version of the question do you prefer, and what makes it preferable?
Q16
In the past 12 months, did you try to get help with any of the following tax-related issues? This help could have been from the IRS or any other source.
Get help obtaining a form or publication
Get help obtaining transcripts online or by mail (i.e., copies of your previous tax returns)
Get help with tax law such as information on withholding, dependents, deductions, tax credits, or estimated taxes
Get answers to tax return preparation questions such as which forms to file, record keeping, filling out forms, how to file or how to get more help
Get information about making payments
RESPONSE OPTIONS
1—Yes
2—No
To what extent is the question clear?
To what extent are each of the issues clear?
What does the phrase “get help with tax law” mean to you?
To what extent do the examples help you understand what the question is asking about?
When in the process of preparing and filing your return would you be likely to seek out help with tax law? While preparing your return? After filing your return? Perhaps both?
What does the phrase “tax return preparation questions” mean to you?
How is getting answers on tax return preparation difference than getting help with tax law?
To what extent is it easy or difficult to respond to the question?
Q57b3.
You indicated that you tried to get assistance to make a payment. What type of payment were you trying to make?
One-time payment to the IRS made at the time you filed
One-time payment to the IRS made after you filed
One-time payment to the IRS made when filing an extension
Installment Agreement Payments
Quarterly Estimated Payments
Other (please specify): ______[SMALL TEXTBOX]______
To what extent is the question clear?
To what extent are each of the response options clear?
What does the phrase “when filing an extension” mean to you?
To what extent is it easy or difficult to respond to the question?
Q26a
The IRS Free File program allows taxpayers who meet certain criteria to file their federal taxes online for free. To use this program, taxpayers can visit the program page on IRS.gov, confirm their eligibility, choose an online tax preparation company (for example: TaxAct, 1040NOW, etc.), and file their federal return online for free. The IRS Free File program is different from IRS fillable forms and from free tax preparation available outside of the IRS.gov website.
Were you aware of the IRS Free File program described above?
1 Yes
2 No
To what extent is the description of the IRS Free File program clear? What, if anything is confusing about the description?
To what extent is it easy or difficult to respond to the question?
FF_Elig
As far as you know, do you qualify to use the IRS Free File program?
Yes
No
Not sure
What does the phrase “qualify to use” mean to you?
In considering whether you “qualify”, what timeframe are you thinking about? Last year? This year? Just in general?
To what extent is it easy or difficult to respond to the question?
FF_Access
What is the first website you would go to try to access the IRS Free File program?
A search engine (e.g., Google, Bing)
IRS.gov
A tax software company’s website
Another site (please specify): ____[SMALL TEXTBOX]____
Not sure
What does the phrase “access the IRS Free File program” mean to you?
To what extent are all the response options clear? Should anything be removed? Should anything be added?
To what extent is it easy or difficult to respond to the question?
Q26a1
Have you ever used tax software that you found through the IRS Free File program on IRS.gov to file your federal taxes?
1 Yes
2 No
3 Not sure
In your own words, what does this question mean to you?
To what extent is it easy or difficult to respond to the question?
Q26b
Did you use tax software that you found through the IRS Free File program on IRS.gov to file your federal taxes this year?
1 Yes
2 No
3 Not sure
In your own words, what does this question mean to you?
To what extent is it easy or difficult to respond to the question?
DEF1.
The following questions are about your expectations for and your level of interest and trust in an IRS-provided online tool used to prepare and electronically file your tax returns directly to the IRS, free of charge. The online preparation and filing tool would include features such as safeguards to protect taxpayer data, ease of accessibility, and “interview-style” preparation (meaning the tool will ask tax-related questions to fill in the return), in multiple languages, with access from mobile devices. This tool would be an alternative to commercial and IRS-provided tax preparation software currently available to taxpayers (e.g., TurboTax, FreeTaxUSA, etc.).
Based on the description, how interested would you be in using this online tool to prepare your taxes if it becomes available?
Very interested
Somewhat interested
Not very interested
Not interested at all
To what extent is the description of the tool clear?
Could you describe the tool in your own words?
To what extent is it clear that the tool is different than other tax preparation options?
To what extent is it easy or difficult to respond to the question?
DEF7.
Imagine you are using this online tool and run into a problem with the system. How important are each of these customer service options to you?
Online chat with an automated bot
Online chat with a live representative
Text chat with an automated bot
Text chat with a live representative
Toll-free help line
RESPONSE OPTIONS
Very important
Somewhat important
Not very important
Not important at all
What does the phrase “problem with the system” mean to you? What kind of problems were you thinking about, or what kind of problems might you expect to encounter?
IF NECESSARY: Are you thinking about technical problems related to using the tool? Are you thinking about problems related to your taxes?
To what extent is it easy or difficult to respond to the question?
Q73.
The next set of questions asks about your activities online.
Please indicate the extent to which you agree or disagree with each of the following statements.
I am comfortable performing financial activities (i.e., paying bills, credit cards, insurance, mortgages, etc.) online
I am comfortable making purchases online
I can usually find what I need online
RESPONSE OPTIONS
Strongly disagree
Disagree
Neither agree nor disagree
Agree
Strongly agree
The items ask about being “comfortable”. What does it mean to you to be comfortable performing financial activities online?
What does the phrase “usually find what I need online” mean to you? What do you include when you consider that phrase?
What does the response option “strongly agree” mean to you? What about the “strongly disagree”?
D2_3.
Do you have health insurance?
1 Yes
2 No
How easy or difficult is it to respond to this question?
D2_4.
Through whom do you or any other members of your household have health insurance coverage?
Select all that apply
1 Directly from an insurance company
2 Employer-based health insurance plan
3 Health Insurance Marketplace from healthcare.gov
4 Medicare
5 Medicaid or CHIP
6 State-specific health insurance plan
7 TRICARE or other military health coverage
9 Other
77 I don’t know
To what extent is the question clear or unclear?
Please consider the response options? To what extent are the response options clear? Is there anything that should be removed? Is there anything that should be added?
How easy or difficult is it to respond to this question?
QB1
Do you currently have a checking or savings account?
Yes
No
How easy or difficult is it to respond to this question?
QB2
How long has it been since you used your checking or savings account?
Less than a month
More than 1 month but less than 6 months
More than 6 months but less than 1 year
A year or more
The question asks about “using” your checking or savings account. What does it mean to you to use checking or savings account? What does that make you think of, or what do you include in “using” you checking or savings account?
How easy or difficult is it to remember how long it has been since you used your checking or savings account?
Would you have different answers about your checking vs. your savings account?
COGNITIVE INTERVIEW CLOSING
That concludes the interview. Thank you for your time, and for your helpful feedback on the survey questions. AmeriSpeak will deliver your incentive for participating in this interview. Note that the points will take longer to show up than for a regular survey, but you should get it within two weeks or so, probably less. On behalf of the IRS, and my company ICF, I wish to thank you for your participation today. Have a great [day/afternoon/evening].
Page
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Sumner, Julia |
| File Modified | 0000-00-00 |
| File Created | 2024-08-05 |