TES Cognitive Interview Guide_English_2021-12-28

Wage and Investment (W&I) Taxpayer Experience Survey 2022

Cognitive Interview Questions

12/28/2021

INTERVIEW INTRODUCTION

Thank you very much for agreeing to participate in this interview. I want to start out by introducing myself: my name is _______________ and I work for a research company called ICF, which is headquartered near Washington, D.C. I am not an employee of the IRS.

My company has been hired by the IRS to conduct interviews with taxpayers such as yourself. We are interested in finding out how people understand and interpret questions in an IRS customer satisfaction survey. We want to know if the way the questions are worded and the answer choices provided in the survey make sense to you.

I am going to ask you some questions based on the survey questions and invite you to give your interpretation, thoughts, and opinions. There are no wrong answers. Also, while we are interested in your survey responses, we are actually more interested in your reactions to and interpretations of the questions.

IF APPLICABLE: We also have additional team members observing this interview. They have all signed confidentiality agreements in order to observe.

[SLIDE 2]

Before we begin, did you review the informed consent that was sent in your invitation to this interview? I am now sharing it on the screen.

If YES: Obtain verbal consent to record interview and ask if they have any questions.

If NO: I’d like to go over the informed consent. [Obtain verbal consent to record interview and ask if they have any questions].

INTERVIEWER: VERBAL CONSENT TO PARTICIPATE MUST BE RECORDED.

[SLIDE 4]

Also I also am required to let you know that for the IRS to speak with the public, we are required to have approval from the Office of Management and Budget. The approval number for this project is 1545-2274. If you have any comments regarding this study, please write to: IRS, Special Services Committee, SE:W:CAR:MP:T:M:S – Room 6129, 1111 Constitution Avenue, NW, Washington, DC 20224.

Do you have any questions?

• Yes

• No

INTERVIEWER: IF RESPONDENT ANSWERS NO, PROCEED TO START THE INTERVIEW.

[SLIDE 5]

The questions you will see are from a national IRS survey that will be administered online and over the phone. During this discussion I am going to guide you through the survey question through the Teams screenshare. As we go through, I’d like you to read each survey question and answer choice out loud before letting me know which answer choices to select. As you answer the questions, please also think out loud. For example, if you are surprised by an answer choice, not sure what the choice means, or unsure of what we are asking, please let me know. After you respond to each question, I will ask you a few questions about how you arrived at your answer. I might ask you if anything was unclear or confusing about the question or what certain words/phrases mean to you. Additionally, please let me know if you have thoughts or questions about something even if I don’t ask you about it.

INTERVIEWER QUESTIONS APPEAR IN RED.

[SLIDE 6, etc.]

W&I Taxpayer Experience Survey 2022

D2.

How often do you access the internet, apart from taking surveys?

1. Several times a day or more

2. Once a day

3. Several times a week

4. Once a week

5. Less than once a week

6. Never

1. To what extent is this question confusing?

2. To what extent is this question difficult to answer?

   1. Are any options missing to you?

W1.

What devices do you use to access the internet for personal use?

Select all that apply

1. Smartphone or web-enabled cell phone

2. Laptop, netbook, or notebook computer

3. Desktop computer

4. Tablet or eReader

5. Smart TV (web-enabled TV)

6. Game console

7. Handheld gaming device

8. Smartwatch

9. Something else

1. What does “access the internet for personal use” mean to you? To what extent is this phrase confusing?

2. What do you think of the term “devices”? Is that the best term to use in the question?

3. Please consider the list of devices.

   1. Is there anything unclear in this list of devices?

   2. Is there anything missing in this list of devices?

   3. Is there anything that should be removed from this list of devices?

W2 (New question)

How often do you use each of these devices to access the internet?

(Will only show the ones selected in the previous question)

1. Smartphone or web-enabled cell phone

2. Laptop, netbook, or notebook computer

3. Desktop computer

4. Tablet or eReader

5. Smart TV (web-enabled TV)

6. Game console

7. Handheld gaming device

8. Smartwatch

9. Something else

[RESPONSE OPTIONS]

1. Several times a day or more

2. Once a day

3. Several times a week

4. Once a week

5. Less than once a week

1. How easy or difficult would it be for you to answer this question?

2. To what extent is this question confusing?

3. An alternate version of the question might be, “What percentage of time do you use the following devices to access the internet?”

   1. Please consider that version and then talk me through how you would answer it.

4. Another version of this question might be, “Select the three devices you use the most to access the internet.”

   1. Please consider that version and then talk me through how you would answer it.

   2. To what extent did you have difficulty selecting up to three devices? Would it be easy or difficult to select two devices or only one?

W3 (New question)

Why don’t you access the internet more often?

(Only asked of people who say they access the internet once a day or less)

TEXTBOX

1. If this applies to them: How would you answer this?

2. If this does not apply to them: I’d like you to try to think of someone you know or know of who doesn’t get online very often or doesn’t do video calls. What are some reasons you think they might have for not accessing the internet very often?

D1_2

Which of the following social media do you use?

Select all that apply

1. Facebook

2. Twitter

3. YouTube

4. Instagram

5. Pinterest

6. LinkedIn

7. Reddit New

8. Other (please specify): ______[SMALL TEXTBOX]________

9. None

1. Do you use any other social media that you think should be included in this list?

2. What, if anything, would you suggest we take off this list?

D1_3 (New)

How do you use social media?

Select all that apply

1. Daily, for work

2. Daily, socially

3. Post more than once a day

4. Post daily

5. Post 2-3 times per week

6. Post once per week

7. Read only, no posting

1. How easy or difficult is it to answer this question?

2. Do you use social media in other ways that you think should be added to this list?

3. What, if anything, would you suggest we take off this list?

FOR SPANISH INTERVIEWS ONLY

S1e.

Original

Would you say you can carry on a conversation in English, both understanding and speaking, very well, well, not well, or not at all?

1. Very well

2. Well

3. Not well

4. Not at all

Revised

Can you carry on a conversation in English, both understanding and speaking…

1. Very well

2. Well

3. Not well

4. Not at all

1. Do you prefer one version of this question over the other?

FOR SPANISH INTERVIEWS ONLY

S1f.

Original

Would you say you can read a newspaper or book in English very well, well, not well, or not at all?

1. Very well

2. Well

3. Not well

4. Not at all

Revised

Can you read a newspaper or book in English…

1. Very well

2. Well

3. Not well

4. Not at all

1. Do you prefer one version of this question over the other?

S6a.

Were any of the following included with your federal tax return this year?

(Note: This survey will be after the 2022 filing season and ask about this year’s return.)

Form/Schedule	Yes	No	Don’t Know
Itemized deductions including home mortgage deductions, charitable contributions, and other types of deductions (Schedule A)
Earned Income Tax Credit (EITC/EIC) (e.g., Schedule EIC)
Child Tax Credit/Additional Child Tax Credit (CTC/ACTC) New
American Opportunity Tax Credit (AOTC) New
Child and Dependent Care Credit (CDCC) New
Other refundable credit (please specify): ______ New
Interest and dividend income (Schedule B)
Small business or self-employment income (Schedule C)
Capital gains or losses (Schedule D)
Supplemental income, such as rental income, royalties, and trusts (Schedule E)
Farm income (Schedule F)
Employee business expenses (Form 2106)
Partnerships or S Corp (e.g., Forms 1120, 1120s, 1065)

1. To what extent is the question clear?

2. To what extent are the forms and schedules clear?

   1. To what extent are you familiar with these forms and schedules?

   2. What about “Child Tax Credit/Additional Child Tax Credit”? To what extent is that clear?

   3. What about “American Opportunity Tax Credit”? To what extent is that clear?

   4. What about “Child and Dependent Care Credit”? To what extent is that clear?

   5. What about “Other refundable credit”? To what extent is that clear?

      • Do you know of any other refundable credits?

3. To what extent is the question easy or difficult to respond to?

Q1a_1

Which of the following IRS.gov (online) services/tools have you heard of?

[Note: Cog testing includes only new services/tools.]

Services/Tools	Yes, I’ve heard of this	No, I haven’t heard of this
Submit Forms Online (allows tax practitioners to submit Forms 2848 and 8821 online to the IRS)
Tax Pro Account (online authorization request)
Child Tax Credit Update Portal (online application to manage Advance Child Tax Credit payments)
IDVerify (allows taxpayers to authenticate their identity and complete the Taxpayer Protection Program (TPP) verification questions)
Secure Access Digital Identity (identity-verification process that helps protect taxpayer data and IRS systems from automated cyber-attacks)

1. To what extent is the question clear?

2. What does it mean to you to have “heard” of these services/tools?

3. To what extent are all of the services/tools clear?

   1. Are any of the descriptions unclear?

4. To what extent is it easy or difficult to respond about each service/tool?

(These questions only asked of people who say they used the IDVerify tool)

Q5c_4.

Were you able to verify your identity?

1. Yes

2. No

[If no] Why were you not able to verify your identity?

TEXTBOX

Q5c_4a.

Was IDVerify easy to use?

1. Yes

2. No

[If no] Please describe any issues you had using IDVerify.

TEXTBOX

1. To what extent are the questions clear?

2. To what extent is it easy or difficult to respond to each question?

(These questions only asked of people who say they used the Secure Access Digital Identity (SADI) tool)

Q5c_5.

Were you able to successfully access SADI?

1. Yes

2. No

[If no] Why were you unable to access SADI?

TEXTBOX

Were you able to complete verification using SADI?

1. Yes

2. No

[If no] Why were you not able to complete verification using SADI?

TEXTBOX

Was SADI easy to use?

1. Yes

2. No

[If no] Please describe any issues you had using SADI.

TEXTBOX

1. To what extent are the questions clear?

2. To what extent is it easy or difficult to respond to each question?

Q2n1a.

Did you need help for any of the following issues?

1. Find out if you are eligible for the Earned Income Tax Credit (EITC)

2. Find out if you are eligible for the Advance Child Tax Credit New

2 Determine if your child/children meet the tests for a qualifying child

3 Estimate the amount of your Earned Income Tax Credit

[RESPONSE OPTIONS]

1—Yes

2—No

1. To what extent is the question clear?

2. To what extent are the response options clear?

3. To what extent is it easy or difficult to respond to the question?

“Direct e-File” is a proposed IRS-provided and administered online tax return filing system for taxpayers to complete and submit tax returns to the IRS. Direct e-File will be at no cost to tax filers. Direct e-File would be an alternative to current tax preparation software available to the taxpayer. The system will be different from the IRS Free File program, IRS fillable forms, and from free tax preparation available outside of the IRS.gov website. The following questions are about your expectations, your level of interest, and level of trust for such a free Direct e-File system.

DEF1.

Based on the description, how interested would you be in using the Direct e-File system to prepare your taxes when it becomes available?

1. Very interested

2. Somewhat interested

3. Not very interested

4. Not interested at all

1. To what extent is the description of direct e-file clear?

   1. Could you describe direct e-file in your own words?

2. To what extent is it clear that direct e-file is different than other tax preparation options?

   1. To what extent does the inclusion of other tax preparation options help you to understand what direct e-file is?

3. To what extent is the question clear?

4. To what extent are the response options clear?

5. What does [Response] mean to you?

6. To what extent is it easy or difficult to respond to the question?

DEF2.

Based on the description, how much would you trust the Direct e-File system to keep your information safe?

1. Very much

2. Somewhat

3. Not very much

4. Not at all

1. To what extent is the question clear?

2. To what extent are the response options clear?

3. What does [Response] mean to you?

4. To what extent is it easy or difficult to respond to the question?

DEF3.

Compared to other tax preparation software, how easy or difficult to use do you expect the Direct e-File system to be?

1. Much easier than other tax preparation software

2. Somewhat easier than other tax preparation software

3. About the same

4. Somewhat more difficult than other tax software

5. Much more difficult than other tax software

1. To what extent is the question clear?

2. To what extent are the response options clear?

3. What does [Response] mean to you?

4. To what extent is it easy or difficult to respond to the question?

DEF4.

How important to you are the following aspects of tax preparation software?

1. User Friendliness

2. Adequate Security

3. Prepopulated Forms

4. Mobile Friendliness

5. Free or Low Cost

6. Step-by-step Process

[RESPONSE OPTIONS]

1. Very important

2. Somewhat important

3. Not very important

4. Not important at all

1. To what extent is the question clear?

2. To what extent are each of the aspects clear?

   1. Should any aspects be added to the list?

   2. Should any aspect be removed from the list?

3. To what extent are the response options clear?

4. What does [Response] mean to you?

5. To what extent is it easy or difficult to respond to the question?

DEF5.

How likely are you to switch from your current tax preparation software to the Direct e-File when it becomes available?

1. Very Likely

2. Likely

3. Unlikely

4. Very Unlikely

1. To what extent is the question clear?

2. To what extent are the response options clear?

3. What does [Response] mean to you?

4. To what extent is it easy or difficult to respond to the question?

DEF5a.

(Only asked of people who say they are unlikely to switch)

Why are you unlikely to switch from your current tax preparation software to the Direct e-File?

Select all that apply.

1. I already file for free

2. I do not think the Direct e-File system would be as good as tax software I pay for

3. I would have more confidence/trust in the previous method I used than the Direct e-File system

4. I prefer not to make changes on the tax filing method/software I used previously

5. Another reason (please specify)

1. To what extent is the question clear?

2. To what extent are the response options clear?

   1. Should any response options be added to the list?

   2. Should any response options be removed from the list?

3. To what extent is it easy or difficult to respond to the question?

COGNITIVE INTERVIEW CLOSING

That concludes the interview. Thank you for your time, and for your helpful feedback on the survey questions. AmeriSpeak will deliver your incentive for participating in this interview. Note that the points will take longer to show up than for a regular survey, but you should get it within two weeks or so, probably less. On behalf of the IRS, and my company ICF, I wish to thank you for your participation today. Have a great [day/afternoon/evening].

Page 23