Chatbot personality unmoderated usability testing

Participant screener: Chatbot personality unmoderated usability testing

Screener instructions:

The Internal Revenue Service (IRS) is conducting a study to learn about your preferences and expectations around an IRS digital feature. This online study will take up to 15 minutes and will be conducted during the week of August 26. If you’re interested in participating, please complete this questionnaire. If you qualify for this study, we’ll share the link to the study with you. Compensation of $10 will be provided to those who participate. 

Next are a few questions to gather background information prior to the study. The questionnaire should take no more than 5 minutes to complete. Your responses to these questions will remain private and will only be used to determine if you qualify for the study. The following Privacy Act Statement guides the handling of your responses.  

Participation in this study is voluntary; you’re not required to participate or share any identifying information, including name or email address, and you may withdraw participation at any time. However, if you don’t include the requested information, you may not be eligible to participate in the study. 

Thank you again for your interest!  

*Privacy Act Statement  

The authority requesting this information is 5 USC 301. The primary purpose of asking for the information is to determine steps the IRS can take to improve our services to you. The information may be disclosed as authorized by the routine uses published in the following Privacy Act System of Records: (1) IRS 00.001 Correspondence Files and Correspondence Control Files and (2) IRS 10.004 Stakeholder Relationship Management and Subject Files. Providing the information is voluntary. Not answering some or all of the questions will not affect you.  

Paperwork Reduction Act Burden Statement:

The Paperwork Reduction Act requires that the Internal Revenue Service display an OMB control number on all public information requests along with the address where you can send comments regarding the study. You are not required to respond unless a currently valid OMB approval number is displayed. The OMB number for this study is 1545-2256. This study is estimated to average 15 minutes. If you have any comments regarding this study, please write to: Internal Revenue Service, Office of Online Services, User Experience Services, [email protected].

Screener questions:

Demographic information

[Recruit a mix of demographics for the study across ages, US locations, races/ethnicities, education levels, genders, and incomes]

//DO NOT DISPLAY RED TEXT TO USERS//

1. How old are you? [Open text field]

[Terminate if under 18] 

2. Have you primarily lived in the United States in the past year? This includes territories and districts like Puerto Rico, the U.S. Virgin Islands, Guam, the Northern Mariana Islands, American Samoa, and the District of Columbia (Washington, D.C.).

   1. Yes, and I am currently living in the United States

   2. Yes, but I am not currently living in the United States [Terminate]

   3. No [Terminate]

3. What state, territory, or district have you lived in? [Open text field]

4. Which terms best describe your race or ethnicity? Select all that apply. 

   1. Asian 

   2. African American or Black 

   3. White 

   4. Hispanic or Latin(o/a) 

   5. Native Hawaiian or Other Pacific Islander 

   6. American Indian or Alaska Native 

   7. Prefer not to answer

5. How well do you speak English? 

   1. Very well 

   2. Well 

   3. Not well [Terminate] 

   4. Not at all [Terminate] 

6. What is the highest level of school or degree you have completed? 

   1. No schooling completed 

   2. Completed schooling through grade [text box] 

   3. Received high school diploma, GED, or alternative credential for high school graduation 

   4. Associate degree or some college 

   5. Bachelor’s degree  

   6. Professional degree beyond a bachelor’s degree (for example: MD, DDS, DVM, LLB, JD) 

   7. Doctorate degree (for example: PhD, EdD) 

7. I identify my gender as: 

   1. Female 

   2. Male 

   3. Non-binary/gender non-conforming 

   4. Prefer not to say 

8. What is your annual household income before taxes? 

   1. Under $25k 

   2. $25k-$74,999 

   3. $75k-99,999

   4. $100k + 

   5. Prefer not to say

Tax information 

For the next series of questions, we’re only interested in your individual tax returns, not any business tax returns you may have been involved in.

9. Within the past three years, have you filed federal income taxes for yourself or your household?

   1. Yes

   2. No [Terminate]

10. What will be/is your federal tax filing status for the current tax season? 

    1. Single 

    2. Married filing jointly 

    3. Married filing separately 

    4. Head of household 

    5. Qualifying widow(er) with dependent child 

    6. I don’t plan to file taxes this year

11. Typically, how involved are you in filing the taxes for your yourself/household?  

    1. I file taxes without help from others 

    2. Very involved 

    3. Somewhat involved 

    4. Not at all involved [Terminate] 

Technological familiarity 

12. How often do you use the internet? 

    1. Daily 

    2. Weekly  

    3. Monthly 

    4. Less than once a month 

    5. Almost never 

[Prioritize monthly or more frequently]

13. How would you rate your level of computer proficiency? 

    1. 1 (zero expertise) [Terminate] 

    2. 2 [Terminate] 

    3. 3 

    4. 4 

    5. 5 (expert) 

[Recruit a mix of participants between 3-5]

14. Which of the following devices do you have access to that you could use to participate in the study? Select all that apply.

    1. Apple/Mac laptop or desktop computer 

    2. Windows-based desktop or laptop computer  

    3. Chromebook 

    4. Smart phone (Apple or Android)

15. Do you use any assistive technologies like screen readers or braille displays when browsing the web?

    1. Yes [Terminate]

    2. No

16. If selected, do you have access to an internet connection to participate in this study? 

    1. Yes 

    2. No [Terminate] 

    3. I don’t know [Terminate] 

Contact information 

17. What is your name? [Open text field] 

18. So that we can contact you about participating, what is your email? [Open text field] 

Ineligible message:

//DO NOT DISPLAY HEADER LANGUAGE TO USERS//

[NEW SCREEN]

Thank you for your time and interest! Unfortunately, you do not qualify for the current study.

Eligible message:

//DO NOT DISPLAY HEADER LANGUAGE TO USERS//

[NEW SCREEN]

Thank you for your interest in this study. If qualified, we’ll be in touch via email to share the study link.