Credential Service Provider Usability Research for Low Income Population

1. Recruiting Screener

   1. Opening Script

Hello, I'm [name] from [company name], a [company description].

We are conducting a research study investigating the usability of a new method to create an account for online government services. I would like to ask you ova few questions to learn if you might be interested in participating. I want to assure you that we are not looking to sell you anything. Is this a good time to speak with you?

This research study will take approximately [90 minutes] of your time and can be completed at your convenience between [dates]. All participants who complete the study will receive an incentive of [$100] at the end of the session. The research study consists of creating an account for a secured service used to register and identity proof citizens by US federal agencies, and then filling out a feedback survey.

Some participants will also be recruited for an additional 30 minute follow up interview by phone and receive an additional incentive for their time. Are you interested in participating?

IF AVAILABLE:

Great! I have to ask a series of questions to check if you qualify to participate. Please note that we maintain absolute confidentiality; all information you provide us will be kept strictly confidential between us and our client.

IF NOT RECRUITING IN REAL TIME:

Recruiting Screener: Please complete the following questionnaire to participate in the Digital Identity Usability Survey. The survey takes approximately [60 – 90 minutes] to complete [and is followed by a 30-minute phone interview]. Participants will receive [$100] in [compensation format].

2. Demographics

What is your age? (Recruit a mix. Terminate if younger than 18 years old. Limit participants to 24 years or younger to no more than ten.)

What is your gender? (Recruit a mix.)

• Male

• Female

• Other

Including yourself, how many people live in your household?

IF ASKED FOR FURTHER CLARIFICATION ON HOUSEHOLD: Individuals do not have to be related in any way to be considered members of the same household.

Which of the following best describes your total estimated household income before taxes for 2019? (Recruit a balanced mix from first four options.)

• Less than $10,000

• $10,000 to less than $15,000

• $15,000 to less than $20,000

• $20,000 to less than $25,000

• $25,000 to less than $50,000 [Terminate Call Script]

• $50,000 or more [Terminate Call Script]

IF ASKED FOR FURTHER CLARIFICATION ON INCOME: We’re interested in your estimated household income before taxes for 2019, which means your income not adjusted, and before any deductions. Household income is the combined gross, or total, income of all members of a household who are 15 years or older. Individuals do not have to be related to be considered members of the same household.

What was your filing status in 2019?

• Single

• Married filing jointly

• Married filing separately

• Head of household

• Qualifying widow(er) with dependent child

I’m going to ask a question about US citizenship. I am only asking this for the purpose of determining if you are eligible to participate in this study, since only US citizens are eligible. If you answer, “no or prefer not to say,” your answer will not be shared with anyone.

Are you a US citizen?

• Yes

• No or Prefer not to say [Terminate Call Script]

Which terms best describe your race or ethnicity? (They may answer with more than one, record all answers.) (Recruit a mix.)

• American Indian or Alaska Native

• Asian

• Black or African American

• Hispanic or Latino

• Native Hawaiian or Other Pacific Islander

• White

What is the highest level of school or degree you have completed? (Recruit a mix.)

1. No schooling completed

2. Completed schooling through Grade ___.

3. Received High School Diploma, GED or alternative credential for high school graduation

4. Associate degree or Some College

5. Bachelor’s Degree

6. Master’s Degree (for example: MA, MS, Meng, MEd, MSW, MBA)

7. Professional degree beyond a bachelor’s degree (for example: MD, DDS, DVM, LLB, JD)

8. Doctorate degree (for example: PhD, EdD)

3. English Proficiency

How well do you speak English?

• Very well

• Well [Terminate Call Script]

• Not well [Terminate Call Script]

• Not at all [Terminate Call Script]

How well do you read and write English?

• Very well

• Well [Terminate Call Script]

• Not well [Terminate Call Script]

• Not at all [Terminate Call Script]

4. Other Eligibility Requirements

FOR INTERVIEW CANDIDATES ONLY: These interview sessions will be audio recorded for internal use only. Therefore, you will be asked to sign a consent form stating that you agree to be recorded; would you be agreeable to that?

• Yes

• No [Terminate Call Script]

Are you currently a representative or an employee of a State or Federal Government Agency?

• Yes [Terminate Call Script]

• No

Are you currently, or have you ever been employed in any field in the tax services industry? (For example, as an enrolled agent, certified public accountant, tax attorney, or other tax professional.)

• Yes [Terminate Call Script]

• No

5. Disability

Do you have a diagnosis for a disability or impairment that currently applies to you? This includes difficulty with learning capabilities.

• I have a current diagnosis for a disability or impairment

• I do not currently have a diagnosis for a disability or impairment

• I prefer not to say

[If “have”] Which of the following describes your diagnosis for a disability or impairment?

• Blind or serious difficulty seeing, even when wearing glasses

• Deaf or serious difficulty hearing

• Serious difficulty with mobility or other physical capabilities

• Serious difficulty with learning capabilities (e.g., ADHD, dyslexia)

• Serious cognitive difficulty concentrating, remembering, or making decisions because of a physical, mental, or emotional condition

• A disability/impairment not listed above

[If “have”] Please list the primary assistive technologies that you use to access the internet: [Capture up to five] Clarification: Assistive technologies are products, equipment, and systems that enhance learning, working, and daily living for persons with disabilities.

For “Deaf or serious difficulty hearing” interview candidates only: Do you require an ASL interpreter?

• Yes

• No

6. Technical Requirements for Study

Do you have access to a computer?

• Yes

• No

• I don’t know

Do you have a personal smartphone with a camera?

• Yes

• No [Terminate Call Script]

• I don’t know [Terminate Call Script]

Is the cellular plan for your personal smartphone registered in your name?

• Yes

• No [Terminate Call Script]

• I don’t know [Terminate Call Script]

Do you have a valid email address?

• Yes

• No [Terminate Call Script]

• I don’t know [Terminate Call Script]

Do you have access to a quiet, private, space with an internet connection to complete an electronic survey? Clarification: a private space where you have WIFI and can work on a survey.

• Yes

• No [Terminate Call Script]

• I don’t know [Terminate Call Script]

FOR INTERVIEW CANDIDATES ONLY: Do you have availability on weekdays, between 8am and 6pm EST, for a 30-minute interview?

• Yes

• No [Not a candidate for interview]

• I don’t know [Not a candidate for interview]

7. Identity Proofing Requirements

In this study, you will be asked to test out an authentication service. An authentication service proves who you are you say you are. To do so, you will be asked to create an account and provide personal documents verifying your identity, such as a driver’s license and a social security number. Note that this is through a secure and reputable authentication platform. Additionally, information from your documents will not be retained and immediately purged upon deleting your account at the conclusion of the survey. Are you willing to create an account with this type of service using your personal documents? 

• Yes

• No [Terminate Call Script]

Do you possess a current state issued photo identification card? (such as a Driver’s license or a photo identification card issued by a U.S. state)

• Yes

• No [Terminate Call Script]

Do you know your social security number?

• Yes

• No [Terminate Call Script]

8. Internet and Financial Activity

Do you currently use a bank? A bank is a financial institution where customers can save or borrow money. It can be an online or mobile only bank.

• Yes

• No

What do you use the internet for on a smartphone, tablet or computer? (Must select a minimum of 2)

1. Stream or download music, radio or podcasts

2. Participate in an online social networking website

3. Use online financial services (such as check a balance, pay a bill, or investing)

4. Make travel arrangements or get directions

5. Search or apply for jobs

6. Purchase products, groceries or restaurant food from an online store

7. Participate in an online course or online job training

8. Find recipes, instructions or other educational content

[Terminate Call Script if one or less are selected]

Which of the following is true for you? (True/False)

1. I know how to switch between two or more tabs in a browser like chrome, safari, internet explorer, edge or firefox

2. I know how to complete online forms

3. I know how to install apps on a smartphone

4. I know how to take a photo with the camera on a smartphone

[Terminate if not true for one or more]

9. Terminate Call Script

Thank you for your participation in our recruitment screener. At this time, we will not require your participation, but appreciate your time in answering our qualifying questions. We are looking for people who fit a specific demographic and it doesn’t appear that you are eligible for this usability survey.

Thank you very much for your time. Goodbye.

2. Informed Consent

   1. Survey Informed Consent (click-through)

PRINCIPAL INVESTIGATOR(S): Rebecca Scollan

STUDY TITLE: Credential Service Provider Usability Study

We invite you to take part in a research study conducted by the MITRE Corporation (MITRE) on behalf of the IRS. We want you to know that taking part in MITRE research is entirely voluntary. You may choose not to take part, or you may withdraw from the study at any time.

If you choose to participate in this study, the data from your survey will be anonymized using a participant ID code, aggregated with other survey responses and reported on to the IRS. If you decide not to participate in this study or if you withdraw from participating at any time, your survey account and any survey responses will be removed.

Before you decide to take part, please take as much time as you need to review the research description and discuss this study with family, friends or any of your advisers.  If you have any questions for the MITRE research team, you may send them through your point of contact at [company], [name] at [email] or [phone].

Research Description

MITRE is conducting usability research on the Credential Service Provider [CSP] on behalf of the IRS. The IRS has requested this research to ensure that taxpayers have a positive user experience with identity proofing, in recognition that it can be a frustrating experience. Your feedback is paramount to gaining a better understanding of how the experience works for citizens and is the motivation behind this research.

A Credential Service Provider (CSP) identity proofs and issues electronic credentials to consumers for government and businesses. [CSP description.]

Neither MITRE nor the IRS is affiliated with [CSP], and you use the system at your own risk. Please review the [CSP] privacy policy at [link] prior to making your decision to participate in this study.

In this study you will be asked to create an account with [CSP] and answer questions about your experience. Participation is expected to take between [60 – 90 minutes]. You may complete the study at your own pace and use any support that [CSP] offers. Some questions are intended to be answered while you are working in your [CSP] account, so we ask that you follow the instructions carefully.

You are expected to complete all four parts of the survey within sixteen days of receiving access to the survey. You are not expected to be able to successfully create an account with [CSP] but are expected to work on creating an account until the point that you either succeed or can no longer proceed. You may complete the survey whether or not you successfully create you [CSP] account. If you experience a delay and need an extension, you may contact [POC] to request one. You must complete the survey to receive the incentive payment.

Risks/Discomforts 

There is an inherent risk to data security and privacy associated with entering and transmitting private and/or identifying information. The MITRE Corporation is not affiliated with [CSP]. MITRE is administering a survey on the usability of the service on behalf of the IRS. MITRE makes no assurances about the data security and privacy of [CSP].

There is the potential for experiencing slight emotional discomfort uploading, displaying, and answering questions about identity documents. There is the potential for experiencing slight emotional discomfort providing a self-photo (“selfie”) for identity proofing purposes.

There is a chance of experiencing slight discomfort associated with using a laptop computer or a smartphone. Risk of this is very low. If you are experiencing wrist or hand fatigue, you may take a break or end the session as you choose. 

Data and Privacy 

All data shared between [recruiter] and MITRE about you will be stored in encrypted files. Access to the data shared with [recruiter], including any of your personally identifying data, will be limited to the MITRE research team only.

All data concerning the study, will be stored in an encrypted directory within MITRE. The data will include demographics (e.g., age, gender, income, education) and survey entries. Collected data will be identified only by a participant ID code in any research reporting within MITRE and the IRS. The data will be aggregated with other survey responses in reporting to the IRS. Participant ID codes will never be linked with the participant identities in reporting on this research.

Other Pertinent Information

1. Confidentiality. All data shared between [recruiter] and MITRE about you will be stored in encrypted files. Access to the data shared with [recruiter], including any of your personally identifying data, will be limited to the MITRE research team only. All data concerning the study, will be de-identified and stored in an encrypted directory within MITRE. The de-identified data will include demographics (e.g., age, gender, income, education), survey and diary entries. Collected data will be identified only by a participant ID code in any research reporting within or external to MITRE. Participant ID codes will never be linked with the participants identities in reporting on this research.

2. Remuneration. As a token of appreciation, you will receive a [$100] [prepaid gift card] upon completion of this survey.

3. Problems or Questions. If you have any problems or questions about this study, or about your rights as a research participant, or about any research-related injury, contact the Principal Investigator, Rebecca Scollan, at [phone].

4. Consent Document. Please keep the PDF copy of this document provided by [recruiter] in case you want to read it again.

5. IRS Privacy Act Statement. The authority requesting the information is 5 USC 301. The primary purpose of asking for the information is to determine steps the IRS can take to improve our services to you. The information may be disclosed as authorized by the routine uses published for the following Privacy Act System of Records: (1) IRS 00.001 Correspondence Files and Correspondence Control Files and (2) IRS 10.004 Stakeholder Relationship Management and Subject Files. Providing the information is voluntary. Not answering some or all of the questions will not affect you.

6. Paperwork Reduction Act: OMB No: 1545-2256. This report is authorized under the Paperwork Reduction Act. Data collected will be shared with IRS staff, but your responses will be used for research and aggregate reporting purposes only and will not be used for other non-statistical or non-research purposes such as direct enforcement activities. The information that you provide will fully be protected allowable under the Freedom of Information Act (FOIA). Public reporting burden for this collection of information is estimated to average 120 minutes, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Special Services Section, SE:W:CAR:MP:T:M:S, Room 6129, 1111 Constitution Avenue, NW, Washington, DC 20224.

The Paperwork Reduction Act requires that the IRS display an OMB control number on all public information requests. The OMB Control Number for this study is 1545-2256.

Participant’s Consent

I have read the explanation about this research study and have been given the opportunity to discuss it and to ask questions. I hereby consent to take part in this study.

Digital Participant Signature

Please check the box below to consent.

[checkbox] I consent to the above items, checking this box acts as my digital signature.

Date

Please select today’s date.

[Date selection tool]

[Submit button]

2. Interview Informed Consent (verbal)

The research we are doing is sponsored by the IRS and is exploring a potential new method to register for online services. During this feedback session, we will discuss the survey and talk more in depth about your experience using [CSP]. Your participation is entirely voluntary, and you are free to end our session at any time. You will receive compensation of [$50] for your participation in this interview.

This interview will take approximately 30 minutes. We will record the audio of our conversation. The recording is for research purposes only and your name will not be associated directly with the content in the recording or with anything in our session notes. Recordings will be kept in a secure location and shared only with persons with a valid need to know. The content of all interviews will be aggregated to identify key issues and any references to individual participants will be non-attributional. Your name and contact details are stored in an encrypted file that is only available to the MITRE research team and is only used for scheduling. The file will be destroyed on completion of this study. Your name will not be linked to anything you say in any reporting within MITRE or to the IRS.

There is some risk to privacy in participating in the interview. We mitigate this risk by only capturing your first name for the purposes of scheduling interview sessions and deleting all first names within 30 days of completing the study.

You are free to end the session at any time; there will be no negative consequences if you choose to do so, and you will receive the full compensation amount even if you choose to end the session early.

MITRE will take appropriate precautions to protect your privacy, as explained in the Privacy Act Statement and Privacy and Civil Liberty Impact Assessment. Your input today in the study will help the IRS create future products and services to better serve taxpayers such as yourself. I will now read the IRS Privacy Act Statement aloud to you: The authority requesting the information is 5 USC 301. The primary purpose of asking for the information is to determine steps the IRS can take to improve our services to you. The information may be disclosed as authorized by the routine uses published for the following Privacy Act System of Records: (1) IRS 00.001 Correspondence Files and Correspondence Control Files and (2) IRS 10.004 Stakeholder Relationship Management and Subject Files. Providing the information is voluntary. Not answering some or all of the questions will not affect you.

The Paperwork Reduction Act requires that the IRS display an OMB control number on all public information requests. The OMB Control Number for this study is 1545-2256.

Do you have any questions about this research?

[FACILITATOR: Pause to answer and note any questions.]

Do you agree to participate?

[FACILITATOR: If “no,” STOP recording, confirm with participant and end interview session.]

Do you consent to the use of the recording by MITRE?

[FACILITATOR: If “no,” STOP recording, confirm with participant and end interview session.]

Paperwork Reduction Act: OMB No: 1545-2256. This report is authorized under the Paperwork Reduction Act. Data collected will be shared with IRS staff, but your responses will be used for research and aggregate reporting purposes only and will not be used for other non-statistical or non-research purposes such as direct enforcement activities. The information that you provide will fully be protected allowable under the Freedom of Information Act (FOIA). Public reporting burden for this collection of information is estimated to average 120 minutes, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Special Services Section, SE:W:CAR:MP:T:M:S, Room 6129, 1111 Constitution Avenue, NW, Washington, DC 20224.

3. Survey and Diary

The CSP Usability Survey is divided into 4 sections. A dashboard with a progress meter will show participants their progress and allow repeated access to instructions and the voluntary consent. The following are illustrative screenshots:

1. Welcome

Your task for this study is to create an account with [CSP] and provide feedback on the experience. We will ask you to create a [CSP] account as if you were going to use it to connect to IRS online services.

In creating your [CSP] account, you are not creating an IRS account. Neither MITRE nor the IRS is affiliated with [CSP].

Before you get started on your task, find a space that is private and free of distractions, where you have access to an internet connection. You should plan to set aside approximately one hour to work on the survey. You are allowed to take breaks and complete the survey according to your own schedule, however, please note that you will be asked to provide an estimate for the time spent on two tasks. You may work on the survey and the task on any device you like. Be prepared to keep the survey open while you work on the task.

To complete the task, you will be asked to gather and use personal documents that verify you are who you say you are. For your personal privacy and data security, make sure your documents are organized and properly stored in a safe location before, during and after completing the task. MITRE has no access to any of your personally identifiable information at any point of this study. Instructions on deleting your account are provided in the [name] section of the survey, which may be accessed at any time.

[CSP description and link]

Please keep this survey page open and available as you work on the task. This survey comes in four steps, which must be completed in order. All four steps can be accessed from the home page at any time.

Step one, Registration Survey, must be filled out first.

Step two, Collect Identification Documents, occurs before creating your account with [CSP].

Step three, Register for an Account, is completed while you create your account with [CSP]. We ask that you attempt to complete part two on one sitting. Set aside [time range] for this.

Step four, Feedback Survey, is completed after you create your account with [CSP].

If you encounter any problems while completing Collect Identification Documents or Register for an Account, describe your problem in the survey and use any support options you might normally use from [CSP]. You are allowed to use any help options provided by [CSP], such as FAQs available on their website. You are allowed to research any questions you have online.

The MITRE research team is not affiliated with [CSP] but is available to answer some questions. You may send questions to the MITRE team through your [recruiter] point of contact.

If you are unable to create your [CSP] account, complete the survey within Register for an Account and then complete the Feedback Survey.

2. Registration Survey

Have you used an account such as a Google or Facebook account to log into a different website or app, for example, Spotify, Medium, a mobile phone game, etc.? A different website or app means a website or application that is different from the service providing the log-in. For example, using a Facebook account to log in to Spotify. Not using a Google account to log into Google. The option to log into third-party sites is typically done using buttons that look like:

• Yes

• No

• Unsure / Don’t know

A Credential Service Provider (CSP) identity proofs and issues electronic credentials to consumers for government and businesses. After identity proofing and setting up authentication, the experience of using a CSP is similar to using a Google or Facebook account to log into a different website or app. Note that companies like Google and Facebook are not a CSP since they do not truly verify your identity.

For the following questions, please imagine using a CSP for websites that contain sensitive information and where it would be very important to know that it is really you that is logging in. For example, using the online account for your bank to log in to other websites or applications like insurance, a mortgage or credit cards.

I find services like these easy to understand. [7-point scale:]

1	Strongly agree
2	Agree
3	Somewhat agree
4	Neither agree nor disagree
5	Somewhat disagree
6	Disagree
7	Strongly disagree
	Unsure / Don’t know
	No opinion

In a general sense (meaning, not specific to a certain company or organization), this method of logging in using a third party keeps my information secure. [7-point]

In a general sense (meaning, not specific to a certain company or organization), services that enable me to login are concerned about keeping my data secure. [7-point]

I find a service such as this useful. [7-point]

A service such as this saves me time when I use it. [7-point]

A service such as this does everything I would expect it to do. [7-point]

A service such as this is easy to use. [7-point]

I would feel comfortable using a similar service [again] in the future. [7-point]

3. Collect Identification Documents

Get your Identification Documents

Instructions: Before you create an IRS account using [CSP], collect all of the information you need to create the account in one sitting. Use the form below to record how long it takes to find the information you need and answer some questions about what you selected. Please make sure to keep track of your documents and keep them secured in a safe place. When you feel that you have time to do this, click on the button below.

I’m ready to get started. [button - click captures time]

CSP

Please select and gather either two primary documents, or one primary and two secondary documents. Note that you may not be asked to provide all documents you select when registering.

Primary Identification Documents

• [list]

Secondary Identification Documents

• [list]

I’m done! [button - click captures time]

About your Identification Documents

Approximately how long did you spend gathering your documents (in minutes)? [number select]

How easy or difficult was it to find and collect all the documents needed to create an IRS.gov account using [CSP]?

1	2	3	4	5	6	7
Very difficult	Difficult	Somewhat difficult	Neither difficult nor easy	Somewhat easy	Easy	Very easy

[C.04] What documents did you choose?

• CSP:

  • Primary (drop down)

  • Primary (drop down – includes “not applicable”)

  • Secondary (drop down – includes “not applicable”)

  • Secondary (drop down – includes “not applicable”)

• [Not asking this question for CSP]

From which state is your driver's license or state identification card issued? (State drop down)

How comfortable are you using the following documents to create an IRS.gov account using [CSP]? [7-point]

1	2	3	4	5	6	7
Very uncomfortable	Uncomfortable	Somewhat uncomfortable	Neither uncomfortable nor comfortable	Somewhat comfortable	Comfortable	Very comfortable

• [list]

Optional: Please describe any issues you encountered gathering the document required to create and [CSP] account. Please do not share any personal information or anecdotes that could be used to identify you. [free text]

4. Register for an Account

   1. Register Diary

Task: Please imagine that you need to sign up for an IRS account; for example, you need to create an account in order to review important information from the prior tax year. You choose to visit IRS.gov, where you see that the IRS has partnered with [CSP] to register secure accounts.

Your goal is to create a [CSP] account today. Please create your [CSP] with an IRS account in mind, however you are not creating an IRS account while working on this task. Neither MITRE nor the IRS is affiliated with [CSP]. When you have completed the task and surveys, you will be instructed to delete your newly created [CSP] account.

Use a “Note” to jot down notes if you are particularly pleased or frustrated at any time while working on creating an account using [CSP]. If you encounter any errors or items you find confusing, please use a “Diary Entry” to describe it.

Please note what time you start the task and if you pause for a break. Grab a sheet of paper to help with this if needed. You will be asked to estimate your time spent once you’ve completed the task. When you have completed the task, or reach a point where you cannot proceed, note the end time.

Find a quiet, private space with an internet connection and have all the documents you gathered on hand. Attempt to complete the task in one sitting. You are allowed to use any help options provided by [CSP], such as FAQs or [customer phone support]. You are allowed to research any questions you have online.

I’m ready to get started. [button - click captures time]

[R.02] Add diary entry [button - click captures time]

[R.02.02] Title on current page [drop down]

CSP

• [list of potential pages]

Note down your thoughts on what you are experiencing. Please do not share any personal information or anecdotes that could be used to identify you.

• What was good or easy about your experience? [free text]

• What was bad or difficult about your experience? [free text]

Submit Diary Entry

Can’t Proceed [button - click captures time]

Task Completed [button - click captures time]

2. Task Completed

Instructions: Now that you have an account, we’d like you to log out and complete one final task. Please log out of [CSP] and click “I’m ready to get started” when you are ready to work on the next task. Set aside approximately five minutes to complete the task.

I’m ready to get started. [button - click captures time]

Instructions: Log back in to [CSP] with your account.

Can’t Login [button - click captures time]

Task Completed [button - click captures time]

Overall, how difficult or easy was creating an IRS.gov account using [CSP] to complete? [7-point scale from very difficult to very easy]

1	2	3	4	5	6	7
Very difficult	Difficult	Somewhat difficult	Neither difficult or easy	Somewhat easy	Easy	Very easy

3. Can’t Proceed

Instructions: You indicated that you reached a point from which you are not able to proceed.

Please describe what issues you encountered with [CSP]. Please do not share any personal information or anecdotes that could be used to identify you. [free text]

CSP If you were unsuccessful multiple times, were you able to "talk to an agent" for assistance?

• Yes

• No

• Not Applicable

4. Task Success

Instructions: Reflect back on your experience creating an account using [CSP] and answer the following.

[CSP example step 1] Sign Up and Confirm Email

Overall, how difficult or easy was Sign Up and Confirm Email to complete? [7-point]

1	2	3	4	5	6	7
Very difficult	Difficult	Somewhat difficult	Neither difficult or easy	Somewhat easy	Easy	Very easy

Describe your experience with Sign Up and Confirm Email.

• What is good or easy about your experience? [free text]

• What is bad or difficult about your experience? [free text]

[Repeat for all CSP workflow steps]

5. Feedback Survey

Instructions: Please answer the following questions about your experience using [CSP].

UMUX-Lite [CSP] capabilities meet my requirements. [7-point]

1	Strongly agree
2	Agree
3	Somewhat agree
4	Neither agree nor disagree
5	Somewhat disagree
6	Disagree
7	Strongly disagree

[CSP] is easy to use. [7-point: 1 Strongly agree to 7 Strongly disagree]

Abbreviated CSP Perceptions Survey

A Credential Service Provider (CSP) identity proofs and issues electronic credentials to consumers for government and businesses. For the following questions, please respond based on your experience using [CSP] CSP to create an IRS account.

The method of logging in to an IRS account using [CSP] keeps my information secure. [7-point: 1 Strongly agree to 7 Strongly disagree with unsure / no opinion]

1	Strongly agree
2	Agree
3	Somewhat agree
4	Neither agree nor disagree
5	Somewhat disagree
6	Disagree
7	Strongly disagree
	Unsure / Don’t know
	No opinion

Branch on 6, 7: Optional: Please comment why: [free text]

A service such as [CSP] is concerned about making sure my data is secure. [7-point: 1 Strongly agree to 7 Strongly disagree with unsure / no opinion]

A service such as [CSP] does everything I would expect it to do. [7-point: 1 Strongly agree to 7 Strongly disagree with unsure / no opinion]

Based on my experience using [CSP] to log in to an IRS account, I would feel comfortable using a similar service again in the future, for IRS or other government websites. [7-point: 1 Strongly agree to 7 Strongly disagree with unsure / no opinion]

Technology Used

What type of device did you use to start creating an account with [CSP]?

• Smartphone / Mobile phone

• Desktop computer

• Laptop computer

• Tablet

If you chose to use a desktop or laptop computer, what internet browser did you use?

• Chrome

• Safari

• Firefox

• Internet Explorer

• Edge

• Other

• Not applicable

What brand of Smartphone do you own?

• Apple

• Samsung

• LG

• Lenovo

• OnePlus

• Google Pixel

• Other

Authentication

Which login option did you use?

• [options]

Which second factor authentication option(s) did you select?

• [options]

Identity Proofing

What did you use to take a picture of your driver's license, state identification card, passport, or passport card?

• Smartphone / Mobile phone camera

• Desktop computer webcam

• Laptop computer webcam

• Tablet camera

Overall, how difficult or easy was uploading your selfie to complete? [7-point scale from very difficult to very easy]

Did you have to edit the information [CSP] captured from your driver's license or state identification card?

• No

• Yes

Technical Support

Did you look up information online about [CSP] prior to creating an account?

• No

• Yes

Overall, how difficult or easy was finding technical support for [CSP]? [7-point scale from very difficult to very easy]

Optional: Please describe any technical support you used while creating an [CSP] account. This includes anything from searching for information to calling a customer support line. [free text]

IRS Online services

Please select any services you have used on IRS.gov in the past:

• Get Transcript

• IRS Free File

• Authorized e-file Provider

• Volunteer Income Tax Assistance Locator

• Directory of Federal Tax Return Preparers

• Where’s My Amended Return?

• Where's My Refund?

• View Tax Account

• Direct Pay

• Online Payment Agreement

• Offer in Compromise Pre-Qualifier

• Credits and Deductions

• Earned Income Tax Credit Assistant

• Exempt Organizations Select Check

• First Time Homebuyer Credit (FTHBC) Account Look-up

• Tax Withholding Estimator

• Sales Tax Calculator

• Alternative Minimum Tax Assistant

• None

What is your preference for creating an account with the IRS? For example, in order to review your payment history and other important information from the current tax year.

• Online, using a third party like [CSP] to register an account

• Online, using IRS to register an account

• On the phone, with an IRS customer service representative

• In person, at an IRS taxpayer assistance center

Please comment on your choice preference for creating an account with the IRS. [free text]

6. Thank You and Wrap Up

• Thank you

• You are allowed to review your answers

• Contact [name] at [Recruitment Company] with questions

4. Interview Facilitator Script

   1. Introduction

• Welcome participant

• Provide researcher and notetaker names and affiliation

• Topic of research, sponsor name (IRS): We are conducting research sponsored by the IRS on a potential new method to register for online services.

• High-level overview of the session: I would like to use our time today to discuss the survey itself and talk more in depth about your experience using [CSP]. Just a quick reminder, we are not affiliated with [CSP]. Your input today will help the IRS create future products and services to better serve taxpayers such as yourself.

• Consent: Before we get started, I’d like to read our informed consent agreement to you and record your decision. If you choose not to consent, I will delete the recording. Ready?

START SKYPE RECORDING

2. Consent for Recording

See Appendix B.2 Interview Informed Consent (verbal).

3. Feedback Interview

Thank you! I’d like to start with a few questions about the survey itself.

[INT.1] Approximately how long did it take for you to complete the entire survey, including registering with [CSP]?

[INT.2] Were there any portions of the survey itself that were overly complicated or difficult to complete?

[FACILITATOR: If “yes,” ask participant to recall those sections and ask any required follow up to confirm the issue(s) encountered.]

[INT.3] Do you have any additional feedback or suggestions on the survey itself?

Ok, thank you! Now I would like to discuss your experience with [CSP].

[INT.4] How would you describe your overall experience using [CSP]?

[INT.5] You were asked to provide [required documents for system]. What was it like for you to get everything required to create an account? [Probe on any specific issues identified within survey responses.]

[INT.6] If you found yourself in a situation where you needed to create an account on IRS.gov, how would the requirement to use [required documents for system] affect your decision?

[FACILITATOR: in advance of session, select diary entries of interest and prepare brief summaries. If the participant encountered any major issues, prepare questions to confirm details of the issue(s). If there were no diary entries, skip.]

[INT.7] I’d like to learn more about the notes you took while creating an account. You described [summary of diary]. Can you tell me more about what you experienced here?

[FACILITATOR: Only ask the following if the participant could not complete registration.]

[INT.8] You reported that you got to [workflow step]. Can you tell me more about what you encountered at this step? [Ask probing questions to learn what occurred and how it affected their experience.]

[INT.9] You answered [participants answer on 7-point scale] for the following question: “Based on my experience using [CSP] to log in to an IRS account, I would feel comfortable using a similar service again in the future, for IRS or other government websites.” Can you tell me more about how you feel using a service like [CSP] to access your personal information on IRS.gov?

[INT.10] What do you think about using a third party to log into an IRS account?

[INT.11] Would you feel more comfortable using a CSP operated by the US federal government or a private company?

[INT.12] Is there anything important I missed about your experience with [CSP]? [If yes, ask any probing questions needed to clarify.]

4. Closing

• Thank the participant.

• Answer any additional questions.

• Provide timeline on incentive the Recruitment Company contact details.

• Close the session.