Supporting Statement

Request for Approval under the “Generic Clearance for the Collection of Routine Customer Feedback” (OMB Control Number: 1545-1432)

TITLE OF INFORMATION COLLECTION: Automated Chatbot Interaction Survey

PURPOSE:

This survey measures how well IRS chatbots serve taxpayers across eight automated services. We collect feedback from people who use our chat services for Refunds, Amended Returns, Identity Theft, Employer Identification Numbers, Change of Address, Advance Child Tax Credit, Earned Income Tax Credit, and Extension to File issues.

The data helps us understand what works well and what frustrates taxpayers when they interact with our automated systems. We use this feedback to improve chatbot responses, fix common problems, and make the overall experience smoother. Better chatbots mean taxpayers can resolve issues faster without waiting on hold or visiting our offices. This survey directly supports our goal of providing excellent digital service that meets taxpayers where they are.

DESCRIPTION OF RESPONDENTS:

Our respondents are taxpayers who recently completed interactions with any of our eight IRS chatbot services. These users span all income levels, geographic regions, and tax situations—from individuals checking refund status to small business owners getting employer identification numbers.

The respondent pool includes both frequent IRS users and people who rarely interact with us. Some respondents use simple chatbot functions like checking account balances, while others navigate complex issues like identity theft resolution. This diversity ensures we capture feedback across the full spectrum of taxpayer needs and technical comfort levels.

Respondents range from young adults filing their first tax returns to retirees managing estate issues. They access chatbots through computers, tablets, and smartphones, giving us insights into how our services work across different devices and technical environments. Geographic distribution mirrors national tax filing patterns, with higher volumes from populous states and metropolitan areas.

TYPE OF COLLECTION: (Check one)

[ ] Customer Comment Card/Complaint Form [X ] Customer Satisfaction Survey

[ ] Usability Testing (e.g., Website or Software [ ] Small Discussion Group

[ ] Focus Group [ ] Other: ______________________

CERTIFICATION:

I certify the following to be true:

1. The collection is voluntary.

2. The collection is low-burden for respondents and low-cost for the Federal Government.

3. The collection is non-controversial and does not raise issues of concern to other federal agencies.

4. The results are not intended to be disseminated to the public.

5. Information gathered will not be used for the purpose of substantially informing influential policy decisions.

6. The collection is targeted to the solicitation of opinions from respondents who have experience with the program or may have experience with the program in the future.

Name:_ _signed Ben Shackleford________________________________________

To assist review, please provide answers to the following question:

Personally Identifiable Information:

1. Is personally identifiable information (PII) collected? [X] Yes [ ] No

2. If Yes, is the information that will be collected included in records that are subject to the Privacy Act of 1974? [X ] Yes [ ] No

3. If Applicable, has a System or Records Notice been published? [ X ] Yes [ ] No

Gifts or Payments:

Is an incentive (e.g., money or reimbursement of expenses, token of appreciation) provided to participants? [ ] Yes [ X] No

BURDEN HOURS

Category of Respondent: Individuals	No. of Respondents	Participation Time	Burden
Soliciting Potential Participants	1,201,251	1 minute	20,020
Expected Participants	30,848	2 minutes	1,028
Totals	30,848		1,028

FEDERAL COST: The estimated annual cost to the Federal government is $42,848

STATISTICAL METHOD:

We use stratified random sampling with proportional allocation to ensure representative feedback from all eight chatbot services. Our annual target is 30,848 responses, distributed based on each chatbot's usage volume and seasonal patterns.

Sample Allocation: The largest service (Refunds) handles 64% of interactions and receives 19,719 response targets. Medium-volume services like Amended Returns get 2,804 targets, while smaller services like Extension to File receive 996 targets. This proportional approach ensures adequate sample sizes for reliable analysis while reflecting actual usage patterns.

Invitation Strategy: We send approximately 1.2 million invitations annually, expecting response rates between 2.26% and 2.69% based on pilot testing and chatbot type. Services used during tax filing season (February-May) typically see higher participation because taxpayers are more engaged with tax issues during this period.

Precision Targets: This design achieves quarterly margins of error between ±1% to ±4.5% for individual chatbots, depending on usage volume and season. Monthly aggregate analysis across all chatbots maintains ±2% precision year-round.

Weighting Methodology: We apply design weights to adjust for different selection probabilities across chatbots and time periods. Base weights equal the inverse of selection probability within each stratum (chatbot type × quarter). We then apply non-response adjustments using chatbot usage patterns, demographics when available, and seasonal factors. Final weights ensure our results properly represent the full population of chatbot users, not just survey respondents.

Analysis Plan:

• Descriptive Statistics: We calculate satisfaction scores, completion rates, and demographic breakdowns for each chatbot service, all properly weighted to reflect population usage patterns.

• Regression Analysis: We use ordinary least squares (OLS) regression with survey weights to test relationships between user characteristics, chatbot features, and satisfaction outcomes. This helps identify which factors most influence user experience.

• Trend Analysis: Monthly and quarterly weighted comparisons track service improvements over time and identify seasonal patterns in user satisfaction.

Text Analysis: For open-ended feedback, we employ Natural Language Processing through Structural Topic Modeling. This computer-assisted method automatically identifies common themes and concerns in thousands of text responses without human bias. The model discovers hidden topics—like "wait times," "confusing language," or "helpful features"—and shows how frequently respondents mention each theme. This approach lets us quickly spot emerging issues and track whether problems improve after we make system changes.

If you are conducting a focus group, survey, or plan to employ statistical methods, please provide answers to the following questions:

The selection of your targeted respondents

1. Do you have a customer list or something similar that defines the universe of potential respondents and do you have a sampling plan for selecting from this universe? [x] Yes [ ] No

If the answer is yes, please provide a description of both below (or attach the sampling plan)? If the answer is no, please provide a description of how you plan to identify your potential group of respondents and how you will select them?

Universe Definition: Our sampling frame includes all taxpayers who complete interactions with any of our eight chatbot services during the survey period. The IRS chatbot dashboard automatically captures each session, creating a complete record of approximately 4.8 million annual interactions. We exclude incomplete sessions, test interactions, and system errors to ensure we only survey genuine users.

Stratification Strategy: We stratify our universe by chatbot type and time period to account for dramatic seasonal variations. Tax filing season (February-May) generates 57% of annual activity, while summer months see much lower usage. Each chatbot also has distinct usage patterns—Extension to File peaks in October, while Refunds dominates January through April.

Selection Process: Within each stratum, we use systematic random sampling with quarterly selection periods. Our system generates randomized invitation lists every three months, with sample sizes calculated to meet precision targets while adjusting for expected response rates. For example, during filing season we increase sample sizes for chatbots that typically see lower response rates, ensuring adequate data for analysis.

Quality Controls: We monitor invitation delivery, track response patterns, and adjust sampling rates if needed to maintain target precision levels. The system prevents duplicate invitations and allows respondents to opt out of future surveys. We also validate that our achieved sample matches the target population demographics and usage patterns.

This systematic approach ensures every chatbot user has a known, calculable chance of selection while maintaining proportional representation across all service types and time periods.

Administration of the Instrument

1. How will you collect the information? (Check all that apply)

[x ] Web-based or other forms of Social Media

[ ] Telephone

[ ] In-person

[ ] Mail

[ ] Other, Explain – Microsoft TEAMS or ZOOM for sharing visuals.

2. Will interviewers or facilitators be used? [ ] Yes [x ] No

Please make sure that all instruments, instructions, and scripts are submitted with the request.

Instructions for completing Request for Approval under the “Generic Clearance for the Collection of Routine Customer Feedback”

TITLE OF INFORMATION COLLECTION: Provide the name of the collection that is the subject of the request. (e.g. Comment card for soliciting feedback on xxxx)

PURPOSE: Provide a brief description of the purpose of this collection and how it will be used. If this is part of a larger study or effort, please include this in your explanation.

DESCRIPTION OF RESPONDENTS: Provide a brief description of the targeted group or groups for this collection of information. These groups must have experience with the program.

TYPE OF COLLECTION: Check one box. If you are requesting approval of other instruments under the generic, you must complete a form for each instrument.

CERTIFICATION: Please read the certification carefully. If you incorrectly certify, the collection will be returned as improperly submitted or it will be disapproved.

Personally Identifiable Information: Provide answers to the questions.

Gifts or Payments: If you answer yes to the question, please describe the incentive and provide a justification for the amount.

BURDEN HOURS:

Category of Respondents: Identify who you expect the respondents to be in terms of the following categories: (1) Individuals or Households;(2) Private Sector; (3) State, local, or tribal governments; or (4) Federal Government. Only one type of respondent can be selected.

No. of Respondents: Provide an estimate of the Number of respondents.

Participation Time: Provide an estimate of the amount of time required for a respondent to participate (e.g. fill out a survey or participate in a focus group)

Burden: Provide the Annual burden hours: Multiply the Number of responses and the participation time and divide by 60.

FEDERAL COST: Provide an estimate of the annual cost to the Federal government.

If you are conducting a focus group, survey, or plan to employ statistical methods, please provide answers to the following questions:

The selection of your targeted respondents. Please provide a description of how you plan to identify your potential group of respondents and how you will select them. If the answer is yes, to the first question, you may provide the sampling plan in an attachment.

Administration of the Instrument: Identify how the information will be collected. More than one box may be checked. Indicate whether there will be interviewers (e.g. for surveys) or facilitators (e.g., for focus groups) used.

Please make sure that all instruments, instructions, and scripts are submitted with the request.