OMB #1545-1349
SUPPORTING STATEMENT
Approval Request to Conduct Customer Satisfaction Research
W&I CAS Adjustments FY2011 Usability Testing Research
W&I CAS Adjustments FY2011 Usability Testing Research
A. JUSTIFICATION
Circumstances Making the Collection of Information Necessary
The Internal Revenue Service (IRS) enlists a balanced measurement system consisting of business results, customer satisfaction, and employee satisfaction. This initiative is part of the Service-wide effort to maintain a system of balanced organizational performance measures mandated by the IRS Restructuring and Reform Act (RRA) of 1998. This is also a result of Executive Order 12862 that requires all government agencies to survey their customers.
The Adjustments operation within the Customer Accounts Services (CAS) Operating Unit of the W&I Business Operating Division (BOD) is responsible for responding to written customer technical and account inquiries, resolving customer account issues, providing account settlement (payment options), and working related issues. W&I require feedback from customers to continually improve the Adjustments operations. W&I solicit this feedback through customer satisfaction research.
IRS recently revised the Adjustments Customer Satisfaction Survey, and this research is needed to ensure that the revised instrument is respondent friendly.
Purpose and Use of the Information Collection
The objective of this task is to do usability testing on the CAS Adjustments Customer Satisfaction Survey.
The key goal of this research is to determine whether or not changes are needed to improve the flow of the survey and the reader’s comprehension of its language.
Consideration Given to Information Technology Sample Design
This task will be conducted to supplement the information and results found during the main data collection phase of the Adjustments Customer Satisfaction Survey. This task will provide richer detail to Adjustments on their customer satisfaction survey and their customers’ experience with the program. IRS will be testing the current survey as well as gaining insight on the taxpayers’ experiences with the Adjustments Program.
To provide this critical information, our contractor, Fors Marsh Group (FMG) will conduct usability testing. In usability testing, participants interact with a product (e.g., a website, survey, form) as they think aloud, and the researcher assesses the effectiveness (ability to complete tasks/goals), efficiency (time to complete tasks/goals), and satisfaction of users.
The FMG Team will conduct remote usability testing (see explanation below) to facilitate in-depth qualitative analysis and ensure taxpayer experiences and needs are understood and integrated into the research program. The testing will also serve to ensure that the current survey reflects customer concerns and the wording facilitates comprehension. This testing will also ensure that the current survey reflects the customer experience and comprehension (e.g., includes “taxpayer friendly” terminology rather than industry jargon).
Recruitment for the usability testing will be performed at the conclusion of survey transactions, asking participants their willingness to participate in future research. FMG will make follow-up phone calls with recruited participants to ensure the desired number of participants is recruited. FMG will recruit participants with varying characteristics such as customer type, census region, and/or population density (high, medium, low).
All usability testing will be led by an experienced FMG Team facilitator, who will lead discussions on established activities defined within a moderator guide. A second team member will participate in all usability sessions and will assist the facilitator, take notes, and support reporting and analysis efforts. The overall approach to this testing is structured and clinical. The moderator guide will be pre-tested by FMG staff. The IRS will have the opportunity to observe all usability tests via live streaming.
The FMG Team will analyze the data, interpret the findings, and present the findings. Findings and analyses will indicate the usability issues with the survey in terms of quantitative (e.g., percent completion, time to complete, satisfaction ratings to items) and qualitative (e.g., participant positive, negative, and open-ended comments) data, as well as the prevailing opinions of the participants. FMG will focus on maximizing the quantitative measurement, and specifically, will highlight recommendations to improve programs, procedures, and future research efforts based on actual end user (i.e., taxpayer) input. Recommendations will be prioritized (high, medium and low) based on participants’ difficulties and the relative importance to the IRS. FMG will deliver a draft template of the report, and a revised report within five (5) business days of receiving IRS comments.
Remote Usability Testing. The FMG Team conducts remote usability testing with participants who live in geographically diverse areas. To accomplish this, FMG will use screen sharing software that allows them to interact with the participant. A view of the participant can also be captured when the participant has a webcam attached to their computer desktop or laptop. Adjustments customers can participate remotely from anywhere in the country by using a program such as the GoToMeeting™ online platform. In addition to the many benefits to the participant, this technology has a wide variety of advantages for researchers:
Representativeness: Sample comprised of customers around the country rather than around a city
Cost: Fewer resources spent on travel and facility costs and more spent collecting customer data
Timing: More sessions can be conducted in shorter times with less set-up and travel time
Moderator control: Moderators can facilitate conversation, show videos and other multimedia, while also letting participants show their screens
Client Access: Clients can watch and listen to sessions remotely anywhere in the world.
4. Efforts Not to Duplicate Research
This will be the only usability research associated with the 2012 Adjustments survey and the Adjustments survey is the only study currently conducted that provides the opportunity to hear from Adjustments customer base on a broad range of customer service issues, with the objective of identifying how to prioritize strategies for improving customer satisfaction.
5. Reducing the Burden on Small Entities Expected Response Rate
The usability research has been designed to minimize burden on respondents. The time to participate in the study has been carefully considered. Each respondent will be allowed to participate in only one usability study. This will aid in decreasing respondent burden.
6. Consequences of Not Conducting Collection
Without conducting the Adjustments survey, CAS would be unable to meet corporate goals and would fall short of meeting the IRS Mission of a Balanced Measurement System. Without the usability research, the survey will be less effective at providing IRS with information about Adjustments customers. The usability research provides an opportunity to drill down into why respondents gave particular survey responses and will add the richness of a qualitative component to the overall research endeavor.
7. Special Circumstances
There are no special circumstances. The information collected will be voluntary and will not be used for statistical purposes.
8. Consultations with Persons Outside the IRS
The same contractor that administered the Adjustments survey will conduct this usability research.
9. Payment of Gift
$40 per participant. Without this payment of gift, the contractor may not be able to attract subjects to participate in this study. Offering incentives for participating in this type of research is a standard industry practice.
10. Confidentiality
All participants will be subject to the provisions of the Taxpayer Bill of Rights II during this study and the vendor will ensure that all participants are treated fairly and appropriately.
The security of the data used in this project and the privacy to the extent allowed by law of taxpayers will be carefully safeguarded at all times. Security requirements are based on the Computer Security Act of 1987 and Office of Management and Budget Circular A-130, Appendices A & B. Physical security measures include a locked, secure office. Notes are stored in locked cabinets or shredded. Data security at the C-2 level is accomplished via the Windows NT operating system. Systems are password protected, users profiled for authorized use, and individual audit trails generated and reviewed periodically.
The IRS will apply and meet fair information and record-keeping practices to ensure privacy to the extent allowed by law protection of all taxpayers. This includes criteria for disclosure—laid out in the Privacy Act of 1974, the Freedom of Information Act, and Section 6103 of the Internal Revenue Code—all of which provide for the protection of taxpayer information as well as its release to authorized recipients.
The usability research will not contain or collect tax return or taxpayer information. Usability research participants will not be identified in any of the documents or files used for this project. We will limit and control the amount of information we collect to those items that are necessary to accomplish the research questions. We will carefully safeguard the security of data utilized as well as the privacy to the extent allowed by law of the respondents. We will apply the fair information and record-keeping practices to ensure protection of all survey respondents. The criterion for disclosure laid out in the Privacy Act, the Freedom of Information Act, and section 6103 of the Internal Revenue Code provides for the protection of information as well as its releases to authorized recipients.
11. Sensitive Nature
No questions will be asked that are of a personal or sensitive nature.
12. Burden of Information Collection
The usability research has been designed to minimize burden on the respondents. The time that a respondent takes to participate in the usability research has been carefully considered and only the most important areas are being discussed. The respondent will participate in only one usability research study. This will aid in decreasing respondent burden. We estimate contacting 52 participants to recruit 10 needed for the usability sessions. The total burden estimate for recruiting and administering the 10 usability sessions is 12.4 hours.
BURDEN HOURS
Category of Respondent |
No. of Respondents |
Participation Time |
Burden Hours |
Recruiting 13 participants |
13 |
5 minutes |
1.08 |
Recruiting 39 non-participants |
39 |
2 minutes |
1.3 |
Participation by 10 participants |
10 |
60 minutes |
10 |
Totals |
|
|
12.38 hrs |
Total minutes = 743 minutes or 12.38 hours
Total Burden = 12.38 hrs
13. Costs to Respondents
N/A
14. Costs to Federal Government
$10,404.80
15. Reason for Change
N/A
16. Tabulation of Results, Schedule, and Analysis Plans
Feedback collected provides useful information, but it does not yield data that can be generalized to the overall population. Findings will be used for survey improvement, but are not for publication or other public release.
The recruiting of the sample is the responsibility of the contractor.
The 10 participants will provide qualitative insights to supplement survey responses.
The contractor will use the analysis of this qualitative information to augment the information obtained from of the survey responses and to improve the usability of the survey.
The contractor will hold the identities of respondents anonymous to the extent permitted by law. The contractor will not provide the IRS with data or status updates that are linked to individual respondents. This data will not include any individually identifying information such as name, address, or taxpayer identification number.
17. Display of OMB Approval Date
N/A
18. Exceptions to Certification for Paperwork Reduction Act Submissions
N/A
19. Dates collection will begin and end
Collection will begin as early as August 1, 2012 and will end prior to December 1, 2012.
B. STATISTICAL METHODS
Data collection will not apply statistical methods. A qualitative analysis of respondents’ experiences with the survey will be performed. There are no plans to publish or otherwise release this information.
Universe and Respondent Selection
The sampling frame consists of Adjustments customers.
Procedures for Collecting Information
The contractor will recruit participants and will be responsible for establishing procedures for data collection. The contractor will prepare the data and conduct data analysis in accordance with accepted industry standard procedures.
The contractor will conduct usability testing by telephone. The contractor will recruit 10 participants; that quantity is sufficient to discover ways to make the survey clearer and more user friendly.
Methods to Maximize Response
Standard procedures will be used in order to obtain the highest response rate possible for the usability testing. The contractor will recruit usability testing participants and conduct the usability testing research. To encourage participation, the contractor will be offering a $40 incentive.
Testing of Procedures
The purpose for this OMB approval application is to conduct usability testing research on the CAS Adjustments survey.
Contacts for Statistical Aspects and Data Collection
For questions regarding the usability test, contact:
Jeffery McConnell, COR
Program Analyst
401 West Peachtree St, Suite 2030
Atlanta, GA 30308
Telephone: 404-338-9056
E-Mail: [email protected]
The Paperwork Reduction Act requires that the IRS display an OMB control number on all public information requests. The OMB Control Number for this study is 1545-1349. Also, if you have any comments regarding the time estimates associated with this study or suggestions on making this process simpler, please write to the, Internal Revenue Service, Tax Products Coordinating Committee, SE:W:CAR:MP:T:T:SP, 1111 Constitution Ave. NW, Washington, DC 20224.
Attachments
A: Usability Testing Screener’s Guide
B: Usability Testing Moderator’s Guide
| File Type | application/msword |
| File Title | OMB SUPPORTING STATEMENT |
| Author | sherri.a.settle |
| Last Modified By | mdsloa00 |
| File Modified | 2012-06-12 |
| File Created | 2012-06-07 |