SOI-476 Consumer Tipping Usability Study

Supporting Statement

Information Collection (ICR) Approval Request to Conduct Customer Satisfaction Research (OMB #1545-1349)

Title: Consumer Tipping Survey Usability Study

1. JUSTIFICATION

1. Circumstances Making the Collection of Information Necessary

The IRS is charged with collecting revenue legally owed to the federal government. One important category of income comes in the form of tips. Previous empirical research has shown income from tips is significantly underreported¹, limiting IRS’s ability to collect the proper amount of tax. Since three decades have passed since the last (and only) study of tip income reporting was conducted, the IRS believes a new study of consumer tipping practices is needed in order to better understand current tip reporting behavior so tax administrators and policy makers can make the tax system fairer and more efficient. In support of this mission, the IRS is seeking clearance to conduct a usability test in preparation of the fielding of a consumer tipping survey.

This initiative flows from Goal 1 of the IRS Strategic Plan for FY 2014-2017: Deliver high quality and timely service to reduce taxpayer burden and encourage voluntary compliance. This is also a result of Executive Order 12862 (Setting Customer Service Standards, issued on September 11, 1993) that requires all government agencies to survey their customers. Pursuant to Executive Order 12862, agencies that provide significant services to the public must survey customers to determine the kind and quality of services they want and their level of satisfaction with existing services. Executive Order 13571 (Streamlining Service Delivery and Improving Customer Service, issued on April 27, 2011) expands the definition of "customer" and encourages the use of a broader set of tools to solicit actionable, timely customer feedback to capture insights and identify early warning signals. As used in Executive Order 13571, the term "customer" refers to any individual or to any entity, including a business, tribal, state or local government, or other agency, to which the agency directly provides significant services.

2. Purpose and Use of the Information Collection
   The IRS will conduct usability testing of a prototype of a web-based consumer tipping survey. The IRS will focus on developing a set of questions which maximizes the usefulness of individual responses while minimizing respondent burden. Specifically, the IRS will design tasks and questions that help to determine the optimal question wording, layout, question hierarchy/dependencies, and recall period for the consumer tipping survey. To do so, the IRS will test prototypes to assess users’ understanding of the questions as well as how users perceive the concepts.

3. Consideration Given to Information Technology
   The usability study will be conducted in two parts. In the first week, respondents will perform cognitive interviews with mock versions of the survey tool on paper in order to determine if the survey language and format is unclear or ambiguous. Immediate changes will be made to questions based on respondent feedback in order to refine the final survey language.
   
   In the second and third weeks, respondents will complete updated drafts of the web-based survey which have been revised based on feedback obtained from the prior week’s cognitive testing. Various usability measures, including eye-tracking technology, completion time, and error rates will be collected during this phase. Recall length will be a focus in the second phase in order to determine the best period of recall to suggest for the pilot study of this project. Respondents will only be eligible to participate in a single session of testing for this experiment, so no one respondent could participate at multiple points of the experiment.

4. Duplication of Information
   This study is the only one investigating the usability of a web-based tipping study for the IRS.

5. Reducing the Burden on Small Entities
   N/A

6. Consequences of Not Conducting Collection
   Failure to conduct usability testing prior to launching the pilot study could result in poor survey design that yields poor quality data due to inaccurate recall length, confusing or ambiguous questions, or incomplete service categories, among other reasons. The ability to draw policy lessons regarding the true nature of tip underreporting would be compromised by using a survey instrument that had not been given proper usability testing.

7. Special Circumstances
   There are no special circumstances. This usability effort is completely voluntary and is in support of a pilot tipping study that will be conducted at a later time.

8. Consultations with Persons Outside the Agency
   IRS is working with Fors Marsh Group (FMG) to conduct the usability testing.

9. Payment or Gift
   FMG will be providing an incentive of $40 to participants they recruit for usability testing. While sessions will last about 45 minutes, participants are required to arrive 10 minutes early to sign in, and to ensure sessions begin on time. Upon completion of the session, participants are required to sign out and receive their incentive. Thus, they are in the office for about 60 minutes. Further, many participants have to travel 30-60 minutes to and from the facility.

10. Confidentiality
    
    

The security of the data used in this project and the privacy of survey respondents will be carefully safeguarded at all times. Security requirements are based on the Computer Security Act of 1987 and Office of Management and Budget Circular A-130, Appendices A & B.  Physical security measures include a locked, secure office.  Notes are stored in locked cabinets or shredded. 

The IRS will apply and meet fair information and record-keeping practices to ensure privacy protection of all respondents to the extent allowed by law.  This includes criteria for disclosure—laid out in the Privacy Act of 1974, the Freedom of Information Act, and Section 6103 of the Internal Revenue Code—all of which provide for the protection of respondent information as well as its release to authorized recipients.

The survey data will not contain any respondent names, Social Security Numbers, or Taxpayer Identification Numbers. Participants will not be identified in any of the documents or files used for this project.  FMG will limit and control the amount of information we collect to those items that are necessary to accomplish the research questions.  FMG will carefully safeguard the security of data utilized as well as the privacy of the survey respondents to the extent allowed by law.  FMG will apply the fair information and record-keeping practices to ensure protection of all survey respondents. The criterion for disclosure laid out in the Privacy Act, the Freedom of Information Act, and section 6103 of the Internal Revenue Code provides for the protection of information as well as its releases to authorized recipients.

11. Sensitive Nature
    No questions will be asked that are of a personal or sensitive nature.

12. Burden of Information Collection

The IRS will attempt to obtain 35 total participants over the course of three weeks. Roughly 200 participants will likely read one form of the invitation of the study, between social media, local, and panel postings. It is expected that a relatively small percentage (35%) of those 200 will likely contact FMG to complete the web-based screener. Roughly 70 participants will need to complete a web-based screener in order to gather the required 35 final participants, as there are no restrictions on participation other than the respondent being at least 18 years of age. Once the web-based screener is completed, respondents will be contacted via email to schedule appointments. Some respondents will drop out at this point due to scheduling conflicts or fail to show for sessions, requiring 70 screened participants to gather the final group of 35 completed sessions. Phone calls will be used to confirm appointment times with respondents, if necessary. WIRA estimates that the total number of burden hours for this project will come to 33.08 hours.

The breakdown is as follows:

Type of Collection	Participation	Response Time (minutes)	Total Burden (Hours)
Read recruitment material	200	1	3.33
Web-Based Screener	70	3	3.5
Usability Sessions	35	45	26.25
Grand Total (column 4)			33.08 hrs.

13. Costs to Respondents

Transportation costs including fuel, bus/metro fare, etc. could be incurred by the respondent but these are factored into the incentive offered by FMG.

14. Costs to Federal Government

The estimated cost is $51,615. This cost is a portion of the total contract cost, which includes survey questionnaire development, development of a pilot study research and analysis plan, and purchase and analysis of electronic point of sale (POS) data.

15. Reason for Change

Not applicable.

16. Tabulation of Results, Schedule, Analysis Plans
    The final report will cover a number of issues, including changes that were made to the survey tool for the pilot study throughout testing and various usability metrics (eye-tracking information and error-rates). The final report is due to IRS no later than December 13, 2014.

17. Display of OMB Approval Date

Not applicable

18. Exceptions to Certification for Paperwork Reduction Act Submissions

These activities comply with the requirements in 5 CFR 1320.9.

19. Dates Collection of Information will Begin and End

The data collection period for the usability study is scheduled for October 15, 2014 through November 30, 2014.

B. STATISTICAL METHODS

1. Universe and Respondent Selection

The opportunity to participate will be advertised in the local community and online social media. Community postings will include such methods as posting flyers in local coffee shops and similar establishments with tear-away slips that participants can use to contact FMG if interested. Social media postings will be an advertisement on Craigslist asking for research participants. Individuals who have opted into contact databases maintained by Fors Marsh Group will be notified via email about the opportunity to participate. Individuals who express an interest in participating will be emailed the web-based screener. If they qualify based on responses to the web-based screener, they will be contacted via email to confirm an appointment. Phone calls will be used to confirm appointment times, when necessary. All individuals over the age of 18 will be considered qualified to participate in the study and will be scheduled via email communication. The final sample of participants will be a convenience sample, and will not necessarily be representative of the population. However, close observation of their behavior during the usability tests as well as pre- and post-test interviews should generate insights about the effects of survey design on participant burden which can generalize to the population of interest.

2. Procedures for Collecting Information

Data will be recorded by the moderator as well as by video recording and eye tracking devices after obtaining participant consent.

The usability study will be conducted in two parts. In the first week, respondents will perform cognitive interviews with mock versions of the survey tool on paper in order to determine if the survey language and format is unclear or ambiguous. Immediate changes will be made to questions based on respondent feedback in order to refine the final survey language.

In the second and third weeks, respondents will complete updated drafts of the web-based survey which have been revised based on feedback obtained from the prior week’s cognitive testing. Various usability measures, including eye-tracking technology, completion time, and error rates will be collected during this phase. Recall length will be a focus in the second phase in order to determine the best period of recall to suggest for the pilot study of this project. Respondents will only be eligible to participate in a single session of testing for this experiment, so no one respondent could participate at multiple points of the experiment.

The IRS will attempt to obtain 35 total participants over the course of three weeks. Roughly 200 participants will likely read one form of the invitation of the study, between social media, local, and panel postings. It is expected that a relatively small percentage (35%) of those 200 will likely contact FMG to complete the web-based screener. Roughly 70 participants will need to complete a web-based screener in order to gather the required 35 final participants, as there are no restrictions on participation other than the respondent being at least 18 years of age. Once the web-based screener is completed, respondents will be contacted via email to schedule appointments.

3. Methods to Maximize Response

Participants have already opted into FMG’s list of potential Usability participants or have contacted FMG after have read one of the community-based flyers. In addition, the incentive that FMG offers offsets transportation expenses and compensates them for their time.

4. Testing of Procedures

N/A

5. Contacts for Statistical Aspects and Data Collection

For questions regarding the study or questionnaire design or statistical methodology, contact:

Brian K. Griepentrog, Ph.D.

Director of Research Studies

Fors Marsh Group LLC

[email protected]

Survey instruments include the following and are attached in a separate file.

• Survey questionnaire

• Moderator instructions for usability study

• Local, online, and email advertisements and recruitment materials

• Web-based participant screener

• Informed Consent Form

1 See IRS Publication 1530 (1990), Tip Income Study, “A Study of Tipping Practices in the Food Service Industry for 1984” and Robert B. Pearl and Kevin F. McCrohan, (1983), Estimates of Tip Income in Eating Places, 1982, IRS Research Bulletin, pp. 49-53.

5