IRS Tipping Report

IRS Tipping Report on Cognitive
and Usability Testing

Prepared for Internal Revenue Service
Prepared by Fors Marsh Group LLC

January 2015
Version 1.1

The views, opinions, and/or findings contained in this report are those of Fors Marsh Group LLC and should not
be construed as official government position, policy, or decision unless so designated by other documentation.
This document was prepared for authorized distribution only. It has not been approved for public release.

Table of Contents
Executive Summary of Findings and Survey Changes ............................................................................... 3 
Cognitive Testing .......................................................................................................................................... 5 
Design and Lab Setup ............................................................................................................................. 5 
Cognitive Test Findings ............................................................................................................................ 5 
Language and Response Option Findings .............................................................................................. 6 
Other Findings .......................................................................................................................................... 8 
Usability Testing ......................................................................................................................................... 10 
Design and Lab Setup ........................................................................................................................... 10 
Usability Testing Findings ...................................................................................................................... 12 
Survey Confusion ................................................................................................................................... 12 
Language and Response Option Findings ............................................................................................ 13 
Other Findings ........................................................................................................................................ 16 
Summary and Conclusions ........................................................................................................................ 18 
Appendix A: Original Survey Drafts ............................................................................................................ 20 
Appendix B: Survey Edits from Cognitive Testing ..................................................................................... 25 
Appendix C: Survey Edits from Usability Testing ...................................................................................... 27 

Page 2

Executive Summary of Findings and Survey Changes
The primary aim of the IRS is the lawful collection of taxable revenue in the United States. This
mission is complicated by the existence of certain business and personal services that engage in
substantial cash-based transactions or have other "off-the-books" income. Service industries where
tipping occurs is the kind of economic activity that poses a challenge for tax administrators. Current
IRS estimates of tipping income are based upon primary research conducted decades ago. In this
case, the estimated tipping rate in the restaurant industry that was reported in 1982, 14.5%,1 has
likely risen over time because of increased use of electronic payment methods such as credit and
debit cards, the use of which have been shown to result in higher tip rates than cash payments.
Furthermore, much of the previous research on this topic was limited in scope, focusing only on the
restaurant industry. Though some estimates exist for other, commonly tipped services in the United
States, such as taxis or barbers2, the literature existing on such services is relatively scant and needs
expansion. One of the primary aims of this project is to determine the frequency of use of other
tipped services by respondents.
To remedy these issues of scope and accuracy, the IRS began a series of task orders aimed at
determining what scholarly literature had uncovered concerning tipping-related behaviors in the
United States prior to launching a large consumer expenditure survey effort to revise these
estimates. A report from Fors Marsh Group3 (FMG) identified several themes and key findings in the
research that had been conducted on tipping-related behaviors. In addition, this report discussed
various methodology approaches that could be used for a large-scale survey effort to update these
estimates, including advantages and drawbacks to each approach.
To ensure that the findings of this survey effort produce accurate tipping estimates, it is imperative
that the final survey instrument’s language and structure be as refined as possible in order to
maximize accuracy and scope and minimize confusion and burden. Two drafts of a proposed survey
instrument were constructed by IRS and FMG researchers for usability and cognitive testing. These
proposed variants included all services that were originally considered as commonly tipped but
varied in how respondents were asked to record their recent expenditures. All elements of the
instrument had to be rigorously examined, including the final list of services to include, how the
survey should be organized and worded for minimum confusion, cross-platform compatibility, and
recall length. To accomplish these goals, testing was conducted in two phases. The first phase
consisted of a week of cognitive testing that focused on the users’ comprehension of survey focus,
wording, and organization. The second phase consisted of two weeks of usability testing that focused
more on device usage (e.g., desktop, smartphone, tablet) and recall length (i.e., 1-, 3-, and 5-day
periods). In both phases, changes to the survey were made in a rapid, iterative fashion at the end of
each week and sometimes each day. Each week of testing had 12 unique participants, for a total of
36 participants in total. Participants that were involved in the first week of cognitive testing were not
eligible to join the usability testing phase.

1Pearl,

R. B., & McChrohan, K. F. (1984). Estimates of tip income in eating places, 1982. Statistics of Income Bulletin, 3(4),
49–53.
2 Pearl, R. B., & Sudman, S. (1983). A survey approach to estimating the tipping practices of consumers. Final report to the
Internal Revenue Service under Contract TIR-81-21, Survey Research Laboratory, University of Illinois.
3
Estimating Consumer Tipping Behavior: Review and Recommendations, February 2014. Internal report prepared for the
Internal Revenue Service by Fors Marsh Group under contract TIRNO-13-Z-00021-001.

Page 3

Although numerous edits and changes were made to the original draft (See Appendix A for original
draft), the findings did not indicate that any questions had to be fundamentally altered or that there
were severe comprehension issues. Most language edits were made during cognitive testing, while
usability testing focused on device preference and recall findings. The primary findings and
improvements are listed below.






Survey Language and Tipped Services
o Added a new category for “moving and household maintenance” services, and
revised services for all major categories based on participant feedback.
o Revised language and examples provided for non-monetary gift questions because of
respondent confusion. Some could not easily grasp the concept or understand when
such a situation might occur: “I can’t think of a scenario that a non-monetary tip
would occur.”
o Improved service-specific instructions were included to help respondents understand
how to fill out the survey accurately.
Survey Construction and Device Usage
o Determined which of the proposed survey variants was preferred by respondents.
Consensus was toward the version that asked for records service by service, with
many stating reasons such as the following: “I like this one (Version 2). Having
everything listed out there (in Version 1) is just a lot there and I feel that people might
skip over some of the lesser expenses.”
o Crafted draft survey instrument on two survey platforms to find one that was more
compatible for completion on mobile devices.
o Conducted usability testing on three web-capable devices (smartphone, tablet,
laptop) and found no significant issues with survey completion on any device.
o Most respondents reported they would complete the survey on a computer or
smartphone.
Recall Accuracy
o Discovered that all respondents often rely on estimation heuristics, even for short
periods of recall, using language such as “I usually tip 20% and the bill was about
$20; therefore, I probably gave a $4 tip.”
o Varied recall time during usability testing to provide recommendation for survey
completion. Determined that 1-day recall is preferable to maximize accuracy for all
estimates.

Page 4

Cognitive Testing
Design and Lab Setup
Goals: The goals of the cognitive test of the survey were to (1) determine the optimal paradigm for
the survey (i.e., grouping categories as one item or breaking them out into separate items); (2)
identify and repair problematic survey language and instructions; (3) ensure the response options
accurately captured respondents’ tipping behaviors and service-related expenditures. These goals
aim to optimize the user experience and statistical reliability and validity of the survey.
Recruitment: We contacted FMG panel members about the participation opportunity. Panel members
were recruited in and around the Washington, DC, metropolitan area through traditional and social
media marketing. People interested in participating in the study completed a web screener. The
recruiter then rescreened respondents over the phone to confirm the information they submitted. All
participants were 18 years of age or older. Participants were scheduled for the interview at a time
convenient for them. A total of 12 participants were interviewed during cognitive testing. The
interviews were all 45 minutes in length or less.
Method: The moderator read an introduction to participants which informed them about the planned
activities during the test as well as their rights as participants. Participants were either first provided
with a combined version (Version 1) or expanded version (Version 2) of the survey (see Appendix A
for original survey materials with both proposed survey versions). The combined version asked
participants to recall tipping expenditures for each of the broad categories before asking them to
select an appropriate subcategory for the service received. The expanded version broke out the
survey into individual categories. After completing one version of the survey, participants reviewed
the other version and gave their feedback. The order in which the surveys were presented was
rotated across participants.
The moderator facilitated the cognitive test with the participant in the same room. As the participant
responded to the survey, the moderator asked probing questions about the survey language and
response categories. After the survey was completed, participants completed a satisfaction
questionnaire that asked them questions about their experience completing the survey. The
moderator then conducted a debriefing interview that went into more depth about participants’
experience and perceptions about the survey.
Cognitive Test Findings
Optimal Paradigm: A primary goal of cognitive testing was to determine the optimal paradigm for the
survey. Version 1 consisted of grouping all of the service categories into a single item, asking
respondents to provide details for any expenditures or payments made at a restaurant or other
prepared food/drink service, casino, personal grooming or beauty service, moving or household
cleaning/maintenance, hotel/motel, or a taxi/limousine/shuttle service prior to selecting the
appropriate service from each broad category. Version 2 consisted of breaking out the service
categories into separate items. For example, Version 2 asked respondents to provide details for any
expenditures or payments made at a restaurant or other prepared food/drink service. If they
responded that they had, they received a follow-up question asking them to select the appropriate
Page 5

service type and fill out their transactions. This question was repeated for each of the service
categories.
Version 2 of the survey was clearly preferred. 10 of 12 participants preferred Version 2 (separated
items), one participant preferred Version 1 (combined item), and one participant was indifferent.





“This [Version 2] flows a little bit easier. It’s easier for my brain to think about it.”
“I like this one [Version 2]. Having everything listed out there [in Version 1] is just a lot there
and I feel that people might skip over some of the lesser expenses. With this version [Version
2], it goes over every category so I think it would be more difficult for someone to skip over
any expenses. This one [Version 1] is just overwhelming. It’s too much information at once.”
“This first one is totally confusing. I think that the biggest problem might be that it could
make people feel stupid. This one [Version 2] is already laid out, it’s already delineated. It
just helps the individual focus and answer what is being requested.”

Language and Response Option Findings
Restaurant or Other Prepared Food Service: Numerous minor edits were necessary for the original
language and service options in this category. Participants considered “Bar” to be a separate
response option from “Full-Service Dining (e.g., traditional restaurants).” One participant said, “If it
said something like ‘bar and grill’ or something like that, then I think it would be OK.” As a result, the
response option “Bar” was added to the category, which itself was changed from “Restaurant or
other prepared food service” to “Restaurant or other prepared food/drink service.”
Additional testing determined that more service categories were needed. One participant expected to
see “Coffee Shops” as a separate response option in the “Restaurants or other prepared food/drink
service” category. This participant said, “I would put it in its own category because it’s so ubiquitous.”
As a result, “Coffee Shops” was added as a response option. Two participants expected to see “Food
Truck” as a response option under the “Restaurant or other prepared food/drink service” category.
As a result, “Food Cart/Truck” was added as a response option. Finally, the category “SelfService/Cafeteria/Buffet” was added because some participants indicated that such food options
would not fit well into the previous categories.
Other participants commented it was odd to see “Ice Cream” as a response option under the
“Restaurant or other prepared food/drink service” category. One participant said, “Ice cream—that
sounds weird.” Another said, “I find it interesting that they put the ‘Ice Cream’ one in there. I mean, I
guess there are ice cream stores out there, but I would have never thought to have put that in there.”
As a result, “Ice Cream” was collapsed with “Smoothie Shops” in order to reduce the number of
service categories.
Hotel/Motel: Participants had difficulty reporting tipping behaviors for the “Hotel/Motel” category. If
they tipped for housekeeping or another service in a hotel/motel, they were unclear what to indicate
in the “Amount you paid for total bill payment” section. Participants said they thought they should
record the amount paid for the room or the total cost for the stay at the hotel. One participant said: “I
Page 6

don’t see anything that would be easily identifiable as a payment for the room. After I paid $130 for
the room, I would leave a little money for the housekeeping. I didn’t pay for the housekeeping, per
se, but then I tip on the housekeeping.” This issue was reexamined and a resolution was
implemented during usability testing.
Another issue that arose was that participants expected to see dining options under the
“Hotel/Motel” category for situations when the hotel had a restaurant, bar, or similar option. It was
decided that services that could be encountered in multiple service categories should be repeated
under each category. As a result, service options for “Full-Service Dining (e.g., traditional
restaurant)”; “Bar”; and “Self-Service/Cafeteria/Buffet” were added to the “Hotel/Motel” category.
Our concern for capturing duplicate responses was outweighed by the possibility of not capturing this
service-related expenditure. To reconcile this concern, respondents were instructed not to record
service-related expenditures that were previously recorded in other survey items. Duplicate
responses recorded can simply be determined during analysis.
Hair Stylist/Barber becomes Personal Grooming, Beauty, or Massage Services: Numerous additional
services were added to the original two that were proposed for this service category after
participants identified many other beauty-related services that could receive tips. Additional services
were added for “Manicurist/Pedicurist,” “Massage Therapist,” “Waxing/Hair Removal,” and
“Facial/Skin Care.” Furthermore, to better reflect the new services, the category was renamed
“Personal Grooming, Beauty, or Massage Services.”
Moving or Household Maintenance: Participants expected to see a category that captured tipping
activities for people in moving, cleaning, plumbing, and repair occupations. As a result, the category
“Moving or household maintenance” was added to the survey. The services added to this category
during cognitive testing were “Professional Movers,” “Maid or Cleaning Service,” “Lawn/Gardening
Service,” and “Handyman/Repairman.”
Casino: After “Bar” was added to the restaurant category, one participant also expected to find the
response option “Bar” under “Casino.” This participant explained how a person could have an
expenditure at a bar or restaurant while visiting a casino. When participants were asked what
response options they expected to see under the “Casino” category, others said that they expected
to see “Bar” and “Restaurant” there. Ultimately, the three food services that were added to the
“Hotel/Motel” category were also added to the “Casino” category.
Taxi, Limousine, Rideshare, or Shuttle Service: The only edit required for this category was changing
“App-Based Taxi” to “Ride-Share service (e.g., Uber or Lyft).” Originally, respondents were not clear
that “App-Based Taxi” meant to refer to businesses such as Uber or similar services. Participants
indicated that they would record the payment type for Uber as a “credit” or “debit” transaction.
Participants were unaware that Uber charges an automatic gratuity, so this was not likely to be
captured in the survey. Participants consider a “Smartphone credit or app” payment type to be
money that has already been loaded into the smartphone or app. Participants explained this
payment type to be “Google Wallet.” When the moderator asked participants to describe a
smartphone or app-based payment, one participant said, “I know you have apps where you put in
your information and when you use it, it just takes the money from your account.” Although these
Page 7

responses indicated an understanding of the concept, such records might need to be examined
during the pilot study to ensure that this option is being selected in conjunction with appropriate
services.
Other Findings
Likelihood to Use Receipts and Financial Statements: When the moderator asked participants,
“Would you look up any records/receipts or complete it on the spot from memory?” participants
provided varying responses. Five of 12 participants indicated that they would check their receipts
and bank statements if they had difficulty recalling the transactions from memory. Two of 12
participants said whether they checked their receipts and statements would depend on the
incentive. One of these participants said: “Fifty-fifty, maybe. It would depend on convenience of doing
so and the incentive.” The other participant similarly said, “If there was an incentive, I would go
through my accounts to try and fill it out.” Three of 12 participants indicated that it would depend on
the purpose of the survey. One of these participants said: “If I thought it was for a specific purpose, I
would check my bank statement. If it was clear that they were investigating the tipped minimum
[wage] versus non-tipped minimum wage, then I would be more diligent in how I reported
information.” Two of 12 participants said they would only recall transactions by memory regardless
of the incentive or purpose of the survey.
Accuracy of Recalling Payments: Based on the records given by participants during cognitive testing,
survey respondents are likely to use heuristics for calculating payment and tip amounts. For
example, one participant indicated that he used a 20% estimate when calculating the tip amount.
Another participant explained that he knows he tips around or about 20%, so he just moves the
decimal over and doubles it. Such responses indicate that respondents are not likely to provide
precise tipping amounts and will try to calculate the amount they believe they tipped using their
recollection of the bill, rather than an actual recollection of the tip amount itself.
However, participants indicated that transactions that are frequently incurred for the same amount
are more likely to be accurately recalled. For example, one participant explained that his barber
always charges him $16 and he always tips her $4 to bring the bill to an even $20. Similarly, another
participant explained how he always pays the same amount for the barber, so this transaction was
easy to recall. This same participant explained that he just remembers what the total bill was for
restaurants and always tips the same percentage of the bill.
During these sessions, the moderator asked all participants, “Regarding the tipping expenditures
that you have just recorded, how many days prior would you think you could accurately remember
(within a half dollar) your tipping expenditures.” Participants commented that it was easy to recall if
the expense was incurred in the last day. Most commented that they could remember accurately up
to 3 to 4 days ago, while a few indicated that they could remember expenses incurred from a week
ago or longer. The longer the recall period, the more likely that participants would use heuristics to
estimate tipping amounts. For example, one participant said: “You know something? It wouldn’t
really be that difficult for me to recall since I usually tip about 15%.” Such responses might indicate

Page 8

that a short recall time is required for use in the survey in order to minimize respondents’ use of
such estimation tactics as much as possible.
Non-Monetary Tip: The term “Non-monetary tip” and the initial description for it were confusing to
most participants. All participants in the cognitive testing round indicated that they had never given a
non-monetary tip and most did not know what the term meant before they read the description. One
participant said, “I can’t think of a scenario that a non-monetary tip would occur.” One participant
was offended that the IRS was asking about a non-monetary payment. She said, “That feels very IRS,
very in your face.… ‘Pay me because I say to.’ That feels invasive.” The participant went on to explain
that she does provide a gift to her hair stylist; however, this is not done as part of any transaction
and is a part of a personal relationship, not just a service one. Because of this, the description of
“Non-monetary tip” was streamlined to remove excess verbiage and to emphasize that items given
as personal gifts were not meant to be recorded.
Splitting and Separating Payments: When asked, “Have you ever left a tip for someone and split the
tip across payment methods, such as cash and credit card?” all 12 participants indicated that they
could not recall a time when they had split a tip across payment methods.
Participants were also asked how they would record multiple expenditures at the same
establishment. The moderator asked participants, “If you filled out this survey for an occasion where
you went to a restaurant and had a drink at the bar before going to your table for your meal, how
would you record that in this survey?” Ten of 12 participants indicated that they would record these
as separate transactions. The remaining two indicated that they would record this as one transaction
because it occurred at the same establishment. Instructions were added to the survey to explain that
such situations should be recorded as separate transactions.
Influences on Respondent Behavior: Additional debriefing questions were asked of participants to
determine if they could guess the intended use of the survey and whether such knowledge might
influence their likelihood to report their transactions accurately. The moderator asked participants,
“What do you think the purpose of this survey effort is?” Six of 12 participants correctly assumed
that the study was being conducted for the IRS to determine tipping rates for different industries.
After those six participants responded correctly, they were asked a follow-up question: “Would that
knowledge make you more or less likely to fill out the survey accurately?” All six participants
indicated that knowing the purpose of the study would either increase or have no impact on the
likelihood that they would complete the survey accurately. Furthermore, these participants wanted
more detailed information about the purpose of the study. One participant said: “I would be more
inclined to try harder if I knew that it was part of some kind of decision or policy. If I saw this was
from…I don’t know, Verizon, I would not be very inclined to try very hard to remember.” Another
participant who wanted more information said: “I guess it depends on what they are going to use
that information for. I would still fill it out.”
Finally, the moderator asked, “Have you ever worked in a job that receives tips for your service?” If
the participant responded “yes,” the moderator then asked, “Does that influence how much or how
you tip?” Four of 12 participants indicated that they had worked in a tipping-based occupation. Only
one of these four participants said that he thought that working in a tipping-based occupation
resulted in him tipping more generously.
Page 9

Usability Testing
Design and Lab Setup
Goals: In addition to any wording edits that were found to be necessary based on respondent
confusion, there were two new goals in the usability testing phase: (1) test the survey on multiple
devices to ensure that respondents are able to complete the survey on common web-based devices,
and (2) examine responses across different recall periods in order to make a recommendation about
the recall frame used for the pilot study.
Survey Design and Platform Choice: Prior to usability testing, two different survey platforms were
used to create a draft of the survey in order to determine which one might best allow for completion
on mobile devices. Survey drafts were created on both Verint and Confirmit systems, two commercial
survey products frequently used by FMG. The Verint platform had participants enter numerous
pieces of information for a transaction on one page, while Confirmit had participants enter one piece
of information on each page.
Mobile screen pictures of the Verint survey platform.

In order to facilitate completion of the survey on mobile devices, particularly smartphones, the
decision was made to set up the survey on the Confirmit system. Because there are numerous
pieces of information that have to be entered for each transaction, the Confirmit survey could place
additional burden on the participant. However, this was judged to be an acceptable trade-off given
the formatting issues present on mobile devices with the Verint survey.

Page 10

Mobile screen pictures of the Confirmit survey platform.

The moderator was present to assist with any technical difficulties but tried not to assist participants
in completing the survey, and often asked participants to complete the survey as if the moderator
was not in the room. A few minor technical difficulties required moderator intervention, but these did
not represent difficulties with the survey language or setup. Specific difficulties that arose are
discussed under “Device-specific difficulties.”
For each session, participants completed the survey three times, with differing lengths of recall time
and different devices. While the device rotation was randomized, recall time increased gradually
from 1 day to 3 days and then 5 days. To minimize burden, participants were asked to record any
additional expenditures they had made during the expanded time frame and not rerecord
expenditures they had listed in earlier recall periods.
Recruitment and Lab Setup: 12 participants were recruited each week for both weeks of usability
testing, as was done in the cognitive testing phase. Two rounds of usability testing were conducted
that took place during weeks 2 and 3 of the overall testing period. In this section, round 1 of usability
testing will refer to week 2 of overall testing, while round 2 of usability testing will refer to week 3 of
overall testing. Recruitment incentives and participant burden were unchanged from the cognitive
testing phase.
After participants were briefed of their rights, the session began. Each participant completed the
survey three times, with differing lengths of recall time and on different devices. Participants
completed surveys with 1-day recall before completing surveys with 3- and 5-day recall. For the sake
of time, participants were instructed only to record new transactions during the longer recall times.
Unlike recall time, device order was randomized among participants in order to get novel reactions to
the survey on each device. Three devices were used: a Windows laptop computer, an Android
smartphone, and an iPad tablet.

Page 11

Usability Testing Findings
Device Preference: The majority of participants across both rounds of testing indicated that they
preferred completing the survey on the personal computer the most out of the three devices.
However, eight participants during week 1 said that they would likely complete the survey on their
smartphone. The stated reason was often that they were not at home to use their personal computer
or that they always had their phone with them. Furthermore, although participants said that it was
more difficult to complete the survey on the smartphone, they said that completing the survey on all
three devices was easy. The overall feedback for all devices was positive and did not leave any
concerns that completing the survey on the smartphone or tablet would be an impediment to
completion.
Device-Specific Difficulties: Fortunately, most participants did not encounter any serious difficulties
with any of the devices, though the need for some small areas for improvement did arise. When
using the smartphone, some participants indicated that they had difficulty selecting options that
were at the bottom of the screen. Participants discovered that this issue could be resolved by
scrolling up and then down again before trying to select options at the bottom of the screen. This
issue should be tested prior to the full launch of the survey to determine if the problem is universal
or if it was related to the specific device used in testing.
Two participants during week 1 had difficulty viewing the entire website unless the tablet was
oriented horizontally in “landscape” mode. At other points, participants indicated that they had to
“zoom in” with the tablet to select a bubble or write in a response, but this did not present any major
issues for any participant.

Survey Confusion
Survey Introduction: The introduction text to the survey was identified as being too lengthy, with
participants saying they would not pay attention to it because it was too wordy. This led to some
minor changes to the introduction language between rounds 1 and 2 of usability testing. The revised
text reads: “Welcome to the 2015 Survey on Consumer Behaviors. The purpose of this survey is to
explore consumer’s behavior with respect to specific goods and services in the United States.”
Multiple Record Instructions: One of the greater areas of confusion concerned parts of the
instructions added to the survey to clarify how the records would be entered. Specifically, the
language that explained to participants in the web-based survey that they would be entering their
information one record at a time caused notable difficulty.




“I have to read it again because I want to be sure of what you’re asking. I think you’re saying
if I ate at two different places to record them separately. Like if I ate at Chipotle twice, I
should record them both.”
“I wish there was a button where you know you could do a second transaction. At the end,
you get the option for the second transaction, which is good and it was good they said in
between to record them in the same way.”

Page 12



“After a couple of times reading it, I figured it out. Especially for immigrants like me, they
should have made it simpler. When you read it, it’s easier … they could make it easier.”

Although it appeared that most participants were able to eventually understand the instructions,
either by rereading it carefully or by moving through and trying to complete the survey, it was felt that
the language was causing too much cognitive burden. We revised the language after the first week of
usability testing to try to reduce participant confusion. All services except for the “Taxi, Limousine,
Rideshare, or Shuttle Service” category had specific language written to better explain how to record
multiple transactions within that service, with appropriate examples for each service category. The
“Taxi, Limousine, Rideshare, or Shuttle Service” category did not receive updated language because
it was determined that there were no likely scenarios in which a respondent could enter multiple
payments for the same cab, so revised language was not necessary.
During the second week of testing, 10 of 12 participants demonstrated the ability to comprehend
the instructions to only enter one transaction at a time. Of the two that did not understand at first,
one participant was able to figure out as he went along that he was supposed to enter only one
transaction at a time. The other participant explicitly said that he was trying to enter transactions at
both a fast food establishment and an ice cream parlor, and he only entered one of the two
transactions, as he was unable to comprehend how to enter more than one. However, despite
leaving one of his transactions out, his data was still valid for the transaction that he was able to
enter. Therefore, we decided to leave the survey language as-is, as there is no further revision to be
made that would result in more participants taking the time to read all the instructions.

Language and Response Option Findings
Restaurants or Other Prepared Food/Drink Services: No major changes or additions were identified
during round 1 of usability testing; participants in this round by and large understood all of the
categories and did not think that any categories were missing. However, two participants near the
beginning of round 2 expressed confusion over where to put restaurants such as Chipotle Mexican
Grill and Panera Bread. One participant said, “I’d put it under fast food, but it’s not fast food; it’s fast
casual.” The other said, “I guess you’d call [Chipotle] a cafeteria.” Given that two participants said
they would classify the same establishment as two different categories, a “Fast Casual” category was
added halfway through round 2 of testing for these types of establishments.
Hotel/Motel: There was some minor confusion regarding how to log certain hotel-related tips and
expenses. The room fee, for example, is not a “service” in the same manner as housekeeping or
room service. This caused one participant to try to record his hotel bill as the “total bill amount”
before recording a tip he had left for the “housekeeping” service. Before week 2 of usability testing,
the term “payment” was changed to “transaction” to help participants consider all of these financial
interactions as service transactions instead of just the direct payments to the establishment.
Additionally, participants were asked to record the number of days they stayed and the room fee if
they reported that they had stayed in a hotel prior to recording any other services they might have
used. This was meant to remove the desire to report the room bill under any service record.

Page 13

Personal Grooming, Beauty, or Massage Services: This service category required only minor edits
overall, because most participants had no major suggestions based on the subcategories that had
been added during cognitive testing. Before round 2 of usability testing, a service was added for
“Makeup artist” based on the comments of two participants in round 1 who indicated these were
professionals who work in makeup stores or individuals who do makeup for groups at events such as
weddings. One of these participants reported that she had one such job in the past and had received
tips for her service. No additional changes were uncovered during round 2 of usability testing.
Moving or Household Maintenance: No major changes or additions were identified during round 1 of
usability testing; participants in this round by and large understood all of the categories and did not
think that any categories were missing. Halfway through round 2 of testing, the response options for
this section were revised further, as one participant expressed confusion about how to classify
ordering and paying for moving boxes. Though this expenditure is clearly a moving expense, none of
the four categories presented accounted for this type of expense. Therefore, an additional category
called “Equipment Rental” was established midway through round 2 of testing. Although equipment
rental is not an expense where people would typically leave a tip, it does fall under the “Moving or
Household Maintenance” category, so providing a category for participants to place this type of
expense is necessary to ensure the quality of the data is not compromised by participants who may
put these expenses improperly into one of the other categories.
Casino: The subcategories for this group of services was somewhat difficult to evaluate because
several participants indicated that they did not have enough experience with casinos to be able to
speak with confidence to any of the subcategories listed under “Casino.” However, the categories
that were listed did not confuse participants, even among those who did not have much experience
with casinos. That being said, two participants in round 1 felt that another option should be included
for valet services, so this option was added before round 2.
Halfway through round 2 of testing, the “Casino” section was revised further after one participant
expressed confusion about the transaction amount for the dealer. After selecting that he paid cash
for the transaction or service, he was confused about the amount, and asked, “Do I put the amount
of money that I lost?” He was unable to figure out the amount of the actual bill because one does not
receive a bill for gambling services; one only pays the amount that he or she loses at the table. For
this type of transaction, it was decided not to ask whether and how much participants paid for the
service. Instead, the survey skips this question for anyone who says that their service was a casino
dealer and proceeds directly to the questions regarding automatic and voluntary tips.
Taxi, Limousine, Rideshare, or Shuttle Service: There was some minor confusion expressed
concerning the term “Rideshare” in round 1 when discussing services such as Uber and Lyft. One
participant stated that he would have thought the survey was referencing carpooling or similar
services if the examples for Uber and Lyft were not provided. Another participant felt that Uber would
have its own category. When these participants were asked about how the language should be
revised and what they would call such services if not “Rideshare,” they had no clear suggestions.
Given that all participants were able to understand the purposes of that subcategory upon reading,
only small revisions were made to the language to emphasize Uber as the primary service. During
round 2, language was revised to read “Uber, Lyft, or other Ride-Share service,” rather than “Ride-

Page 14

Share service (e.g., Uber or Lyft).” Participants in round 2 who were asked all said that they
understood what the category entailed.
Paying for a Product or Service: Some participants in round 1 indicated that they would sometimes
go somewhere and they would not pay any portion of the bill (e.g., a friend or a parent would pay the
bill). In the first version of the survey, participants were expected to click “Not Applicable – There was
no bill for this service” when asked how they paid for the service. However, this caused confusion, as
there oftentimes was still a bill for the service; it was just paid by someone else (i.e., not the person
taking the survey). Before round 2 of testing began, a follow-up question was added that read, “Did
you pay for this product or service?” If participants answered “yes,” they were asked the appropriate
follow-ups about the bill payment method and amount. If they answered “no,” they were no longer
asked about the amount and method of payment of the bill and were instead directly routed to the
tipping questions. The response option “Not Applicable – There was no bill for this service” was
removed from the payment method screen.
“I don’t remember”: In round 1 of usability testing, participants had the option to select “I don’t
remember” if they could not remember the amount of their bill or their tip. One participant selected
this option, and upon probing, revealed that she was in a group of people and they each paid their
portion of the tip. She could not recall the exact amount, so she selected “I don’t remember.” After
discussion, it was decided that rounded amounts in these instances were preferable to no numerical
data at all, so the decision was made to remove the “I don’t remember” response option from these
questions before round 2 of testing. No participants in round 2 expressed a desire for an “I don’t
remember” option.
Total Bill, Automatic Tip, and Voluntary Tip Language: For the question regarding the total bill
amount, there were clear indications that some participants did not fully read the instructions
concerning the total bill amount, and that it was meant to exclude any tips, whether voluntary or
automatic.
The question concerning automatic tips caused some minor confusion among round 1 participants.
Some indicated that while they knew that some places did this, they were confused by its inclusion in
the survey. One participant in round 1 selected the option on the assumption that there was an
automatic tip of some kind included in the bill for the shuttle service. This indicated that some
respondents might report that there were automatic tips added by the business when they were not
sure. The chance for respondents to report that a business was including an automatic tip without
knowing if this occurred was concerning, because it could lead to false estimates about how often
businesses add such tips. Before round 2 of testing, this response option “Yes, but amount not
provided” was dropped from the survey, leaving respondents with the option to either provide an
amount of automatic gratuity or report that one was not included.
Non-Monetary Gifts: The question concerning non-monetary gifts and payments continued to cause
participants trouble during week 1 of usability testing. Some confusion centered on what a nonmonetary gift was and there were indications that examples could be clearer. Specifically, the
language for “coupons” as a non-monetary gift was an area of confusion. One participant read that
and started to believe it was for reward points that you can get from restaurants and other
establishments. Despite this confusion, most were able to understand it if they gave sufficient effort
Page 15

to reading and interpreting the definition, but it was determined that streamlining the language and
question flow would still be necessary for round 2 of testing. Therefore, in round 2 of testing,
participants were no longer asked, “Did you leave a non-monetary tip?” as a separate question.
Instead, “Non-monetary*” was added as a payment option under the question, “What payment
type(s) did you use to pay the voluntary tip?” The examples given were also changed from “coupon or
event tickets” to “concert tickets, a bottle of wine, or a meal.” Explanatory text was added to the
bottom of the response option bank that read: “*Examples of non-monetary gifts are: concert tickets,
a bottle of wine, or a meal. Note that non-monetary gifts should only be recorded if they were used to
compensate for the service. Non-monetary gifts that are given as personal tokens of appreciation
should not be recorded.” If participants selected this option, they were then asked to estimate the
value of the portion of the tip that was non-monetary in a follow-up question.
Another potential flaw with this item was discovered with the round 1 participant who had given nonmonetary gifts to her movers. In this case, she gave several gifts and listed them, but when it came
to evaluating the worth of the items, she was only able to successfully value the price of the food she
had provided. She determined that evaluating the worth of the furniture that she had given away was
not possible to do accurately and did not include those items in her evaluation. This issue serves as
evidence that this item will lead to some problematic data in the survey, but there is no clear manner
in which it can be fixed, because evaluating the dollar value of used items is cognitively challenging.
Ultimately, while there were some signs that respondents were able to grasp the concept of the
question, there were numerous signs of respondent confusion despite multiple revisions. Although
this information about non-monetary tips could be useful for providing estimates of income that is
not currently captured by the other survey measures, there are numerous opportunities for error that
could enter such records. Serious discussion about whether this item is necessary to the survey
effort will be required prior to the pilot test. Respondent error, burden, and low incidence rates could
make the inclusion of the item more problematic than the benefits of this information warrant.
Recording Non-Tipping Occasions: Some participants indicated that they wanted to list payments for
services that fell within the realm of some of the major service categories but were not services that
could be considered tipping events. Participants in some instances indicated that they thought they
should be able to record transactions such as buying groceries or purchases made at the pharmacy.
Another participant, when looking at the options for the “Transportation” category, thought that there
should be subcategories for public transit options such as buses and metro. A different participant
recorded a transaction under the “Personal Grooming, Beauty, or Massage Services” category for a
few dollars. When asked about this, he indicated that the purchase was for hair gel and that the
grooming category should be clear that it was not meant for store-bought purchases.

Other Findings
Participant Tipping Experience: Seven of the 12 participants in round 1 and 6 of 11 participants in
round 2 indicated that they had worked for a job at some point that received tips for service. Of
those, nearly all of them (five of seven in round 1, four of six in round 2) said that experience
influenced their tipping behaviors, either by increasing their knowledge about how important tips are
Page 16

to some professions or from an increased sense of empathy for people who work those professions
because of their shared experience.
Participant Perceptions of Survey Focus: Perceptions of the survey intent were mixed. Some
participants felt that the aim of the survey was to better understand consumer spending patterns
and what people were spending their money on. Two participants thought that the survey might be
used to create changes in legislation affecting the minimum wage of certain tipped jobs. Four of 12
round 1 participants and two of 10 round 2 participants seemed to have a good sense of the
purpose of the study. Of those six participants, one said that knowing the purpose of the study would
negatively affect how truthful he would be when filling out the survey.
Banking: Across both rounds of usability testing, there was no consensus concerning the use of
banking records to help complete the survey, although nearly all reported that they had a banking
profile or records they could check. Seven participants in round 1 and four in round 2 indicated that
they had a banking profile and might look up their records to complete the survey, while five in round
1 and two in round 2 indicated that they would not look up bank records or receipts. A few
participants indicated that they would be more likely to use a banking profile to check their records
for longer periods of recall, such as for the 5-day condition.

Page 17

Summary and Conclusions
Survey Variant Selection: Although numerous changes came from the testing process, the first and
most pressing issue was to determine which of the two proposed survey versions would be the better
option. Ten of the 12 participants indicated that they preferred thinking about each service
separately and felt that it was easier for them to complete the second version. Although the
increased survey burden is a concern, the benefits clearly outweighed the costs given the
preference, leading to the decision to move forward with the second version for usability testing.
Online Survey Design: The next major design decision was about the setup for the web version of the
survey. Two drafts were created using two different survey platforms. The Verint version showed a
greater amount of questions on the screen at once but was ultimately deemed not intuitive enough
to complete on mobile devices. The Confirmit platform required respondents to answer one question
at a time but was much more intuitive to complete on mobile devices and was thus selected for use.
However, the negative consequence of its selection was an increase in survey burden due to the
format of answering one question at a time. However, final specifications of the pilot survey will
depend on the survey platforms operated by the population vendors, Ipsos and GfK. FMG will work in
consultation with both to ensure that there is common programming specifications and that the final
product aims to minimize burden across devices.
Device Preference: One of the main concerns in the survey design stage during usability testing was
ensuring that the prototype was compatible not only with computers but also with smartphones and
tablets. The thought was that many survey respondents would complete the survey with the device
that was most accessible and that in numerous situations that would be a smartphone or other
mobile device. Respondents generally indicated some preference for the laptop and phone versions
of the survey, and most reported that they would complete the survey on such a device, either
because of preference or accessibility reasons. Fortunately, no respondent felt that the survey was
much more difficult to complete on a smartphone than on the other devices, and a few commented
on how it seemed that the survey was well designed for such platforms.
Recall and Accuracy: Although respondents seemed to indicate that a 3-day recall period would be
the maximum amount of time that they could accurately record their expenditures, it was determined
that the pilot study should proceed with a 1-day recall period rather than expanding to a 3-day period
because the maximum recall that could be tested would likely increase difficulty significantly. This
determination was also made because of concerns about survey burden and from a lack of
irrefutable proof that respondents could accurately record expenditures from multiple days earlier.
Although there were respondents who indicated that they might check their banking records for
greater accuracy, this was not observed in the lab and cannot be expected during the survey fielding.
Although some respondents will not have any records because of the shorter recall time, this is
considered an acceptable consequence given the importance of maximizing the accuracy in the
records that are gathered.
Wording Changes and Service Additions: Numerous questions from the original draft of the survey
required revision to enhance respondent comprehension and include directions specific for the webbased version. Most items received revisions at some point of the cognitive or usability testing
process, but no major items were added or removed throughout beyond the addition of instructions
Page 18

or additional service categories. The language edits did not change the original intent of the
questions in any manner. The primary revisions were the instructions logging multiple entries,
revised and new service categories, and clarification to the definition of “non-monetary” gift.
Other Findings: Other noteworthy findings from usability testing included participant thoughts on the
use of banking statements or other records to accurately fill out the survey and the general intent of
the survey. Most participants reported that they had access to some kind of online banking profile or
other means of tracking their expenditures, and roughly half of those felt that they might check those
records to complete such an online survey. However, no participants appeared to actually check
these records while in the lab setting, so it is difficult to gauge how often this might occur during the
pilot test, if at all. Finally, a notable portion of participants were able to successfully guess at the
general intent of the survey (i.e., that it was an IRS effort aimed at better determining the tipped
income that might not be reported), but nearly all claimed this knowledge would not negatively
influence whether they would accurately fill out such a questionnaire (one respondent acknowledged
that this might negatively influence how truthfully he filled it out). This feeling was endorsed by
others when they were told the true purpose of the study. A few participants acknowledged that
others might not be inclined to fill out the survey properly if they knew the purpose.
Outstanding Issues to be Resolved Prior to Pilot Study: The two major issues that remain prior to
launching the pilot survey are the questions about non-monetary gifts and survey platform design.
Given the predicted low-incidence rate and high degree of confusion presented by the questions and
language concerning non-monetary gifts, it is FMG’s recommendation that these questions be
removed from the survey. Even after multiple attempts to clarify the language there were still
difficulties in interpreting this type of gift, in addition to other issues such as properly valuing gifts.
The survey design issues will be addressed in coordination with the panel vendors, Ipsos and GfK, as
they will likely both separately program the final survey. For this reason, it will be important to give
both detailed specifications about how the survey should be designed and programmed before
attempting to resolve any discrepancies between their unique systems. Respondent burden and
mobile accessibility are the two greatest design elements that need to be addressed during this
process.

Page 19

Appendix A: Original Survey Drafts

Consumer Tipping Draft Survey for Usability Testing
Welcome to the 2015 Survey on Consumer Tipping Behaviors. This survey is aimed at
determining average expenditures amongst consumers, particularly tipping related
expenditures. In this short survey, we will ask you about your expenditures within the past XX
days. This survey is being conducted by a third party research group, Fors Marsh Group, LLC.
This survey should only take 8 minutes to complete.
Screener1) In the past XX days, have you made any expenditures at a restaurant, casino,
hair stylist, hotel/motel, taxi/limousine service, or on a cruise ship.
1A)
Please provide details for any expenditures made in the past at a
restaurant, casino, hair stylist, hotel/motel, taxi/limousine service, on a cruise ship, or at an
auto mechanic. If you have made multiple expenditures on a given type of service in the past
, provide separate details for each. If you have not made any expenditures
on one of the listed services in the past , select “No Expenditure”.
Establishme
nt/Service
Type
(restaurant,
casino, hair
stylist,
hotel/motel,
taxi/limo,
cruise ship)

Drop down
menu

Sub-Type
(e.g., for
restaurants:
Café/FamilyStyle/Diners,
Traditional
Restaurants/Casu
al Dining, Upscale
Casual Dining,
Fine Dining, Fast
Food, Delivery, Ice
Cream, Coffee
Shops, Smoothie,
SelfService/Cafeteria/
Buffets )
Drop down menu

Total bill
amount
(after
tax,
before
automati
c or
voluntary
gratuity)

Payment
type for bill
(cash, debit,
credit, check,
gift card,
smartphone
credit or app,
paper or
online
coupon {e.g.,
Groupon},
nonmonetary,
other)

Amount
of
automati
c gratuity
added by
establish
ment

Amoun
t of
volunt
ary tip

Payment
type for
Voluntar
y tip
(same
options
as
column
4)

Amount
of nonmonetar
y gift*

Description
of tip if nonmonetary
(text field)

Text

Multiple
choice
(select all
that apply)

Text

Text

Multiple
choice
(select
all that
apply)

Text

Text

Page 20

*If a portion of the gratuity or tip took the form of a non-monetary payment (e.g., a coupon or
event tickets) indicate the cash equivalent amount in this column. Note that non-monetary
transfers should only be counted as tips if they were used as payment for immediate service
and are used as a substitute for a monetary tip. Non-monetary transfers that are used as
gifts/personal tokens of appreciations should not be counted as tips.
[Note: 1B is an alternative question format that could be tested during the usability phase. This
method would ask a variant of question 1 for each of the services and establishments of interest.
The goal for this approach is to improve participant recall and have them actively consider each type
of establishment we are interested with. If they answer yes, they get a follow-up question asking
them to list their expenditures for that type of establishment]
1B)
In the last , have you purchased/visited a ?
1. No
2. Yes

[If Q1 is yes, list the table below for the service from the prior question]
Please answer the following questions regarding the amount spent and the amount tipped on this
purchase/visit.
Total bill
Sub-Type
Payment
Amount of
Amount
Amount Description
(e.g., for restaurants:
amount
type for bill
automatic
of
Payment
of nonof tip if
Café/Family-Style/Diners, (after tax,
(cash,
gratuity
voluntary type for monetary
nonTraditional
before
debit,
added by
tip
voluntary
gift*
monetary
Restaurants/Casual
automatic
credit,
establishment
tip
(text field)
Dining, Upscale Casual
or
check, gift
(same
Dining, Fine Dining, Fast
voluntary
card,
options
Food, Delivery, Ice Cream,
gratuity) smartphone
as
Coffee Shops, Smoothie,
credit or
column
Selfapp, paper
4)*
Service/Cafeteria/Buffets)
or online
coupon
{e.g.,
Groupon},
nonmonetary,
other)
Drop down menu
Text
Multiple
Text
Multiple
Text
Text
choice
choice
(select all
(select
that apply)
all that
apply)

Page 21

*If a portion of the gratuity or tip took the form of a non-monetary payment (e.g., a coupon or
event tickets) indicate the cash equivalent amount in this column. Note that non-monetary
transfers should only be counted as tips if they were used as payment for immediate service
and are used as a substitute for a monetary tip. Non-monetary transfers that are used as
gifts/personal tokens of appreciations should not be counted as tips.
[Note: Demographic items 2-4 will be captured by the frame file of both survey panels and will not be
asked of participants in the actual survey.]
2) What is your age?

3) In which do you live?

4) What is your gender?
1. Male
2. Female
5) Are you of Hispanic or Latino origin (ethnicity)?
1. Yes, of Hispanic origin
2. No, not of Hispanic origin
6) What is your race? Please select one or more. Are you…
1.
2.
3.
4.
5.

White
Black or African-American
Asian
Native Hawaiian or Other Pacific Islander
American Indian or Alaskan Native

7) Please indicate your highest level of educational attainment:
1.
2.
3.
4.
5.
6.
7.

No formal education
1st, 2nd, 3rd, or 4th grade
5th or 6th grade
7th or 8th grade
9th grade
10th grade
11th grade
Page 22

8. 12th grade NO DIPLOMA
9. HIGH SCHOOL GRADUATE - high school DIPLOMA or the equivalent GED)
10. Some college, no degree
11. Associate degree
12. Bachelors of degree
13. Master’s degree
14. Professional or Doctorate degree
8) Please indicate your employment status:
1.
Working - as a paid employee
2.
Working - self-employed
3.
Not working - on temporary layoff from a job
4.
Not working - looking for work
5.
Not working – retired
6.
Not working – disabled
7.
Not working – other
9) Please indicate your annual household income:
1. Less than $5,000
2. $5,000 to $7,499
3. $7,500 to $9,999
4. $10,000 to $12,499
5. $12,500 to $14,999
6. $15,000 to $19,999
7. $20,000 to $24,999
8. $25,000 to $29,999
9. $30,000 to $34,999
10. $35,000 to $39,999
11. $40,000 to $49,999
12. $50,000 to $59,999
13. $60,000 to $74,999
14. $75,000 to $84,999
15. $85,000 to $99,999
16. $100,000 to $124,999
17. $125,000 to $149,999
18. $150,000 to $174,999
19. $175,000 or more
The Paperwork Reduction Act requires that the IRS display an OMB control number on all
public information requests. The OMB Control Number for this survey is 1545-1349. We
estimate the time required to be eight minutes. Also, if you have any comments regarding the
time estimates associated with this study or suggestions on making this process simpler,
please write to:
Internal Revenue Service
Tax Product Coordinating Committee
Page 23

1111 Constitution Avenue NW
Washington, DC 20224

Page 24

Appendix B: Survey Edits from Cognitive Testing














Question 1 originally read: “Please provide details for any expenditures or payments made in
the past week at a restaurant or other prepared food service, casino, barber or hair stylist,
hotel/motel, or a taxi/limousine service. If you have made multiple expenditures on a given
type of service in the past week, provide separate details for each. For example, if you stayed
at a hotel and had tipped room service and a concierge, please record those separately.”
Question 1 was changed to: “Please provide details for any expenditures or payments made
in the past week at a restaurant or other prepared food/drink service, casino, personal
grooming or beauty service, moving or household cleaning/maintenance, hotel/motel, or a
taxi/limousine/shuttle service. If you have made multiple expenditures (e.g., bill, tip) at a
given establishment or type of service in the past week, provide separate details for each.
For example, if you stayed at a hotel and had tipped room service and a concierge, please
record those separately.”
Question 2A originally read: “In the last week, have you made any expenditures at a
restaurant or other prepared food service?” This question was changed to: “In the past week,
have you made any expenditures at a restaurant or other prepared food/drink service?”
For Question 2A, the following response options were added to the “Restaurant or Other
Prepared Food/Drink Service” category:
o “Bar”
o “Coffee Shops”
o “Food Cart/Truck”
o “Self-Service/Cafeteria/Buffets”
For Question 2B, the following response options were added to the “Casino” category:
o “Bar”
o “Full-Service Dining”
o “Self-Service/Cafeteria/Buffets”
For Question 2C, the question originally read: “In the last week, have you visited a barber or
hair stylist?” This question was changed to: “In the last week, have you made any
expenditures on personal grooming, beauty, and massage services?”
For Question 2C, the following response options were added to the new “Personal Grooming,
Beauty, and Massage Services” category:
o “Manicurist/Pedicurist”
o “Massage Therapist”
o “Waxing/Hair Removal”
o “Facial/Skin Care”
For Question 2C, there originally were no instructions informing participants to record
transactions separately. These instructions were added: “Provide separate details for each
expenditure. For example, if you had tipped a hair stylist in addition to tipping a manicurist,
please record those separately.”
Question 2D was added to the survey: “In the last week, have you made any expenditures on
moving or other household maintenance services?” The following response options were
added to the “Moving or Household Maintenance” category:
o “Professional Movers”
o “Maid or Cleaning Service”
o “Lawn/Gardening service”
Page 25









o “Handyman/Repairman”
For Question 2E, the following response options were added to the “Hotel/Motel” category:
o “Bar”
o “Full-Service Dining”
o “Self-Service/Cafeteria/Buffets”
For Question 2F, the question originally read: “In the last week, have you used a taxi or
limousine service?” This question was changed to: “In the last week, have you used a taxi,
limousine, rideshare, or shuttle service?”
For Question 2F, the response option “App-Based Taxi” was changed to “Ride-Share service
(e.g., Uber or Lyft)”
First non-monetary gift question changed from “Value of non-monetary gift* you provided” to
“Did you give a non-monetary gift* for this service? If so, can you estimate its value?”
Non-monetary gift description was changed from the following: “*If a portion of the gratuity
or tip took the form of a non-monetary payment (e.g., a coupon or event tickets), indicate the
cash equivalent amount in this column. Note that non-monetary transfers should only be
counted as tips if they were used as payment for immediate service and are used as a
substitute for a monetary tip. Non-monetary transfers that are used as gifts/personal tokens
of appreciation should not be counted as tips.” The revised description: “*A non-monetary
gift could be something like a coupon or event tickets. Note that non-monetary gifts should
only be recorded if they were used to compensate for service. Non-monetary gifts that are
given as personal tokens of appreciation should not be recorded.”

Page 26

Appendix C: Survey Edits from Usability Testing





Changed introduction from the following: “Welcome to the 2015 Survey on Consumer Tipping
Behaviors. The purpose of this survey is to determine payments for commonly tipped
services in the United States.” The revised introduction: “Welcome to the 2015 Survey on
Consumer Behaviors. The purpose of this survey is to explore consumer’s behavior with
respect to specific goods and services in the United States.”
Changed “In the last [TIME], have you made any service-related payments at a…” to “In the
last [TIME], have you made any transactions at a…” for each service category question.
Changed the uniform language for multiple payments for most services to a unique question
for each with examples. Text for “limousine, taxi, or shuttle service” service category was not
altered because it was determined that it was not reasonable to expect that someone could
make multiple payments to the same cab. Language changed from the following: “On the
next page, we will ask you to record one transaction you have made for [SERVICE
CATEGORY]. Do not record transactions for which you have already provided information.”
o “On the next page, we will ask you to record one transaction you had at a restaurant
or other prepared food/drink service. If you have had multiple transactions at the
same establishment (even if during the same visit), please record each transaction
separately. For example, if you made separate payments for a drink at the bar and a
meal at the table, please record these transactions separately.”
o “On the next page, we will ask you to record one transaction you have made at a
hotel/motel. If you have engaged in multiple transactions at the same establishment
(even if during the same visit), please record each transaction separately. For
example, if you engaged in separate transactions for valet service and luggage
assistance during the same visit, please record these transactions separately. Do not
record transactions for which you have already provided information.”
o “On the next page, we will ask you to record one payment you have made at a
personal grooming, beauty, or massage service. If you have made multiple payments
at the same establishment (even if during the same visit), please record each
transaction separately. For example, if you made separate payments to your hair
stylist and your manicurist during the same visit, please record these transactions
separately. Do not record transactions for which you have already provided
information.”
o “On the next page, we will ask you to record one payment you have made at a moving
or household maintenance service. If you have made multiple payments at the same
establishment (even if during the same visit), please record each transaction
separately. For example, if you made separate payments to your gardener and your
landscaper during the same visit, please record these transactions separately. Do not
record transactions for which you have already provided information.”
o “On the next page, we will ask you to record one transaction you have made at a
casino. If you have engaged in multiple transactions at the same establishment (even
if during the same visit), please record each transaction separately. For example, if
you engaged in separate transactions to your casino dealer and your floor server

Page 27



















while playing at the same table, please record these transactions separately. Do not
record transactions for which you have already provided information.”
Added “Fast Casual” as a service for the “Restaurant or Other Prepared Food/Drink Service”
category.
Added questions asking “What was the average nightly rate for the room?” and “How many
nights did you stay at this hotel?” after indicating that they had a transaction at a
hotel/motel.
Added “Makeup Artist” as a service for the “Personal Grooming, Beauty, or Massage
Services” category.
Added “Equipment Rental” as a service for the “Moving or Household Maintenance Services”
category.
Added “Valet” as a service for the “Casino” category.
Changed “Ride-Share service (e.g., Uber or Lyft)” to “Uber, Lyft, or other Ride-Share service.”
Added question after they select their service for “Did you pay for this product or service?
(Yes/No).” If “yes,” they move to the payment options for the bill. If “no,” they move to the
question asking if an automatic tip was added by the business.
Removed response option “Not Applicable – there was no bill for this service.” from the
question asking what payment type they used to pay their portion of the bill.
Bolded the language “(after tax, before automatic or voluntary tip)” for the question about
the amount of the bill paid.
Removed the response option for the automatic tip question stating, “Yes, but amount not
provided.”
Added question “Did you leave a voluntary tip for this service?” after the question for the
automatic tip. If “yes,” they move forward to the question about the type of tip.
Removed the response option for the voluntary tip type question “There was no tip for this
service.”
Included description of non-monetary gift for the question of voluntary tip payment type.
Removed description from follow-up questions about non-monetary tip value. “*Examples of
non-monetary gifts are: concert tickets, a bottle of wine, or a meal. Note that non-monetary
gifts should only be recorded if they were used to compensate for the service. Non-monetary
gifts that are given as personal tokens of appreciation should not be recorded.”
Follow-up non-monetary gift question changed from “Did you give a non-monetary gift* for
this service? If so, can you estimate its value?” to “Estimate the value of the part of the tip
that is non-monetary.”

Page 28