Attachment L - Cash Incentive Study Proposal for the ATUS

Incentive Study proposal for the American Time Use Survey

October 2, 2019

The American Time Use Survey uses debit card incentives to reach no-telephone-number households, but doing so has become costly. ATUS management is thus exploring other options, including the possibility of sending cash incentives rather than debit cards to these households. Doing so may save the program money that could be spent on other projects, such as expanding the use of incentives to include difficult-to-reach cases. This paper summarizes ATUS’s use of incentives and proposes a study to test the use of cash incentives.

Who receives incentives?

Since the survey began in 2003, the American Time Use Survey (ATUS) has mailed $40 incentive cards to sampled DPs (designated persons) for whom there are no phone numbers available from the Current Population Survey (CPS). This is done to minimize the probability of nonresponse bias due to the survey’s design as a telephone-based survey. DPs without phone numbers available from CPS are sent a brochure about the survey, a $40 debit card incentive, and a letter that contains an appeal to call an ATUS interviewer and complete the survey. After participating in the survey, these respondents are provided with the PIN (Personal Identification Number) they need to cash their debit cards. In this discussion, I refer to these no-telephone-number cases as regular incentive cases. In 2014-16, 6.2 percent of the ATUS sample were regular incentive cases.

With OMB permission, ATUS expanded its definition of no-telephone-number households first in 2008, and then again in 2011. Since mid-2011, when the second initiative was implemented, cases assigned one of the following call outcome codes after the first of eight weeks in sample are converted to incentive cases: 108 Number not in service; 109 Number changed, no new number given; 124 Number could not be completed as dialed; and 127 Temporarily not in service. These expanded incentive cases are sent the same materials that regular incentive cases receive. Interviewers also continue to try to reach the DPs associated with expanded incentive cases by phone, using the number provided in CPS and any new numbers identified through research. In 2014-16, 2.1 percent of the ATUS sample were expanded incentive cases.

Compared to cases with phone numbers¹, both regular and expanded incentive case DPs were more likely to be Black, Hispanic, less-educated, and to have lower annual household incomes. Expanded incentive case DPs were more likely to be younger (ages 15 to 34) compared to all cases with phone numbers. Table 1 provides information about the characteristics of DPs for cases with phone numbers, regular incentive cases, and expanded incentive cases in 2014-16.

Table 1. Descriptive info about telephone and no-telephone number cases in the ATUS sample for Jan-Dec sample months in 2014-16

	Total sample	Cases with phone numbers	Regular incentive cases	Expanded incentive cases
Number of cases per year	24,596	22,548	1,533	515
Percent of ATUS sample	100.0	91.7	6.2	2.1
Number of completes per year	11,000	10,260	569	172
Response rates	44.7	45.5	37.1	33.4
Percent of completes	100.0	93.3	5.2	1.6
Percent distribution, total	100.0	100.0	100.0	100.0
Age
15 to 24	12.5	12.4	12.2	18.3
25 to 34	18.1	17.9	19.7	22.4
35 to 44	19.1	19.2	16.8	18.3
45 to 54	16.8	16.9	16.2	15.0
55 to 64	14.6	14.7	14.4	11.8
65 to 74	10.6	10.6	10.5	8.8
75 and over	8.3	8.2	10.2	5.5
Race
White only	75.6	76.4	68.1	63.3
Black or African American only	17.5	16.8	25.0	29.0
Other	6.8	6.8	6.9	7.7
Hispanic or Latino ethnicity
Hispanic or Latino	17.6	17.3	21.0	23.1
Non-Hispanic	82.4	82.7	79.0	76.9
Educational attainment
High school diploma or less	43.6	42.5	54.4	60.0
Some college or associate degree	27.6	27.9	24.1	25.9
Bachelor's degree or higher	28.8	29.7	21.5	14.1
Annual household income
Less than $25,000	26.3	25.0	40.1	45.9
$25,000 to $49,999	26.0	25.8	27.7	28.3
$50,000 to $74,999	17.7	18.0	15.3	12.3
$75,000 or more	30.0	31.2	16.9	13.5

How much do debit card incentives cost?

In fiscal year 2018, ATUS debit card incentives cost the program approximately $152,000. This included $82,000 of Census staff time associated with managing the debit cards, including the time involved in contracting with the bank that administers the cards, funding and defunding cards, checking in card shipments, assembly and mailing of the cards, managing electronic files with card information, and creating and maintaining card tracking reports. (See table 2.)

An additional $70,000 expense was associated with backing the value of the incentive cards. After each card was issued, $44 in ATUS funds were set aside to provide the recipient with $40 and to cover the ATM fee (up to $4) should the recipient cash the card. These funds were tied up until the cards expired and, upon expiration, any unused funds were returned to the ATUS account and used to purchase new debit cards.² In FY 2018, about $25,000 was spent covering the face value of the debit cards and ATM fees. The remaining $45,000 cost to the program was incurred when we crossed from one fiscal year into the next. Because Congress provides the Bureau of Labor Statistics with single-year funding only, money associated with debit cards that are issued in one fiscal year and expire in the next cannot be recycled to purchase debit cards in the new fiscal year. This single-year funding restriction led to a loss to the program of approximately $45,000 in FY 2018, and accounts for the majority of the $70,000 cost associated with backing the cards.³

Table 2. Cost associated with $40 debit card incentives in FY 2018
	Annual costs
Census staff management costs	$82,000
Funds sent to bank vendor a	$70,000
Expired funds that bank vendor returned	$45,000
Approximate cost of cards cashed	$25,000
Total cost of debit cards	$152,000
a. Includes face value of debit cards, ATM fees, expired funds, and associated management fees.

What costs are expected with the use of cash incentives instead of debit cards?

In fiscal year 2017, when the U.S. Treasury contracted with a new debit card vendor, the costs associated with administering ATUS debit card incentives rose substantially. This was primarily because the new contract does not allow ATUS debit card funds to be carried across fiscal years. However, Census staff management costs also increased as the new bank vendor requires Census staff to perform functions that previously had been the responsibility of the debit card vendor. In addition to cost concerns, a general dissatisfaction with the new vendor and their unwillingness to resolve ongoing processing issues has motivated management to consider alternatives to sending debit card incentives.

ATUS management is considering the possibility of sending $5 or $10 cash incentives instead of prepaid $40 debit cards to incentive cases. As shown in table 3, the estimated annual costs associated with sending $5 incentives amounts to $82,500, and those costs associated with $10 incentives are approximately $96,000. In both cases, the largest expense is that associated with Census’s management of the incentives. Staff would need to order the cash delivery, check in cash shipments, secure funds at the National Processing Center (NPC), and assemble and mail the incentives. Because Census staff no longer would need to track funds and generate reports after the incentives are mailed, the management costs associated with cash incentives are expected to be approximately $57,000 per year, which is $25,000 less than the costs associated with managing the debit cards.

The ATUS program would incur bank fees each time cash is delivered to NPC. Banks charge approximately $12,000 per delivery, but a couple of factors could help to minimize these costs. First, NPC allows up to one year’s worth of incentive cash to be delivered at a time, and so at most ATUS would need to fund one delivery per year. Second, deliveries and their associated costs can be shared as long as ATUS is able to coordinate this with other surveys. Finally, approximately $13,500 per year would be needed to purchase $5 cash incentives, or twice this—$27,000 per year—would be needed to purchase $10 cash incentives.

The use of cash incentives is expected to save between $56,000 per year (if only $10 cash incentives are used) and $69,500 per year (if only $5 cash incentives are used) in costs associated with the ATUS program’s use of incentives. These funds could be spent on other aspects of the survey, such as expanding incentives to target groups with the lowest survey response rates.

Table 3. Approximate annual costs associated with sending $5 and $10 cash incentives
	Annual costs
Costs associated with use of only $5 incentives
Census staff management costs	$57,000
Funds sent as cash incentives a	$13,500
Bank fees associated with one cash delivery per year to Census b	$12,000
Total cost of cash incentives	$82,500
Costs associated with use of only $10 incentives
Census staff management costs	$57,000
Funds sent as cash incentives a	$27,000
Bank fees associated with one cash delivery per year to Census b	$12,000
Total cost of cash incentives	$96,000
a. This is a conservative estimate calculated using Census' current practice of ordering 225 debit cards per month.
b. Funds for up to one year's worth of incentives can be placed in Census' safe at the National Processing Center. Delivery fees are less if ATUS shares the delivery with another survey.

Proposed incentive study

While changing from the use of $40 prepaid debit cards to cash incentives has potential to save program funds, it is not clear whether and how this change might affect survey response. ATUS management thus proposes an incentive study.

The proposed ATUS incentive study has two goals:

1. To test the effectiveness of using $0, $5, and $10 cash incentives, where effectiveness will be measured in terms of survey response.

2. To test whether a $5 or $10 cash incentive can boost survey response among certain under-represented populations. In this case, the focus is on DPs who are 15- to 24-year-olds.

The reasoning behind the first goal was discussed earlier. If testing indicates cash incentives are a cost-effective alternative to $40 debit card incentives, then the ATUS program may implement this change as a cost-saving measure. Both $5 and $10 cash incentives will be tested and compared to a $0 control group, as well as historic data on the effectiveness of using $40 debit card incentives. Using cash incentives rather than debit card incentives may save money that could be used for other purposes, such as to expand the ATUS incentive program to include members of under-represented populations. The literature shows that the use of incentives, especially prepaid incentives, has been demonstrated to improve survey response.⁴ Thus, the second goal of the study is to learn whether sending $5 or $10 cash incentives to members of under-represented populations increases their rates of survey response. In all cases, when incentives are sent to DPs they are included in the informational packets that are mailed to DPs just prior to their first of up to eight weeks in sample.

As shown in table 4, people ages 15 to 19 and those ages 20 to 24 were the least likely to complete the ATUS in 2017. Survey response for 20- to 24-year-olds was 31.0 percent, lower than all other age groups, and those ages 15 to 19 had a response rate of 33.9 percent. Response rates for both of these age groups declined significantly from 2003 to 2017, and the drop was especially steep (down 24.4 percentage points) for 15- to 19-year-olds. The second goal of the proposed study will specifically test whether ATUS survey response rates for 15- to 24-year-olds increase when they are sent cash incentives.

Table 4. Response rates (percentages) by age group, 2003 and 2017
Age group	2003	2017	Change, 2003 to 2017 (percentage points)
15-19	58.3	33.9	-24.4
20-24	45.9	31.0	-14.9
25-29	48.5	35.2	-13.3
30-34	54.0	38.9	-15.1
35-39	55.3	42.2	-13.1
40-44	57.5	42.9	-14.5
45-49	58.5	42.1	-16.4
50-54	59.4	45.9	-13.5
55-59	63.8	50.9	-12.9
60-64	62.6	51.2	-11.5
65-69	62.4	54.5	-7.9
70 and over	53.9	50.3	-3.6

Note: These are unweighted response rates calculated using published ATUS Survey Methods files.

To study the effectiveness of sending cash incentives instead of $40 debit cards, ATUS will divide the regular incentive cases into three groups: a control group in which DPs will receive no incentive, a test group in which each DP will be sent $5 cash, and a second test group in which each DP will be sent $10 cash. In 2014-16, there was an average of nearly 1,540 regular incentive cases per year. Running the study for the full year will allow about 513 cases to be assigned to each of the three conditions. (See table 5.)

To test the second goal, the study will focus on cases that have 15- to 24-year-old DPs who have not been identified as regular incentive cases. The samples of 15- to 19-year-olds and 20- to 24-year-olds also will each randomly be divided into three groups: a control group that will not receive an incentive, a test group that will receive a $5 cash incentive, and a second test group that will receive a $10 cash incentive. Using 2014-16 averages, approximately 507 15- to 19-year-old DPs and 460 20- to 24-year-old DPs would be assigned to each group during the one-year test period. Normally, about 3 percent of 15- to 24-year-old DPs are converted to expanded incentive cases after their first week in sample. However, to ensure the integrity of this experiment, 15- to 24-year-old DPs would not be eligible to be converted to expanded incentive cases for the duration of the study. The costs associated with the cash incentives for each test group (but not other costs associated with the proposed study) also are shown in Table 5.

Table 5. Approximate number of cases assigned to the control and test groups in the ATUS incentive study, and estimated costs associated with each group
	ATUS sample cases per year	Number of cases per group	Costs of incentives (per group)
			$0	$5	$10
Study goal 1.
Regular incentive cases	1,540	513	$0	$2,567	$5,133
Study goal 2.
15- to 19-year-old DPs a	1,520	507	$0	$2,533	$5,067
20- to 24-year-old DPs a	1,380	460	$0	$2,300	$4,600
Total cost			$0	$7,400	$14,800
a Includes only cases that were not assigned regular incentives.

During the testing period, cases that are converted to expanded incentive cases will be sent $5 cash and an appeal to call and complete the ATUS. Expanded incentive cases will be identified consistent with how they have been identified in the past; however, only cases with DPs ages 25 and older will be eligible to become expanded incentive cases. Using the 2014-16 averages shown in table 1, sending $5 cash to expanded incentive cases with DPs ages 25 and over would cost about $2,100 (81.7 percent of 515 expanded incentives were sent to DPs ages 25 and over).

The use of incentives to increase survey response

Survey response rates are down generally, especially in the hard-to-reach populations being targeted in this research (i.e., those with lower educational levels, Black and Hispanic respondents, those with lower annual incomes, and younger respondents, as noted on page 2 above; Couper, 2013). This lowered response rate is even more pronounced in telephone surveys compared to other modes (Keeter et al., 2017). To help boost response and reduce nonresponse bias, some surveys offer prepaid, unconditional incentives to respondents. There is a wide body of research showing that incentives increase survey response in all modes, unconditional incentives perform better than promised incentives, and cash incentives work better than gifts, in motivating survey response (Dillman et al., 2014).

In reviewing the literature on prepaid incentives in Federal surveys, we found that in a 2011 field test of the National Household Education Survey, response rates to the survey screener were about 4 percentage points higher for households that were mailed $5 cash compared with those sent $2 cash. Those who were ethnic and racial minorities were even more responsive to the higher incentive amount: response rates for Black and African American households were 8 percentage points higher, those for Hispanics were 4 percentage points higher, and those for Asian and Pacific Islanders were 7 percentage points higher, compared with an increase of 3 percentage points for White households. They tested higher unconditional cash incentive amounts for the full survey (i.e., those who screened into the survey and participated in it) too, determining that $5 and $10 incentives are the right amounts to send to people who responded early to a screener questionnaire, whereas $10 to $15 cash incentives were appropriate amounts to send to those who were late responders to the screener questionnaire. Sending $20 cash incentives did not have much of an additional effect on survey response. (See McPhee and Hastedt, 2012.) In addition, another 2009 field test of the NHES tested $0, $5, and $15 unconditional, mailed cash incentives. The $15 cash incentive yielded the highest survey response (83.2 percent), and performed substantially better than a $5 cash incentive (77.2 percent). Those that received no incentive had a response rate of 71.6 percent. (See Brick, Williams, and Montaquila, 2011.)

A nationally-representative survey of households for Nielsen Ratings included an experiment that tested the use of mailed $0, $1, $2, $3, $4, $5, $6, $7, $8, and $10 cash incentives in securing response. Similar to the NHES studies, they found that the higher incentive amounts were more effective than the lower amounts. For households that had neither agreed nor refused to participate in the initial stage of the survey, $10 incentives performed much better than incentives of lesser amounts. For households that had refused to cooperate in the initial stage, but were mailed diaries anyway, the $10 incentives out-performed all conditions except $7 and $8. (See Trussell and Lavrakas, 2004.)

In addition to the 2011 NHES study, other studies have shown the effectiveness of unconditional incentives in motivating survey response among difficult-to-reach populations. Knoll et al. (2012) found that unconditional incentives are effective in motivating vulnerable populations to complete telephone surveys, while Felderer et al. (2018) found that unconditional incentives were effective in retaining lower-income respondents in a panel survey. Singer and Ye (2013) note the potential for incentives to increase and reduce non-response bias, especially when the incentives are targeted to sample members who would otherwise fail to respond.

Although some research has shown that token prepaid incentives as low as $2 are enough to boost response to simple screeners, mail, and web surveys (e.g., Singer & Ye, 2013), Dillman, et al., 2014 argue that for hard-to-reach subgroups, token incentives such as $2 are not enough to induce a social exchange, where respondents feel they must reciprocate the gesture and participate in the survey. Rather, these subgroups are more likely to see the incentive as an economic exchange, where they assess the cost and benefit of participating. The closer the incentive matches the income they could earn in the time needed to complete the survey interview, the more willing they are to participate in the survey, and as a result, incentive amounts closer to $10 are more motivating to them.

Thus, the literature shows that while prepaid incentives can help improve response rates and reduce nonresponse bias, they do not work equally well across all populations and subgroups of respondents. In addition to the variability in the effectiveness of incentives for different subgroups, very few studies have been conducted using prepaid incentives in panel or longitudinal surveys with multiple waves. In fact, the most recent meta-analysis on the effects of survey incentives (Mercer et al. 2015) excludes panel and longitudinal surveys. Of the studies we identified that examine the impact of prepaid incentives on response rates and nonresponse bias in panel surveys, all of them have used a minimum of $10 (or equivalent in foreign currency) as a starting point, have shown them to be effective at retaining panel participants over time, and at specifically reducing nonresponse bias in hard-to-reach populations (see Felderer et al., 2018; Yu et al., 2017; Kretschumer & Muller, 2017; Becker & Glasuer, 2018; Becker et al., 2019). Additionally, a review by To (2015) shows a history of Federal surveys experimenting with prepaid incentives with amounts of up to $20 to $40.

In terms of respondent burden, the ATUS is more demanding than a typical screener, mail, or web survey that uses token $2 to $5 incentives. In the ATUS, respondents are asked to participate in a phone interview, and provide interviewers with information about their personal lives, including all of the activities they did the past 24 hours, which can sometimes elicit sensitivity and requires careful focus and memory to reconstruct the previous day. In the case of the proposed research, for an incentive to be effective, it must be an amount that would motivate respondents to participate in a ninth wave of a panel (following 8 waves of the CPS, where some waves were conducted in-person and by a different interviewer), and to take the initiative to call-in to set up an interview since the subgroup of interest does not have a phone number on file. In addition to these barriers to participate, the top reason respondents refuse to participate in the ATUS is that they are tired of surveys, making it more difficult to recruit them into the interview.

Finally, with ATUS researching the possibility of moving away from the use of $40 debit card incentives, and with limited staff resources available to conduct and execute this research, it is critically important to conduct a study with a wide enough dispersion to detect differences in response rates and nonresponse bias for the targeted subgroups. Using token incentives of $2 to $5 would severely limit the ability to detect the optimal incentive amount for these groups and compare the response rates back to data using the $40 debit cards. Thus, ATUS proposes to build on previous research to test $0, $5, and $10 cash incentives, which would save the program money while maintaining the integrity of the research. As discussed earlier, individuals who will receive the incentives are members of difficult-to-reach populations. They are more likely to be racial and ethnic minorities, have less education and lower incomes, and to be younger than the rest of the ATUS sample. Targeting these individuals is an important strategy in minimizing the possibility of nonresponse bias. See the appendix for more information about the literature on incentives.

Implementing the study

Currently, there are six steps in the ATUS sampling process:

1. A stratified sample of households is selected from those that completed the eighth month of the CPS.

2. A DP, age 15 or older, is randomly selected from each household.

3. The DP is assigned a panel month.

4. Regular incentive cases are identified using the variable INCNTFLG. This variable is assigned a value of “1” if there is no phone number available from CPS and a value of “0” otherwise.

5. A reference day is randomly assigned to the case such that 10 percent are assigned a weekday and 25 percent are assigned to each weekend day.

6. Cases are randomly assigned to a panel week that indicates when interviewers will first contact the DP for an interview.

For study goal #1, cases will be assigned to the control and test groups in step 4, at the time regular incentive cases are identified. Assigning the test and control groups at this stage will ensure they are randomly assigned across reference days and panel weeks. Under the test design, no-telephone-number households (identified by INCNTFLG=1) will be divided into three groups, and each group will be re-assigned one of three new values: a “2” will be assigned to cases that will receive no incentive, a “3” will be assigned to cases that will receive a $5 cash incentive, and a “4” will be assigned to cases that will receive a $10 incentive. To accomplish this, cases with INCNTFLG=1 will be sorted by the variable STRATUM, which accounts for factors used to select the stratified sample (the presence of children in the household, race, and ethnicity). Values of INCNTFLG= 2, 3, and 4 will then be sequentially assigned (2, 3, 4, 2, 3, 4, 2, etc.) to each record until all eligible no-telephone-number DPs have had their INCNTFLG=1 value re-assigned to INCNTFLG= 2, 3, or 4.

For study goal #2, 15- to 24-year-old DPs with a phone number on file will be sent no, $5, or $10 incentives. Two groups of 15- to 24-year-olds will be excluded from the test: DPs with INCNTFLG= 2, 3, or 4, and DPs with allocated values of age identifying them as age 25 or more.⁵ Eligible cases will first be sorted by factors used to select the stratified sample and by age, using the variables STRATUM and PUAGE. Values of INCNTFLG= 5, 6, and 7 will then be sequentially assigned (5, 6, 7, 5, 6, 7, 5, etc.) to each record until all eligible DPs have had their INCNTFLG=0 value re-assigned to INCNTFLG= 5, 6, 7. To ensure the integrity of this incentive test, 15- to 24-year-olds will not be eligible to become expanded incentive cases for the duration of the study. Table 6, below, summarizes the different meanings for values of INCNTFLG.

Table 6. Values for INCNTFLG: What do they mean?
Timing	INCNTFLG value	Meaning
Initial value assigned in sampling, step 4	0	Telephone number available from CPS
	1	No telephone number available from CPS
Pre-incentive study	0	Not a regular incentive case
	1	Regular incentive case, debit card mailed
Incentive Study Goal #1	2	No incentive group
	3	$5 incentive group
	4	$10 incentive group
Incentive Study Goal #2	5	No incentive group
	6	$5 incentive group
	7	$10 incentive group

Staff at NPC will implement new procedures to process and track incentive cases chosen to receive $5 and $10 cash incentives. Cases with incentive flag values indicating they should be sent $5 incentives (INCNTFLG=3 or 6) will have their incentive flag values and a code word identifying them as $5 cash incentive cases printed on their label files. Similarly, cases with incentive flag values indicating they should be sent $10 (INCNTFLG=4 or 7) will have their incentive flag values and a code word identifying them as $10 cash incentive cases printed on their label files. Census also will implement a way to track instances in which the advance materials associated with cash incentive cases are returned to NPC because they were undeliverable.

Before the study begins, the materials mailed to DPs in advance of the survey will be modified. Regular incentive cases will no longer be sent a debit card and instructions on how to use the debit card. Additionally, the letter that is mailed to incentive cases will be modified to remove the reference to a $40 debit card. The advance letters sent to 15- to 24-year-olds who are included in part 2 of the study will be modified to ask them to call in and set up an appointment.

During the ATUS interview, respondents are asked whether they received the advance materials that were mailed to them just prior to the first call attempt. Their responses to this question are not recorded and output on the data files; however, Census can obtain this information by reviewing the ATUS audit trails. Census will implement procedures to review the audit trails and create a variable conveying whether respondents received the packet containing the cash incentive, letter, and other information about the ATUS. This information will be helpful in evaluating the effectiveness of the cash incentives.

Data for the proposed incentive study will be collected for 12 consecutive months, for the entire fiscal year 2020, after which they will be examined to determine answers for the study’s two goals and to conduct additional analysis. Are cash incentives effective in obtaining interviews from DPs for whom ATUS does not have a phone number from CPS (regular incentive cases)? If so, how does the use of cash incentives compare to past use of $40 debit card incentives? Also, do cash incentives boost response among 15- to 24-year-old DPs? If cash incentives perform well, which amount is optimum, and does this vary by the DP’s demographic or household characteristics? Depending on the study’s results, ATUS survey methods may be changed.

It will take some time to process and analyze the data collected during the proposed study, and so ATUS proposes the use of $5 cash incentives for regular and expanded incentive cases after the study’s collection period has ended, and until an analysis of the results can provide insight on the best way to proceed. Study data will be delivered with the fourth quarter 2019 to fourth quarter 2020 ATUS data files, and will be ready for analysis in May 2021. Results of the study will be available to inform the use of incentives in the ATUS in fiscal year 2022 and beyond. Thus, $5 cash incentives would be used for all regular and expanded incentive cases in fiscal year 2021.

Costs associated with the proposed cash incentive study

To implement the proposed cash incentive study at the beginning of fiscal year 2020 requires the following preparations in fiscal year 2019:

• Census will need to arrange and pay for a cash delivery that will cover the cost of the study’s $5 and $10 cash incentives. To ensure continuity in ATUS’s ability to mail incentives as we cross from fiscal year 2020 to fiscal year 2021, the delivery will cover the cost of incentives for 13 months, from October 2020 through October 2021.

• Sampling programs will need to be modified to identify and assign cases to the incentive test groups.

• Letters sent to incentive cases will need to be modified. A new letter will need to be drafted for 15- to 24-year-old DPs who are selected to receive cash incentives.

The costs associated with this work appear in table 7, below.

Table 7. Fiscal year 2019 costs associated with the cash incentive study
	Cost
Total cost of cash incentives for Oct. 2019 through Oct. 2020	$27,000
Cash incentives for study groups 1 and 2	$22,200
$5 cash expanded incentives for DPs ages 25 and over	$2,100
Cash incentives for regular and expanded cases in Oct. 2020	$900
Additional funds a	$1,800
Bank fees associated with one cash delivery to Census	$12,000
Programming to set up and assign indicators to test groups	$5,000
Modifying letters	$3,000
Total	$47,000
a. Calculations were based on 2014-16 averages. These funds provide a cushion in case the number of incentive cases in FY 2020 exceeds these averages.

Completion of this work will ensure ATUS is prepared to study the use of cash incentives, as outlined in this proposal, for the duration of fiscal year 2020. These costs will be offset by a reduction in FY 2020 funds needed for debit card incentives. The costs of managing the incentive study and compiling and processing the study’s data will be covered with annual ATUS funding set aside for the management of incentives. Incentive study variables will be added to ATUS survey methods files and delivered to BLS for analysis and publication.

Appendix: Relevant literature on unconditional incentives

More Money? The Impact of Larger Incentives on Response Rates in a Two-Phase Mail Survey

Cameron McPhee and Sarah Hastedt, 2012 FCSM.

https://s3.amazonaws.com/sitesusa/wp-content/uploads/sites/242/2014/05/Hastedt_2012FCSM_I-A.pdf

• Discusses National Household Education Survey, 2011 field test

• Two-phase mail survey (phase 1 is a screener, phase 2 is the main survey)

• Odds of response were 1.2 times higher with $5 incentive compared with $2 incentive for screener

• Final screener response rate was about 4 percentage points higher for the households that received $5 compared with the households that received $2

• For minority households, $5 incentive performed better than the $2 incentive; in terms of survey response:

  • +8 percentage points for Black/African American households

  • +4.3 percentage points for Hispanics

  • +7.3 percentage points for Asian/Pacific Islanders

  • For comparison, +3.3 percentage points for White households

• $5 and $10 is an appropriate amount to send with the phase two survey for those who responded early in phase one

• $10 to $15 is an appropriate amount to send with the main survey for those who were late respondents to the screener

• Increasing the incentive to $20 for the main survey had little additional effect on the response rate

Address-Based Sampling for Subpopulation Surveys

J. Michael Brick, Douglas Williams, Jill Montaquila

Public Opinion Quarterly, Vol. 75, Issue 3, Fall 2011, pages 409-428

https://doi.org/10.1093/poq/nfr023

• Discusses National Household Education Survey, 2009 field test

• Two-phase mail survey (phase 1 is a screener, phase 2 is the main survey)

• Cases were randomly assigned to the $0, $5, or $15 incentive groups; incentive was included with the initial mailing of the main topical questionnaire

• The $15 cash incentive helped the study achieve an 83.2 percent response rate; this was substantially better than the $5 cash incentive response rate of 77.2 percent; and it was a statistically higher (p< 0.05) response rate than the $0 case which had a response rate of 71.6 percent

• Response rates increased with the amount of the incentive, but at a diminishing rate

The Influence of Incremental Increases in Token Cash Incentives on Mail Survey Response: Is There an Optimal Amount?

Norm Trussell and Paul Lavrakas

The Public Opinion Quarterly, Vol. 68, No. 3 (Autumn, 2004), pp. 349-367

Published by: Oxford University Press on behalf of the American Association for Public Opinion Research

Stable URL: https://www.jstor.org/stable/3521675

• Many studies confirm the positive impact of cash incentives on response rates, but few study the optimum amount for maximizing response at an affordable level

• Study is a large-scale experiment that is a nationally-representative sample of households (for Nielsen ratings)

• Tests the effects of varying amounts of prepaid, noncontingent cash incentives on the quantity and quality of mail responses

• Tested cash incentives of $0, $1, $2, $3, $4, $5, $6, $7, $8, $10

• Households had neither agreed nor refused to participate in the mail (second) stage of the survey:

  • No significant difference in response between $0 condition and any of the $1 to $4 conditions

  • $5 to $8 conditions were not different from each other in terms of response; however, with the exception of the $7 condition, all showed a higher response than the $0, $2, and $4 conditions.

  • $10 condition was significantly more likely than any of the incentives below $8 to yield completed surveys

  • $10 condition performed much better than $8

• Households that were mailed diaries even though they refused to cooperate during the RDD (first) stage:

  • No difference between the $0 and $1 conditions in returning completed surveys

  • $2 and $3 conditions were more likely than the $0 condition to return completed diaries

  • $4 to $10 conditions were statistically more likely to return completed diaries than the conditions with lesser amounts of cash

  • $7 to $10 conditions were more likely to return completed diaries than conditions with incentives less than $7

  • Households in the $10 condition outperformed all conditions except $7 and $8

The Use and Effects of Incentives in Surveys

Eleanor Singer and Cong Ye

The Annals of the American Academy of Political and Social Science, Vol. 645, The Nonresponse Challenge to Surveys and Statistics (Jan. 2013), pp. 112-141

Published by Sage Publications, Inc.

Stable URL: https://www.jstor.org/stable/23479084

• Article is a systematic review of literature on survey incentives appearing since 2002

• Cantor, O’Hare, and O’Connor (2008) report: (1) prepaid incentives between $1 and $5 increase response rates from 2 to 12 percentage points compared to no incentives; (2) larger prepaid incentives lead to higher response rates, but at a decreasing rate; (3) although the effects of incentives have not declined over time, the baseline response rate without incentives has dropped substantially

• Findings reported in Cantor, et al, are consistent with findings from other interviewer-mediated surveys

• Relevant conclusions on the influence of incentives on survey response:

• Incentives increase response rates to surveys in all modes, and in cross-sectional and panel studies

• Monetary incentives increase response rates more than gifts, and prepaid incentives increase them more than promised incentives or lotteries

• There is no good evidence for how large an incentive should be; generally, response rates increase as the size of the incentive increases, but at a declining rate

• Incentives have clear potential for both increasing and reducing non-response bias, especially if targeted to sample members who would otherwise fail to respond

How much gets you how much? Monetary Incentives and Response Rates in Household Surveys

Andrew Mercer, Andrew Caporaso, David Cantor, Reanne Townsend

Public Opinion Quarterly, Vol. 79, No 1, Spring 2015, pp. 105-129

• Meta-analysis of research on the use of survey incentives, for studies conducted since 1987

• Seek to quantify a dose-response relationship between the value of a monetary incentive offered to respondents and response rates in general-population household surveys

• Estimate the effect of mode and other survey incentive characteristics on the dose-response relationship

• Shows a strong, nonlinear effect of incentives across all three modes of data collection (phone, mail, in-person) included in the analysis

• Prepaid incentives for mail surveys exhibited the largest increase in response rates on a per-dollar basis; prepaid incentives in telephone surveys exhibited the second largest per-dollar increase

The use of incentives in vulnerable populations for a telephone survey: a randomized controlled trial

Megan Knoll, Lianne Soller, Moshe Ben-Shoshan, et al.

BMC Research Notes. 2012; 5: 572.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3503563/

• 2008-09 Telephone survey, Surveying Canadians to Assess the Prevalence of Common Food Allergies and Attitudes towards Food Labeling and Risk

• Publically-funded survey conducted by McGill University researchers

• Tested the use of unconditional incentives to target vulnerable populations (lower education, lower socioeconomic status)

• Tested $0 versus $5 unconditional incentives (used $5 coupons for coffee or food); response rates were 36.1 percent for the incentive group and 28.7 percent for the no incentive group

• Unconditional incentives are effective in motivating vulnerable populations to complete telephone surveys

The Effect of Differential Incentives on Attrition Bias: Evidence from the PASS Wave 3 Incentive Experiment

Barbara Felderer, Gerrit Muller, Frauke Kreuter, and Joachim Winter

Field Methods, 2018, Vol. 30(1) 56-69

https://journals.sagepub.com/doi/pdf/10.1177/1525822X17726206

• An unconditional ~$10 prepaid incentive was effective at reducing attrition bias and increased retention in a Wave 3 of a panel survey in Germany

• The unconditional cash incentive was especially effective at retaining those with a lower propensity to participate in surveys (e.g., lower income respondents)

Other research cited:

Becker, R., Möser, S., & Glauser, D. (2019). Cash vs. vouchers vs. gifts in web surveys of a mature panel study––Main effects in a long-term incentives experiment across three panel waves. Social science research, 81, 221-234.

• Compared a ~$10 prepaid incentive to a ~$20 conditional incentive at Wave 6 of panel survey. Found that prepaid incentives led to higher response rates, especially those with lower education

Couper, M. P. (2013, December). Is the sky falling? New technology, changing media, and the future of surveys. In Survey Research Methods (Vol. 7, No. 3, pp. 145-156).

Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: the tailored design method. John Wiley & Sons.

Keeter, S., Hatley, N., Kennedy, C., & Lau, A. (2017). What low response rates mean for telephone surveys. Pew Research Center, 15, 1-39.

Kretschmer, S., & Müller, G. (2017). The wave 6 NEPS adult study incentive experiment. Methods, data, analyses: a journal for quantitative methods and survey methodology (mda), 11(1), 7-28.

• Compared a ~$10 prepaid incentive to a ~$20 conditional incentive at Wave 6 of panel survey. Found that prepaid incentives led to higher response rates, especially those with lower education

To, Nhien (2015). Review of Federal Survey Program Experiences with Incentives (unpublished report).

Yu, S., Alper, H. E., Nguyen, A. M., Brackbill, R. M., Turner, L., Walker, D. J., ... & Zweig, K. C. (2017). The effectiveness of a monetary incentive offer on survey response rates and response completeness in a longitudinal study. BMC medical research methodology, 17(1), 77.

• $10 incentive was effective at increasing Wave 4 response rates, especially in encouraging initially reluctant respondents

1 That is, cases that provided phone numbers in CPS and were not converted to incentive cases.

2 Approximately one-third of ATUS debit cards are cashed.

3 These funds were returned to the U.S. Treasury, as required by law.

4 For example, see Andrew Mercer, et al. “How much gets you how much? Monetary incentives and response rates in household surveys,” Public Opinion Quarterly, Spring 2015, pp. 105-129. This study is a meta-analysis of literature on the topic of incentives and household surveys.

5 Eligible cases will be identified by INCNTFLG=0, PUAGE values of 15 to 24, and PRTAGE values of 15 to 24.

19