U. Incentives Experiments

Appendix U. Incentive Experiments

Household survey incentive experiments

FNS is requesting a pre-paid survey incentive of $5 and a post-survey incentive of a $35 Visa gift card for participants completing the 35 minute survey instrument, as described in SSA. FNS is also proposing an experiment, an increased post-pay incentive amount of $45, rather than $35, among subpopulations of nonrespondents to further reduce nonresponse bias, .

Post-Pay Experiment. FNS proposes increasing the post-incentive amount to $45, instead of $35, among subpopulations of nonrespondents in subsequent releases to reduce nonresponse bias. The current study design includes three separate releases of sample throughout data collection, where sample members are randomly assigned to a release. After the first sample release, the study team would review response rates by county to determine if there are significant differentials between subgroups of SNAP participants compared to SNAP nonparticipants. We hypothesize there may be differences in response rates between SNAP participants and SNAP nonparticipants given that nonparticipants are not receiving FNS benefits. If a response rate differential within individual counties exists, such that SNAP nonparticipants are less likely to participate, the team would increase the post-incentive value among SNAP nonparticipants randomly selected to be included in the second or third release. This approach attempts to use a preexisting characteristic of survey nonresponders (SNAP participation status) to maximize the benefit of incentive payments.

For these subgroups, FNS proposes a post-paid incentive of $45, instead of $35 to further encourage survey response and ultimately reduce nonresponse bias. The literature shows a positive relationship between higher post-pay incentives and higher response rates.^1,2,3 However, it is not always cost effective or beneficial to increase incentive amounts for the entire sample. Not all participants require an increased incentive to participate;⁴ further, simply increasing survey response rates does not necessarily reduce nonresponse bias in survey estimates.⁵ Research suggests that differential incentives are most effective in reducing nonresponse bias when they are offered among otherwise underrepresented study groups.^1,6 The study’s randomized multiple release sample design offers the study team a unique opportunity to learn firsthand about the sample population during the first release and provides the opportunity to tailor the data collection approach in subsequent releases based on known sample characteristics. If it is found that the subgroup of SNAP nonparticipants are less likely to participate in the study during the first sample release, increasing the post-incentive provided to these sample members might help increase response rates and reduce nonresponse bias in the most cost effective way. We will include subgroup analyses (including income within each state) as we review the effects of the experiment on completion rates to confirm the experiment did not have a disproportional impact by income, which could add additional bias.

1 Singer E, Ye C. The use and effects of incentives in surveys. The ANNALS of the American Academy of Political and Social Science. 2013;645(1):112–41.

2 Singer, Eleanor, et al. "The effect of incentives on response rates in interviewer-mediated surveys." Journal of official statistics 15.2 (1999): 217.

3 Singer, E., and R.A. Kulka. “Paying Respondents for Survey Participation.” In Studies of Welfare Populations: Data Collection and Research Issues. Panel on Data and Methods for Measuring the Effects of Changes in Social Welfare Programs, edited by Michele Ver Ploeg, Robert A. Moffitt, and Constance F. Citro. Committee on National Statistics, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press, 2002, pp. 105–128.

4 Groves RM, Singer E, Corning A. Leverage-saliency theory of survey participation: description and an illustration. The Public Opinion Quarterly. 2000;64(3):299–308.

5 Groves RM. Nonresponse rates and nonresponse bias in household surveys. Public opinion quarterly. 2006;70(5):646–75.

6 Lepkowski JM, Mosher WD, Groves RM, et al. Responsive design, weighting, and variance estimation in the 2006–2010 National Survey of Family Growth. National Center for Health Statistics. Vital Health Stat 2(158). 2013.