Incentive Rationale

Rationale for Increasing Incentives and Measuring Ghost Awareness

Rationale for Increasing Incentives

RTI is seeking OMB approval for two changes to the respondent incentive procedure. First, we would like to include $1 in the Panel Maintenance Mailing, in order to boost returns of address updates and confirmations from households. Second, we would like to offer a higher incentive to respondents who complete the follow-up survey online during a three week early release period, in order to reduce cost of in-person interviewing and reduce nonresponse. These practices have been demonstrated to increase response rates and reduce costs and nonresponse in other studies.

Prepaid Incentive for Panel Maintenance Mailing

RTI plans to send a locator mailer to participating households, informing them of the upcoming ExPECTT data collection and asking them to update their contact information if planned moves have occurred since the baseline survey. Respondents can update their contact information by calling a toll-free number or by completing and returning an enclosed postcard. In order to encourage response to this mailing, RTI recommends including a $1 cash incentive in the mailing.

Studies show that prepaid incentives are effective, and that they work better than promised incentives.(National Youth in Transition Database 2010; Singer 2012) Even small prepaid incentives can have a meaningful influence on response rates. For example, one study found that a $1 prepaid incentive yielded a response rate of 71%, while a $50 promised incentive yielded a response rate of 57%.(James 1992) More recent studies show that prepaid token incentives of $1 or $2 have been demonstrated to increase response rates by 8% to 31% compared with no incentive,(Dillman 2009) and that a $5 prepaid incentive increased the response rate by 14% overall, and by 42% within the first week of survey administration.(LeClere 2012) Another study showed that a $1 prepaid incentive had no influence on the overall completion rate, but that it dramatically boosted the rate of completion within the first two weeks of survey administration compared with no incentive.(Ward 2014)

Prepaid incentives have been used by RTI in the past. For example, RTI achieved a 37% return rate on the UNC Renters and Homeowners Study when a cash incentive was included in the locator mailer. In comparison, the National Survey of Child and Adolescent Well-being, which provided no incentive, achieved a 15% to 20% return rate.

Incentive for Completing the Survey Online During an Early Response Period

A top priority for the ExPECTT follow-up data collection is to maximize response rate. This is critical for achieving sufficient power to answer key questions about campaign effectiveness. The design of the follow-up survey administration—in which respondents choose to complete the survey online or through an in-person interview—creates an opportunity to boost the response rate by means of an “early bird” incentive. Early bird incentives are when respondents are offered a higher incentive for completing a survey during a specified period early in the data collection. RTI would like to provide an incentive of $30 (rather than $20) to respondents who complete the online version of the ExPECTT follow-up survey during a three week early release period.

Two experimental studies examine the influence of early bird incentives on survey response.(LeClere 2012; Ward 2014) LeClare et al. found that offering respondents a $5 prepaid incentive as well as an additional promised $20 for completing a telephone survey within the first week of administration yielded an overall response rate of 65.1% compared with 57.9% for the $5 prepaid incentive only and 50.9% for a no incentive condition.(LeClere 2012) This represents an “early bird + prepaid incentive” response rate 12% greater than the prepaid-incentive-only condition, and 28% greater than the no incentive condition. Moreover, the early bird incentive yielded a response rate of 36.7% within the first seven days of administration, compared with 25.0% for the $5 prepaid incentive only and 17.6% for a no incentive condition.(LeClere 2012) This represents an “early bird + prepaid incentive” response rate 47% greater than the prepaid-incentive-only condition, and 100% greater than the no incentive condition. A similar study by Ward et al. found that early bird incentives did not increase the overall response rate, but that they markedly increased the rate of completions within the first two weeks of survey administration (p<0.05).(Ward 2014) Increasing response rate in the early weeks of the ExPECTT follow-up survey will result in a higher proportion on online interviews, and fewer resource-intensive in-person interviews. A higher rate of completes in the early weeks of survey administration will enable RTI to focus resources on respondents who are the most difficult to convert.

RTI has used the early bird incentive strategy in an effort to reduce nonresponse and cost for several studies, including the Beginning Postsecondary Students Longitudinal Study (BPS), the National Postsecondary Student Aid Study, and the Baccalaureate and Beyond Longitudinal Study field tests. An experiment conducted with 3,700 respondents from the 2012/14 BPS varied incentives to complete the survey during a four week early response period from $0 to $50. On the last day of the early response period, those who were offered less than $30 had a response rate of 19.7%, and those who were being offered $30 or more had a response rate of 36.2%. This means the response rate associated with an early bird incentive of $30-$50 was 84% greater than the response rate associated with an early bird incentive of less than $30.

Rationale for Measuring Ghost Awareness

The ExPECTT Media Tracking Brief Report, which is based on data collected during the first two months of the Real Cost campaign, shows that 21% of respondents report awareness of the Real Cost brand. To place this estimate in context, 23% report awareness of the fake brand Digital Youth Against Tobacco (DYAT) and 20% report awareness of the fake brand Rebellion. These data suggest that current estimates of Real Cost brand awareness are less a function of campaign exposure than of respondent error of some type. However, as the Real Cost campaign becomes more established, we expect brand awareness to exceed that of fake brands. The inclusion of fake brands in the survey, therefore, is designed to provide context for youth reports, which are influenced by inaccurate recall, intentional misreports response and other factors. These fake brands will enable RTI to contextualize brand awareness estimates and better track actual campaign awareness trajectories.

In the ExPECTT follow-up survey we measure youth awareness of the Real Cost brand and other real (truth, Tips) and fake (DYAT, Rebellion) brands. We also measure awareness of Real Cost and other real (truth, Tips) ads. In order to better place these awareness estimates in context, we would like to add a fake advertisement to the follow-up survey. We envision that a fake ad would be one developed for but not aired as part of the campaign or a real state ad that has not aired in some years. Such state ads are available through the CDCs Media Campaign Resources Center. Measuring reported awareness of an ad youth could not have seen will contribute to our understanding of actual Real Cost ad awareness.

References

Dillman, D. A. S., J.D.; and Christian, L.M. (2009). Internet, mail, and mixed-mode surveys: The tailored design method. Hoboken, NJ, John Wiley & Sons.

James, J. M., and Bolstein, R. (1992). "Large monetary incentives and their effect on mail survey response rates." Public Opinion Quarterly(56): 442-453.

LeClere, F., Plumme, S., Vanicek, J., Amaya, A., Carris, K. (2012). Household Early Bird Incentives: Leveraging Family Influence to Improve Household Response Rates. American Statistical Association Joint Statistical Meetings, Section on Survey Research.

National Youth in Transition Database (NYTD) (2010). Technical Assistance Brief No. 6 Understanding Incentives and Motivators for Participation in the Youth Outcome Survey. . http://www.nrccwdt.org/resources/nytd/docs/NYTD_TA_Brief_6_Understanding_Incentives_081010.pdf.

Singer, E. (2012). The Use and Effects of Incentives in Surveys. National Science Foundation. Washington DC.

Ward, C., Stern, M., Vanicek, J., Black, C., Knighton, C., Wilkinson, L. (2014). Evaluating the Effectiveness of Early Bird Incentives in a Web Survey. Federal Computer Assisted Survey Information Collection (CASIC) Workshops.

2