D. Use of Incentives_clean_responsetoPRAO

Appendix D. Use of Incentives

OMB No. 0584-XXXX

Modernizing Channels of Communication
With SNAP Participants

March 2, 2020

Project Officer: Andrew Burns

Office of Policy Support

Food and Nutrition Service

U.S. Department of Agriculture

1320 Braddock Place

Alexandria, VA  22314

703.305.1091

[email protected]

Overview of Incentives for the Study

Individuals currently participating in the Supplemental Nutrition Assistance Program (SNAP) and who participate in a focus group for the Modernizing Channels of Communication With SNAP Participants study will receive a token of appreciation or incentives to offset child care and travel costs in the form of a $45 Mastercard, Visa, American Express or Discover gift card. Since focus group participants will be required to use their mobile device during the focus group (see Appendix K. SNAP Participants Focus Group Protocol), the incentive could also offset any data charges incurred. To ensure each focus group starts on time, participants who arrive to the focus group at least 10 minutes early will be eligible to receive an additional $10 Mastercard, Visa, American Express or Discover gift card given at the end of the focus group. The tokens of appreciation for focus group respondents are an essential component of the multiple methods that will be used to ensure the necessary sample sizes are met, improve data quality, and reduce nonresponse bias. The primary objectives of focus groups with SNAP participants, which will be conducted in English, are to assess the usability of mobile communication strategies (MCS) and to evaluate client satisfaction with MCS. USDA FNS along with its contractor Insight Policy Research (the Research team) will work with local SNAP office staff to recruit participants for the focus group who have experience using the State’s MCS and will hold the focus groups on site during the site visits.

SNAP applicants sitting in local office waiting rooms will be randomly selected to receive one of two $10 Mastercard, Visa, American Express or Discover gift cards if they complete the waiting room questionnaire. The purpose of the waiting room questionnaire is to assess the awareness and usefulness of MCS among potential SNAP participants. The study team will recruit applicants to complete the waiting room questionnaire in person at the local offices during the site visits.

1. Factors Supporting the Use of Incentives

This section provides more detail on the decision to provide an incentive and the amount of the incentive.

1. Improve data quality

An incentive is essential to obtain the sample sizes needed to understand the full range of various SNAP participants’ experiences with MCS in each State. Recruiting for two separate focus groups will be particularly important in States where multiple MCS are being utilized and where the study team wants to gain opinions from participants with a variety of experiences. Response rates for research have declined in recent years (Brick & Williams, 2013; Curtin, Presser, & Singer, 2005). Offering incentives is an effective way to improve response rates without compromising data quality (Singer & Ye, 2013). The knowledge that respondents will receive a monetary incentive to participate in a focus group is expected to increase their likelihood of spending the time to respond. In a recent study, Mercer and colleagues (2015) conducted a meta-analysis of the dose-response association between incentives and response and found a positive relationship between incentives with a greater monetary value and response rates for household telephone surveys offering incentives. Monetary incentives provided to respondents are particularly effective at increasing response rates for populations similar to SNAP participants, which include people with low income and lower educational levels.

2. Improve coverage of hard-to-reach or specialized respondents

The target population for the focus groups is low-income SNAP participants, who are considered hard to reach (Bonevski et al., 2014). Households in the study are specialized respondents because they are limited in number and difficult to recruit, and their lack of participation jeopardizes the outcomes evaluation. These groups may also have transportation concerns and childcare schedules that make it difficult to reach SNAP participants. Incentives may encourage greater participation among these groups. Similarly, focus groups may take place after working hours, to increase participation for working respondents, and

3. Reduce respondent burden

The incentives will reduce respondent burden because they can help offset the costs associated with participation, including childcare that may be needed while respondents complete the focus group, transportation costs to travel to the focus group venue, and fees associated with mobile phone usage during focus group

4. Reduce no-shows and diversify the sample

People who participate without incentives may have characteristics that are less common among, and therefore less generalizable to, other SNAP participants. Incentives reduce efforts to recruit low-income study participants and lower overall costs and time to achieve completion rates without affecting data quality (Dillman, 2000; Singer, 2006). In a meta-analysis, Mercer and colleagues (2015) estimated that surveys that promised $10 generated response rates 5 percentage points greater than surveys that offered no incentive. Fredrickson and colleagues (2005) found a $10 contingent incentive increased responses by 20 percentage points among Medicaid recipients, a similar population to those being recruited for this study. (Children’s income eligibility for Medicaid is at least 133 percent of the Federal Poverty Guidelines (FPG) but is higher in many States, and SNAP income eligibility is set at 130 percent of the FPG for the household size.)

Providing focus group participants and waiting room questionnaire participants with a monetary incentive reduces the number of no-shows and ensures a full range of experiences is reflected in the data, especially in populations defined as being in poverty; monetary incentives are particularly effective relative to other types (Yancey, Ortega, & Kumanyika, 2006; Singer, 2002; James, 1997; Groves et al., 2009; Singer & Ye, 2013; Parker & Tritter, 2006). Incentives can lower the rate of no-shows and offset the barriers to participation (Krueger, 2014; Groves, Singer, & Corning, 2000; Messer & Dillman, 2011) by encouraging those less interested in research to participate, including low-income respondents (Groves et al., 2006; Singer & Kulka, 2002). Several studies have shown offering incentives may improve representation for ethnic minority subgroups with low income and a lower level of education. Response rates among minorities and individuals with less education are generally lower for all types of surveys, especially those conducted by mail. For example, in a survey for the National Cancer Institute, the sample was divided into two strata: (1) one with a greater percentage of minority individuals, and (2) one with a smaller percentage. The response rate to the strata with a greater proportion of minority individuals was 12 percentage points lower than the rate for the strata with a smaller proportion (23 percent versus 35 percent; Westat, 2017). When testing within household selection procedures, Olson and colleagues (2014) similarly found all procedures led to underrepresenting non-Whites, Hispanics, individuals with less education, and individuals in the lowest income groups. The population of interest in the current study is comparable to the populations discussed earlier in this section. This study will be recruiting participants who currently participate or are eligible to participate in SNAP.

2. Past Experience Using Incentives in Focus Groups With Similar Low-Income Populations

The study team is proposing that respondents who complete the 90-minute focus group (see appendix K: SNAP Participants Focus Group Protocol) receive a $45 incentive, an amount based on incentives that were approved for similar data collections in other recent studies with similar burden among low-income populations. For example, The Examination of Cash Nutrition Assistance Program Benefits in Puerto Rico study for FNS (Office of Management and Budget [OMB] Control Number 0584-0597; expiration date March 31, 2015) used $50 gift cards to increase participation for in-person focus groups among low-income participants of Puerto Rico’s Nutrition Assistance Program. The WIC Cost Containment Study (OMB Control Number 0584-0627; expiration date September 30, 2020) provided a $30 incentive to Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) participants to complete a 30-minute telephone survey and a $30 incentive to former WIC participants to complete a 20-minute telephone survey. For the Evaluation of the Pilot Project for Canned, Frozen, or Dried Fruits and Vegetables in the Fresh Fruit and Vegetable Program for FNS (OMB Control Number 0584-0598; expiration date September 30, 2017), $50 incentives were offered to community members, including low-income parents, who completed 1-hour telephone interviews. The Evaluation of the Summer Food Service Program Participant Characteristics used $25 prepaid gift cards to increase the participation of parents/caregivers of participants and eligible nonparticipants in 30-minute telephone interviews (OMB Control Number 0584-0595; expiration date April 30, 2016).

The incentives for the current study will be offered equally to all potential focus group and waiting room questionnaire participants who are deemed eligible by the respective screeners. Based on the evidence discussed earlier in this section FNS/USDA plans to offer all focus group participants a $45 Mastercard, Visa, American Express or Discover incentive, which will be distributed to participants in-person at the end of the focus group. As discussed in appendix C (Pretest Methods and Summary of Findings), pretest focus group respondents received a $45 gift card for their participation. Pretest respondents are not part of the study and cannot receive the incentive again.

In summary, the planned incentives for the current and former SNAP participant focus group and waiting room questionnaires are designed to without enticing promote cooperation and improve data quality by minimizing nonresponse bias and improving sample representativeness.

References

Bonevski, B., Randell, M., Paul, C., Chapman, K., Twyman, L., Bryant, J., . . . Hughes, C. (2014). Reaching the hard-to-reach: A systematic review of strategies for improving health and medical research with socially disadvantaged groups. BMC Medical Research Methodology, (14)42.

Brick, J. M., & Williams, D. (2013). Explaining rising nonresponse rates in cross-sectional surveys. Annals of the American Academy of Political and Social Science, 645(1), 36–59.

Curtin, R., Presser, S., & Singer, E. (2005). Changes in telephone survey nonresponse over the past quarter century. Public Opinion Quarterly, 69(1), 87–98.

Dillman, D. (2000). Mail and internet surveys: The tailored design method (2nd ed.). New York, NY: John Wiley & Sons.

Fredrickson, D. D., Jones, T. I., Molgaard, C. A., Carman, C. G., Schukman, J., Dismuke, S. E., & Ablah, E. (2005). Optimal design features for surveying low-income populations. Journal of Health Care for the Poor and Underserved, 16, 677–690.

Groves, R., Fowler, F., Couper, M., Lepkowski, J., Singer, E., & Tourangeau, R. (2009). Survey methodology (2nd ed., pp. 205–206). New York: Wiley.

Groves, R., Singer, E., & Corning, A. (2000). Leverage-saliency theory of survey participation: Description and an illustration. Public Opinion Quarterly. 64(3), 299–308.

Groves, R. M., Couper, M. P., Presser, S., Singer, E., Tourangeau, R., Acosta, G., & Nelson, L. (2006) Experiments in producing nonresponse bias. Public Opinion Quarterly, 70(5), 720–736.

James, T. L. (1997). Results of wave 1 incentive experiment in the 1996 survey of income and program participation. Proceedings of the Survey Research Methods Section (pp. 834–839). Washington, DC: American Statistical Association.

Krueger, R. A. (1994). Focus groups: A practical guide for applied research. Thousand Oaks, CA: Sage Publications, Inc.

Mercer, A., Caporaso, A., Cantor, D., & Townsend, R. (2015). How much gets you how much? Monetary incentives and response rates in household surveys. Public Opinion Quarterly, 79, 105–129.

Messer, B., & Dillman, D. (2011). Surveying the general public over the internet using address-based sampling and mail contact procedures. Public Opinion Quarterly, 75, 429–457.

Olson, K., Stange, M., & J. Smyth. (2014). Assessing within-household selection methods in household mail surveys. Public Opinion Quarterly, 78, 656–678.

Parker, A., & Tritter, J. (2006). Focus group method and methodology: Current practice and recent debate. International Journal of Research & Method in Education, 29, 23–37.

Singer, E. (2002). The use of incentives to reduce non response in households surveys. In R. Groves, D. Dillman, J. Eltinge, & R. Little (Eds.), Survey non response (pp. 163–177). New York: Wiley.

Singer, E. (2006). Introduction: Nonresponse bias in household surveys. Public Opinion Quarterly, 70(5), 637–645.

Singer, E., & Kulka, R. A. (2002). Paying respondents for survey participation. In M. Ver Ploeg, R. A. Moffitt, & C. F. Citro, Committee on National Statistics, Division of Behavioral and Social Sciences and Education (Eds.), Studies of welfare populations: Data collection and research issues. Panel on data and methods for measuring the effects of changes in social welfare programs (pp. 105–128). Washington, DC: National Academy Press.

Singer, E., & Ye, C. (2013). The use and effectives of incentives in surveys. Annals of the American Academy of Political and Social Science, 645(1), 112–141.

Westat. (2017). Health Information National Trends Survey 5 (HINTS 5), Cycle 1 methodology report. Retrieved from https://hints.cancer.gov/docs/methodologyreports/HINTS5_Cycle_1_Methodology_Rpt.pdf

Yancey, A., Ortega, A. & Kumanyika, S. (2006). Effective recruitment and retention of minority research participants. Annual Review of Public Health, 27, 1–28.