Change Justification Memo

United States

Department of

Agriculture

Food and

Nutrition

Service

3101 Park

Center Drive

Alexandria, VA

22302-1500

TO: Julie Wise

OMB Desk Office

Office of Information and Regulatory Affairs

Office of Management and Budget

THROUGH: Rachelle Ragland-Greene, FNS Information Collection Officer

Planning and Regulatory Affairs Branch

Food and Nutrition Service

and

Ruth Brown, OCIO Desk Officer

FROM: Hoke Wilson

Project Officer

Office of Research and Analysis

Food and Nutrition Service

SUBJECT: Justification for Changes to the 2012, Evaluation of the Household-Based Summer Demonstrations on Food Insecurity Among Children (SEBTC), OMB# 0584-0559

As part of its efforts to end child hunger, the Food and Nutrition Service (FNS) of the U.S. Department of Agriculture (USDA) is studying alternative approaches to providing food to children in the summer months. The 2010 Agriculture Appropriations Act (P.L. 111-80) authorized and provided funding for the USDA to implement and rigorously evaluate the Summer Food for Children Demonstrations, one component of which is the Summer Electronic Benefits Transfer for Children (SEBTC). The first year of the SEBTC, 2011, served as a “proof-of-concept” (POC) test of the administration and evaluation methodologies of five demonstration sites in preparation for the “full implementation” of the demonstration in 2012. In 2012, the number of demonstration sites will expand from five to 14 sites and the number of low-income child beneficiaries will increase from 12,500 to 75,000. The changes recommended here are intended to ensure a successful full implementation and evaluation study and are derived from the lessons learned from the POC year.

• Increasing the incentive for the summer survey from $10 to $25 and distributing the baseline and summer survey incentives on a pre-paid Visa card rather than a retailer specific gift card. In the POC year approximately 28% of households who completed a baseline survey did not complete a summer survey. In order to rectify this we propose to increase the incentive for completing the summer survey from its current level of $10 to $25. Study participants would be informed in the Advance letter that they would receive $10 for completing the spring survey and $25 for the summer survey, for a total of $35. With 27,000 households responding, we expect to incur an expense of $405,000, well within our budgeted reserve funds for remedial data collection actions.

To enhance the attractiveness of survey participation further, we also propose to provide incentives for both the baseline and summer surveys on pre-paid Visa cards at a cost of $1 per card for a total expense of $54,000. During the POC year incentives were provided on two specific retailers’ gift cards. Especially in rural areas where the distance to the specified retailer might be great, this made the redemption of the incentive card less than fully attractive. The spending flexibility provided by the Visa card will further incentivize survey response.

• Reducing respondent burden by making the baseline (spring) survey shorter and more “respondent friendly”. We believe that a contributing factor to the lower than desired summer survey response rate in the POC year may have been a perception that the baseline survey was too burdensome. Instrument pre-testing indicated that the baseline survey should take 25 minutes to complete. Administration during the POC year indicated that it averaged about 35 minutes (and almost 40 minutes for Spanish Speakers). To reduce burden we propose dropping some baseline content, such as food frequency consumption of the focal child, and streamlining questions on household food expenditures as these items are principally used as summer outcome measures to determine the relative impacts between treatment and control groups. While much shorter, the survey also includes several new questions that were added after completing the impact analysis in order to improve the impact analysis for next year. To facilitate the review of proposed changes, Table 1 lists the section headings of the original and proposed baseline instruments and the reasons for the proposed changes.

Table 1- Changes and Reasons Spring Baseline Questionnaire (attached) provides a crosswalk between the original baseline survey (attachment A) and the revised baseline survey (attachment B).

Attachment A provides a copy of the currently approved baseline survey with text highlighted in yellow to indicate that it will be deleted. Text highlighted in green will be reworded.

Attachment B provides a copy of the 2012 proposed baseline (spring) survey. New questions/text are highlighted in blue. In addition, because some respondents found the language asking for permission to review EBT transaction data at the end of the baseline survey unsettling, we propose rewording it and moving it to the initial survey participation recruitment letter.

Attachment C provides a copy of the 2012 proposed Advance letter to the baseline survey with the additional reworded language highlighted in green.

• Modifying the introduction and Advance letter for the summer survey. Because the introduction to the summer survey is similar to that of the baseline survey, some respondents during the POC year believed that they had already completed the survey. To remedy this problem we propose modifying the introduction to the summer survey, as well as its Advance letter to make it more distinct from the baseline survey. In addition, as with the baseline survey, some questions were modified or added in order to improve next year’s impact analysis. Some questions were dropped to stay within the proposed average administration time of 30 minutes, keeping in mind that not all sections are answered by all respondents. The control group does not answer questions about SEBTC benefits in Section G (Program Participation – Household), and respondents to the baseline survey do not answer questions I1-I14 in Section I (Caregiver Demographics). The summer instrument took an average of 25 minutes to administer in the POC year, requiring slightly less time for those who completed the baseline survey. Table 2 lists the section headings of the original and proposed summer instruments and the reasons for the proposed changes.

Table 2- Changes and Reasons Summer Questionnaire (attached) provides a crosswalk between the original summer survey (attachment D) and the revised summer survey (attachment E).

Attachment D provides a copy of the currently approved summer survey with text highlighted in yellow to indicate that it will be deleted, and the language to be changed highlighted in green.

Attachment E provides a copy of the 2012 proposed summer survey. New questions/text are highlighted in blue.

Attachment F provides a copy of the 2012 proposed Advance letter with reworded language highlighted in green and annotated comments describing the proposed changes.

• In addition, the process study instruments were revised and streamlined for three reasons.  First, since in the full demonstration year there will be 10 grantees who will collectively be implementing SEBTC in 14 demonstration areas, the protocols were customized. Secondly, in the full demonstration year, there will be two rounds of data collection and the original protocols were developed for three rounds of data collection. Finally, some sections were shortened and some questions eliminated as some information was found not to be needed for the process study in the POC year. Table 3 provides a crosswalk between the POC year protocols and the proposed protocols for 2012 and the reasons for the proposed changes.

Attachment G includes the original process study instruments, with annotated headings describing the changes to be made for 2012.

Attachment H includes the proposed revisions to the process study instruments for 2012.

All of the proposed changes are requested to improve the quality and/or efficiency of the evaluation. In particular, changes to the baseline survey and increased incentive in the summer are critical to insuring that response rates for the 2012 full implementation are high enough to provide the level of statistical power and precision necessary for a definitive analysis and evaluation of the impact of the SEBTC demonstrations.

Attachments

2