SEBTC Revised OMB Part B 3-27-14

Contents
B.1

Respondent Universe and Sampling Methods ............................................................................... 1
B.1.1 Sampling Frame and Household Identification ..................................................................... 1
B.1.2 Consent .................................................................................................................................. 2
B.1.3 Random Assignment ............................................................................................................. 2
B.1.4 Sample Size ........................................................................................................................... 2
B.1.5 Two-Phase Sampling Plan..................................................................................................... 3
B.1.6 Response Rates ...................................................................................................................... 3
B.1.7 Analysis of Nonresponse Bias ............................................................................................... 3

B.2

Procedures for the Collection of Information ................................................................................ 4
B.2.1 Statistical Methodology for Stratification and Sample Selection .......................................... 4
B.2.2 Estimation Procedures ........................................................................................................... 4
B.2.3 Degree of Accuracy Needed: Statistical Power and Minimum Detectable Differences ....... 5
B.2.4 Unusual Problems Requiring Specialized Sampling Procedures .......................................... 6
B.2.5 Use of Periodic Data Collection Cycles to Reduce Burden .................................................. 6

B.3

Methods to Maximize Response Rates and Deal with Nonresponse ............................................ 6
B.3.1 Advance Letters ..................................................................................................................... 6
B.3.2 Field Follow-Up .................................................................................................................... 6

B.4

Tests of Procedures or Methods to be Undertaken ....................................................................... 7

B.5

Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing
Data .................................................................................................................................................... 7

References .................................................................................................................................................... 8

January 24, 2011

i

Part B of the Justification for this information collection activity, the FNS Summer Electronic Benefits
Transfer for Children (SEBTC) Demonstration: Evaluation Findings for the Fourth Implementation Year,
addresses the five points outlined in Part B of the OMB guidelines.

B.1

Respondent Universe and Sampling Methods.

Describe (including a numerical estimate) the potential respondent universe and
any sampling or other respondent selection method to be used. Data on the
number of entities (e.g., establishments, State and local government units,
households, or persons) in the universe covered by the collection and in the
corresponding sample are to be provided in tabular form for the universe as a
whole and for each of the strata in the proposed sample. Indicate expected
response rates for the collection as a whole. If the collection had been conducted
previously, include the actual response rate achieved during the last collection.
In this section, we describe the procedures that will be used to obtain existing data from vendors and
School Food Authorities (SFAs) on the sample of households and children that participate in the SEBTC
benefit demonstrations, including:
•

Procedures for obtaining informed consent;

•

Random assignment to the treatment and control conditions;

•

Sample sizes;

•

Use of a two-phase sampling plan; and

•

Response rates.

B.1.1 Sampling Frame and Household Identification
The 5 States (Delaware, Michigan, Nevada, Washington, and Oregon) included in the current evaluation
have been previously selected and participated in the 2013 demonstrations. For the 2013 evaluation, the
households within each site were sampled for the demonstration and evaluation from sampling frames
that were comprised of school children in grades K-12 certified for free or reduced-price NSLP meals in
the SFAs in the respective demonstration areas. A household is eligible to participate in the demonstration
if it includes at least one child certified for free and reduced price lunch (FRP) in grades K-12. For 2014
evaluations, individuals/households will not be re-contacted.

1

B.1.2 Consent
Obtaining consent for participation in the 2014 evaluation is not needed because the households that will
participate are the same households that participated in the demonstration and evaluation in 2013, at either
the $60 or $30 benefit level, respectively, and will not be re-contacted. The households will continue to
receive the same level of benefit as they did in 2013. Data collection will only involve obtaining EBT
transaction data from grantees and EBT vendors as well as conducting qualitative interviews with the
same grantees (Appendix C).

B.1.3 Random Assignment

Random assignment was used in 2013 to assign households to either $60 SEBTC benefit group or the $30
benefit group. Random assignment is not applicable for this evaluation because the data collection will
involve obtaining EBT transaction data only on those same households and conducting qualitative
interviews with grantees.

B.1.4 Sample Size

The 2014 Evaluation of the SEBT for Children will include the 6 sites that participated in the 2013 year.
The total sample size will be transaction data from approximately 22,800 households that participated
(11,400 were issued $60 benefits and 11,400 were issued $30 benefits) for the 2013 demonstration.

2

B.1.5 Two-Phase Sampling Plan

In 2013, participating SFAs constructed lists of households with children certified for FRP meals. After
working with SFAs and others to obtain consent from families (by an active process), each grantee sent a
list of consenting households to the evaluation team. The team then randomly assigned their families to be
in either the $60 SEBTC benefit group or the $30 benefit group, with the objective of assigning 5,000 to
14,000 children (depending on the site) to receive a benefit—half to receive $60 per child and half to
receive $30 per child. In all six sites, all households that were randomly assigned to receive either the $60
or $30 benefit were included in the evaluation sample and were contacted to participate in the household
survey. Sampling for the 2014 demonstration and evaluation is not needed because
individuals/households will not be contacted in this phase of this information collection request. The data
collection will involve receiving EBT transaction data from the same households and conducting
qualitative interviews with grantees on implementation.

B.1.6 Response Rates

Some households that received benefits in 2013 may not be included in the 2014 demonstration because
they are no longer eligible or they have moved out the of the area. Response rates for obtaining EBT
transaction data from grantees and EBT vendors and qualitative interviews with grantees will be 100%
because these grantees vendors and households participated in the 2013 demonstration and evaluation.

B.1.7 Analysis of Nonresponse Bias

Response rates for data collection will be 100% because it involves obtaining EBT transaction data and
from grantees and EBT vendors who participated in the demonstrations in 2013 as well as conducting
qualitative interviews with the same grantees.

3

B.2

PROCEDURES FOR THE COLLECTION OF INFORMATION

Procedures for the collection of information addressed below include:
•

Statistical methodology for stratification and sample selection;

•

Estimation procedure;

•

Degree of accuracy needed for the purpose described in the justification;

•

Unusual problems requiring specialized sampling procedures; and

•

Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

B.2.1 Statistical Methodology for Stratification and Sample Selection
Stratification and sample selection are not applicable for this study as it is a follow-up to a previous study.
Data collection will involve obtaining EBT transaction data from grantees as well as conducting
qualitative interviews with the same grantees who participated in the 2013 SEBTC demonstrations. The
grantees (States and Tribal Organizations) will be asked to sign a Memorandum of Understanding that
describes the data collection required for the evaluation (Appendix F).

B.2.2 Estimation Procedures

In this section, we discuss our plan for presenting descriptive statistics that will describe the use of
benefits by households that participate in the 2014 demonstrations. Households will be not be contacted;
the information will be obtained from the EBT transaction data.
Descriptive Analysis

For the 6 demonstration sites and administrative data from households that is collected in 2014
demonstrations, we will present several tables and graphs to provide descriptions of household use of
benefits and spending patterns. These descriptive analyses will provide answers to the research questions
listed in Part A.

4

The analysis will use SEBTC transaction data collected from the evaluation sites. Grantees and EBT
vendors will provide data on benefit issuances, redemptions, and other transactions (such as returns and
reversals) for each month of the SEBTC benefit period. Data from the SNAP and WIC systems provided
the date, time, and total dollar value of each purchase transaction. In addition to the purchase-level data,
the WIC data provided separate transactions for each category of food issued and redeemed, allowing for
the analysis of redemptions at the aggregate and food category levels for the WIC-model sites. The dollar
value of WIC benefits issued was determined for each site by multiplying the quantity issued by the
average unit price in the site for each food category, and then summing across the food categories. Thus,
the value of the benefit per child varied across sites, both within and between States, based on differences
in food prices.
The transactions for each household will be aggregated to produce net amounts for benefits issued and
redeemed for each issuance cycle, and then summed the monthly benefits issued and redeemed for the
summer (taking into account benefits carried over from month to month in the SNAP sites).
Subgroup Analysis

The evaluation provides descriptive information on the patterns of benefit use by households in the 2014
SEBTC demonstration, focusing on variation across sites. The evaluation will also examine use of
benefits across such subgroups such as demonstration type (SNAP vs. WIC), type and size of store, and
types of food purchased. The findings will be comparable to those presented in the 2013 SEBTC
demonstration evaluations.

B.2.3 Degree of Accuracy Needed: Statistical Power and Minimum Detectable
Differences

Tests of significant differences in levels of use across subgroups will be performed. For the qualitative
interviews, statistical power and MDD are not needed.

5

B.2.4 Unusual Problems Requiring Specialized Sampling Procedures

No specialized sampling procedures are involved in this evaluation.

B.2.5 Use of Periodic Data Collection Cycles to Reduce Burden

This is a one-time study.

B.3

METHODS TO MAXIMIZE RESPONSE RATES AND DEAL WITH NONRESPONSE

On behalf of FNS, the contractor has contact information for grantees (State agencies and school districts)
from the 2013 demonstrations. These stakeholders have provided EBT transaction data, other
administrative data, and similar interview responses previously so non-response should not be an issue.
Again, the grantees (States and Tribal Organizations) and contractor will sign a Memorandum of
Understanding that describes the data collection required for the evaluation (Appendix F).

B.3.1 Advance Letters

The use of advance letters will describe the 2014 evaluation and will go out to grantees and EBT vendors
(Appendices A and B). These letters will also be customized for each site and request the administrative
data and interviews required for the evaluation (Appendix C). Since individuals/households will not be
contacted, letters to households will not be needed.

B.3.2 Follow-Up

Because the stakeholders (grantees and EBT vendors) are familiar with previous evaluations, informal
follow up will be used if needed.

6

B.4

TESTS OF PROCEDURES OR METHODS TO BE UNDERTAKEN

The procedures and instruments to be used in the current evaluation are similar to components that have
been developed, tested, and administered for previous evaluations of the SEBTC Demonstrations,
particularly the 2013 evaluations, conducted by Abt Associates and its subcontractor, Mathematica.

B.5

INDIVIDUALS CONSULTED ON STATISTICAL ASPECTS AND INDIVIDUALS
COLLECTING AND/OR ANALYZING DATA

The FNS’ Office of Policy Support has reviewed the study design and instruments. Dr. Melissa Abelev of
FNS can be reached at (703) 305-2209 or [email protected]. The contractor, Abt Associates
Inc. and its subcontractor, Mathematica, are responsible for all data collection and analysis for this study.

7

REFERENCES
Briefel, R., Collins, A., Bellotti, J., Klerman, J., Logan, C. W., Cabili, C., Rowe, G., Greece, J., Owens,
C., Weiss, A. (2011). 2011 Status Report: Summer Electronic Benefits Transfer for Children.
Prepared by Abt Associates, Mathematica Policy Research, and Maximus under Contract No.
AG-3198-C-11-0002. Project Officer: Hoke Wilson. Alexandria, VA: United States Department
of Agriculture, Food and Nutrition Service.
www.fns.usda.gov/ora/menu/DemoProjects/SummerFood/Default.htm
Briefel, R., Collins, A., Rowe, G. Wolf, A. Klerman, J., Logan, C. Enver, A., Smither Wulsin, C., Owens,
C., Jacobson, J., Bell, S., Bein, E., Juras, R., Weiss, A. (2012). Summer Electronic Benefits
Transfer for Children (SEBTC) Demonstration: 2012 Congressional Status Report. Prepared by
Abt Associates, Mathematica Policy Research, and Maximus under Contract No. AG-3198-C-110002. Project Officer: Hoke Wilson. Alexandria, VA: United States Department of Agriculture,
Food and Nutrition Service.
Briefel, R., Collins, A., Rowe, G. Wolf, A. Lyskawa, J., Logan, C., Klerman, J., Fatima, S., (2013).
Summer Electronic Benefits Transfer for Children (SEBTC) Demonstration: 2012 Congressional
Status Report. Prepared by Abt Associates, Mathematica Policy Research under Contract No.
AG-3198-C-11-0002. Project Officer: Dr. Joseph F. Robare. Alexandria, VA: United States
Department of Agriculture, Food and Nutrition Service.
Collins, A., Briefel, A., Klerman, J., Bell, S., Bellotti, J., Logan, C., Gordon, A., Wolf, A., Rowe, G.,
McLaughlin, S. M., Enver, A., Fernandes, M., Wolfson, C., Komarovsky, M., Cabilli, C., Owens,
C. (2012) Summer Electronic Benefits Transfer for Children (SEBTC) Demonstration: Evaluation
Findings for the Proof-of-Concept Year. Prepared by Abt Associates, Mathematica Policy
Research, and Maximus under Contract No. AG-3198-C-11-0002. Project Officer: Hoke Wilson.
Alexandria, VA: United States Department of Agriculture, Food and Nutrition Service.
www.fns.usda.gov/ora/MENU/Published/CNP/FILES/eSFSP_FY2011.pdf
Collins A., Briefel R, Klerman J., Rowe G., Wolf A., Logan C., Gordon A., Wolfson C., Enver A., Owens
C., Cabili C., Bell S. (2013). Summer Electronic Benefits Transfer for Children (SEBTC)
Demonstration: Evaluation Findings for the Full Implementation Year. Prepared by Abt
Associates, Mathematica Policy Research, and Maximus under Contract No. AG-3198-C-110002. Project Officer: Hoke Wilson. Alexandria, VA: United States Department of Agriculture,
Food and Nutrition Service.
Collins, A., Briefel, A., Klerman, J., Wolf, A., Rowe, G., Enver A., Logan, C., Fatima, S., Komarovsky,
M., Lyskawa, J., Bell, S. (Forthcoming, 2014) Summer Electronic Benefits Transfer for Children
(SEBTC) Demonstration: Evaluation Findings for the Third Implementation Year. Prepared by
Abt Associates, Mathematica Policy Research, and Maximus under Contract No. AG-3198-C-110002. Project Officer: Joseph Robare. Alexandria, VA: United States Department of Agriculture,
Food and Nutrition Service.

8