Statement A

Contract Number:
AG-3K06-D-09-0212/GS-10F-0050L

Supporting Justification for OMB

Mathematica Reference Number:
06687.070

the National Household Food

Submitted to:
USDA Economic Research Service
1800 M Street
Washington, DC 20036
Project Officer: Mark Denbaly
Submitted by:
Mathematica Policy Research
955 Massachusetts Avenue
Suite 801
Cambridge, MA 02139
Telephone: (617) 491-7900
Facsimile: (617) 491-8044
Project Director: Nancy Cole

Clearance of the Field Test for
Acquisition and Purchase Survey
Part A: Justification
September 23, 2010

CONTENTS

A. Justification ............................................................................................. 1

A1.  Circumstances Making the Collection of Information Necessary .......................1 
A2.  Purpose and Use of the Information .................................................................2 
A3.  Use of Information Technology and Burden Reduction ....................................7 
A4.  Efforts to Identify Duplication and Use of Similar Information ........................8 
A5.  Impacts on Small Businesses or Other Small Entities ........................................ 9 
A6.  Consequences of Collecting the Information Less Frequently ........................... 9 
A7.  Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 ...................9 
A8. Comments in Response to the Federal Register Notice and Efforts to
Consult Outside the Agency .............................................................................. 9 
A9.  Explanation of Any Payment or Gift to Respondents ..................................... 11 
A10. Assurance of Confidentiality Provided To Respondents .................................. 13 
A11. Justification for Sensitive Questions ................................................................ 14 
A12. Estimates of Annualized Burden Hours and Costs .......................................... 14 
A13.Estimates of Other Total Annual Cost Burden to Respondents or Record
Keepers ........................................................................................................... 17
A14. Annualized Cost to Federal Government ........................................................ 17 
A15. Explanation for Program Changes or Adjustments ......................................... 20 
A16. Plans for Tabulation and Publication and Project Schedule ............................. 20
A17. Reason(s) Display of OMB Expiration Date is Inappropriate .......................... 23 
A18. Exceptions to Certification for Paperwork Reduction Act Submissions .......... 23 
Appendix A: Technical Work Group Recommendations
Appendix B: Cognitive Test Results
Appendix C: Advance Letter and Study Brochure
Appendix D: Household Screener
Appendix E: Short Form for Nonrespondents
Appendix F: Consent Form
Appendix G: Household Interview #1
Appendix H: Meal Matrix and Calendar Magnet
Appendix I: Single Book for Reporting Food Acquisitions
Appendix J: Scanner Instructions and Barcode Book

ii

CONTENTS (CONTINUED)
Appendix K: Adult Food Booklet
Appendix L: Youth Food Booklet
Appendix M: Sample Scripts for Telephone Calls Collecting
Information Recorded in Food Booklets
Appendix N: Household Interview #2
Appendix O: Household Interview #3
Appendix P: Pre-test Results and Recommendations for Adjustments
Appendix Q: Image of Handheld Scanner
Appendix R: CIPSEA Pledge, Long Version
Appendix S: Confidentiality Agreement Signed by all Employees of
Mathematica Policy Research
Appendix T: Sample Table Shells

iii

A. Justification
A1. Circumstances Making the Collection of Information Necessary
The Economic Research Service (ERS), U.S. Department of Agriculture (USDA), is requesting
Office of Management and Budget (OMB) approval to conduct the Field Test for the National
Household Food Acquisition and Purchase Survey (aka National Food Study) in preparation
for a later full-scale implementation of the survey in 2012.1 The mission of ERS is to provide timely
research and analysis to public and private decision makers on topics related to agriculture, food, the
environment, rural America, and the impacts of USDA’s food and nutrition assistance programs on
clients’ well-being. To achieve this mission, ERS requires a variety of data, including the availability
and price of food at the point of sale, household demand for food products, household access to
healthy food, and quality of household food choices.
A number of existing databases provide data relevant to ERS’ data needs. For example, the
National Health and Nutrition Examination Survey (NHANES) collects data on individuals’ food
consumption; the Consumer Expenditure Survey collects aggregate data on food expenditures. No
current data source provides detailed household-level information about food acquisitions. The
absence of adequate data has made it difficult for ERS to provide accurate and timely economic
information on food demand factors, such as income and price elasticities of demand for food, and
nutritional characteristics of household food choices. In addition, the structure of the U.S. food
economy has changed dramatically in the past decade, making older surveys and estimates of food
demand increasingly outdated and irrelevant. The increase in rates of overweight and obesity have
also led to a demand for better data for understanding the relationship between food acquisition
patterns and diet quality.
ERS plans to conduct a full-scale National Food Study designed to collect household information
and food acquisition data from a nationally representative sample of 5,000 households over a sixmonth period from March 2012 through August 2012. The sample for the National Food Study will
include three strata: households participating in the Supplemental Nutrition Assistance Program
(SNAP);2 low-income households not participating in SNAP and with income below 185 percent of
the poverty guidelines; and higher-income households with income above 185 percent of the
poverty guidelines. This survey will provide data not currently available to program officials and
researchers, thereby broadening the scope of economic analyses of food choices made by U.S.
households and how those choices influence diet quality and decisions about participation in food
assistance programs.
If approved, the Field Test for the National Food Study will be conducted in winter 2011. The
field test will collect data from a sample of 400 low income households selected from the two lowincome strata defined for the full-scale survey. Higher income households will not be included in the
Field Test because the cognitive task of reporting food acquisitions over a seven-day period is not
expected to be a substantial burden for that population.
1

A separate Supporting Justification for the full-scale National Food Study will be submitted to OMB in 2011.

2
Households are eligible for SNAP if gross household income is at or below 130 percent of the federal poverty
guidelines, and income net of deductions is below the federal poverty guidelines (households with an elderly or disabled
member are not required to meet the net income test). Most households must also meet certain resource tests.

1

The primary purpose of the Field Test is to provide methodological information about two
different approaches for collecting food acquisition data from households over a seven-day period.
This information is needed because no prior survey has collected similarly detailed information
about food acquisitions in both the “food-at-home” and “food-away-from-home” categories.
Households will be randomly assigned to two alternate survey protocols and two alternate incentive
levels. Data collected from the randomly assigned subgroups will be used to provide a two-by-two
test of the estimated differences in response rates and data quality associated with different survey
protocols and different incentive levels. Data quality will be measured by adherence to survey
protocols and by reporting accuracy on items that can be validated. Information about differences in
response rates and data quality among randomly assigned subgroups will be used to assess the
efficacy of alternative approaches for the full-scale survey.
The Field Test will include all aspects of the data collection process and most aspects of data
processing that are planned for the full-scale survey, thereby providing a test of all the systems
designed for gathering data from study participants.
Section 17 [7 U.S.C. 2026] (a)(1) of the Food and Nutrition Act of 2008 provides legislative
authority for the planned data collection. This section authorizes the Secretary of Agriculture to
enter into contracts with private institutions to undertake research that will help to improve the
administration and effectiveness of SNAP in delivering nutrition-related benefits. Although ERS is
the lead agency for implementing the National Food Study, the Food and Nutrition Service (FNS)
of USDA is providing both staff and financial support. FNS is responsible for administration of
SNAP at the Federal level.
A2. Purpose and Use of the Information
The full-scale National Food Study will collect information about household food acquisitions,
including foods purchased and foods obtained at no cost (e.g., home-grown vegetables).
Information also will be collected about household characteristics, including demographics, income,
assets, major categories of nonfood expenditures, food security, health status (including heights and
weights), and dietary knowledge. This survey will provide ERS with a comprehensive database to
support the analysis of a wide variety of research questions, including patterns of shopping behavior
and food choice; the influence of access and retailer choice on dietary quality; the magnitudes of
income and price elasticities of demand for food; the influence of dietary knowledge on purchase
patterns and food choice; the relationship between food acquisition patterns and levels of food
security; and differences in food acquisition patterns for SNAP households and low-income
households not participating in SNAP. ERS is requesting permission to conduct a 400-case
methodological field test of data collection procedures in preparation for carrying out the full-scale
survey in 2012.
The Field Test for the National Food Study will use the same survey procedures planned for
the full-scale survey. These include: sampling addresses from an address-based sampling frame,
screening households in person to determine survey eligibility, administering household interviews,
training households to report food acquisitions using food booklets and a handheld scanner,
matching survey respondents with SNAP eligibility data to confirm SNAP participation status,
matching scanned barcodes with food descriptions from a Universal Product Code (UPC) data

2

dictionary and extant price data.3 The field test will provide critical information about rates of
screening eligible households from the address-based sample. It will also provide estimates of
response rates from a large scale test of survey methods. Field test data will be matched with SNAP
caseload data to assess sampling rates among SNAP households. Scanned barcodes from food items
will be matched with extant data to assess the efficacy of using handheld scanners and extant data to
obtain complete product information.
The field test differs from the full-scale survey in that households will be randomly assigned to:
(a) one of two survey protocols for collecting food acquisition data, and (b) one of two incentive
levels (high versus low). One protocol includes a single set of food acquisition instruments for the
household, with a primary respondent for the household; the second protocol includes individual
food booklets for each member of the household (age 11 and older), and individual reporting. Data
from the field test will be used to test the hypotheses that reporting compliance is greater with one
primary respondent, and with higher incentive levels. The first hypothesis will be tested by
estimating the difference in household response rates and within-household participation rates for
the two survey protocols. The second hypothesis will be tested by estimating the marginal impact of
the higher incentive on household response. If there is no statistically significant difference in
response and participation by survey protocol, the full-scale survey will use the lower cost protocol.
Similarly if the marginal impact of the higher incentive is not statistically significant, the full-scale
survey will use the lower incentive level.
ERS convened a Technical Working Group (TWG) to assist with the design of the National
Food Study. Many of the TWG recommendations (Appendix A) have been incorporated in the
design of the field test. At the recommendation of the TWG, ERS conducted cognitive tests of
different versions of instruments for collecting food acquisition data from households (Appendix B).
Findings from the cognitive tests were incorporated in the final instruments.
The field test will be conducted over a two-month period from January 2011 through March
2011. The field test will complete data collection with 400 households selected across two chosen
primary sampling units (PSUs) (see Table B.1 for distribution) purposely selected from the midAtlantic or Northeast regions to provide a mix of rural and urban locations and varying food
environments. These regions were chosen because they also provide proximity to the study’s survey
operations offices, and thus save on field test costs. Within the two PSUs, households will be
selected from an address-based sampling frame obtained from a commercial vendor. The address
frame will be matched with SNAP participants’ addresses obtained from the state SNAP agencies.
After matching these data, addresses will be sampled from two strata: addresses with SNAP
households and addresses without SNAP households. Sampled addresses will be randomly assigned
to one of two survey protocols for collection of food data, and to one of two incentive levels (high
or low).
All sampled addresses will be sent an advance letter with a study brochure (Appendix C).4
Sampled addresses will be visited by a field interviewer. Households residing at selected addresses
3

Food acquisition data from the full-scale survey will be matched with a nutrient database. Nutrient coding will
not be done for the field test because nutrient coding procedures are well-established and the field test does not provide
nationally representative data for addressing research questions regarding dietary quality.
4

Appendix C contains two versions of the advance letter. These versions differ only in the stated incentives that
may be earned for participation. Addresses will be randomly assigned to the low and high incentive levels.

3

will be asked to complete a brief screener to determine income eligibility for the survey (Appendix
D). The screener includes a question to identify the primary food shopper in households determined
eligible for the survey. As eligible households are identified, the interviewer will ask to speak with the
primary food shopper; ask the food shopper to participate as the primary respondent for their
household; explain what is expected of participants; and explain that participation is voluntary.
Eligible households that refuse to participate will be asked to respond to a brief interview of
nonrespondents (Appendix E), which will provide data to help assess nonresponse bias.
All survey materials will be available in both English and Spanish, and interviews will be
conducted in either language according to the primary respondent’s preference. Households that do
not have a member who is fluent in either English or Spanish will not be included in the field test.
For the full-scale National Food Study, survey materials will be prepared in two languages in addition
to English and Spanish.
For households that agree to participate, the primary respondent (identified as the primary food
shopper) will be asked to sign a consent form (Appendix F) and complete the first interview
(administered via CAPI5), which includes questions about household demographics, food assistance
program participation, and food acquisition patterns (Appendix G). The primary respondent will
then be given a meal matrix to complete during the week (Appendix H) and the field interviewer will
train the respondent to use the survey protocols for tracking food acquisitions during the survey
week, including foods brought home and foods obtained and consumed away from home.
The survey protocols include three tools for tracking food acquisitions during the survey week:
(1) a handheld scanner for scanning all food acquisitions with barcodes; (2) a “scanner” book with
written instructions for scanning foods brought home, data collection forms for reporting foods
brought home, and pictures and barcodes to scan variable weight items that might not have attached
barcodes; (3) food booklets data collection forms for reporting foods obtained for consumption
away from home.
The two survey protocols to be tested in the Field Test will include each of the three tools
described above. The first protocol, known as the “Single Book” protocol, will provide the
household with one binder containing all instructions and data collection forms for reporting food
acquisitions―both food-at-home and food-away-from-home―for all members of the household
(Appendix I). The second protocol, known as the “Multiple Book” protocol, provides the
household with one binder containing instructions and data collection forms for reporting food-athome (Appendix J), and with multiple booklets so that each household member can report his or
her own food-away-from-home acquisitions, with separate versions of the booklet for adults
(Appendix K) and children ages 11–18 (Appendix L). The Field Test will determine the relative data
quality and response rates from individual household members under two scenarios: (1) primary
respondent is responsible for reporting all food acquisitions for the household in one book, and (2)
multiple household members are individually responsible for reporting their own food acquisitions
in individual books (except for children ages 10 and under, whose food acquisitions (e.g., meals at
school) will be recorded in the primary respondent’s booklet).

5

CAPI is computer-assisted personal interviewing.

4

Household members will be asked to use the binder or booklets to store receipts and provide
detailed information about food acquisitions, including place where they got food, method of
payment, use of discounts, and tip amount. Descriptions and quantities of food-at-home items will
be collected with scanners, and respondents will be asked to write this information on data
collection forms only if items cannot be scanned. Description and quantities of food-away-fromhome items will be collected by attaching receipts to data collection forms, and respondents will be
asked to write this information on data collection forms only if items are not listed on a receipt.
On days two, five, and seven of the survey week, the primary respondent for each household
will be asked to complete a brief telephone interview to report food-away-from-home acquisitions
from receipts and data collection forms. Telephone interviewers will conduct this interview using a
custom designed data collection website. The website will have embedded Internet queries so that
interviewers can confirm the precise locations of places where food was acquired via a Google
search. This system will also be preloaded with name-brand and generic menu items so that
interviewers can probe for exact names of foods or menu items acquired and consumed away from
home (Appendix M).
On day three of the survey week, the primary respondent will be asked to complete a telephone
interview to report nonfood expenditures, income, assets, and recent major life events (Appendix
N). Questions about expenditures, income, and assets were taken from, or modeled after, questions
in the Survey of Income and Program Participation (SIPP) ) and the Consumer Expenditure Survey
(CES). Item nonresponse will be compared with rates of nonresponse in the SIPP and the CES.
On the day after the completion of the survey week, a field interviewer will visit the household to
review and collect survey instruments, and interview the primary respondent (via CAPI) about
consumer behavior; knowledge and attitudes on diet, health, and nutrition; special dietary needs;
food security; and health status, including height and weight (Appendix O). Most questions in the
third household interview were taken from the National Health and Nutrition Examination Survey
(NHANES).
Field staff will transmit the two interviews conducted by CAPI to the contractor’s office via
secure data transfer. Field staff will transfer scanned barcodes from the handheld scanners to their
secure computers via USB cable and then transmit these files to the contractor’s office. All hardcopy forms, including the household screeners and food booklets, will be transmitted to the
contractor’s office via traceable Federal Express delivery.
The purpose of the field test is to test alternate designs of the food booklets for collecting food
acquisition data. Testing is needed because no other surveys have collected the level of detail about
food acquisitions that ERS/FNS plans for the National Food Study.6 ERS has conducted tests with
three versions of food booklets for reporting food data (Appendix P). Two alternate instrument
designs were developed based on findings from those tests. The field test will randomly assign
households to the two food booklet designs. Response rates and data quality will be compared
across the randomly assigned subgroups.

6
The Consumer Expenditure Survey collects information about aggregate spending for broad categories of
expenditures. Commercial surveys, such as The Nielsen Company’s Homescan, collect price and quantity information
about grocery purchases, but not food away from home. Neither collect information about foods acquired at no cost.

5

The field test also will test the effectiveness of two different incentive levels. The National
Food Study is estimated to impose an average estimated total household burden of 4.7 hours for
each household completing all data collection activities. The incentive is designed to offset the
burden of participation and encourage complete participation throughout the data collection week.
As discussed in Section A.9, households will be randomly assigned to one of two incentive levels.
Each of the two incentive levels will be tiered according to household size to account for the added
burden of reporting acquisitions of additional household members.
The field test will be designed to answer the following questions:
• What is the household response rate and how does it vary for the subgroups defined by
food booklet version and the subgroups defined by incentive level? Information
gathered from responses to this question includes
-

Percentages of sampled households that agree to participate

-

Percentages of sampled households that complete all survey protocols

-

Percentages of sampled households that complete all household interviews, scan
food at home, complete the food booklets, but do not complete the telephone
reporting of food away from home on days two, five, and seven

-

Average percentages of individual household members reporting food
acquisitions

• How does item response compare for the two food booklet versions and two incentive
levels?
-

What are the estimated differences between subgroups in average weekly number
of food-at-home acquisitions, food-away-from-home acquisitions, and total food
acquisitions?

• How does data quality compare for the two food booklet versions and two incentive
levels?
-

What are the estimated differences between subgroups in the number of
acquisitions with complete information for location, total cost, and item prices?

-

What is the estimated difference between subgroups in the number of food items
reported with sufficient detail for nutrient coding?

• How does total household burden compare for the two food booklet versions and two
incentive levels?7
• How does the actual survey cost per case-week compare for the two food booklet
versions and two incentive levels?

7

Different incentive levels could impact total household burden if, for instance, the higher incentive encourages
more members of the household to provide complete information on food acquisitions.

6

Appropriate sampling weights and statistical methods will be used to estimate differences
between subgroups and determine the statistical significance of estimates. As noted above, the
primary hypothesis is that reporting compliance and data quality will be greater with one primary
respondent, and with higher incentive levels. If there are no statistically significant differences in
response and participation by survey protocol, the full-scale survey will use the lower cost protocol.
Similarly the higher incentive will be used for the full survey only if the higher incentive is associated
with a statistically higher response rate.
Analyses of field test data will be compiled in internal documents and distributed to the survey’s
technical working group who will form recommendations for methodological improvements for the
full-scale survey. Improvements may include minor modification to survey instruments,
improvements to the data collection website to improve interviewer interaction with the site and
reduce interview time, and changes to the sampling design to improve screening rates or reduce
design effects. Field test data will not be disseminated to researchers or the general public. Results
will be used to plan the full-scale National Food Study.
A3. Use of Information Technology and Burden Reduction
In compliance with the E-Government Act, 2002, information technology has been
incorporated into this field test wherever possible to reduce respondent burden. Computer-assisted
telephone interviewing (CATI) and computer-assisted personal interviewing (CAPI) will be used to
conduct all interviews except the household screener and short form for refusals.8 Both CATI and
CAPI interviewing will automate skip logic to improve the pace and flow of the interviews.
A handheld barcode scanner will be given to each household for reporting food acquisitions
(Appendix Q). The scanner provides a “point-and-click” method for collecting universal product
codes (UPCs) and other barcodes. UPCs can be linked to a UPC dictionary to obtain precise
product names and product sizes of manufactured items; the GS1 Databar barcode precisely
identifies produce items.9,10 The scanning process eliminates measurement error and the timeconsuming process of writing down each food item. Respondents will also be given a Book of
Barcodes containing pictures and barcodes for variable-weight products including produce, service
deli items, and bulk foods. Variable-weight products often do not have attached barcodes. Scanning
a barcode from the Book of Barcodes reduces respondent burden compared with the alternative of
writing down the names of these items. The use of hand-held scanners to record UPC/barcodes will
provide accurate and complete information about most foods that households acquire for home
preparation and consumption, thereby enabling more accurate matching of food items to nutrient
information.

8

The screener is a brief interview that will be administered to more than 1,000 households to determine their
income eligibility for the survey. The short form will be administered to households screened as eligible that decline to
participate. The burden associated with these instruments is minimized by paper administration.
9
The GS1 Databar barcode is designed to condense information in a barcode suitable for printing on small
packages, such as produce.
10

Scanned barcodes that are not UPCs or GS1 Databar, such as store-specific codes on deli items and packaged
store bakery items, will be discarded and information about the product will be obtained from the receipt.

7

Information technology also will be employed in the telephone interviewers conducted on days
two, five, and seven. During these interviews, respondents will report information about food-awayfrom-home acquisitions that are reported in food booklets. Interviewers will use a custom CATI
system designed as a data collection website with access to the Google Maps search engine for
obtaining and confirming the precise location of food acquisitions (address of store or restaurant).
Interviewers will enter foods reported by respondents by choosing from a list of menu items
preloaded in the data system for the top 30 full-service restaurants, top 30 fast-food restaurants,
school menu items (reported to the School Nutrition and Dietary Assessment Study-III), and
generic meal items compiled from NHANES. This “pick-list” approach will reduce interviewer
recording time and the length of these interviews, and improve the precision of collected data.
Text messaging was considered as an alternative to telephone reporting of food-away-fromhome on days two, five, and seven. Text messaging may be less burdensome and less intrusive, thus
encouraging response (with albeit less data quality), especially among teenagers within sampled
households. Texting has been successfully used as a means of contacting respondents (with outgoing
texts to respondents)11, but there are no known studies of its use as a reporting tool (with incoming
texts from respondents). In addition, this alternate response mode would introduce bias in response
rates because those with cell phones, texting capabilities, and familiarity with texting would be more
likely to respond. Thus, this additional use of technology was rejected.
In addition to use of technology, respondents are encouraged to save receipts for all food
acquisitions. Reporting burden for food-away-from-home is reduced when respondents save
receipts, as the data collection form instructs respondents to write food items and prices that do not
appear on the receipt. Receipts for food-away-from-home will be used by respondents as recall aides
during telephone interviews on days two, five, and seven. Receipts for food-at-home (groceries) will
be used to extract prices when extant data are not available.
Information for the field test will be gathered from existing data sources when feasible. The
sampling frame of addresses within survey areas will be constructed from a commercially available
address-based sample, matched with SNAP administrative data to facilitate sampling of SNAP and
non-SNAP households and reduce the number of households screened for the survey. To assess
data quality, scanned barcodes collected from households will be matched with two data sources.
First, barcodes will be matched with an existing database from Gladson Interactive to obtain
product names, package sizes, and nutrient information. Second, for households shopping at stores
that participate in the The Nielsen Company’s Scantrack survey, barcodes will be matched with
Nielsen price files to obtain the price in effect during the data collection week at the store where
food was obtained.12 These uses of existing data will significantly reduce respondent burden and data
processing costs.
A4. Efforts to Identify Duplication and Use of Similar Information
There is no similar data collection available. Every effort has been made to avoid duplication.
ERS has reviewed existing federal government data collections. Current government data collections
11

Maher et al. (2001). “Using text messages to contact difficult-to-reach study participants,” American Journal of
Public Health, April 15, 2010.
12

Additional price data will be obtained from the store receipts that households save in their food booklets.

8

do not provide data for analyzing food acquisitions by households. For example, NHANES collects
data on individual’s food consumption through 24-hour recalls. NHANES does not collect data for
all members of a household; it collects only limited information on the source of food consumed
and does not collect price or expenditure data. The Consumer Expenditure Survey collects aggregate
data on food expenditures in broad food categories for food at home and food away from home,
but detailed information about purchased food items is not available for analyses of food choices
and the nutritional quality of acquired foods.
A5. Impacts on Small Businesses or Other Small Entities
Information being requested or required in this field test has been held to the minimum
required for the intended use. No small businesses will be involved in this field test.
A6. Consequences of Collecting the Information Less Frequently
The Field Test for the National Food Study is a one-time data collection. Without this field test,
ERS will not have sufficient information to ensure that best procedures are used to maximize data
quality and minimize respondent burden in the full National Food Study of 5,000 households. The
full survey is needed to achieve ERS’ mission to provide research on food demand and economic
behavior of participants in USDA food programs.
A7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
There are no special circumstances. The collection of field test information is conducted in a
manner consistent with the guidelines in 5 CFR 1320.5.
A8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside
the Agency
The Economic Research Service (ERS) published a notice in the Federal Register on May 18,
2010, Volume 75, Number 95, pp. 27701–27703 seeking public comment on the field test . No
public comments were received during the 60-day comment period. ERS also consulted with nonAgency experts who provided input on the field test research design, data needs, survey content, and
survey protocol. Individuals who contributed to these consultations include employees of the
contractor, Mathematica Policy Research:
Nancy Cole
Project Director
Mathematica Policy Research
955 Massachusetts Ave, Suite 801
Cambridge, MA 02139
(617) 674-8353
[email protected]

Laura Kalb
Survey Director
Mathematica Policy Research
955 Massachusetts Ave, Suite 801
Cambridge, MA 02139
(617) 301-8989
[email protected]

Mary Kay Fox
Technical Reviewer
Mathematica Policy Research
955 Massachusetts Ave, Suite 801
Cambridge, MA 02139
(617) 301-8993

John Hall
Senior Statistician
Mathematica Policy Research
P.O. Box 2393
Princeton, NJ 08543
(609) 275-2357
9

[email protected]

[email protected]

In January 2010, a technical working group of academic experts was convened to discuss the
proposed field test data collection instruments and survey methodology of the National Food Study.
Individuals that attended this meeting, and provided comments on subsequent versions of the field
test instruments include the following:
Steven Heeringa
Senior Research Scientist
Institute for Social Research
University of Michigan
426 Thompson
Ann Arbor, MI 48104
(734) 647-4621
[email protected]

Helen Jensen
Professor, Department of Economics
Food and Nutrition Policy
Center for Agricultural and Rural Development
Iowa State University
578E Heady Hall
Ames, IA 50011-1070
(515) 294-6253
[email protected]

Suzanne Murphy
Professor
Cancer Research Center of Hawaii
1236 Lauhala St, Suite 407
Honolulu, HI 96813
(808) 564-5861
[email protected]

Sarah Nusser
Professor and Director
Center for Survey Statistics & Methodology
Iowa State University
220 Snedecor
Ames, IA 50011-1210
(515) 294-9773
[email protected]

Roger Tourangeau
Survey Methodologist
Director, Joint Program in Survey
Methodology
University of Maryland
1218 LeFrak Hall
College Park, MD 20742
(301) 314-7984
[email protected]

Parke Wilde
Associate Professor
Friedman School of Nutrition Science and Policy
Tufts University
150 Harrison Ave
Boston, MA 02111
(617) 636-3495
[email protected]

The technical working group meeting was also attended by representatives of government
agencies. These individuals provided verbal comments during the meeting:
Lori G. Borrud
Nutritional Epidemiologist
National Center for Health Statistics-CDC
[email protected]

John Eltinge
Associate Commissioner for Survey Methods
Research
Bureau of Labor Statistics
202-691-7404
[email protected]

Kelly Kinnison
Nutritionist
Food and Nutrition Service, USDA
703-305-2124

Susan Krebs-Smith
Chief of the Risk Factor Monitoring and
Method
National Cancer Institute, NIH
10

[email protected]

301-436-8500
[email protected]

Donna Rhodes
Agricultural Research Service, USDA
301-504-5040
[email protected]
The complete OMB clearance package was reviewed by staff of the USDA National
Agricultural Statistics Service (NASS) and their comments were addressed and incorporated in this
document.
Materials about the field test study and the draft survey instruments will be reviewed and
approved by the Public/Private Ventures institutional review board (IRB).13 Materials were provided
to the IRB in September 2010. The IRB contact person is
Margo Campbell
(215) 557-4446
[email protected]
A9. Explanation of Any Payment or Gift to Respondents
The plan for the field test, and for the full-scale survey, is to offer households an incentive to
participate in the study. It is essential to include an incentive in order to maximize the response rate,
and it is particularly important with a challenging population and a demanding data collection. In a
seminal meta-analysis, Singer, et al. (1999) found that incentives in face-to-face and telephone
surveys were effective at increasing response rates, with a one dollar increase in incentive resulting in
approximately a one-third of a percentage point increase in response rate, on average. They found
some evidence that incentives were useful in boosting response rates among underrepresented
demographic groups, such as low income and non-white (Singer, et al., 1999). This is a significant
consideration for this study of the SNAP and SNAP-eligible population. Another important
consideration is the burden posed by this data collection, which asks respondents for seven days of
food acquisition data, including a one hour training, two in-person interviews, four telephone
interviews (three for reporting about foods acquired away from home and one about household
finances), keeping records and receipts for all food acquired by anyone in the household, and
scanning groceries brought into the home. A previous data collection for the School Nutrition
Dietary Assessment (SNDA) IV offered a $50 incentive to school food managers for five days of
record keeping only. As described below and shown in Table A.1, we propose a range of incentives
for this field test, from a low of $50 for a single person household in the low incentive group to a
high of $250, including an initiative bonus, for a 6-person household in the high incentive group.
The goal of this variation is to test the effect of incentive on participation (both household level and
individuals within the household) and data quality.
In order to determine the optimal incentive protocol for the full-scale study, the field test will
randomly assign respondents to one of two incentive levels. The incentive is designed to encourage
13

P/PV is a 501(c)(3) nonprofit, nonpartisan research organization. Mathematica Policy Research regularly uses
this external IRB and one Mathematica staff member sits on the P/PV IRB.

11

respondents to complete the interviews and track food acquisitions during the data collection week.
The amount of time required for the interviews varies by household size: (1) many questions in the
household interviews are asked with regard to each household member and (2) the number of food
acquisitions and number of food items per acquisition are expected to vary with household size.
The two incentive levels used for the field test are labeled “low” and “high” in Table A.1. The
low and high incentives are tiered to reflect the additional burden for larger households. Both
incentive schemes include a base amount for the primary household respondent to offset the burden
of responding to the screener, three household interviews, and tracking food acquisitions for one
person. Incentives increase by a fixed amount for each additional household member to reflect the
additional burden of the household interviews and the burden of tracking an additional person’s
acquisitions. The increased incentive amount for additional household members will be offered
contingent upon participation of other household members in reporting food acquisitions.
Individual household member participation is measured by completed food booklets under the
Multiple Book protocol; and reported acquisitions by each household member under the Single
Book protocol. This design is intended to encourage responses from household members other than
the primary respondent. Findings from the cognitive tests indicate that teenagers, in particular, might
be reluctant to participate without a targeted incentive.
The incentive scheme also includes a “telephone bonus” to be paid if the primary respondent
initiates the telephone reporting of food acquisitions on days two, five, and seven. This bonus is
designed to reduce overall data collection costs for the survey. Interviews initiated by incoming calls
from respondents are completed at significantly lower cost than outgoing calls with multiple
callbacks to obtain these responses.
Table A.1. Incentive Levels to be Tested in the National Food Study Field Test

Household
Size

Percentage
of Samplea

Low
Incentive

1

45.5

$50

2

19.8

$70

3

15.6

4

10.1

5

5.5

6

Note:
a

Telephone
Bonus

Total Low
Incentive

High
Incentive

Telephone
Bonus

Total High
Incentive

$25

$75

$100

$25

$125

$25

$95

$125

$25

$150

$90

$25

$115

$150

$25

$175

$110

$25

$135

$175

$25

$200

$130

$25

$155

$200

$25

$225

2.3

$150

$25

$175

$225

$25

Average

$72

$97

$128

$250
$152

The low incentive is equal to $50 + (hhsize-1)* $20. The high incentive is equal to $100 + (hhsize-1)* $25.

The distribution of the sample by household size is based on the distribution of the SNAP caseload in 2008 (Source:
USDA, FNS. Characteristics of Supplemental Nutrition Assistance Program Households: Fiscal Year 2008.

Respondents will be informed of the tiered incentive structure in the advance letter and again
upon determination of eligibility for the study. The additional incentive per household member will
be paid for each household member age 5 and above for whom food acquisitions are reported. This
age limit is designed to encourage adults to fully report school meals obtained by children. As shown
in Table A.1, 98.8 percent of low-income households have six or fewer members; there will be no
cap on the incentive for the remaining households with more than six members.
12

Incentives will be delivered using a mixed delivery mode. Field interviewers will provide the
base incentive for the primary respondent by check at the end of the data collection week.
Additional earned incentives, per household member and for the telephone bonus, will be provided
in the form of gift certificates mailed to households within 4 weeks of the end of the data collection
week. This mixed delivery mode will be communicated to respondents by the field interviewers and
in the written consent form. The mixed delivery modes serve three purposes: (a) field interviewers
are protected from making determinations of incentive amounts, (b) gift certificate are more easily
distributed to family members, compared to a check for the total amount, and (c) households will
not expect a large sum at the end of the week and are less likely to alter their behavior during the
data collection week in anticipation of the incentive payment.
A10. Assurance of Confidentiality Provided To Respondents
Data for the Field Test and full-scale National Food Study will be collected under the
Confidential Information Protection and Statistical Efficiency Act of 2002 (CIPSEA). Field test
participants will receive assurances of confidentiality in the advance letter and study brochure
(Appendix C). The consent form includes the shorter version of the CIPSEA pledge (Appendix F),
and the long version of the CIPSEA pledge will be included in the study’s informational website at
www.usdafoodstudy.org (Appendix R).14 Throughout the field test, participants will be informed
that their participation is voluntary and that they have the option to refuse to answer any questions.
They will also be told that neither their participation nor their responses will be identifiable, nor will
they affect their eligibility in any government programs.
All respondents’ information collected during the field test will be kept confidential and not
disclosed to anyone other than the analysts conducting this research, except as otherwise required by
law. The contractor will not disclose any information that permits identification of respondents,
except as otherwise required by law. The field test data will not be released in a public data set and
respondents will not be identified individually in any reports.
Personally identifying information (names of household members and telephone numbers) will
be collected by the contractor’s field staff upon screening eligible households for the survey.
Identifying information, including address, will be maintained on separate forms and files that are
linked only by sample identification number. After a sample case is completed in the field, access to
the file linking sample identification numbers with respondent contact information is limited to a
small number of staff that has a need to know this information. All hard-copy field materials will be
transmitted to Mathematica Policy Research’s Survey Operations Center via traceable Federal
Express delivery. All members of the study team having access to the data are trained on the
importance of confidentiality and data security. Additionally, all permanent and temporary
employees of Mathematica are required to sign a confidentiality agreement (Appendix S)
emphasizing the importance of confidentiality and describing their obligation.
After field test survey materials are returned from the field, hard-copy documents will be stored
in secured file cabinets and rooms; electronic data will be maintained on secured, passwordprotected computer servers. Discarded materials containing confidential information will be
shredded. Both sources of data will be accessible only by approved contractor staff; such
14

This site is not yet active.

13

accessibility is limited to those who have direct responsibility for providing and maintaining sample
locating information.
Field test data will be processed and stored on the contractor’s password-protected local area
network (LAN). The contractor, Mathematica, protects its LAN with several security mechanisms
available through the network operating system. Access to private information stored on LAN
directories is restricted to authorized project staff by means of IDs and passwords. In addition,
network servers containing private information are kept in a locked area.
A11. Justification for Sensitive Questions
The field test information collection includes questions that some respondents might find
sensitive. All respondents will be informed that they can decline to answer any question they do not
wish to answer and there are no negative consequences for not participating. Sensitive questions
include household income, assets, citizenship status, food security, and body weight. Information
about these potentially sensitive topics is important to statistical uses under the study. Household
income is needed to determine eligibility for the field test; income, assets, and citizenship status are
needed to determine SNAP eligibility among nonparticipants. Food security and body weight are
important outcomes indicating household health and well-being. The National Food Study will
enable ERS to examine the relationship of these outcomes with food access and food acquisition
patterns. The field test will collect these items in order to assess item nonresponse and to mirror the
full study protocol.
A12. Estimates of Annualized Burden Hours and Costs
A total of 2,885.34 burden hours are estimated for this study. The burden estimate has been
decreased over the estimate presented in the 60-day notice in the Federal Register (3,400.67 hours).
Compared with estimates presented in the 60-day notice, the pre-test of instruments obtained lower
average burden for the household screener, each of the three household interviews, reporting of
food obtained for home preparation and consumption, and telephone reporting of food obtained
away from home. On average, households acquired food for home preparation and consumption on
two of seven days and obtained food away from home on three of seven days, thus lowering the
estimated reporting burden for food acquisitions.15 These lower burden estimates were partly offset
by accounting for two additional sources of burden: (1) households will be trained on how to report
food acquisitions (average of 70 minutes per household) and (2) field interviewers will conduct a
final review of data collection materials with respondents during the final household visit (average of
19 minutes per household).
Data collection for the field test will occur from January 2011 through March 2011. Table A.2
reports the expected number of respondents, frequency of response, hours per response, and the
total burden hours for the data collected. Estimates of the percentages of respondents who will
agree to complete the forms are based on other national field study cooperation rates.16 Burden
15
Reporting burden for the food diary is estimated to average 15 minutes on three days when foods are acquired
and 2 minutes on four days when no food is acquired.
16

National Health and Nutrition Examination Survey (NHANES) 2007-2008 interview response rate for all ages
was 78.4 percent; National Health Interview Survey (NHIS) 2009 total household response rate was 82.2 percent.

14

estimates for completed interviews were obtained from a pretest of the full data collection protocol
with six households. Burden estimates for attempted interviews reflect our best estimate of the
amount of time spent trying to convince non-respondents to participate.
Table A.3 reports the expected number of respondents, frequency of response, hours per
response, and the total burden hours for the full survey, using the same expected response rates and
burden estimates that appear in Table A.2, applied to the larger sample size. All of these numbers
for the full survey will be adjusted based on the burden estimates gathered from the field test.
Table A.2. Reporting Burden for the National Food Study Field Test

Instrument

Estimated
Number of
Respondents

Responses
Annually per
Respondent

Total Annual
Reponses

Estimated
Average
Number of
Hours per
Response

Estimated Total
Annual Hours of
Response
Burden

Household Screener
Completed interviews
Attempted interviews

1,063
413

1.00
1.00

1,063
413

0.13
0.08

141.73
34.42

523
62

1.00
1.00

523
62

0.32
0.08

165.62
5.17

47
15

1.00
1.00

47
15

0.13
0.03

6.27
0.50

Household Interview #1
Completed interviews
Attempted interviews
Short Form for Refusals
Completed interviews
Attempted interviews

Household Training for Reporting Food Acquisitions
Completed interviews
Attempted interviews

503
20

1.00
1.00

503
20

1.17
0.25

586.83
5.00

453
50

1.00
1.00

453
50

0.43
0.05

196.30
2.50

402
101

1.00
1.00

402
101

0.30
0.05

120.60
5.05

Household Interview #2
Completed interviews
Attempted interviews
Household Interview #3
Completed interviews
Attempted interviews

Reporting Food Obtained for Home Preparation or Consumption
Completed reports
Attempted reports

453
50

2.00
1.00

906
50

0.25
0.05

226.50
2.50

1,086
120

7.00
3.00

7,602
360

0.13
0.05

962.92
18.00

3.00
1.00

1,206
101

0.22
0.08

261.30
8.42

402
101

0.32
0.08

127.30
8.42

Food Booklets1
Completed reports
Attempted reports

Telephone Reporting of Food Away from Home
Completed interviews
Attempted interviews

402
101

Review of Food Booklets During Final Household Visit
Completed interviews
Attempted interviews

402
101

1.00
1.00

15

Total Responding Burden

1,476

9.40

13,877

1

0.21

2885.34

Estimated average hours for food booklet reporting is .25 hours on three days when food is acquired, plus .03
hours on 4 days with no food acquisitions.

16

Table A.3. Reporting Burden for Full-scale National Food Study in 2012

Instrument

Estimated
Average
Number of
Hours per
Response

Estimated Total
Annual Hours of
Response
Burden

Estimated
Number of
Respondents

Responses
Annually per
Respondent

Total Annual
Reponses

13,221
5,137

1.00
1.00

13,221
5,137

0.25
0.08

1,762.80
428.08

6,505
771

1.00
1.00

6,505
771

0.42
0.08

2,059.92
64.25

578
193

1.00
1.00

578
193

0.13
0.03

77.07
6.43

Household Screener
Completed interviews
Attempted interviews
Household Interview #1
Completed interviews
Attempted interviews
Short Form for Refusals
Completed interviews
Attempted interviews

Household Training for Reporting Food Acquisitions
Completed interviews
Attempted interviews

6,256
249

1.00
1.00

6,256
249

0.58
0.05

7,298.67
62.25

5,634
622

1.00
1.00

5,634
622

0.58
0.05

2,441.40
31.10

5,000
1,256

1.00
1.00

5,000
1,256

0.33
0.05

1,500.00
62.80

11,268
622

0.17
0.05

2,817.00
31.10

Household Interview #2
Completed interviews
Attempted interviews
Household Interview #3
Completed interviews
Attempted interviews

Reporting Food Obtained for Home Preparation or Consumption
Completed reports
Attempted reports

5,634
622

2.00
1.00

13,507
1,492

7.00
3.00

94,549
4,476

0.25
0.08

11,976.21
223.80

3.00
1.00

15,000
1,256

0.25
0.08

3,250.00
104.67

Food Booklets
Completed reports
Attempted reports

Telephone Reporting of Food Away from Home
Completed interviews
Attempted interviews

5,000
1,256

Review of Food Booklets During Final Household Visit
Completed interviews
Attempted interviews
Total Responding Burden

5,000
1,256

1.00
1.00

5,000
1,256

0.25
0.08

1,583.33
104.67

1,8358

9.37

172,593

0.25

35,885.54

17

Table A.4 shows estimated annualized hourly costs to respondents during the field test.
Respondents to the field test will include households in two survey strata: (1) households
participating in SNAP and (2) nonparticipating households with incomes below 185 percent of the
federal poverty guidelines. Estimated annualized cost to field test respondents is based on the
federal minimum wage rate, which provides earnings equivalent to 128 percent of the poverty
guidelines for a full-time worker. SNAP quality control data for 2008 (the most recent year available)
indicates that 29 percent of SNAP households had earned income. Those with earned income had
average monthly earnings equivalent to full-time work for one person at just slightly above the
minimum wage in effect that year.
Table A.5 shows estimated annualized hourly costs to respondents during the full survey.
Respondents to the full survey will include households in three survey strata: (1) households
participating in SNAP (N=1500), and (2) nonparticipating households with incomes below 185
percent of the federal poverty guidelines (N=2000), and (3) nonparticipating households with
incomes above 185 percent of the federal poverty guidelines (N=1500). Estimated annualized cost
to respondents in the first two strata are based on the federal minimum wage rate ($7.25), which
provides earnings equivalent to 128 percent of the poverty guidelines for a full-time worker.
Estimated annualized costs to respondents in the third strata are based on average earnings of
earners in households with incomes above 185 percent of the federal poverty guidelines ($22.62).17
The weighted average of these two estimates ($11.86) is used in Table A.5.
A13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers
There are no capital/start-up or ongoing operation/maintenance costs associated with this
information collection.
A14. Annualized Cost to Federal Government
The total estimated cost of the field test information collection is $2,823,257 over a 21-month
period, for an annualized cost of $1,613,289. This includes the costs associated with the contractor
conducting the project and the salary of the assigned ERS project director. This cost includes all
study tasks, including design, cognitive tests, pretests, sample selection, recruitment, information
collection, analysis, and report writing.
USDA’s Food and Nutrition Service is supporting this effort by contributing 15.7 percent of the
overall cost and by providing staff expertise.

17
Average earnings of earners in households with income above 185% of poverty is based on the distribution of
households by income level, and the average number of earners per household, from the 2008 Current Population
Survey (http://www.census.gov/hhes/www/cpstables/032009/hhinc/new01_001.htm).

18

Table A.4. Annualized Cost to Respondents for the National Food Study Field Test

Instrument

Estimated
Number of
Respondents

Estimated
Average
Number of
Hours per
Response

Estimated Total
Annual Hours of
Response Burden

Estimated
Hourly Wage
Rate

Estimated
Respondent
Cost

Household Screener
Completed interviews
Attempted interviews

1,063
413

0.13
0.08

141.73
34.42

7.25
7.25

1,926.69
249.55

523
62

0.32
0.08

165.62
5.17

7.25
7.25

1,519.46
49.52

47
15

0.13
0.03

6.27
0.50

7.25
7.25

58.94
5.08

Household Interview #1
Completed interviews
Attempted interviews
Short Form for Refusals
Completed interviews
Attempted interviews

Household Training for Reporting Food Acquisitions
Completed interviews
Attempted interviews

503
20

1.17
0.25

586.83
5.00

7.25
7.25

1,915.81
18.13

453
50

0.43
0.05

196.30
2.50

7.25
7.25

1,915.81
18.13

402
101

0.30
0.05

120.60
5.05

7.25
7.25

971.50
36.61

Household Interview #2
Completed interviews
Attempted interviews
Household Interview #3
Completed interviews
Attempted interviews

Reporting Food Obtained for Home Preparation or Consumption
Completed reports
Attempted reports

453
50

0.25
0.05

226.50
2.50

7.25
7.25

1,642.13
18.13

1,086
120

0.13
0.05

962.92
18.00

7.25
7.25

13,778.92
217.50

0.22
0.08

261.30
8.42

7.25
7.25

2,185.88
61.50

Food Booklets
Completed reports
Attempted reports

Telephone Reporting of Food Away from Home
Completed interviews
Attempted interviews

402
101

Review of Food Booklets During Final Household Visit
Completed interviews
Attempted interviews
Total Responding Burden

402
101

0.32
0.08

127.30
8.42

7.25
7.25

2,185.88
61.50

1,476

0.21

2,885.34

7.25

24,654.86

19

Table A.5. Annualized Cost to Respondents for the Full-Scale National Food Study in 2012

Instrument

Estimated
Number of
Respondents

Estimated
Average
Number of
Hours per
Response

Estimated Total
Annual Hours of
Response Burden

Estimated
Hourly Wage
Rate

Estimated
Respondent
Cost

Household Screener
Completed interviews
Attempted interviews

13,221
5,137

0.25
0.08

1,762.80
428.08

11.86
11.86

20,907.23
5,077.17

6,505
771

0.42
0.08

2,059.92
64.25

11.86
11.86

24,431.10
762.06

578
193

0.13
0.03

77.07
6.43

11.86
11.86

914.03
76.30

Household Interview #1
Completed interviews
Attempted interviews
Short Form for Refusals
Completed interviews
Attempted interviews

Household Training for Reporting Food Acquisitions
Completed interviews
Attempted interviews

6,256
249

0.58
0.05

7,298.67
62.25

11.86
11.86

86,563.93
738.30

5,634
622

0.58
0.05

2,441.40
31.10

11.86
11.86

28,955.93
738.30

5,000
1,256

0.33
0.05

1,500.00
62.80

11.86
11.86

28,955.59
368.85

Household Interview #2
Completed interviews
Attempted interviews
Household Interview #3
Completed interviews
Attempted interviews

Reporting Food Obtained for Home Preparation or Consumption
Completed reports
Attempted reports

5,634
622

0.17
0.05

2,817.00
31.10

11.86
11.86

17,790.36
7,444.82

13,507
1,492

0.25
0.08

11,976.21
223.80

11.86
11.86

33,410.29
368.85

0.25
0.08

3,250.00
104.67

11.86
11.86

142,040.67
2,654.32

Food Booklets
Completed reports
Attempted reports

Telephone Reporting of Food Away from Home
Completed interviews
Attempted interviews

5,000
1,256

Review of Food Booklets During Final Household Visit
Completed interviews
Attempted interviews
Total Responding Burden

5,000
1,256

0.25
0.08

1,583.33
104.67

11.86
11.86

38,545.79
1,241.37

1,8358

0.25

35,885.54

11.86

425,611.14

20

A15. Explanation for Program Changes or Adjustments
This is a new collection of information; therefore, there are no changes to burden or to any
ERS/FNS program.
A16. Plans for Tabulation and Publication and Project Schedule
The contractor will analyze the field test data collected and prepare two memoranda and a
briefing for ERS/FNS. The two memoranda are:
•

Memorandum on Data Collection for the Field Test – Report on response rates and
burden for the subgroups defined by survey protocol and incentive level. The memo
will also report any unanticipated problems with the data collection.

•

Memorandum on Data Processing for the Field Test - Report item nonresponse and
data quality, comparing subgroups defined by survey protocol and incentive level. This
memorandum will also report on rates of matching to extant data.

The contractor will analyze response rates and data quality to answer the questions enumerated
in section A2. Table shells that illustrate the types of tabulations planned are provided in Appendix
T. The power of the significance tests for differences between treatment groups is discussed in
Section B.
The table shells included in Appendix T present our approach to testing our main hypotheses:
1. The single book protocol will result in higher household response rates, compared with
the multiple book protocol. Response rates will be measured by the proportion of
households completing all survey protocols, and the proportion with partial responses.
2. The single book protocol will result in greater individual participation within
households, compared with the multiple book protocol. Individual participation will be
measured by the proportion of all individuals, age 5 and older, in each treatment group
who report food acquisitions or are reported to have no food acquisitions.18,19
3. The single book protocol will result in lower reporting burden for food acquisitions
compared with the multiple book protocol. Burden will be measured by the total
completion time for telephone reporting of food-away-from-home acquisitions.
4. The single book survey protocol will result in better data quality, compared with the
multiple book protocol. Where data quality is measured by:
a. Higher average weekly number of reported food acquisitions per household
18

In the cognitive tests and pre-tests, primary respondents willingly identified household members who refused to
participate in the survey.
19

This measure will be subject to a higher design effect than household measures, due to within household
clustering. However the sample size of individuals is larger than the sample size of households, so that the “effective
sample size” (sample size divided by design effect), and thus power, is approximately equivalent for the individual and
household level measures of response rates. Estimated design effects for the survey are discussed in section B.

21

b. Higher average weekly number of food item acquisitions per household
c. Higher proportion of SNAP shopping trips reported by SNAP households
measured by the ratio of reported SNAP shopping trips to the number of SNAP
shopping trips observed in SNAP electronic benefit transfer (EBT) data
d. Higher proportion of acquired food items scanned by respondent households,
measure by the ratio of scanned food items to the number of food items observed
on receipts for transactions with receipts
5. The high incentive will result in higher household response rates and greater individual
participation within households, compared with the low incentive.
6. The high incentive will result in better data quality, compared with the low incentive,
where data quality is measured as for (2) above.
Additional hypotheses were formed during development of the sampling design presented in
Section B. The field test data will provide tests of these hypotheses; failure to confirm these
hypotheses indicates a need to modify the sampling design for the full–scale survey. These
hypotheses include:
1. Among sampled addresses that were matched to SNAP households (the SNAP strata),
the expected eligibility rate will be at least 87 percent, where the screening rate is defined
as the percentage of sampled addresses contacted and for which eligibility is determined.
2. Among sampled addresses that were not matched to SNAP households, the expected
screening rate will be at least 67.5 percent where the screening rate is defined as the
percentage of addresses (that are sampled, not-matched to the SNAP frame, and
considered deliverable) that are contacted and for which eligibility is determined
3. Among SNAP households, the expected completion rate will be at least 71 percent,
where completion is defined by completion of all household interviews and reporting
food acquisition data in booklets (regardless of whether telephone reporting was
complete).
4. Among non-SNAP households, the expected completion rate will be at least 55 percent.
The analysis will follow six key steps:
1. Data Entry. Data entry is needed for field test data collected on paper. Data will be
entered from the household screener and short form for refusals. Price data will be
entered from respondents’ saved receipts from purchases of food that they brought
home and scanned (except for stores in the The Nielsen Company Scantrak database).
2. Match Files. Field test data will be matched with three extant data sources: a UPC
database containing product names and packages sizes (matched on UPC code); Nielsen
store-specific price files (matched by store, UPC, and date); and SNAP records of
electronic benefit redemptions during the sample period for the SNAP household
(matched by SNAP caseid). The final match will be used to validate SNAP households’
reporting of SNAP food purchases.

22

3. Prepare Analytic Files. Each field test data file will be checked for missing or
inconsistent data and outliers, and then cleaned and recoded as necessary. For the CATI
and CAPI surveys, data cleaning will be minimal because of established response options
and controlled skip patterns. Prices entered from receipts will be compared with average
prices for the same items in the Nielsen database to identify outliers that are potential
data entry errors.
4. Prepare Sampling Weights. The field test data will be weighted to produce tabulations
representative of low-income households in the sampled PSUs. Weights will be adjusted
for nonresponse to the survey.
5. Specify Tabulations. The field test data will be tabulated to address each field test
research question (described in Section A.2). Tabulations will be prepared for
households overall and for subgroups defined by food booklet version and incentive
level. Tabulations will include household response rates; item response rates; average
number of food-at-home acquisitions; average number of food-away-from-home
acquisitions; average number of food items acquired in food-at-home and food-awayfrom-home acquisitions; and distribution of food sources.
Although the field test sample is not designed to be nationally representative, we will
nonetheless compare it with national benchmarks to assess its representativeness. SNAP
household characteristics will be compared with the USDA report of Characteristics of
SNAP Households; SNAP household use of SNAP benefits will be compared with
national SNAP electronic benefit transaction statistics; characteristics of the entire field
test sample will be compared with characteristics of low-income households from the
Current Population Survey.
6. Estimate Descriptive Statistics, Including Cross-Tabulations, Using Appropriate
Statistical Methods. The planned analyses of field test data will involve straightforward
frequencies, means, and cross-tabulations. Analyses will be conducted using statistical
software such as SUDAAN or STATA to compute standard errors that adjust for the
complex sample design (See Section B). Statistical tests will be conducted for differences
between households randomly assigned to (1) different food booklet designs and (2)
different incentive levels.

Project Schedule
The planned schedule for the field test of the National Food Study, assuming receipt of OMB
clearance by late November or early December 2011, is as follows:
Activity

Schedule

Select PSUs for field test

September 2010

Train Data Collectors for Field Test

January 2011

Conduct Data Collection

Mid-January through mid-March 2011

Memo of Data Collection for Field Test

April 2011

Memo on processing of Field Test Data

May 2011

Memo Summarizing Field Test

June 2011

23

A17. Reason(s) Display of OMB Expiration Date is Inappropriate
The agency plans to display the expiration date for OMB approval of the information collection
on all instruments.
A18. Exceptions to Certification for Paperwork Reduction Act Submissions
There are no exceptions to the certification statement.

24