Study Requesting Memo

MEMORANDUM

March 28, 2016

To: Robert Sivinski, Desk Officer

Office of Management and Budget

From: John Kirlin, Economist

United States Department of Agriculture, Economic Research Service

Via: Pheny Weidman, Clearance Officer

United States Department of Agriculture, Economic Research Service

Subject: Request to conduct two field pretests under generic clearance for survey research studies (OMB Control # 0536-0073)

The purpose of this memorandum is to obtain OMB clearance for two small-scale field pretests to test a new data collection system to collect food purchases and acquisitions and to test new survey questions about household income.

BACKGROUND AND PURPOSE

In 2012, the National Household Food Acquisition and Purchase Survey (FoodAPS1, OMB Control # 0536-0068) was conducted on behalf of the U.S. Department of Agriculture’s (USDA) Economic Research Service (ERS), as the first nationally representative survey of American households to collect unique and comprehensive data about household food purchases and acquisitions, along with factors that influence household food choices. FoodAPS1 aims to fill a critical data gap and support research that informs policymaking on key national priorities, including health and obesity, hunger, and nutrition assistance policy.

Though FoodAPS1 was successful, the ERS hopes to take steps to improve the quality and efficiency of data collected in the subsequent fielding of FoodAPS2, in 2018 or 2019. ERS has done much to evaluate the procedures and protocols used in FoodAPS1 in order to suggest potential improvements for FoodAPS2, however, it was determined that a pilot test of suggested changes would be necessary to evaluate their feasibility and impact.

ERS has contracted with Westat to conduct a pilot study to recommend, develop, and test an alternate data collection method (ADCM) for collecting FoodAPS data. This study is referred to as a pilot study that will recommend viable improvements for FoodAPS2. The pilot study will be implemented in the fall of 2016. To prepare for the pilot, Westat plans two field pretests of the procedures and systems.

The first will be implemented in April and May 2016. The second pretest will be implemented about three months before the start of the Pilot Study. The findings from the pretests will help us finalize data collection instruments and procedures for the Pilot.

The first pretest will test questions and procedures of the ADCM study that are different from FoodAPS1:

1. We will fully test the web-based reporting system that allows all members in eligible households to report food or drinks obtained during their assigned 7-day data collection period.

2. We will evaluate respondents’ comfort using different electronic equipment to access the web-based system and scan product barcodes. These include:

   1. Using smartphones to scan product barcodes in grocery stores and other locations where food is acquired and complete their electronic diaries,

   2. Using hand-held traditional barcode scanners attached to laptops or computes to scan barcodes and complete their diary,

3. We will evaluate the use of smartphones to record food away from home food acquisitions. We will encourage respondents to record mementos of food away from home events such as taking photos, creating text reminders, or voice memos. The pretest will help us understand how often respondents use these tools and how to train respondents to use these tools.

4. We will test newly developed income questions and investigate the feasibility of asking the primary respondent within the household to answer questions about income received by all members of the household during the past 30 days through the same web-based reporting system that captures their food acquisitions.

The purpose of the second pretest is to test the full integration of all the data collection and field management systems that will be implemented in the Pilot. These include three CAPI instruments, the web-based food reporting system and barcode scanning application, and the interviewer management systems. Special attention will be paid to new changes made to the food reporting system after the first pretest.

RESEARCH PLAN FOR PRETEST 1

Pretest 1 is to be conducted on 10 purposively selected households.

Selection of households for Pretest 1

We will recruit three households with active SNAP participants and three households with active WIC participants. The four remaining households will be recruited according to other characteristics such as smartphone ownership or age of the household members.

We will recruit by posting flyers, online advertisements on Craigslist, online advertisements on the Westat Intranet, and using a database of research participants from prior projects at Westat. The advertisements used to recruit participants are shown in Appendix A. The recruiting script in Appendix B will be used to screen respondents.

Data collection tasks for Pretest 1

Screened participants will be scheduled for a visit by Westat interviewers. Interviewers will be survey methodologists and project staff responsible for developing the training materials for the Pilot. An interviewer will visit the household at the scheduled interviewing day and time. The interviewer will meet with the participant and will give the respondent a brief introduction to the FoodAPS study. A copy of the disclaimer statement (Appendix G-1) is provided to the household. The interviewer will then identify the primary respondent (i.e., the main food shopper or the meal planner) and household members eligible for food diary collection (i.e., household members aged 11 or older). Adult household members will be asked to read through the consent form in Appendix D-1 and sign it. The interviewer will then ask parents to assent for minors between the age of 11 and 17 to access the web instrument themselves and participate in the study (consent form is shown in Appendix D-2). The primary respondent will report for minors when consent is not provided for them to access the web instrument.

The interviewer will then train all household members who are available and have given consent on how to use the web-based system to report food or drinks obtained during the data collection week. Household members are asked to access the web-based system from their own device. Households without the necessary equipment will be provided a smartphone and/or a laptop and MiFi. A Mifi is a wireless router that acts as a mobile Wifi hotspot providing Internet to the household.

The interviewer will also train the primary respondent as well as adult members of the household who are available to complete the income worksheet online during the data collection week. The interviewer will then make an appointment with the primary respondent for a second visit.

At the end of the seven-day data collection period, the interviewer will visit the household for the second time. During this visit, interviewers will administer the new income questions to the primary respondent, conduct debriefings with the primary respondents and any household members who are present at the second visit (see Appendix E), collect the smartphone or laptop loaned to the household, any receipts and materials respondents have saved, and provide the promised incentives that the household has earned during the week of data collection.

Training for Pretest 1

For this pretest, interviewers will be survey methodologists and project staff responsible for developing the training materials for the Pilot. The training will be completed on a single day and will involve each staff member walking through the data collection protocol and using the electronic surveys and hard copy materials while role playing all the procedures that need to be implemented during each visit with the households. Interviewers will record notes about what is needed to communicate the protocol to households. They will update these notes as the pretest progresses and they learn more from debriefing with households and a review of the data. These notes will be used to inform the training for the Pilot study

Research goals for Pretest 1

With this pretest, we will focus on learning from pretest respondents the following issues:

• How often do household members have problems reporting price or quantity information?

• How helpful is the predefined list of stores and locations?

• How likely are respondents to find item descriptions for their food acquisitions?

• Do respondents have any problems with the web log in procedures?

• Are respondents using mementos as expected? Are the tools given to them helpful for keeping mementos?

• Did the primary respondents use a smartphone or another device to review the household’s food acquisitions at the end of the day?

• How do younger household members (aged 11 to 17) access the web-based reporting system?

• How well does communication with each eligible household member work?

• Are there any issues with the instrument, technology, or other data collection procedures that households had during the week?

• Did household members find the data collection procedures burdensome? What suggestions did they have for improvements to the data collection procedures?

Westat will provide a memo reviewing the results from the pretest and use the results to inform the development of Pretest 2 as well as the training materials that will be used to train both field interviewers and eligible respondents.

RESEARCH PLAN FOR PRETEST 2

Pretest 2 will also be conducted on 10 purposively selected households.

Selection of households for Pretest 2

Westat will attempt to recruit four households with their own iPhones and six households with Android phones. Households will be recruited by posting flyers (shown in Appendix A), online advertisements on Craigslist, online advertisements on the Westat Intranet, and using a database of research participants from prior projects at Westat.

Data collection tasks for Pretest 2

Once recruited, Westat interviewers will visit the selected households to identify a primary respondent within each household who is defined as either the household’s main food shopper or the meal planner and introduce the ADCM Study. A copy of disclaimer statement is provided to the household (see Appendix G-2). Adult household members will be asked to read through the consent form in Appendix D-3 and sign it. The interviewer will then ask parents to assent for minors between the age of 11 and 17 to access the web instrument themselves and participate in the study (consent form is shown in Appendix D-2). The primary respondent will report for minors when consent is not provided for them to access the web instrument.

Interviewers will conduct the Household Screener Interview (see Appendix C-5) and Initial Interview (see Appendix C-6) and then train all household members aged 18 and older on how to load the food data collection apps onto household members’ smartphones and on how to use the app to report food or drinks obtained during the data collection week. With an adult household member’s consent they will also train children between the ages of 11 and 17 on how to access the app on their own smartphones. All eligible household members will be asked to record food acquisitions over a 7-day period using own smartphone or a computer.

At the end of the 7-day period, interviewers will visit the household for the second time to conduct the Final Interview (see Appendix C-7) and provide the promised incentive.

All eligible household members will be sent e-mail communications with regard to their food entry status and the amount of incentive accumulated at the household level. If necessary, all eligible household members will also be contacted by interviewers at different points in the week to discuss any issues or problems that they might be having using the web-based system or with reporting their food acquisitions. Interviewers will leave their phone numbers with the households so that household members can call them if they encounter problems or have questions.

Training for Pretest 2

For this pretest, interviewers will be survey methodologists and project staff responsible for developing the training materials. The interviewers will test early drafts of the electronic training materials. As in Pretest 1, they will record notes about enhancements that could improve the training for the pilot study

Research goals for Pretest 2

With this pretest, we will focus on learning from pretest respondents the following issues:

• How easy or difficult it is to load the food-reporting apps to household members’ own smartphones?

• Do respondents have any problems with the downloading and installations of the apps?

• Do respondents encounter problems with the web log-in procedures?

• Do respondents encounter problems entering food and drink acquisition to the app?

• Are respondents using mementos as expected? Are the tools given to them helpful for keeping mementos?

• How do younger household members (aged 11 to 17) access the app?

• Are there any issues with the instrument, technology, or other data collection procedures that households had during the week?

• Did household members find the data collection procedures burdensome? What suggestions did they have for improvements to the data collection procedures?

Westat will provide a memo reviewing the results from the pretest and use the results to inform the development of the pilot test as well as the training materials that will be used to train both field interviewers as well as eligible respondents.

DATA COLLECTION INSTRUMENTS FOR PRETESTS

The survey instruments for reporting food acquisitions are web-based tools that collect information about respondents’ food acquisitions. The web-based tools can be accessed on a computer or on a smartphone. Draft versions of screenshots of the web version are shown in Appendix C-1. The instrument starts by asking participants to report some event level information about the food purchase and then participants are asked to report each food or drink item that was purchased. Participants are also asked to upload photos of receipts. Participants will receive reminder e-mail or text messages throughout the week if they do not report at the end of each day.

The survey instrument for reporting income questions during the data collection week is a web-based instrument. The web instrument can be accessed on a computer or on a smartphone. The instrument starts by asking whether or not household members received income from select income sources and if received, asks about details on the amount of income received. Screenshots of the web version are shown in Appendix C-2. Household members will receive reminder e-mail messages to complete the income worksheet.

Participants will also be asked to report the meals and snacks that they ate each day of the week. An example of the meals and snacks form from FoodAPS1 is shown in Appendix C-3. An electronic version of this form will be used during the pretest.

Interviewers will also ask the primary respondents to answer income questions again on their second visit to the household. The question wordings are mostly the same as those used in the web instrument. The income questions are shown in Appendix C-4.

Four new instruments will be used in Pretest 2 that will not be used in Pretest 1: Household Screener Interview (Appendix C-5), Initial Interview (Appendix C-6), Final Interview (Appendix C-7), and Respondent Feedback Form (Appendix F). We will also test the integration of these instruments with the food acquisition web instrument. For example, the household roster is created during the household screening interview and this information is passed on to the web instrument. In addition, income information that is collected in the web instrument using the income worksheet (Appendix C-2) is passed on to the Final Interview.

CONFIDENTIALITY

Respondents will be provided the following with all materials.

All information which would permit identification of an individual, a practice, or an establishment will be held confidential, will be used for statistical purposes only, will be used only by USDA staff, contractors, and agents authorized by USDA to perform statistical activities only when required and with necessary controls, and will not be disclosed or released to other persons without the consent of the individual or establishment in accordance with the Confidential Information Protection and Statistical Efficiency Act (PL-107-347). By law, every employee as well as every agent has taken an oath and is subject to a jail term of up to five years, a fine of up to $250,000, or both if he or she willfully discloses ANY identifiable information about you.

BURDEN INFORMATION

The anticipated burden levels associated with this information collection request are shown in Table 1.

Table 1. Reporting burden for Pretest 1 and Pretest 2.

Reporting Burden for Pretest 1
Instrument	Estimated Number of Respondents	Responses Annually per Respondent	Total Annual Responses	Estimated Average Number of Hours per Response	Estimated Total Annual Hours of Response Burden
Disclaimer	25	1	25	.08	2.00
Household recruiting	10	1	10	.08	0.80
Consent form	25	1	25	.08	2.00
Household training for reporting food acquisitions	25	1	25	.75	18.75
Income Worksheet	25	1	25	.25	6.25
Final Interview – income questions only	10	1	10	.17	1.70
Food Reporting System	25	7	175	.42	73.50
Meals and snacks form	25	7	175	.05	8.75
Debriefing	25	1	25	1.00	25.00
Total burden	25				138.75
Reporting Burden for Pretest 2
Disclaimer	25	1	25	.08	2.00
Consent Form	25	1	25	.08	2.00
Household Screener	10	1	10	.20	2.00
Initial Household Interview	10	1	10	.50	5.00
Household training for reporting food acquisitions	25	1	25	.75	18.75
Income Worksheet	25	1	25	.25	6.25
Final Household Interview	10	1	10	.50	5.00
Food Reporting System	25	7	175	.42	73.50
Meals and snacks form	25	7	175	.05	8.75
Respondent Feedback Form (see App. F)	10	1	10	.08	0.80
Total burden	25				124.05
TOTAL BURDEN FOR BOTH PRETESTS	50				262.8

The following statement will be displayed on the web data collection instrument where the respondents are reporting their food acquisitions, income, and meals and snacks.

NOTIFICATION TO RESPONDENT OF ESTIMATED BURDEN: Public reporting burden for this collection of information is estimated to average 5.6 hours per respondent for Pretest 1 and 5 hours per respondent for Pretest 2, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The valid OMB control number for this information collection is #0536-0073. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to: ERS, 1400 Independence Avenue, SW, Mail Stop 1800, Washington, DC 20250-1800, ATTN: John Kirlin (202-694-5661). Do not return the completed form to this address.

INCENTIVE PAYMENTS

For both pretests, recruited households will accumulate $3 per day for each household member whose food acquisition behavior is recorded in the web system. Food acquisition behavior includes affirmation of no food acquisitions that day. The households will also receive an additional $50 at the end of the week if all household members record food acquisitions on each of the 7 days. In addition, $5 will be provided to the household if they complete the income questions online.

If approved by ERS, $75 will be provided for completing the debriefings with the interviewers at the end of the 7-day data collection for the first pretest.

For Pretest 2, $50 will be provided to the households on completion of the Initial interview.

A summary of the proposed incentive levels is shown in Table 2.

Table 2. Summary of proposed incentives for 2016 FoodAPS pretests

	First Pretest		Second Pretest
Component	Incentives for Food Diary	Special Incentive for Pretest	Incentives for Food Diary	Special Incentive for Pretest
Debriefing interview		$75
Completion of initial interview				$50
Income worksheet	$5		$5
Purchasing behavior	$3 a day per person (max $21 per person)		$3 a day per person (max $21 per person)
HH reports all days; all members, all questionnaires	$50 bonus		$50 bonus
Maximum post-Screener Total 1 person HH	$151		$126
Maximum post-Screener Total 2 person HH	$172		$147
Maximum post-Screener Total 3 person HH	$193		$168
Maximum post-Screener Total 4 person HH	$214		$189

The incentive levels shown in Table 2 are comparable to those used in FoodAPS1. Table 3 shows a summary of the incentives used in FoodAPS1.

Table 3. Incentives levels for different types of households in FoodAPS 1.

Type of household	Incentive Amount
Single Adult households
One person household	$130
No youth or teens	$130
Youth only	$142
Teens only	$153
Youth and teens	$166
Multiple Adult households
Adults, no youth or teens	$151
Adults and youth	$162
Adults and teens	$181
Adults, youth, and teens	$197

CONTACT INFORMATION

The contact person for questions regarding this data collection is:

John Kirlin

202-694-5661

[email protected]

Appendixes

• Appendix A: Advertisements

• Appendix B: Recruiting Script

• Appendix C-1: Screenshots of Web-based Food Reporting System

• Appendix C-2: Income worksheet Web Instrument

• Appendix C-3: Meals and Snack form

• Appendix C-4: Income questions in Final Interview

• Appendix C-5: Household Screener Interview for Pretest 2

• Appendix C-6: Initial Interview for Pretest 2

• Appendix C-7: Final Interview for Pretest 2

• Appendix D-1: Adult Consent Form for Pretest 1

• Appendix D-2: Minor Consent Form for Pretest 1 and 2

• Appendix D-3: Adult Consent Form for Pretest 2

• Appendix E : Pretest Debriefing Protocol for Pretest 1

• Appendix F: Respondent Feedback Form for Pretest 2

• Appendix G-1: Disclaimer for Pretest 1

• Appendix G-2: Disclaimer for Pretest 2

• Appendix H: Study Information Sheet

14