Memorandum

Memorandum

May 9, 2017

To:	Robert Sivinski, Desk Officer Office of Management and Budget
From:	Michele Ver Ploeg, Economist US Department of Agriculture, Economic Research Service
Subject:	Request to administer questionnaires on users of National Household Food Acquisition and Purchase Survey dataset to evaluate the survey

The purpose of this memorandum is to obtain OMB clearance for a research study that will evaluate the strengths and weaknesses of the National Household Food Acquisition and Purchase Survey (FoodAPS-1) from a data user’s perspective. This memorandum describes the research study, including the recruitment strategy and survey instrument.

Background and Purpose

We seek to collect feedback from FoodAPS-1 researchers as part of the preparation for a possible FoodAPS-2. This is a one-time data collection. The overall goal of this study is to evaluate the data and research of the FoodAPS-1 Geography Component and to identify other data needs for further research in FoodAPS-2. In partial fulfillment of this goal, we will survey researchers who have used FoodAPS-1 to evaluate strengths and weaknesses of the broader dataset. The survey will capture the following elements:

• Difficulties encountered while using FoodAPS-1

• Variables that were most useful or in need of improvement

• Adequacy of supporting documentation such as users’ guides and codebooks

• Suggestions for modifications in design and data collection for FoodAPS-2

Survey results will contribute to a broader evaluation of FoodAPS-1 data components. With the help of evaluation feedback, FoodAPS-2 data will be able to respond more effectively to intense public policy interest around food expenditure and acquisition, consumer food shopping behaviors, the food retail environment, and healthy food access.

Research Plan

This section describes the research plan and instruments. These procedures and instruments were informed by a pre-test of 6 respondents, as described in Appendix A.

Sample and Recruitment

For each of the 40 research teams that has used or is using FoodAPS-1, we will recruit 2 team members as respondents, one investigator and one data analyst. For investigators, our target population is non-governmental investigators who either have previously used or are currently using FoodAPS-1 data. To construct our contact list, we obtained the names and contact information of principal investigators, including recipients of National Bureau of Economic Research (NBER) grants, University of Kentucky Center for Poverty Research grants, external researchers with ERS cooperative agreements, and other external researchers suggested by ERS staff (see Appendix E). No payment in any form will be offered to respondents.

For recruitment purposes, we will make up to four attempts to contact potential respondents. We first will communicate with investigators via email (see Appendix B, Exhibit B-1). The email describes the purpose of the survey and includes the survey as an attachment. Each responding investigator will identify the team member to serve as the data analyst respondent, which may be the investigator himself or herself. If we do not receive a response, we will follow up within six business days by email (see Appendix B, Exhibit B.2). If we do not receive a response via email, we will follow up via telephone, making two attempts to reach the investigator (see Appendix B, Exhibits B-3 and B-4). After the fourth attempt, if we still do not receive a response, we will it code as a non-response.

We will track our contacts using a recruitment log formatted for Microsoft Excel (see Exhibit 1).

Exhibit 1. Recruitment log template.

Investigator Name

Data Analyst

Email Contacts

Telephone Contacts

Result

Affiliation

Email

Tel.

Date

Outcome

Date

Outcome

Date

Outcome

Date

Outcome

Format

We will offer participants the option of completing the survey as an email-administered survey or telephone-administered survey:

• Email-administered survey. Participants may provide written responses to survey questions via email. Respondents choosing this option will complete and return the questionnaire in Microsoft Word format. As part of their written responses, participants may indicate whether they would like to speak via telephone to answer any follow-up questions. Follow-up questions may arise if written responses lack sufficient detail.

• Telephone-administered survey. Participants may request to schedule an interview. During the telephone interview, we will ask the same questions asked of participants who choose to respond in writing. We also will ask follow-up questions as needed during the interview session. All survey responses collected via telephone will be audio-recorded using WebEx, which is available to us through Tufts University. We will record field notes in writing during follow-up or full telephone interviews as well. Using audio recordings and field notes as guides, we will produce detailed summaries of survey responses, with some sections being transcribed verbatim.

We pre-tested both modes, as described in Appendix A. We expect that completing the survey will take approximately 60 minutes for each respondent. This estimate includes 30 minutes for the survey itself and 30 minutes for reviewing instructions, corresponding with us via email, and answering questions, either via email or telephone. The detailed interview summaries along with written survey responses will provide the basis for an evaluation of strengths and weaknesses of the FoodAPS-1 dataset.

Analysis Plan & Report Outline

Our evaluation of strengths and weaknesses of the FoodAPS-1 dataset will incorporate analysis of published literature, survey responses, and FoodAPS-1 data. For our analysis of survey responses, we will tabulate and summarize results about data that were used, barriers that were encountered, and improvements or modifications that were suggested. We will share these tabulations and summaries as part of a report to ERS. This report will take on the following structure:

1. Introduction

   1. Goals

   2. Methods

2. Literature review

3. Results

   1. Interview results

   2. Post-interview FoodAPS-1 analysis results

4. Conclusion

5. Appendix

   1. Full list of barriers

   2. Full list of suggestions

Survey Instrument

The survey instruments (see Appendix D) include a mix of closed and open-ended questions. Topics include background information, data usage, and recommendations. We summarize revisions to the survey in our overview of the pre-test in Appendix A.

Confidentiality

ERS intends to protect respondent information under the Privacy Act of 1974 and 7 U.S.C. 2276. ERS has decided not to invoke the Confidential Information Protection and Statistical Efficiency Act of 2002 (CIPSEA). Any information collected from interview participants and disclosed to the public will not identify or attribute information to particular respondents. We will report on all information in aggregate form. Participants will receive an explanation of confidentiality in a disclaimer (see Appendix C, Exhibit C-1). We also describe confidentiality and voluntariness in an informed consent form (see Appendix C, Exhibit C-2) and verbal consent form (see Appendix C-3). For participants who choose to respond in writing, we will request that participants return an electronically signed informed consent form to us via email. For participants who choose to respond via telephone, we will send a signed and scanned verbal consent form to them via email.

Burden Information

Our time burden estimates (see Exhibit 2) are based on information gathered from the pre-test phase. We will send the survey to 40 research teams, with a target response rate of 75%, or 30 teams that actually complete the survey. For each of the two respondents in each research team, the survey itself will require 30 minutes. In the pre-test (Appendix A), mean completion time was 30 minutes under each mode. We estimate that another 30 minutes will be needed for the following preparation tasks: email correspondence; reading the disclaimer; refreshing memory about FoodAPS-1 data and code-books; and follow-up questions. Thus, we estimate that this research protocol will require 70 hours (4,200 minutes) on part of all participants.

Exhibit 2. Reporting burden for survey
Survey	Sample Size	Freq	Responses				Non-response				Total Burden Hours
			Resp. Count	Freq x Count	Min./ Resp1	Burden Hours	Nonresp Count	Freq. x Count	Min./ Nonr1	Burden Hours
Preparation Tasks	80	1	60	60	30	30	20	20	30	10	40
Interview or Survey	80	1	60	60	30	30	20	20	0	0	30
Total	80		60			60				10	70
1 We will not ask participants to track time burden from email correspondence, but we estimate average and total time burden based on length and content of email messages observed during the pre-test phase (Appendix A).

Contact Information

The contact person for questions regarding this data collection is:

Michele Ver Ploeg

(202) 694-5372

[email protected]

Appendices

• Appendix A. Pre-Test

• Appendix B. Recruitment Emails and Telephone Scripts

• Appendix C. Disclaimer and Consent Forms

• Appendix D. Questionnaire

• Appendix E. Contact List

5