OMB Supporting Statement B

USAID Feed the Future Initiative: National Survey of U.S. Adults about Global Food Security

Generic Information Collection under Formative Research and Tool Development

OMB #XXXXXX

Section B: Supporting Statement

November 11, 2019

Contact

Jennifer Cupp

Communications Director, Feed the Future

U.S. Agency for International Development, Bureau for Food Security

Principal Investigator

1300 Pennsylvania Avenue NW

Washington, DC 20004

Phone: 202-712-4381

E-mail: [email protected]

TABLE OF CONTENTS

1. Respondent Universe, Sampling Method, and Response Rate 3

2. Procedures for the Collection of Information 5

3. Methods to Maximize Response Rates and Deal with No Response 5

4. Tests of Procedures or Methods to be Undertaken 6

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or

Analyzing Data 6

EXHIBITS

Exhibit 1.1: Sampling Universe, Sampling Frame, and Survey Sample Size 3

Exhibit 1.2: Distribution of Survey Sample Selected 4

Exhibit 5.1: Statistical Consultants and Project Team Members 7

LIST OF ATTACHMENTS

Attachment 1 Authorizing Legislation, Executive Order 12862

Attachment 2 Questionnaire

Attachment 3 NORC AmeriSpeak Privacy Statement

Attachment 4 NORC AmeriSpeak Technical Overview

Attachment 5 NORC AmeriSpeak Standard Profile Variables

Attachment 6 NORC AmeriSpeak Panel Documentation for IRBs

Attachment 7 NORC AmeriSpeak Survey Invitation and Reminder

Attachment 8 Published 60 Day Federal Register Notice

Attachment 9 Published 30 Day Federal Register Notice

1. Respondent Universe, Sampling Method, and Response Rate

The respondent universe for this study includes all residents of the United States who are age 18 or older. This universe is estimated at 253,881,929 individuals (the total US population minus the 22.4% who are minors), based on US Census population estimates from July 1, 2018 (https://www.census.gov/quickfacts/fact/table/US#). The sample sample will be drawn from the AmeriSpeak panel, which is maintained by NORC. It includes approximately 35,000 adult individuals who fit the study’s target criteria. Therefore, the number of individuals who have the possibility of responding to this survey is 35,000. See Exhibit 1.1 for a summary.

Exhibit 1.1: Sampling Universe, Sampling Frame, and Survey Sample Size

Group	Size
Respondent universe	253,881,929
Panel sample	35,000
Survey invitations issued	2,200
Survey competes	1,000

Sampling will be done through AmeriSpeak, a probability-based panel designed to be representative of the U.S. household population.

AmeriSpeak leverages the NORC National Frame, which provides sample coverage for over 97 percent of the U.S. households. The 2010 National Frame used a two-stage probability sample design to select a representative sample of households in the United States. The first stage—the sampling unit—is a National Frame Area (NFA), which is either an entire metropolitan area (made up of one or more counties) or a county (some counties were combined so that each NFA contains a population of at least 10,000). The largest NFAs with a population of at least 1,543,728 (0.5 percent of the 2010 Census U.S. population) were selected with certainty; these areas have a high-population density, and are dominated by tracts with street-style addresses. These areas contain 56 percent of the population within 8 percent of the geographic area of the United States. The remaining areas were stratified into areas where street-style addresses predominate, and the remaining areas, which are less likely to have street -style addresses. The latter stratum (“rural” areas) comprises 81 percent of the geographic area, but only 14 percent of the population.

Within the selected NFAs, the second stage sampling unit is a segment, defined in terms of either Census tracts or block groups, containing at least 300 housing units according to the 2010 Census. A stratified probability sample of 1,514 segments was selected with probability proportional to size. For most of the 1,514 segments, the U.S. Postal Service Delivery Sequence File (DSF) provided over 90 percent coverage of the segments in terms of city-style addresses that are geo-codeable. For the 123 segments where the DSF provided insufficient coverage, we enhanced the DSF address list with in-person listing. The National Frame contains almost 3 million households, including over 80,000 rural households added through the in-person listing.

See Attachment 4 for a full technical overview of NORC’s AmeriSpeak panel.

Exhibit 1.2 shows the distribution of the survey’s intended sample. A side-by-side comparison of AmeriSpeak’s panel composition with the demographic benchmarks established by the U.S. Census Current Population Survey (CPS) and the American Community Survey (ACS) illustrates AmeriSpeak’s sample representativeness. The data are weighted for active AmeriSpeak Panel households and the Census benchmark data sources.

Exhibit 1.2: Distribution of Survey Sample Selected

	AmeriSpeak	Census*	Delta
Variable	Column %	Column %	Percentage Pt.
Age
18-24	11.8	11.8	0.0
25-34	17.4	17.9	-0.5
35-44	16.8	16.3	0.5
45-54	16.2	16.8	-0.6
55-64	17.5	16.9	0.5
65-74	14.0	12.1	1.9
75+	6.4	8.3	-1.9
Race
Non-Hispanic White	63.5	63.5	0.0
Hispanic	11.9	11.9	0.0
Non-Hispanic Black	11.9	11.9	0.0
Non-Hispanic Asian or Pacific Islander	3.6	6.2	-2.7
Non-Hispanic All Other	4.8	2.2	2.7
Gender
Female	51.7	51.7	0.0
Male	48.3	48.3	0.0
Educational Attainment
Less than High School	10.6	10.6	0.0
High School Grad	28.6	28.6	0.0
Some College	28.3	28.3	0.0
College Grad	18.9	20.9	-2.0
Postgrad Work	13.6	11.6	2.0
Household Income
Less than $25K	15.5	15.3	0.2
$25K < $50K	20.9	20.5	0.4
$50K < $75K	17.8	18.3	-0.5
$75K < $100K	13.9	14.0	-0.1
$100K or more	32.0	32.0	0.0

* All data come from Feb 2018 CPS except for Household Income (2016 1-Year ACS). All data provided by NORC:

https://amerispeak.norc.org/Documents/Research/AmeriSpeak%20Panel%20Demographic%20Report%202019%2004%2030_v2.pdf

The overall expected cumulative response rate for this study is 10%, based on NORC’s AmeriSpeak panel and prior nationally-representative studies completed within that panel. For survey completion, the expected rate is 35%. This particular study has not be conducted previously.

2. Procedures for Collection of Information

The sample for a specific study is selected from the AmeriSpeak Panel using sampling strata based on age, race/ethnicity, education, and gender (48 sampling strata in total). The size of the selected sample per sampling stratum is determined by the population distribution for each stratum. In addition, sample selection takes into account expected differential survey completion rates by demographic groups so that the set of panel members with a completed interview for a study is a representative sample of the target population. If panel household has one more than one active adult panel member, only one adult in the household is eligible for selection (random within-household sampling). Panelists selected for an AmeriSpeak study earlier in the business week are not eligible for sample selection until the following business week.

In this study, surveying will be conducted with a mixed-mode approach of both online and telephone interviews. NORC has estimated that approximately 17% of a general population survey using its panel are completed via phone, and the rest are expected to be online. Online surveying necessitates an invitation email, which will be delivered to respondents in the panel. A template for the invitation from NORC is found in Attachment 7 and includes programming language for including the panelist’s name and the specific point incentive for this study.

The AmeriSpeak Panel is a probability-based panel representative of the total US population. Since the panel is not a simple random sample, a design effect reflects the multi-stage panel recruitment design, which includes subsampling of initial non-respondents for the face-to-face, in person recruitment in the non-response follow-up stage. The design effect for this study is expected to be approximately 1.75. Including the design effect, the margin of error for a 50% statistic at the 95% confidence level would be +/- 4.1 percentage points. For example, on a Yes/No question where 50% of the respondents answer “Yes,” we have 95% confidence that the population measurement is between 45.9% and 54.1% (taking into account sampling error). The sampling margin of error decreases for survey statistics below or greater than 50%. Therefore, the sampling margin of error estimate of +/-4.1 percentage points is a conservative estimate of the margin of error.

This relatively high degree of precision is needed in this survey because, as described in the justification in Supporting Statement A, part of the objective is to understand the composition and size of different audience segments who know or support Feed the Future’s mission to varying degrees. To do so, this survey’s sample must be representative. But because the sample is nationally-representative, there are no expected unusual problems requiring specialized sampling procedures in this research. There will be no periodic data collection cycles to reduce burden since this is a one-time survey.

3. Methods to Maximize Response Rates and Deal with No Response

Participants in this study are registered with AmeriSpeak and will be offered survey choice “points” to redeem for prizes which are commonly provided to survey panel respondents who complete online surveys. The points will not be sent to respondents from USAID, but instead will be provided by the online panel provider to respondents who complete the survey. This is part of the business model of the online panel provider.

All panel-based research organizations, whether probability-based such as AmeriSpeak or non-probability, motivate panel members to continue participating in the surveys through an internal rewards program. Relatively small amounts of points are used for AmeriSpeak panel surveys based on an understanding of what would keep the participants engaged and motivated to obtain maximum retention of panelists and survey participation. For this particular study, points worth $6 will be awarded to panelists for completing the survey.

Panelists who do not respond to the initial survey invitation will receive up to two reminders emphasizing the importance of their participation in the project (included in Attachment 7). Telephone panelists will receive multiple follow-up call attempts at strategic times over the course of the survey recruitment period.

NORC will calculate and report unit and item non-response rates and carry out a non-response bias analysis following the guidelines in Standard 3.2 of the OMB Standards and Guidelines for Statistical Surveys. NORC will assess and measure non-response bias by evaluating the demographic and geographic representativeness of the survey participants compared to the Current Population Survey (CPS) population benchmarks.

4. Tests of Procedures or Methods to Be Undertaken

The survey will be tested and reviewed thoroughly both before and after initial deployment. After the survey is programmed, the NORC programming team will conduct quality control reviews to ensure the work is error-free; then, KRC Research will review the study again for quality and accuracy. In its testing, KRC Research tests for missing survey components, incorrect or missing survey or question routing logic, inaccuracy in conveying initial instructions for programming, misspellings, poor formatting, language or content that may be confusing when displayed on the screen for online survey takers, and phrasing that may be unclear or convoluted for phone survey takers.

After testing has been completed and any corrections or revisions made, NORC will deploy the survey to a subset of the total sample size before suspending the survey progress to allow for KRC Research to check the raw data files. This pause after “soft launch” typically occurs at 10% of survey completes. At this point, KRC Research will review the raw data question by question to ensure programming and logic is displaying correct questions to correct audiences (where applicable), all questions have proper base sizes, and there are no highly unusual initial findings to suggest that a question needs rephrasing or clarification. Once this soft launch review has been completed, KRC Research will authorize deployment of the survey to the remaining sample.

5. Individuals Consulted On Statistical Aspects and Individuals Collecting and/or Analyzing Data

Exhibit 5.1 below lists the project team members consulted on aspects of research design and, separately, those who will be collecting and analyzing the data.

• USAID is responsible for oversight of KRC researchers overseeing research.

• KRC Research and NORC are responsible for the design of the research.

• KRC Research and NORC are responsible for implementation.

• KRC Research is responsible for analyzing data, preparing report(s) on findings, and presenting and delivering those findings to USAID.

USAID staff will neither collect data from nor interact with research participants. No individual identifiers will be linkable to collected data, and no individually identifiable private information will be shared with or accessible by USAID staff.

Exhibit 5.1: Statistical Consultants and Project Team Members

Team Member	Organization	Phone	Email
Jennifer Cupp	USAID	202-712-4381	[email protected]
Mark Richards, PhD	KRC Research	202-230-8767	[email protected]
Mike Ruddell	KRC Research	202-585-2946	[email protected]
Suzanne Howard	NORC	312-759-5244	[email protected]

National Survey of U.S. Adults about Global Food Security: Supporting Statement B 16