Part B Revised 2018 01 18 Clean

Supporting Justification for OMB Clearance for the Scanner Capability Assessment of SNAP-Authorized Small Retailers (SCANR) Study

Part B

June 26, 2017 (Revised, 1/18/18)

Office of Policy Support

Food and Nutrition Service

United States Department of Agriculture

3101 Park Center Drive

Alexandria, VA 22302

Project Officer: Jenny Genser

Telephone: 703-305-2559

Email: [email protected]

Table of Contents

Part B: Collection of Information Using Statistical Methods B-3

B.1 Respondent Universe and Sampling Methods B-3

B.2 Procedures for the Collection of Information B-14

B.3 Methods to Maximize Response Rates and to Deal with Non-response B-16

B.4 Test of Procedures or Methods to be Undertaken B-18

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data B-20

References B-21

Tables

Table B1.1.Estimated eligibility rates, response rates, and respondents for each phase of the SCANR Survey B-8

Table B1.2. Sample allocation for the SCANR Survey with stratification by urbanicity and store type B-10

Table B3.1. Estimated response rate by phase and mode B-18

Table B4.1. Number of pretests by instrument B-20

Table B5.1. Individuals consulted on data collection or analysis B-20

Appendices

A.1 The Agricultural Act of 2014, Section 4002

A.2 Sec. 17(a)(1) of the Food and Nutrition Act of 2008

B Research Objectives and Questions

C.1 Industry Interviews Initial Recruitment Email

C.2 Industry Interviews Recruitment Scheduling Email

C.3 Industry Interview Guide

D.1 SCANR Survey Initial Cover Letter

D.2 SCANR Survey Frequently Asked Questions

D.3 SCANR Survey Letter for Second Mailing

D.4 SCANR Survey Letter for Third Mailing

D.5 SCANR Survey Script for Nonresponse Telephone Follow-Up

D.6 SCANR Survey Questionnaire - Mail Version

D.7 SCANR Survey Questionnaire - Web-based version and Screen Shots

D.8 SCANR Survey Questionnaire - CATI version

E.1 Follow-Up Interviews with Stores with Scanning Technologies that Meet the Federal Requirement - Initial Recruitment Letter/Email

E.2 Follow-Up Interviews with Stores without Scanning Technologies that Do Not Meet the Federal Requirement - Initial Recruitment Letter/Email

E.3 Follow-Up Interviews with Stores with Scanning Technologies that Meet the Federal Requirement - Recruitment Scheduling Script

E.4 Follow-Up Interviews with Stores without Scanning Technologies that Do Not Meet the Federal Requirement - Recruitment Scheduling Script

E.5 Follow-Up Interviews with Stores with Scanning Technologies that Meet the Federal Requirement - Interview Guide

E.6 Follow-Up Interviews with Stores without Scanning Technologies that Do Not Meet the Federal Requirements - Interview Guide

E.7 Follow-Up Interview Thank You Letter

E.8 Follow-Up Interview Voicemail Script

F Summary of Pretest Methods and Findings

G National Agricultural Statistics Service Comments

H Privacy and Nondisclosure Agreement

I IRB Approval Notice

J Excel Burden Chart

PART B. COLLECTION OF INFORMATION USING STATISTICAL METHODS

B.1 Respondent Universe and Sampling Methods

Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Respondent Universe for the SCANR Survey. The SCANR Survey is designed to provide nationally representative information on the share of small SNAP-authorized retailers that lack scanning technologies and the barriers to adoption by store type and urbanicity. The respondent universe is small SNAP-authorized retailers defined as small grocery stores, medium grocery stores, convenience stores, and specialty stores. The respondent universe excludes stores that are part of a large chain (i.e., stores that have been previously identified by FNS as having 10 or more outlets under the same owner). Based on previous research (USDA, FNS, 1998) and recent informal contacts, we are assuming that stores that are part of a large chain are very likely to have scanning technology that is integrated with the EBT payment terminal in place.¹ Thus, we made the decision to not survey these stores and exclude them from the sampling frame.

We will use the FNS Store Tracking and Redemption Subsystem (STARS) database to create the sampling frame. The STARS database includes all SNAP-authorized retailers and is an operational system so it is always current. A preliminary STARS dataset was used to create a preliminary sampling frame to inform the sample design. About one month before the start of data collection, we will request a dataset from the STARS database to create the final sampling frame so that it is up-to-date. The starting sampling frame included 144,217 stores for the four store types of interest (small grocery stores, medium grocery stores, convenience stores, and specialty stores). A file with corporate IDs for retailers that were identified as large chains by FNS (SCANR Study Stores in C-Plans) was used to remove retailers from these chains from the frame (n = 18,814), yielding 125,403 small SNAP-authorized retailers in the preliminary sampling frame.

Sampling Method and Stratification for the SCANR Survey. We will select a nationally representative probability-based stratified sample of small SNAP-authorized retailers. As described below, we will stratify the sample by five store types (small grocery/medium grocery/convenience franchise/convenience nonfranchise/specialty) and urbanicity (rural or urban), creating a total of 10 mutually exclusive strata. Within each stratum, retailers will be selected with equal probability. Before selecting the stratified random sample, we will sort the frame by three implicit stratification variables (owner name, zip code, and sales revenue) to ensure the final sample includes a wide range of retailers across the nation ranging in size. We are implicitly stratifying based on owner name to help ensure that we do not randomly select a large number of stores owned by the same owner because some owners own more than one store. The sampling unit and analytic unit are the retail stores, and the respondents will be the retail store owner/manager or regional manager (or other knowledgeable individual) who will complete the survey.

Store type. The STARS dataset includes a code for store type based on the characteristics of the retailer. We assigned records to one of the four store types using the codes shown below:

Store Type Categories for Sampling	STARS Store Codes	STARS Classification
Small grocery	small grocery (SG) and nonprofit food-buying cooperative (BC)	SG = stores that stock a small selection of all four staple food categories. BC = stores that operate as a “cooperative.”
Medium grocery	medium grocery (MG)	Stores that stock a moderate selection of all four staple food categories.
Convenience store	convenience store (CS)	Self-service stores that offer a limited line of convenience items and are typically open long hours to provide easy access for customers.
Specialty store	specialty bakery/bread (BB), fruit/vegetables (FV), meat/poultry products (ME), and seafood products (SE)	Stores specializing in the sale of the specified product. May also carry non-food items or other food items, but such stock is incidental to the primary specialty food stock.

The convenience stores (CS) were further split into convenience franchise and convenience nonfranchise stores because the decision-making process may be different for the two types of convenience stores. Thus, in total, the store type stratification variable will have five levels. A convenience store was categorized as a convenience franchise store if the store name was listed more than 10 times in the sampling frame.²In the preliminary sampling frame, we identified 24,817 convenience franchise stores. The remaining convenience stores were categorized as convenience nonfranchise stores.

Urbanicity. The STARS dataset includes Rural-Urban Commuting Area (RUCA) Codes, which are a Census tract-based classification scheme that uses the standard Bureau of Census Urbanized Area and Urban Cluster definitions in combination with work commuting information to characterize all of the nation’s Census tracts regarding their rural and urban status and relationships (http://depts.washington.edu/uwruca/). The RUCA codes are as follow:

1 Metropolitan area core: primary flow within an Urbanized Area (UA)

1.0 No additional code

1.1 Secondary flow 30% through 49% to a larger UA

2 Metropolitan area high commuting: primary flow 30% or more to a UA

2.0 No additional code

2.1 Secondary flow 30% through 49% to a larger UA

3 Metropolitan area low commuting: primary flow 10% to 30% to a UA

3.0 No additional code

4 Micropolitan area core: primary flow within an Urban Cluster (UC) of 10,000 through 49,999 (large UC)

4.0 No additional code

4.1 Secondary flow 30% through 49% to a UA

4.2 Secondary flow 10% through 29% to a UA

5 Micropolitan high commuting: primary flow 30% or more to a large UC

5.0 No additional code

5.1 Secondary flow 30% through 49% to a UA

5.2 Secondary flow 10% through 29% to a UA

6 Micropolitan low commuting: primary flow 10% to 30% to a large UC

6.0 No additional code

6.1 Secondary flow 10% through 29% to a UA

7 Small town core: primary flow within an Urban Cluster of 2,500 through 9,999 (small UC)

7.0 No additional code

7.1 Secondary flow 30% through 49% to a UA

7.2 Secondary flow 30% through 49% to a large UC

7.3 Secondary flow 10% through 29% to a UA

7.4 Secondary flow 10% through 29% to a large UC

8 Small town high commuting: primary flow 30% or more to a small UC

8.0 No additional code

8.1 Secondary flow 30% through 49% to a UA

8.2 Secondary flow 30% through 49% to a large UC

8.3 Secondary flow 10% through 29% to a UA

8.4 Secondary flow 10% through 29% to a large UC

9 Small town low commuting: primary flow 10% through 29% to a small UC

9.0 No additional code

9.1 Secondary flow 10% through 29% to a UA

9.2 Secondary flow 10% through 29% to a large UC

10 Rural areas: primary flow to a tract outside a UA or UC (including self)

10.0 No additional code

10.1 Secondary flow 30% through 49% to a UA

10.2 Secondary flow 30% through 49% to a large UC

10.3 Secondary flow 30% through 49% to a small UC

10.4 Secondary flow 10% through 29% to a UA

10.5 Secondary flow 10% through 29% to a large UC

10.6 Secondary flow 10% through 29% to a small UC

Consistent with previous research, we classified stores as urban vs. rural as follows: urban (1.0, 1.1, 2.0, 2.1, 3.0, 4.1, 5.1, 7.1, 8.1, 10.1) and rural (4.0, 4.2, 5.0, 5.2, 6.0, 6.1, 7.0, 7.2, 7.3, 7.4, 8.0, 8.2, 8.3, 8.4, 9.0, 9.1, 9.2, 10.0, 10.2, 10.3, 10.4, 10.5, 10.6).

Sample Allocation, Selection, and Precision for the SCANR Survey. We will employ a multimode, two-phase sample that implements a nonresponse survey in the second phase of data collection. As shown in Table B1.1, we will select a sample of 1,377 small retailers and a reserve sample of 750 small retailers for a total sample size of 2,127 (excludes the individuals contacted for the pretest).³ We anticipate 15% of the selected retailers will be ineligible (e.g., no longer SNAP-authorized retailer, out of business), resulting in approximately 1,170 eligible retailers. We assumed a 50% response rate⁴ for Phase I, after adjusting for ineligibility, resulting in 585 survey completes. In Phase II, we will conduct a more intensive approach for all nonrespondents from Phase I (n = 585) and adjust the data collection procedures based on lessons learned from the Phase I data collection. Two weeks after the third survey mailing, interviewers will begin calling all non-responding retailers. Interviewers can complete the survey with retailers over the phone, or if preferred, they can assist retailers in completing the survey by mail or via the Web. Retailers who indicate that they will complete the survey via mail or Web on their own will be flagged for phone follow-up if a completed survey is not received 2 weeks after the initial call. Interviewers will attempt to identify and address obstacles preventing retailers from responding.

We assumed a 60% response rate in Phase II, resulting in 351 survey completes. When combining Phases I and II estimated completes (585 + 351 = 936) and applying ineligibility adjustments, we are targeting an 80% response rate (936/1,170 = 0.80), which will yield the goal of 936 completed surveys (excludes the pretest respondents). If during data collection we find we are not obtaining the number of completed surveys needed to meet the precision requirements or the ineligibility rate is higher than anticipated, we will release additional sample from the reserve sample.

Table B1.1. Estimated eligibility rates, response rates, and respondents for each phase of the SCANR Survey

Starting sample size	1,377
Reserve sample	750
Phase I estimated eligibility rate	85%
Phase I estimated number of eligible retailers in sample	1,170
Phase I estimated response rate	50%
Phase I eligible respondents	585
Remaining nonrespondents for Phase II	585
Phase II estimated response rate	60%
Phase II respondents (eligible and ineligible)	351
Phase II estimated eligibility rate	100%
Required sample size (Phase I and II eligible respondents)	936
Overall response rate	80%

Table B1.2 shows the estimated total counts for the target population,⁵ number of desired completes, number to sample in order to obtain the desired number of completes, and the number of reserve stores to be selected. The sample of 1,377 SNAP-authorized stores (excludes the individuals contacted for the pretest) will be evenly split across the five store types, resulting in 275 sampled SNAP-authorized stores for each store type.⁶ Within each store type, the 275 will be allocated to either rural or urban stores with two-thirds allocated to urban (184 selected stores) and one-third allocated to rural (91 selected stores). In total, 922 will be allocated to urban stores and 455 will be allocated to rural stores. This allocation will be revisited once the final sampling frame is created and we can reassess the final counts for the target population. Given the ineligibility and response rate assumptions discussed above, a sample of 275 for each subgroup level for store type is sufficient to obtain the 187 completes (275 * 85% eligibility rate * 80% overall response rate = 187) needed to meet the desired precision requirements at the subgroup level (margin of error equal to or less than +/− 0.08 percentage points for proportion estimates around 0.5 with 95% confidence and design effects⁷ equal to 1.25). We reviewed previous research conducted with retailers and use of scanning systems and information was not available on design effects (USDA, FNS, 1998). We did not explicitly include effects for multiple stores with the same owner or under the same corporate umbrella; however, implicit stratification by owner name will help minimize the effects of any clustering. Furthermore, it is possible that stores owned by the same owner may operate differently if the owner is not part of a franchise

Table B1.2. Sample allocation for the SCANR Survey with stratification by urbanicity and store type

		Convenience				Grocery				Specialty		National
		Franchise		Nonfranchise		Medium		Small
		Urban	Rural	Urban	Rural	Urban	Rural	Urban	Rural	Urban	Rural
Numbers and estimates from preliminary frame	Number on frame	22,467	2,338	61,954	11,262	7,271	1,175	11,282	1,050	5,483	837	125,119
	Eligibility rate	85%	85%	85%	85%	85%	85%	85%	85%	85%	85%	85%
	Estimated response rate	80%	80%	80%	80%	80%	80%	80%	80%	80%	80%	80%
Sampling allocationa	Number of completes	125	62	126	62	125	62	125	62	125	62	936
	Number sampled	184	91	186	91	184	91	184	91	184	91	1,377
	Reserves	100	50	100	50	100	50	100	50	100	50	750
	Number of completes by store type	187		187		187		187		187		935
	Margin of error (+/− percentage points)	0.080		0.080		0.080		0.080		0.080		0.045
	Number of completes by urban/rural	623	312
	Margin of error (+/− percentage points)	0.055	0.077

^a Equal allocation for store type and over­sampling rural stores (allocate two-thirds of sample to urban and one-third to rural within store type).

agreement that requires a certain type of register system. The design effect of 1.25 is somewhat conservative allowing for some additional design effects for the convenience/nonfranchise, medium grocery, and specialty strata.

For rural estimates, we expect to obtain 312 completes resulting in estimates with a margin of error equal to or less than +/− 0.077 percentage points for proportion estimates around 0.5 with 95% confidence and design effects equal to 2.44. Similarly, for urban estimates, we expect to obtain 623 completes resulting in estimates with a margin of error equal to or less than +/− 0.055 percentage points for proportion estimates around 0.5 with 95% confidence and design effects equal to 1.94.

A total sample size of 1,377 is sufficient to obtain the 936 completes (1,377 * 85% eligibility rate * 80% overall response rate = 936) (excludes the individuals contacted for the pretest and respondents to the pretest) needed to meet the national-level desired precision requirements (margin of error equal to or less than +/− 0.045 percentage points for proportion estimates around 0.5 with 95% confidence and design effects equal to 2.3).

Assumptions for the sample size determination included the above-mentioned design effects. We estimated the design effects for store type, urbanicity, and nationally based on estimates of nonresponse-adjusted sampling weights. We also assumed a 95% confidence level and a proportional estimate of 0.5 (most conservative level) for a particular survey measure when calculating the estimated precision for each estimate.

As previously discussed, we plan to select the sample using a systematic stratified random design. We will sort the sampling frame before sample selection by owner name, zip code, and sales revenue to ensure the final sample includes a wide range of retailers across the nation ranging in size within the group of small chain and independent retailers. The reserve sample and the main sample will be selected at the same time. After the stores are selected we will randomly assign stores to either the reserve sample or the main sample based on the allocations indicated in Table B1.2. The reserve sample will be randomly ordered with each stratum and released in order, as needed. If a store is found to be ineligible in a given stratum a reserve from that specific stratum will be released to replace the ineligible store. Additionally, if the number of completes for a given stratum is less than estimated and we are at risk of not meeting the number of completes needed to obtain the desired precision we will release additional sample from the reserve sample for that specific stratum.

Response Rates and Nonresponse Bias Analysis for the SCANR Survey. For calculating response rates, the numerator is the number of respondents and the denominator is the number of eligible retailers (excludes any retailers that are no longer in operation since selection of the sample and retailers that are no longer SNAP-authorized). If a response rate of 80% is not achieved, we will perform a nonresponse bias analysis as required by OMB. Nonresponse bias is the expected difference between an estimate from the survey respondents and an estimate from the target population. The extent to which nonresponse bias occurs depends on 1) the extent of missing data (both unit and item nonresponse) and 2) the difference in an estimate between respondents and nonrespondents. We will conduct two analyses to examine the potential for nonresponse bias. First, we will compare certain sampling unit characteristics that would be known for both respondents and nonrespondents (e.g., FNS region, store type, and urbanicity). We will calculate the distributions of various sampling characteristics for the respondents and nonrespondents. Next, we will calculate the estimated bias for each characteristic and determine whether they are significant at the 5% level. The estimated bias is the difference between the weighted mean of the respondents and the weighted mean of the selected sample.⁸

After the final analysis weights (adjusted for nonresponse) are calculated, we will recalculate the estimated bias to determine if the nonresponse weight adjustment in the analysis weights reduced the estimated bias. Second, we will calculate the percentage missing for all survey responses. If the percentage missing exceeds 20% (excluding legitimate skips), we will conduct an item nonresponse bias analysis for up to 10 key survey questions and test if the estimated biases are significant at the 5% level.

Respondent Universe and Sampling Methods for the Industry Interviews. The respondent universe for the industry interviews is vendors that sell and install scanning hardware and software that are compatible with EBT systems (e.g., DUMAC Business Systems, Retail Data Systems, Total Retail Solutions). We will develop a candidate list of vendor representatives based on established industry contacts and conduct interviews with nine individuals. These interviews are a convenience sample and are not meant to be a representative sample of vendors. A convenience sample targeting the largest vendors is sufficient because nationally representative data on technology costs is not needed for the cost analysis. The target respondent will be a representative in the sales department who is knowledgeable of prices and technology requirements.

Respondent Universe and Sampling Methods for the Follow-Up Interviews with Retailers. We will select participants for the follow-up interviews from the respondents to the SCANR Survey. Assuming a response rate of 70%,⁹ we will select 71 retailers to complete 50 interviews. We will purposively select a mix of retailers meeting the following criteria:

• Fifty percent of the retailers will be users of scanning technologies that meet the Farm Bill requirement, and 50% of the retailers will be nonusers of scanning technologies that meet the Farm Bill requirement.

• Among nonusers, we will select six to eight that indicated on the survey that they may discontinue as a SNAP retailer if required to have scanning technologies.

• The retailers will include a mix of store types, with approximately 25% of the retailers from each of the four store types (small grocery stores, medium grocery stores, convenience stores, specialty stores).

• The retailers will include a mix of urbanicity, with approximate equal allocation across the levels of urbanicity used for stratification for the SCANR Survey.

• The retailers will include a mix of geographic locations; however, for the users of scanning systems, we will cluster the interviews in five geographic areas because these interviews will be conducted in person (e.g., three urban areas such as Minneapolis, Los Angeles, and Dallas) and two rural areas (e.g., rural areas in Oklahoma, Maine, or Washington State). Within each cluster, combinations of retailers in urban and suburban and/or rural locations will be selected to achieve diversity in geographic type (e.g., within the same geographic area, select retailers in urban and suburban cities or rural and isolated communities).

B.2 Procedures for the Collection of Information

Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the purpose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

The required information is provided for the SCANR Survey. Additional information regarding the items listed above is not relevant for the industry interviews and the follow-up interviews with retailers.

Statistical Methodology for Stratification and Sample Selection for the SCANR Survey. The survey design for the SCANR Survey is a systematic stratified random design. First, we will stratify the sampling frame by store type and urbanicity creating 10 mutually exclusive strata. Next, we will sort the frame by owner name, zip code, and sales revenue to select a systematic random sample of small retailers. Within each stratum, the retailers are selected with equal probability. Additional information on the sample design and sampling approach was provided in Section B.1.

Estimation Procedures and Creating of Sample Weights for the SCANR Survey. We plan to use standard design-based methods for estimation and variance estimation that will lead to confidence intervals on means and percentages. The final analysis weights will reflect the sample design and any nonresponse, which will allow for nationally representative estimates as well as subgroup-level estimates representative of the subgroups of interest. We will use these weights to conduct all statistical analysis.

All sampled retailers will have a sample design base weight reflecting each retailer’s probability of inclusion in the survey. An ineligibility adjustment will be applied to the weights at the end of data collection to account for cases that are deemed to be ineligible (i.e., out of business, no longer participating in SNAP) weights will be the product of the sample design weights, ineligibility adjustment, and nonresponse adjustment. We will use the WTADJUST procedure within SUDAAN 11 to make the nonresponse and post-stratification adjustments (RTI International, 2012). The nonresponse bias analysis (described in Section B.1) will also inform whether any adjustments we may need to make in the final analysis weights.

Degree of Accuracy Needed for the Purpose Described in the Justification for the SCANR Survey. At the national level, assuming a design effect of 2.3 and an 80% response rate, the sample size of 1,377 will yield estimates with a margin of error of ±0.045 percentage points at a 95% level of confidence for proportion estimates around 0.5 (at 0.80 power). For the subgroup analysis of store type (i.e., analysis by the five store types), assuming a design effect of 1.25 and an 80% response rate, the strata sample sizes shown in Table B1.1 will yield estimates with a margin of error of ±0.08 percentage points at a 95% level of confidence for proportion estimates around 0.5 (at 0.80 power). For rural estimates, assuming a design effect of 2.44 and urban estimates, assuming a design effect of 1.94, the allocated sample sizes will yield estimates with a margin of error of 0.077 percentage points and 0.055 percentage points, respectively.

B.3 Methods to Maximize Response Rates and to Deal with Non-response

Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

Our procedures for ensuring high response rates for the SCANR Survey are summarized below.

• Employ a 6-month, two-phase design that uses mail, Web, and computer-assisted telephone interviewing (CATI) to gather data, optimizing the mode and timing of contacts to minimize survey error and maximize response.

• Simultaneously offer a paper questionnaire (Appendix D.6) and an invitation to complete the survey via the Web or mobile device using a unique access code (Appendix D.7).

• Provide a toll-free number and email address for respondents to contact to verify the study’s legitimacy or to ask questions.

• For the initial mailing, send via first class mail a packet to all study participants that will include a cover letter (Appendix D.1), a FAQ document (Appendix D.2), the questionnaire with an invitation to complete the survey via the Web or mobile device using a unique access code, and a postage-paid return envelope.

• Six weeks after mailing the initial survey packet, send all nonrespondents via first class mail a second paper questionnaire with an invitation to complete the survey via the Web (Appendix D.3).

• Twelve weeks after mailing the initial survey packet, send all nonrespondents via Federal Express (or U.S. Priority Mail for retailers with a PO box) a third copy of the paper questionnaire with an invitation to complete the survey via the Web (Appendix D.4). Use of FedEx aims to draw the retailer’s attention to the importance of the study.

• Two weeks after the third survey mailing, call nonrespondents and try to complete the survey over the phone using CATI (Appendixes D.5 and D.8). We will make a minimum of 10 call attempts to each working phone number.

• Use telephone call scheduling procedures designed to call numbers at different times of the day (between 8 am and 9 pm in respondent’s time zone) and week (Sunday through Saturday) to improve the chances of finding respondents at home.

• Leave a generic message on voice mail on the participant’s telephone to let him/her know to call back to complete a telephone interview.

• Implement refusal conversion efforts by trained telephone interviewers.

During data collection, we will monitor outcomes based on paradata (e.g., survey timing data and number of call attempts) and sampling frame information throughout data collection and adjust the survey approach as necessary. For example, if we anticipate a particular subgroup will not reach its targeted number of completes, we can make additional call attempts for those cases, and the cases will be reviewed closely by project management. Trained refusal converters will be used to contact retailers who have previously provided “mild” or “soft” refusals. Additionally, supervisors will closely monitor the number of call attempts for all cases and will prioritize cases, such as those who began the Web survey but did not complete it.

With this approach, the estimated response rate is 50% for Phase I and 60% for Phase II, for an overall targeted response rate of 80%. Table B3.1 shows the estimated response rates by phase and mode.

Table B3.1. Estimated response rate by phase and mode

Overall Starting Sample Size	Phase I Estimated Eligibility Ratea	Phase I Estimated Number of Eligible in Sample
1,377	85%	1,170
Data Collection Stage	Starting Eligible Sample Size	Anticipated Response/Target
Phase I Initial mailing (USPS mail with Web invitation)	1,170	10% mail (n = 117) 15% Web (n = 176)
Second mailing (USPS mail with Web invitation)	878	13% mail (n = 117) 20% Web (n = 176)
Phase I estimated response rate		50% (n = 585)
Phase II Third mailing (FedEx with Web invitation)	585	5% mail (n = 29)
Telephone prompting/follow-up	556	41% CATI (n = 225) 17% Web (n = 97)
Phase II estimated eligibility rate		100%
Phase II estimated response rate		60% (n = 351)
Overall estimated response rate		80% (n = 936)

^a Reserve retailers will be substituted for ineligible retailers as needed during data collection.

B.4 Test of Procedures or Methods to be Undertaken

Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

The instruments for this study were pretested in February and March 2017 with nine individuals (see Table B4.1). For the SCANR Survey, three respondents from the target population were asked to complete the survey instrument prior to the interview, and the interviewer asked debriefing questions to assess whether the questions’ intent was clear, the terminology well defined, and the responses unambiguous. The pretests also provided estimates of participant burden, which was found to be about 15 minutes. We experienced some difficulties with recruiting and follow-through by the respondents which are detailed in Appendix F. In addition, we made informal contacts with one industry expert and five trade associations to obtain their feedback on the draft survey instrument. Most of the changes to the SCANR Survey dealt with simplifying the wording of the question or revising the terminology so that it was more familiar to retailers. No pretest participants found the survey difficult to complete.

For the one industry interview and the five follow-up interviews with retailers, we conducted the interview and then asked debriefing questions to assess understanding of the questions. We found there were no concerns with the flow, timing, and information obtained during the industry interview pretest. We made some changes to the interview guide to reflect that some retailers may make modular upgrades to meet the Farm Bill requirement instead of installing a completely new point-of-sale system. The interview took one hour to complete which is reflected in the burden estimates.

Based on our experience conducting the pretests for the follow-up interviews with retailers, we determined that recruiting stores is likely to be challenging and that flexibility in the recruitment and interviewing approach will be necessary. We also found that structured interviews are not an effective way to collect data from this target group. Rather, conversational interviews with the owners based on the interview guides are likely to achieve the best results. We made some changes to the interview guide to reflect that some retailers may make modular upgrades to meet the Farm Bill requirement instead of installing a completely new point-of-sale system and to clarify some of the terminology. The pretests also provided estimates of participant burden, which was found to be about 30 minutes. The instruments were revised, as needed, based on the pretest findings. Additional information on the pretests, including a summary of changes made based on the pretest findings, is provided in Appendix F.

Table B4.1. Number of pretests by instrument

Instrument	Number
Industry interview guide	1
Questionnaire for SCANR Survey	3
Interview guide for follow-up interviews with retailers	5

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The contractor (RTI) and FNS staff consulted on statistical and other aspects of the design (see Table B5.1). The same staff will be responsible for collecting and analyzing the study’s data.

Table B5.1. Individuals consulted on data collection or analysis

Name	Affiliation	Phone Number	Email Address
Sheryl Cates	RTI International	919-541-6810	[email protected]
Kristen Capogrossi Giombi, PhD	RTI International	919-541-7330	[email protected]
Celia Eicheldinger, MS	RTI International	919-541-6222	[email protected]
Jenny Genser	USDA/FNS	703-305-2559	[email protected]
Barbara Murphy	USDA/FNS	703-305-2532	[email protected]
Marga Ortiz	USDA/FNS	703-305-2546	[email protected]
Josephine Akato	USDA/NASS	202-720-2641	[email protected]

References

Office of Management and Budget. (2006, September). Standards and guidelines for statistical surveys. Retrieved from http://www.whitehouse.gov/sites/default/files/omb/inforeg/statpolicy/standards_stat_surveys.pdf.

RTI International. (2012). SUDAAN user’s manual, release 11. Research Triangle Park, NC: RTI International.

U.S. Department of Agriculture, Food and Nutrition Service. (1998, August). Food stamp EBT systems and program-eligible vs. non-eligible food items. Report to Congress. Retrieved from http://www.fns.usda.gov/sites/default/files/finalscn.pdf.

1 According to the 1998 FNS study, “Trade publications indicate that about 93 percent of chain-based supermarkets use scanning systems.” This statement was confirmed in a recent informal contact with a representative from a scanning system vendor.

2 We matched based on name of store, including matching if the spelling was slightly off or the capitalization was different; for example, we included "7-ELEVEN" as the same as "7-11" and "Bp" as the same as "BP FOOD MART".

3 We will coordinate with Manhattan Strategy Group (MSG), the contractor for the FNS Study on Third Party Processors (TPP) that also involves surveying SNAP-authorized retailers so that no retailer is contacted to participate in both surveys. Any SNAP-authorized retailer selected to be in the SCANR Survey that is also in the TTP study will be replaced with a reserve retailer.

4 We will use the response rate formulas in OMB’s Standards and Guidelines for Statistical Surveys (2006) to calculate response rates at each phase of data collection and at the end of the study to determine the final response rate.

5 The preliminary sampling frame was used to obtain the estimated counts for the target population.

6 To keep the total number of sampled stores the same as originally proposed (1,377), it was necessary to allocate two extra stores to one of the five store types. Because urban convenience nonfranchise is the largest stratum, we allocated 277 stores to that stratum.

7 The design effect (deff) is used to evaluate efficiency of a sample design and is a measure of the precision gained or lost by using the more complex design instead of a simple random sample (deff = 1). The unequal weighting effects will be the main source of the deff.

8 We will use the sampling weights to calculate the estimated bias.

9 The reason for the 70% response rate for the follow-up interviews is that it may be more challenging to get individuals to cooperate in an in-person interview compared to the survey, or to have to take the time to participate in a 30-minute phone interview.