Part B_QRS II_Final 2.25.21

Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The potential respondent universe for this collection includes administrators of USDA Food and Nutrition Service (FNS) Special Nutrition Programs (SNPs) in State and local agencies. No data will be collected from program participants.

Generally, the structure of the SNPs is hierarchical: State Agencies (SAs) provide administrative oversight and technical assistance to local agencies (LAs). Local agencies typically distribute benefits through sites that provide nutritious foods to program participants. For example, State Child Nutrition directors work with school food authorities (SFAs), usually a school district, to provide meals at schools. Similarly, State Agencies (often the State’s health department) work with local WIC agencies who then work with WIC sites to provide benefits to WIC participants. For most SNPs, we refer to three potential types of respondents: the State Agency, the local agency, and the local site. The local agency and site may be either a government or business entity (in the previous submission, all local sites were included as government entities).

Two types of information collection projects are proposed under this generic clearance: 1) collection of local agency and site contact information to build sampling frames and 2) administration of quick response surveys (QRS).

Sampling frame data collection will target all State-level administrators of SNPs for which FNS anticipates a data collection within the year. This collection will provide the necessary information to create sample frames for conducting short, quick turnaround surveys. Including all State administrators is necessary in order to create accurate sampling frames reflecting all local agencies involved in a SNP. No sampling or weighting is required.

Each QRS will be specific to a particular respondent type or types and seek to collect information to answer specific research questions about the SNP under study. Therefore, the particular universe, sample size, and sampling techniques will depend on the SNP and the project’s research questions. Details on sample size and sampling methodology for any specific QRS will be included with each generic clearance memorandum submission.

Table 1 summarizes potential sampling techniques to be used for a QRS of each SNP respondent type and provides approximate upper bounds of the sample size. In Section B2, we provide more detail on the considerations for sample design in each program that lead to the estimated sample size. These estimations assumed a precision for nationally representative estimates of ± 0.05 percent on a mean with 95 percent confidence interval, while maintaining precision for subgroup comparisons of ± 0.05 percent with 90 percent confidence interval. To obtain the upper bound for the sample sizes, we assumed design effects as discussed below.

Table B1.1. Details of potential SNP QRS sampling plans

Type of Respondent	Estimated Universe Size	Potential Sampling Techniques	Sample Size	Sampling Considerations
Special Supplemental Nutrition Program for Women, Infants, and Children (WIC)
State WIC Agency	90	Census, stratification	90	Stratification by ITO/territory status
Local WIC Agency	1,900	Stratification, PPS, SRS, systematic	1,594	Stratification by ITO/territory status, caseload, Region, urbanicity
Local WIC Site	10,000	2-stage, stratification, PPS, SRS	2,750	Stratification by ITO/territory status, Region, caseload. 2-stage may be required to get site contacts. MOS based on caseloads.
National School Lunch Program/School Breakfast Program/Fresh Fruit and Vegetable Program (NSLP/SBP/FFVP)
State NSLP/SBP/FFVP Agency	56	Census	56
SFA	15,000	Stratification, PPS	2,500	Stratification by Region, urbanicity, FRPL eligibility. MOS based on number of students or number of schools.
School	100,000	Stratification, PPS	2,500	Stratification by Region, urbanicity, FRPL eligibility. MOS based on number of students or number of schools.
USDA Foods in Schools
USDA Foods in Schools State Agency	52	Census	52
USDA Foods in Schools Local Agency	15,000	Stratification, PPS	2,500	Stratification by Region, urbanicity, FRPL eligibility. MOS based on number of students or number of schools.
USDA Foods in Schools Provider	100,000	Stratification, PPS	2,500	Stratification by Region, urbanicity, FRPL eligibility. MOS based on number of students or number of schools.
Summer Food Service Program (SFSP)
SFSP State Agency	54	Census	54
SFSP Sponsor	4,800	Stratification, PPS, SSRS	1,876	Stratification by Region, urbanicity, type of entity. MOS based on meals or number of sites.
SFSP Site	42,500	2-stage, stratification, PPS, SRS, systematic	2,626	2-stage may be required to get site contacts. Stratification by meals served, length of operations.
Child and Adult Care Food Program (CACFP)
State CACFP Agency	54	Census	54
CACFP Child Care Center Sponsor	10,240	Stratified, PPS	2,188	Stratification by FNS Region. MOS based on number of enrolled children or number of sites.
CACFP Child Care Center	49,856	Stratified, SRS, PPS	2,375	Stratification by FNS Region, type of center. MOS based on number of enrolled children.
CACFP Head Start Sponsor	1,613	Stratified, PPS	1,219	Stratification by FNS Region. MOS based on number of enrolled children.
CACFP Head Start Center Provider	13,191	Stratified, PPS	2,188	Stratification by FNS Region. MOS based on number of enrolled children.
Family Day Care Home Sponsor	855	Census	855	A sampling strategy is feasible but likely will not appreciably reduce the cost of data collection.
Family Day Care Home Provider	124,068	Stratified, PPS	2,438	Stratification by Region and tier. MOS based on children enrolled.
Adult Day Care Center	5,254	Stratified, SRS	800	Stratification by State, FNS Region, sponsorship type.
The Emergency Food Assistance Program (TEFAP)
State TEFAP Agency	54	Census	54
Eligible Recipient Agency	300	Census	300	Limiting universe to ERAs with direct agreement with SAs
Emergency Feeding Organization	60,000	Multi-stage, stratification, PPS	2,625	2-stage likely required to get EFO contacts. MOS based on quantity of food or people served.
Commodity Supplemental Food Program (CSFP)
State CSFP Agency	57	Census	57
Local CSFP Agency	4,000	Stratification, PPS	1,838	Stratification by Region. MOS based on people served.
Food Distribution Program on Indian Reservations
State Agency	112	Census, stratification	112	Stratification by ITO status, size, Region.

ITO = Indian Tribal Organization; MOS = measure of size; PPS = probability proportionate to size; SRS = simple random sampling

Expected Response Rates

Surveys of local SNP administrators executed under the generic clearance for Special Nutrition Programs Quick Response Surveys (SNP QRS, OMB Number 0584-0613, expiration date 02/28/2021) had survey response rates between 77 and 86% (see Table B1.2). In other surveys of local administrators, FNS contractors have generally achieved at least an 80 percent response rate, and these QRS will be much shorter (no more than 20 questions or approximately 20 minutes) than those used in typical large FNS studies. Based on these experiences, we anticipate an 80 percent response rate for local agencies and sites for all SNP respondents.

Table B1.2 Response rates in previous SNP QRS

Study	SNP & Respondent Type	Response Rate
WIC Child Retention QRS:	WIC local sites	81.2%
Study of Food Safety Needs of Adult Day Care Centers in the CACFP	CACFP local sites	76.5%
USDA Foods Distribution and Fees QRS	USDA Foods State Agencies	100%
Child Nutrition Reporting Burden Analysis Study	NSLP/SBP/FFVP State Agencies	100%
Child Nutrition Reporting Burden Analysis Study	NSLP/SBP/FFVP local agencies (SFAs)	86.4%

Surveys of State Agencies under the current SNP QRS clearance had response rates of 100% (see Table B1.2). In general, FNS surveys of State Agency SNP administrators achieve 100% response rates or very close. FNS anticipates a 100% response rate for QRS of State Agencies and sampling frame data collection.

We anticipate an overall response rate of 80.29% across the entire collection (based on a 100% response rate among 529 State agency information requests and an 80% response rate among 35,670 local agency and site requests). To achieve these response rates we will build on prior FNS study experience, focusing on contacting those sampled, securing their participation in the study, and offering support and completion reminders (see Section B.3 below for more detail).

B2. Procedures for the Collection of Information

Describe the procedures for the collection of information including:

Statistical methodology for stratification and sample selection;
Estimation procedure;
Degree of accuracy needed for the purpose described in the justification;

Unusual problems requiring specialized sampling procedures; and
Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Information to build sampling frames for SNP local administrators will be collected from the relevant SA. For each SNP, all relevant SAs will be asked to provide contact information in order to create accurate national frames, so there will be no sampling of SAs. The FNS National Office will liaise with FNS Regional Offices to alert the relevant Agencies to any sampling frame collection request, and the study team will provide telephone and email support to SAs as requested. Based on previous experience, we expect that SAs will primarily provide the data in Excel files, but we will be prepared to work with SAs should it be more convenient for them to provide the data in a different format.

The specific procedures for the administration of QRS will depend upon the nature and context of each project. Before data collection activities are undertaken for any QRS, FNS will provide OMB with a generic clearance memorandum describing the study, sample design, data collection activities, burden estimates, and a copy of survey instruments. This submission will also include details of the statistical methodology for sample selection and any anticipated estimation procedures. It will also detail quality control practices to be used during data collection and analysis (for example, review of the survey before it enters the field, data checks programmed into the survey itself, and data cleaning strategies to remove errors and outliers).

Statistical Methodology for Stratification and Sample Selection

In general, FNS anticipates that most QRS of SAs will be administered as a census, and there will be no sample stratification or selection. QRS of SAs will be narrowly focused, causing the heterogeneity across the States in operations, structure, and technology to reduce the desirability of sampling, especially since the universe sizes for SAs is relatively small.

For surveys of local agencies and local sites, sample selection will rely on sampling frame information collected from SAs whenever possible. If contact information to create a national sampling frame of local sites is not available, multi-stage data collection designs will sample higher levels first (i.e. local agencies) and then lower levels (i.e., local sites) based on information from the higher levels. Across programs, we anticipate sampling by stratifying along variables such as urbanicity, FNS Region, jurisdiction (i.e., State or Territory), size, and eligibility for free and reduced-price meals. Within the strata, sampling may be done using sampling techniques including simple random sampling (SRS), systematic sampling or probability proportionate to size (PPS). For PPS sampling, the measure of size (MOS) will depend on the unit being sampled. For example, the number of participants in a program at the site level (e.g., students served by an SFA, daily meal count at an SFSP site) may be an appropriate MOS for some studies, while the number of institutions (e.g., schools, Head Start providers) may be appropriate for others. All specific sampling designs for any QRS will be submitted to OMB for approval as part of the individual generic information collection submissions.

Here we present a brief discussion of the main factors that may influence the sampling design for each SNP and respondent type, along with estimates of necessary sample sizes for a QRS.

Special Supplemental Nutrition Program for Women, Infants, and Children (WIC)

At the State level, the sampling frame is the State Agency in all 50 States, the District of Columbia (DC), American Samoa, Guam, the Commonwealth of the Northern Mariana Islands (CNMI), Puerto Rico, the US Virgin Islands, and 34 Indian Tribal Organizations (ITOs). A WIC SA QRS may call for a census of SAs or may require sampling (e.g., for some research questions stratification of SAs by status in a States or ITO may be appropriate).

We expect approximately 1,900 LAs in a national local WIC agency sampling frame. A potential sampling design might ensure representation across ITOs and LA caseload size by using three strata— (1) within a U.S. Territory or ITO, (2) with a caseload greater than 10,000, and (3) remaining (small) LAs. LAs in the first two strata would be treated as certainty units, while the remaining could be sampled using PPS (based on caseload) to reach the desired sample size. A sample of size of 1,000 would satisfy the precision requirements for a nationally-representative sample.

The second design stratifies LAs into six strata by size caseload. The last stratum will contain the largest LAs (approximately 25-30 LAs) and will be treated as certainty units. The remaining strata will be local agencies with (1) fewer than 1000 cases, (2) between 1000 and 2500 cases, (3) between 2500 and 5000 cases, (4) between 5,000 and 10,000 cases, and (5) more than 10,000 cases and fewer than the smallest certainty unit. This design would use systematic sampling within the strata, where the stratum sampling rate is based on the sample size allocation for each stratum. This design allows for some “sorting” variables to promote representation of other subgroups; for example, urbanicity or FNS Region. A sample size of 1,594 satisfies the precision requirements for a nationally-representative sample.

If a sampling frame for the universe of WIC sites is successfully created, then stratified SRS or PPS will be appropriate, depending on the research questions. Relevant strata include FNS Region, location in ITO, Territory, or State, and caseload. With approximately 10,000 sites, a sample of 1,875 allocated over seven strata (e.g., seven FNS regions) would satisfy the precision requirements for a nationally-representative sample, assuming design effects of approximately 1.3.

If no national sampling frame of WIC sites can be created, FNS will need to use a two-stage approach, sampling LAs first and then requesting information on their sites to use for sample selection. Sampling two to four sites from 850 LAs would give a sample of approximately 2,750, which would be sufficient to satisfy the precision requirements, assuming design effects of 1.4. To the extent that not all LAs have multiple sites, we would need to augment the size of the first stage in order to get enough sites for national representation. (Other designs could select SAs first and then LAs, but with the sampling unit being the site, it is not clear that these approaches would offer lower burden.)

National School Lunch Program/School Breakfast Program/Fresh Fruit and Vegetable Program (NSLP/SBP/FFVP)

At the SA level, the sampling frame for the NSLP, SBP and FFVP is SAs in the 50 States, DC, Puerto Rico, Guam, American Samoa, CNMI, and the US Virgin Islands. We assume that any QRS with these SAs as the respondent will call for a census.

For these three SNP, the local agency is typically the SFA and the site is the school. The SFA sampling frame will be constructed using administrative data from FNS (i.e., the FNS-742 Verification Summary Report, 7 CFR Part 245 - Determining Eligibility for Free & Reduced Price Meals and Free Milk in Schools, OMB Number 0584-0026, expiration date 7/31/2023). Sampling will likely be done within strata using PPS procedures, with student enrollment or number of schools as the MOS and potentially considering variables such as FNS region, urbanicity and free and reduced-price lunch (FRPL) eligibility. Given the universe of approximately 15,000 public school SFAs and making conservative assumptions about the outcome and design effects, we estimate a QRS would require at most a sample size of 2,500 SFAs for a nationally-representative sample.

For a QRS at the school level, a national school-level frame can be constructed using the National Center for Education Statistics Common Core of Data school universe file (available at http://nces.ed.gov/ccd/pubschuniv.asp). The research questions will drive the sampling design for school-level sampling. Schools may be stratified to cover, for example, FNS Regions, FRPL eligibility, or urbanicity. Conservatively, we estimate that a sample of approximately 2,500 schools will be required for nationally-representative estimates.

USDA Foods in Schools

At the SA level, the sampling frame for USDA Foods in Schools is SAs in DC, Puerto Rico, the US Virgin Islands, and all 50 States except Kansas (which does not participate in the USDA Foods Program). We assume that any QRS with these SAs as the respondent will call for a census.

The USDA Foods in Schools program is used by most SFAs and schools, so we assume a similar sampling design to that described above for the NSLP, SBP and FFVP.

Summer Food Service Program (SFSP)

At the State-level, the SFSP sampling frame includes SAs in all 50 States, DC, Puerto Rico, Guam, and the US Virgin Islands. We assume that surveys of SFSP SAs will call for a census.

For a QRS of SFSP sponsors, a sampling frame of SFSP sponsors will be constructed including details on size, location, and type of entity (e.g., public school, not-for-profit private school, government, private not-for-profit organization, camp). A likely sample design will involve using stratified PPS procedures on variables such as FNS Region, urbanicity and/or type of entity, in which the MOS is based on the number meals served. Alternatively, a study focused on administrative issues may use a MOS derived from the number of sites that the sponsor operates. Using a stratified PPS design and making conservative assumptions about the outcome and design effects, a sample of 1,875 units should satisfy precision requirements for a nationally-representative sample.

For a QRS of SFSP sites the sample design will depend on the availability of a national sampling frame for the sites. If such a frame can be created, stratified PPS or SRS can be used. Without prior site information, a stratified two-stage sampling approach would be necessary, wherein a nationally representative sample of SFSP sponsors would be selected first and then sites associated with these sponsors. The first stage of sampling would likely be done using stratified PPS procedures, and the second stage would likely be done using either stratified SRS or systematic sampling. Some sponsors may only have one site, which will be sampled with certainty if that sponsor is selected, while other sponsors may have several sites. The sites could be stratified by variables such as number of meals served and/or length of operation. Given a universe of approximately 4,800 sponsors and 42,500 sites, a sample size of 2,625 sites, (allowing for design effects of 1.3) would satisfy the precision requirements for a nationally representative sample. A sample of approximately 500 sponsors with approximately four to six sites per sponsor would provide national representation of sites and allow for major subgroup comparisons.

Child and Adult Care Food Program (CACFP)

CACFP SAs include the 50 States, DC, Guam, the US Virgin Islands, and Puerto Rico. For a QRS at the State level, we anticipate using a census of all SAs.

The design of any QRS of CACFP sponsors or providers (i.e., child care centers, Head Start providers, family day care homes, and adult day care centers) will depend on the specific research questions and program types.

In general, for sponsors of child care centers (CCCs), we anticipate that the sample design would consider size (i.e., enrolled children or number of providers) and FNS region. A potential design would use seven strata, one for each FNS Region, and PPS with MOS based on the number of providers or the number of children enrolled. To preserve representation of the sponsors, the probability could be based on the square root of the number of children. This design would introduce design effects of approximately 1.3, increasing the sample size to approximately 2,188.

For a QRS of CCCs at the provider level, a design that allows for subgroup comparisons by FNS Region and comparisons between independent and sponsored CCCs would require 14 strata—one for each region and, within each region, one for sponsored and one for independent CCCs. In this case, SRS within the strata would call for a sample of approximately 1,815, while with PPS the necessary sample size would be 2,375.

For Head Start sponsors and providers, we anticipate using stratification by the seven FNS Regions and PPS sampling based on number of children enrolled. The size of the samples will be 1,219 and 2,188 for studies of Head Start center sponsors and providers, respectively.

Assuming 855 family day care home (FDCH) sponsors, a national representative QRS would require 690 sampling units. The savings of sampling versus a census of these organizations may not be worth the cost in terms of lost information. Thus, we anticipate that a QRS of FDCH sponsors would be performed as a census, allowing for analysis and adjustments for non-response. A QRS of FDCH providers would likely call for comparisons over FNS regions and payment tiers. Thus, with seven FNS regions and two tiers, there would be 14 strata. PPS sampling could be based on the number of children enrolled in the FDCH, indicating a sample size of 2,438.

Under the current generic clearance, FNS surveyed adult day care centers regarding their food safety practices. This survey used a stratified systematic sampling design, accounting for State, FNS Region and sponsorship type, and sampled 800 adult day care centers.

The Emergency Food Assistance Program (TEFAP)

TEFAP SAs include SAs in all 50 States, DC, Puerto Rico, Guam, and the US Virgin Islands. We assume that surveys of TEFAP SAs will call for a census of all States.

The sampling frame for local TEFAP agencies, known as eligible recipient agencies (ERAs), will be composed of the ERAs with direct agreements with SAs. Because little information is currently available about ERAs, we anticipate a QRS would be a census in order to establish a database about their operations.

The local TEFAP site is known as an emergency feeding organization (EFO). The exact number of EFOs is not known. However, Feeding America works with approximately 60,000 food pantries and meal programs, most of which likely participate in TEFAP.^¹ A two-stage approach will likely be necessary for any QRS of EFOs given the lack of information available. One possibility is to select States first, then ask the ERAs in those States for the contact information and characteristics of their partner EFOs. We could then sample EFOs in a manner consistent with the research questions. MOS may include number of individuals served or quantity of food provided over a defined period. A sample of 2,625 allocated over approximately 20 States will satisfy the precision requirements for QRS, assuming design effects of 1.3.

Commodity Supplemental Food Program (CSFP)

We anticipate that a QRS of CSFP SAs will be a census of the participating States and ITOs, including 48 States, DC, Puerto Rico, and three ITOs. Current data indicate that there are approximately 4,000 local CSFP agencies. Stratification by FNS Region and PPS sampling (for example, by participants served as the MOS) would indicate a sample size of approximately 1,838.

Food Distribution Program on Indian Reservations (FDPIR)

The FDPIR SA sample frame will include the 112 ITOs and States who administer the program. Most QRS of FDPIR SAs will call for a census. Some QRS research questions may support a sample based on, for example, FNS Region, number of participants, or geographic location. It would also be possible to sample SAs randomly but with different probability based on some of these characteristics. However, the relatively large sample sizes necessary to achieve the desired precision rates might make this approach inefficient.

In FDPIR, the SA distributes food directly so there are no local agencies.

Estimation Procedure

Because the sampling frames data collection will represent a census of SAs, no estimation procedures are necessary for this element of information collection.

All specific estimation procedures for any QRS will be detailed in the submission to OMB for approval under the generic clearance of this ICR. In general, we anticipate that QRS using samples will include in their analysis sampling weights reflecting the overall probabilities of selection and differential nonresponse rates. The first step in the weighting process will be to assign a base weight to each sampled unit. The base weight is equal to the reciprocal of the probability of selecting the unit for the study, which will vary by sampling stratum, depending on the sample design. Next, the base weights will be adjusted for nonresponse within cells consisting of units that are expected to be homogeneous with respect to response propensity. To determine the appropriate adjustment cells, we will conduct a nonresponse bias analysis to identify characteristics of the units that are correlated with nonresponse. The potential set of predictors to be used to define the adjustment cells will depend on the SNP and respondent type. Within these cells, a weighted response rate will be computed and applied to the base weights to obtain the corresponding nonresponse-adjusted weights.

To properly reflect the complex features of the sample design, advanced techniques, such as jackknife replication, will be used to calculate the standard errors of the survey-based estimates. Under the jackknife replication approach, subsamples or "replicates" would be formed in a way that preserves the basic features of the full sample design. A set of weights (referred to as “replicate weights”) would then be constructed for each jackknife replicate. Using the full sample weights and the replicate weights, estimates of any survey statistic can be calculated for the full sample and for each of the jackknife replicates. The variability of the replicate estimates is used to obtain the variance of the survey statistic. The replicate weights can be imported into variance estimation software (e.g., SAS, STATA) to calculate standard errors of the survey-based estimates. In addition to the replicate weights, stratum and unit codes can be provided in the data files to permit calculation of standard errors using Taylor series approximations. (Note that while replication and Taylor series methods often produce similar results, jackknife replication has some advantages in reflecting statistical adjustments used in weighting such as nonresponse and post-stratification.²)

Degree of Accuracy Required

Because data will be collected from a census of SAs, this question is not relevant for the collection of sampling frame information.

For QRS using a sample, the degree of accuracy needed for national estimates is ± 0.05 percent with 95 percent confidence interval, while the precision for subgroup comparisons is ± 0.05 percent with 90 percent confidence interval. SNP QRS requests to OMB will include minimum detectable differences at 80 percent power with 95 percent confidence for the tests of interest for each QRS study.

Unusual Problems Requiring Specialized Sampling Procedures

FNS does not anticipate unusual problems requiring specialized sampling procedures.

Use of Periodic Data Collection Cycles to Reduce Burden

Sampling frames will only be assembled when there are forthcoming surveys of those entities during that year. Any SNP respondent type will not receive more than one survey per year. Data collection procedures for any single QRS will only be conducted once.

B3. Methods to Maximize the Response Rates and to Deal with Nonresponse

Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Surveys of SAs and requests for sampling frame contact information are expected to achieve a 100 percent response rate, while surveys of local agencies and sites are expected to achieve an 80 percent response rate. The approach to achieving high response rates builds on prior FNS study experience and depends on contacting those sampled, securing their participation in the study, and offering support and completion reminders.

While the specific data collection procedures will depend on the individual project, for any information collection the team will develop engaging recruiting materials to describe the study, including a variety of appeals to encourage participation. The letters inviting the sample member to participate will be carefully developed to emphasize the importance of this study and how the information will help FNS to better understand and address current policy issues related to the SNPs. Current contact information will be used for initial correspondence and will be updated as needed throughout the data collection period to facilitate communication with the respondents.

FNS Regional Offices may be involved to introduce a sampling frame request or QRS to SAs and build support; similarly, SAs may be involved in reaching out to local-level respondents regarding QRS. Designated FNS regional staff will serve as regional study liaisons and be kept closely informed of the project so that they will be able to answer questions from State and local participants. The study team may copy the Regional Offices on communications with the SAs to promote participation and response.

To encourage participation in a QRS and discourage unit nonresponse, the study team will make sure sample members are aware of what will be asked of them on the survey prior to beginning it. This will help ensure that, if needed, the respondent can gather necessary information before beginning the survey and/or request assistance from a colleague who might have the appropriate knowledge. A toll-free number and study email address will be provided to all participants to receive assistance. Staff will be readily available to clarify survey questions and work with participants to resolve technical issues. Personalized assistance bolsters the perceived legitimacy of the study and will encourage respondents to persist in completing the survey.

The web survey will allow respondents to save and exit the survey at any point, and then return to access and complete the survey later. We anticipate that the short nature of the surveys (no more than 20 questions or approximately 20 minutes to complete) should facilitate completion of the full survey. While the type of information requested will depend on the specific QRS, we anticipate that much of what respondents will be asked to provide will be readily available to them without a great deal of effort needed to find or recall (i.e., basic information or their individual perspective on program operations).

Sampled participants will receive phone and/or email follow-up reminders, including those who have begun but not completed a survey. Follow-up by telephone reminders will be made to all sampled units who do not complete the survey within a specified period. At this point, the respondent may choose to complete the survey by telephone. For State Agencies, continued non-response may call for additional follow-up communications from FNS Regional and/or National Office staff indicating the importance of SA participation.

B4. Test of Procedures or Methods to be Undertaken

Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

All QRS instruments will be pre-tested with fewer than 10 respondents. The specific procedures used to recruit pre-test participants and perform pre-tests will depend on the particular study. In general, pre-test candidates will be identified and recruited via communication with FNS Regional Office staff. Ideally, pre-test participants will reflect the potential respondents along key characteristics such as job title, duties, and years of experience. Pre-testing will generally explore the clarity of the survey instruments to identify any problems with the instrument and suggestions for improvement. It will also inform understanding of the level of burden. The results will be reported when the particular study is submitted to OMB for approval and will be used to update the study instruments accordingly

Pre-testing is not necessary for sampling frame data collection.

B5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Table B5.1 lists individuals consulted on statistical aspects of the design. The staff who will collect or analyze the data will vary by project, depending on the contractor who wins an award to perform a QRS or create a sampling frame.

Table B5.1. Individuals consulted on statistical aspects of study design

Reviewer	Affiliation	Phone	Email
Conor McGovern	FNS/USDA	703-457-7740	[email protected]
Kelley Scanlon	FNS/USDA	571-289-2403	[email protected]
Jennifer Rhorer	NASS/USDA	202-720-3026	[email protected]

1 http://www.feedingamerica.org/about-us/how-we-work/food-bank-network/

2 See Rust, K.F., and Rao, J.N.K., 1996. Variance estimation for complex surveys using replication techniques. Statistical Methods in Medical Research, 5: 283-310.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	Mathematica Standard Report Template
Author	MATHEMATICA
File Modified	0000-00-00
File Created	2021-02-27