Part B - SNP QRS Revised Oct 2017

Part B - SNP QRS Revised Oct 2017.docx

Special Nutrition Programs Quick Response Surveys

OMB: 0584-0613

⚠️ Notice: This form may be outdated. More recent filings and information on OMB 0584-0613 can be found here:

Document [docx]

Download: docx | pdf

FNS Generic Clearance for Special nutrition programs quick response surveys

OMB# 0584-0613

REQUEST FOR OMB Clearance

Supporting Statement Part B

Prepared by:

Food and Nutrition Service (FNS)

U.S. Department of Agriculture

Office of Policy Support
3101 Park Center Drive
Alexandria, VA 22302
Project Officer: Ruth Morgan
Telephone: 703-457-6791
Email: [email protected]
October 4, 2017

B. SUPPORTING STATEMENT

1. Respondent Universe and Sampling Methods 2

2. Procedures for the Collection Of Information 8

3. Methods to Maximize the Response Rates and to Deal with Nonresponse 21

4. Test of Procedures or Methods to be Undertaken 24

5. Individuals Consulted on Statistical Aspects & Individuals Collecting and/or Analyzing Data 24

TABLES

Table 1. Potential Special Nutrition Programs Quick Response Survey Studies* 5

APPENDICES

A. Sections of Authorizing Statutes

B. Public Comment from 60-Day Notice
C. Estimated Burden for Data Collection and Nonresponse for Potential SNP QRS Studies

1. Respondent Universe and Sampling Methods

Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The sample frame data collection will target all State administrators of FNS SNPs for which FNS anticipates a data collection within the year and will provide the necessary information for conducting short, quick turnaround surveys, Surveys conducted under this proposed generic Information Collection Request (ICR), the Special Nutrition Programs (SNPs) Quick Response Surveys (QRS), will target State and local administrators of FNS’ SNPs. No data will be collected from program participants. Generally, the structure of the SNPs is hierarchical with Food and Nutrition Service (FNS) Regional Offices working with State Agencies (SA, State level) who provide administrative oversight and technical assistance to local agencies (LAs, local level) and organizations working directly with the sites (sub-local level) that provide nutritious foods to recipients and program participants. For example, State Child Nutrition Directors work with School Food Authorities (SFAs, usually a school district) to provide meals at schools. Similarly, SAs (often the State’s health department) work with local WIC agencies who then work with WIC sites to provide benefits to WIC participants. Each sample frame and QRS project will be specific to a particular level (i.e., respondent type) and seek to collect information to answer specific research questions about that program. Therefore, the particular universe, sample size, and type of sampling, if any, will depend on the parameters of each sample frame and QRS and will be included with our individual request to OMB to conduct a QRS.

The universe of State agencies for the sample frame data collection will include a census of those agencies, and the sample size will vary (from 50 to 105) with each program and the particular corresponding QRS project. As shown in Table 1 Potential Special Nutrition Programs Quick Response Survey Studies, the universe, potential sample sizes, and potential sample designs for any QRS vary by program. The sample sizes in Table 1 provide approximate upper bound illustrations—the actual size for any particular study will be determined once the specific research questions are known. Similarly, Table 1 includes the anticipated ranges of sample plans. The exact specifications for stratification and sampling methodology will also depend on the parameters of the particular studies. It is possible that two different QRS of the same program would have different strata in order to answer different questions. Moreover, FNS envisions that some studies will rely on sampling according to probability proportionate to size (PPS), employing various measures of size (MOS). While only SAs and LAs will be surveyed, the number of participants in a program at a particular level (students served by a SFA, children in Independent Child Care Center, etc.) may be appropriate MOSs for some studies, while the number of institutions (schools, Head Start Center providers, etc.) may be appropriate for others. Studies of the Commodity Supplemental Food Program (CSFP) may employ the quantity of food as a MOS, while studies of the Summer Food Service Program (SFSP) may employ a MOS that, in part, is based on the number of serving days. For the sample frame data collection, the potential respondent universe will always include the State program agencies that will provide contact information about the local agencies within their purview.

In Table 1, the universe of State agencies (for any sample frame data collection) will be no more than 105, while the largest universe for a survey of providers (for any QRS) is estimated to be approximately 125,000. The representative sample sizes for any QRS were calculated assuming a precision for nationally representative estimates of ± 0.05 percent on a mean with 95 percent confidence interval, while maintaining precision for subgroup comparisons of ± 0.05 percent with 90 percent confidence interval. Minimum detectable differences (MDDs), at 80 percent power with 95 percent confidence, will be specified for the tests of interest for each particular QRS study. To obtain the upper bound for the sample sizes, we assumed design effects as discussed below. The sample size for any sample frame data collection will always equal the number of State agencies for that program.

Expected response rates. When collecting data from SNP LAs using web surveys, FNS contractors have achieved at least an 80 percent response rate. Moreover, the QRS will be much shorter and more focused than usual, multi-year studies. Therefore, we anticipate that the response rates will be at least 80 percent for LAs. For SAs, FNS expects and strives to achieve 100 response rates to surveys and to the sample frames data collection. The overall response rate is expected to be 80.3%. More discussion of response rates is presented in Question 3.

Table 1. Potential Special Nutrition Programs Quick Response Survey Studies*

Program

Universe Size

Sample Designs

Sample Size

Sampling Considerations

Special Supplemental Nutrition Program for Women, Infants, and Children (WIC)

State Agencies

Census

Local Agencies

1,900

Stratified, PPS, QRS, systematic

1,594

Consideration for ITOs and EBT States in stratification; Stratification by caseloads, region, urban, rural.

Sites

10,000

2-stage, stratified, PPS, QRS

2,750

2-stage may be required to get Site contacts; MOS may be based on caseloads.

National School Lunch Program/School Breakfast Program (NSLP/SBP)

State Agencies

Census

Local Agencies

15,000

Stratified, PPS

2,500

Stratification by Region, urban/rural, poverty; MOS based on number of students or number of schools.

Schools

100,000

Stratified, PPS

2,500

Stratification by Region, urban/rural, poverty, school type; MOS based on number of students per school.

Summer Food Service Program (SFSP)

State Agencies

Census

2. Procedures for the Collection of Information

Describe the procedures for the collection of information including:

Statistical methodology for stratification and sample selection,
Estimation procedure,
Degree of accuracy needed for the purpose described in the justification,
Unusual problems requiring specialized sampling procedures, and
Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Statistical methodology for stratification and sample selection

Sample frames for the respondents (i.e., State agencies) will represent a census of all agencies; therefore, no sampling of these entities will take place. For this ICR, the FNS National Office will liaise with FNS Regional offices to collect and verify contact information for its State program offices. The States will provide contact information regarding the local agencies within each program in order to create accurate national sample frames for each program. The statistical methodology for stratification and sample selection will vary by program and each QRS. In general, FNS anticipates that most QRS of SAs will be a census. QRS studies of States will be narrowly focused, causing the heterogeneity across the States in operations, structure, and technology to reduce the desirability of sampling, especially since the universe sizes for SAs is relatively small. For surveys of local and sub-local agencies, the SNP QRS will rely on the sample frame information collected from each SAs for the LAs within each program in order to create accurate national sample frames for each program whenever possible. When not possible, multi-stage data collection designs will sample higher levels first and then lower levels based on information from the higher levels. All specific sampling designs for any QRS will be submitted to OMB for approval under the generic clearance of this ICR. Here we present a brief discussion of the main factors that will determine the sample designs along with estimates of sample sizes based on current information for any QRS. (The upper bound estimates are presented in Table 1.)

Special Supplemental Nutrition Program for Women, Infants, and Children (WIC)

At the State-level, the sampling frame is the SAs in all 50 States, the District of Columbia, American Samoa, Guam, the Commonwealth of the Northern Mariana Islands, Puerto Rico, the Virgin Islands, and 34 Indian Tribal Organizations (ITOs). Some QRS Task Orders will call for a census of SAs, while others may support sampling. For some research questions stratification of SAs into States and ITOs will be appropriate because the ITOs are relatively small compared to States.

Sample frames of Local WIC Agencies (LAs) will be constructed following OMB approval for a QRS of LAs. We expect approximately 1,900 LAs in the frame and two types of sample designs. First, a design that promotes representation within ITOs, Electronic Benefit Transfers (EBT) States, and LA size in terms of caseloads employs four strata— (1) within the U.S. Territories and ITOs, (2) within an EBT State, (3) LAs with caseloads greater than 10,000, and (4) the remaining (small) LAs. Then, the LAs in the first three strata are treated as certainty units, while the remaining are sampled PPS to reach the desired sample size. The MOS used for PPS sampling will be caseload. A sample of size of 1000 satisfies the precision requirements for QRS.

The second design stratifies LAs into six strata. The last stratum will contain the largest (in terms of caseload) and will be treated as certainty units. We estimate that 25-30 LAs will fall in this strata. The remaining will be (1) < 1000, (2) >= 1000 & < 2500, (3) >= 2500 & < 5000, and (4) >= 5,000 & < 10,000, and (5) > 10,000 and less than the size of certainty units. We propose systematic sampling within the strata where the stratum sampling rate is based on the sample size allocation for each stratum. This design allows for some “sorting” variables to promote representation of other subgroups; for example, urbanicity or FNS Region. A sample size of 1,594 will satisfy the QRS precision requirements.

The sample plan for WIC sites also depends on the availability of a national database of sites. If the sampling frame is successfully created, then stratified simple random sampling (SRS) or PPS will be appropriate, depending on the research questions. Relevant strata include FNS Region, ITO/Territory/State, and caseloads. With approximately 10,000 sites, a sample of 1,875 allocated over seven strata would satisfy the precision requirements of QRS, assuming design effects of approximately 1.3.

In some cases, FNS will need to use a two-stage approach, sampling LAs first and then requesting information on their sites. With this information, a small sample of sites from each LA would then be selected. If we use the 850 LAs described above and sampled two - four sites from each, we would have a sample of approximately 2,750, which would be sufficient to satisfy the precision requirements, assuming design effects of 1.4. To the extent that not all LAs have multiple sites, we will need to augment the size of the first stage in order to get enough sites for national representation. Other designs may rely on a selection of States/ITOs first and then LAs but with the sampling unit being the site, it is not clear that these approaches will offer lower burden.

National School Lunch Program/School Breakfast Program (NSLP/SBP)

At the SA level, the sampling frame is all SAs in the 50 States, DC, Puerto Rico, Guam, America Samoa, the Northern Mariana Islands, and the Virgin Islands. We assume that any QRS with NSLP/SBP State agencies as respondents will call for a census of all State agencies. At the local level, there are two possible sampling frames – SFAs and Local Education Agencies (LEAs, also called school districts). These frames are similar—the difference arises because SFAs may serve more than one LEA. The sampling frames will be constructed using administrative data from FNS and the FNS-742 (Verification Summary Report) file for SY 2014-2015 provided by FNS and the NCES Common Core of Data (CCD) LEA universe file (http://nces.ed.gov/ccd/pubagency.asp). Sampling will be done within strata using (PPS) procedures where student enrollment in the SFA or LEA is the MOS.

There are approximately 15,000 SFAs over public schools and 13,500 public school LEAs. Assuming we select SFAs or LEAs using a stratified PPS design, we would likely need at most (that is, making conservative assumptions about the outcome and design effects) a sample size of 2,500.

A QRS may be targeted at the school level. A national school-level frame will be constructed using the CCD school universe file (http://nces.ed.gov/ccd/pubschuniv.asp). Because we cannot survey schools within a SFA or LEA without district-level permission, we will construct the school sampling frames only when QRS requests are made that require school-level sampling and data collection. The research questions will drive the sampling. For example, focus on just elementary schools may indicate SRS because the elementary schools are relatively homogeneous in terms of size. Of course, the schools could be stratified first to cover, for example, FNS Regions or urban/rural locations. This approach will have a smaller design effect, compared to PPS, since the schools in the final sample would have similar weights. On the other hand, high schools may be more heterogeneous in terms of size, indicating PPS to promote the number of students represented by the sample. Conservatively, we estimate that sample of approximately 2,500 schools will be required for national estimates.

With the biennial release of the Private School Universe Survey and the fact that information on the SFAs over private schools is available on the FNS-742 file, it is possible to create a national frame for private schools for QRS. There are approximately 4,000 private SFAs and 31,000 private schools; hence, sample designs could be similar to those proposed above but with somewhat smaller sizes.

Summer Food Service Program (SFSP)

At the State-level, we will construct a frame of State SFSP agency contacts. The frame will include contact information for SAs in all 50 States, the District of Columbia, Puerto Rico, Guam, and the Virgin Islands. We assume that most QRS task orders that involve SAs will call for a census. Separate sampling frames of SFSP sponsors will be constructed using administrative data from SAs. We will compile a descriptive dataset (size, locations, etc.) of all SFSP sponsors that administer the SFSP, including public or nonprofit private schools, units of local, municipal, county, tribal or State government, private nonprofit organizations, public or private nonprofit camps, and/or public or private nonprofit universities or colleges. A likely sample design will involve using stratified PPS procedures where the MOS is based on the number meals served over a period of time. Alternatively, a study focused on administrative issues may use a MOS derived from the number of sites that fall under the sponsor. Recognizing that sponsors may also serve as sites, that they may support additional sites, and that the periods for serving meals can vary by the sites that are sponsored, the MOS may imply differential sampling for different periods over the summer. Of course, such a scheme assumes that information on opening and closing of sites will be available at the sponsor level. Assuming we select SFSP sponsors using a stratified PPS design, we would likely need (i.e., making conservative assumptions about the outcome and design effects) a sample of 1,875 units.

For a QRS of SFSP sites the sample design will depend on the availability of a national frame for the sites. Without prior site information, a stratified two-stage sampling approach will be used, wherein a nationally representative sample of SFSP sponsors will be selected first and then sites will be selected second within selected SFSP sponsors. The contact information for SFSP sites within SFSP sponsors that are selected in the first stage will be updated immediately after selection by contacting the SFSP sponsors. The first stage of sampling (of SFSP sponsors) will likely be done using stratified PPS procedures and the second stage will likely be done using either SRS or systematic sampling of a limited number of sites under each sponsor. Some sponsors may only have one site, which will be sampled with certainty if that sponsor is selected, while other sponsors may have several sites. The sites will be stratified under the sponsors by a combination of size and length of operation and then randomly (or systematically) selected within the strata. With approximately 4,800 sponsors and 42,500 sites, a sample size of 2,625 (allowing for design effects of 1.3) would satisfy the QRS precision requirements noted below. Hence, a nationally representative sample of approximately 500 sponsors with approximately four - six sites per sponsor would provide national representation of sites and allow for major subgroup comparisons.

Child and Adult Care Food Program (CACFP)

SAs include the 50 States, the District of Columbia (DC), Guam, the US Virgin Islands, and Puerto Rico. For a QRS driven by research questions at the State level, we anticipate that the QRS would seek a census of all SAs. Current estimates for Child Care Centers (CCCs) include 10,240 CCC sponsors, 40,961 sponsored CCCs, and 8,895 independent CCCs. For a QRS of CCC Sponsors, we anticipate that the sample design parameters would include, in addition to national estimates of the measures of interest, subgroup comparisons based on size (children or number of sub-local agencies), geographic region, and some classification along the urban/rural spectrum. A stratified design with seven strata, one for each FNS Region, and SRS within the strata would require a sample size of approximately 1,688. If the research questions require promoting the larger sponsors, the sampling could be done with PPS with MOS based on the number of sponsored providers or the number of children participating. To preserve representation of the sponsors, the probability can be based on the square root of the number of children. This design will introduce design effects of approximately 1.3, increasing the sample size to approximately 2,188.

For a QRS of providers, a design that allows for Regional subgroup comparisons and comparisons between independent and Sponsored CCCs, would rely on 14 strata—one for each region and, within each region, one for sponsored and one for independent CCCs. In this case, SRS within the strata requires a responding sample of approximately 1,815, while with PPS the responding sample size would be 2,375.

Information from 2013 on Head Start centers indicates that there were 1,613 Head Start center sponsors and 13,191 Head Start center providers. Using stratification by the seven FNS Regions, samples of size 1,219 and 2,188 for studies of Head Start center sponsors and providers, respectively.

In 2013, there were 855 Family Day Care Homes (FDCH) sponsors and 124,068 FDCH providers. For a national estimate of FDCH sponsors, the responding sample size would need to be 550, which would require 690 sample units, assuming an 80 percent response rate. The savings of sampling versus a census of these organizations may not be worth the cost in terms of lost information. Thus, we anticipate that a QRS of FDCH sponsors would be a census but allow for some analysis and adjustments for non-responses. On the other hand, a national estimate of FDCH providers would likely promote comparisons over FNS regions and tiers. Thus, with seven FNS regions and two tiers, there will be 14 strata. PPS sampling could be based on the number of participants in the FDCH, indicating a sample size of 2,438.

The National Study of Long Term Care Providers (NSLTCP)^¹ identified 5,254 Adult Day Care Centers (ADCCs) in 2012. Of the respondents in this study (4,800), approximately 47 percent served 1 to 25 people, 47 percent served 25 – 100 people and 5 percent served more than 100. Moreover, this study found an approximately even distribution over four regions of the U.S. Thus, this information suggests that a SRS may be sufficient to adequately represent this population. In such a design, the sample size would be 1,488.

The Emergency Food Assistance Program (TEFAP)

The sampling frame for TEFAP SAs will include contact information for SAs in all 50 States, the District of Columbia, Puerto Rico, Guam, the Northern Mariana Islands, and the US Virgin Islands. We assume that QRS task orders that involve surveys of TEFAP SAs to answer questions about TEFAP will call for a census of all States. The sampling frame for Eligible Recipient Agencies (ERAs) will be composed of the ERAs with direct agreements with SAs and be constructed by requesting that the SAs provide the current ERA contact information. ERAs may have agreements with other ERAs such as Emergency Food Organizations (EFOs; for example, food banks and pantries and soup kitchens) but we will treat studies of these secondary organizations as described below. We anticipate that the first QRS of ERAs will be a census to establish a database about their operations with States and EFOs.

EFOs are considered a type of ERA in TEFAP. An EFO is an ERA which provides nutrition assistance to relieve non-disaster situations of emergency food need and distress through the provision of food to individuals in need. Administrative data on EFOs is collected by SAs and/or overarching ERAs. The exact number of EFOs in the States is currently unknown; however, Feeding America works with approximately 58,000 food pantries and meal programs most of which likely participate in TEFAP.^²

Because there are very few existing studies of either ERAs or EFOs, there is some uncertainty about the characteristics that should be represented when designing a sample plan. We assume that QRS data will be needed for geographic representation (FNS Region), urban/rural locations, EFO size, and, perhaps, some characteristics such as prepared meals site versus household distribution site. Ideally, we will be able to enumerate all EFOs and create a national frame for samples driven by QRS task orders.^³ In this case, the likely approach will be stratification by FNS Region and measures of urban/rural location, followed by PPS sampling within the strata where the MOS would reflect the quantity of food distributed by the EFO. As noted above, we may also want to distinguish prepared meals settings. With design effects of 1.3, we would need a responding sample of 1,875. If we are not able to build a national frame of EFOs, a multi-stage approach will be developed. One possibility is to select States first, then ask the ERAs in those States for the contact/characteristics of their EFOs and then sample EFOs in a manner consistent with the research questions. A sample of 2,625 allocated over approximately 20 States will satisfy the precision requirements for QRS, assuming design effect of 1.3.

Food Distribution Program on Indian Reservations (FDPIR)

The SA sample frame will be the 105 ITOs and State contacts who administer the program. Most QRS task orders of FDPIR SAs will call for a census. Some QRS research questions may support a sample. ITOs vary in size (using either number of Tribes or number of participants), geographic location, cultural norms, and accessibility. Therefore, a QRS may target a specific subgroup. It would also be possible to sample SAs randomly but with different probability based on some of these characteristics. However, the relatively large sample sizes necessary to achieve the desired precision rates, might make this approach inefficient.

Fresh Fruit and Vegetable Program (FFVP)

At the State-level, the sampling frame of SAs is all 50 States, DC, Puerto Rico, Guam, America Samoa, the Northern Mariana Islands, and the Virgin Islands. The QRS of these SAs will be a census. At the local level, participation in National School Lunch Program (NSLP) is requisite for the implementation of FFVP; therefore, the sample designs for FFVP could be similar to those used for NSLP described above. However, the FFVP is only available in elementary schools and targets the poorest schools in terms of percentage of elementary students eligible for free and reduced-price meals. Thus, the sampling frame may be restricted to SFAs with schools participating in the FFVP. We anticipate the number of SFAs with schools participating in the FFVP will be approximately 5,000 by the time of a QRS; hence, with design effects equal to 1.3, the study would require a sample of 1,875 SFAs. A sampling frame for elementary schools (as opposed to SFAs) can be constructed from CCD school universe file (http://nces.ed.gov/ccd/pubschuniv.asp), facilitating QRS of schools. With approximately 66,000 public elementary schools in the US, we expect that approximately 15,000 will be participating in the FFVP. Thus, the likely sample size would be 2,250, assuming design effects of 1.3.

Commodity Supplemental Food Program (CSFP)

We anticipate that a QRS of SAs will be a census of the participating States and ITOs at the time of the survey. Currently, 51 CSFP State agencies are funded (48 States, DC, and two ITOs).The SAs provide foods and some funds to LAs for the distribution of CSFP foods to eligible participants. Current data indicate that there are approximately 4,000 local sites. Stratification by FNS Region and PPS sampling (for example, by quantity of food or number of sites served as the MOS) would indicate a sample size of approximately 1,838.

Expected Levels of Precision or Accuracy

The degree of accuracy needed for national estimates in quick-response surveys is ± 0.05 percent with 95 percent confidence interval, while the precision for subgroup comparisons is ± 0.05 percent with 90 percent confidence interval. SNP QRS requests to OMB will include minimum detectable differences (MDDs) (at 80 percent power with 95 percent confidence) for the tests of interest for each QRS study. Because the sample frames data collection will represent a census of State agencies, the point estimates computed (i.e., means, standard deviation, etc.) will always represent those values in the population.

Estimation procedure

There are no estimation procedures beyond those described above for the sample frames data collection. For quick-response surveys with samples, sampling weights reflecting the overall probabilities of selection and differential nonresponse rates will be attached to each data record providing usable data for estimation. The first step in the weighting process will be to assign a base weight to each sampled unit. The base weight is equal to the reciprocal of the probability of selecting the unit for the study, which will vary by sampling stratum, depending on the sample design. Next, the base weights will be adjusted for nonresponse within cells consisting of units that are expected to be homogeneous with respect to response propensity. To determine the appropriate adjustment cells, we will conduct a nonresponse bias analysis to identify characteristics of the units that are correlated with nonresponse. The potential set of predictors to be used to define the adjustment cells will depend on the program. Within these cells, a weighted response rate will be computed and applied to the SFA base weights to obtain the corresponding nonresponse-adjusted weights.

To properly reflect the complex features of the sample design, standard errors of the survey-based estimates will be calculated using jackknife replication. Under the jackknife replication approach, subsamples or "replicates" will be formed in a way that preserves the basic features of the full sample design. A set of weights (referred to as “replicate weights”) will then be constructed for each jackknife replicate. Using the full sample weights and the replicate weights, estimates of any survey statistic can be calculated for the full sample and for each of the jackknife replicates. The variability of the replicate estimates is used to obtain the variance of the survey statistic. The replicate weights can be imported into variance estimation software (e.g., SAS, STATA, SUDAAN, WESVAR) to calculate standard errors of the survey-based estimates. In addition to the replicate weights, stratum and unit codes will be provided in the data files to permit calculation of standard errors using Taylor series approximations if desired. Note that while replication and Taylor series methods often produce similar results, jackknife replication has some advantages in reflecting statistical adjustments used in weighting such as nonresponse and post-stratification (e.g., see Rust, K.F., and Rao, J.N.K., 1996. Variance estimation for complex surveys using replication techniques. Statistical Methods in Medical Research, 5: 283-310).

Unusual problems requiring specialized sampling procedures

FNS does not anticipate unusual problems requiring specialized sampling procedures.

Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

FNS intends to assemble the sample frames during the first year of information collection, and refresh (i.e., update) the information in the two subsequent years of data collection. Sample frames will only be assembled when there are forthcoming surveys of those entities during that year. For the quick-response surveys, FNS does not anticipate that every program/entity-type will be studied each year.

3. Methods to Maximize the Response Rates and to Deal with Nonresponse

Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Overall response rate projections were presented earlier. Surveys directed at SAs (sample frames and quick-response surveys) are expected to achieve 100 percent response rate, while surveys of LAs are expected to achieve 80 percent response rate. The expectations are based on prior studies with these organizations. SAs have working relationships with FNS and, when considering that all QRSs are relatively short, SAs will with high likelihood respond to the survey. In collecting data from LAs, previous studies conducted by FNS often achieve more than 80 percent (for example, the Special Nutrition Operations Study for SY 2013-14). Again, when considering the prior FNS data collection surveys took much longer than 20 minutes, there is high likelihood that the QRS response rates will achieve 80 percent. Maximizing the response rate for particular studies involves a set of procedures described below.

Procedures to be followed to maximize the number of sample members who complete the QRS:

The letters inviting the sampled units to participate will be very carefully developed to emphasize the importance of this study and how the information will help the FNS to better understand and address current policy issues related to the SNPs.
The current contact information will be used for all initial correspondence and be updated as needed throughout the data collection period to facilitate communication with the Contractor.
Designated FNS regional staff will serve as regional study liaisons and be kept closely informed of the project so that they will be able to answer questions from State and local participants.
Provide a toll-free number and study email address so that participants can receive assistance with the study.
Sampled units will have the option of completing web-based surveys as telephone surveys.
Email reminders will be sent to sample members who have not yet completed the survey.
Follow up by telephone reminders will be made to all sampled units who do not complete the survey within a specified period and urge them to complete the survey. At this point, the respondent may choose to complete the survey by telephone.

The following procedures will be used to maximize the completion rates for quick-response surveys that are administered by telephone:

Use a core of interviewers with experience working on telephone surveys, particularly interviewers who have proven their ability to obtain cooperation from a high proportion of sample members.
Conduct a telephone interviewer training session specific to this study.
Use call scheduling procedures that are designed to call numbers at different times of the day (between 8am and 6pm) and days of the week (Monday through Friday), to improve the chances of finding a respondent at work.
Provide a toll-free number and email help address for respondents to verify the study’s legitimacy or to ask other questions about the study.

If a study fails to achieve the expected response rate, the nonresponses will be analyzed as described in Question 2. Adjustments to the weights based on the nonresponse analyses is expected to ensure that the data generate reliable estimates.

4. Test of Procedures or Methods to be Undertaken

Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

All QRS instruments will be pre-tested with fewer than 10 respondents. The results will be reported when the particular study is submitted to OMB for approval. Pre-testing is not necessary for the sample frame data collection.

5. Individuals Consulted on Statistical Aspects & Individuals Collecting and/or Analyzing Data

Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The Contractor, 2M Research Services, and their Subcontractor, Mathematica Policy Research, will conduct this study.

Name	Affiliation	Telephone Number	e-mail
Jim Murdoch	2M	817-856-0863	[email protected]
Nicholas Beyler	Mathematica	202-250-3539	[email protected]
Mary Kay Fox	Mathematica	617-301-8993	[email protected]
Jeff Bailey	NASS/USDA	202-720-4008	[email protected]

1 U.S. Department of Health and Human Services, “Long-Term Care Services in the United States: 2013 Overview, Vital and Health Statistics, 3(7), December 2013.

2 http://www.feedingamerica.org/about-us/how-we-work/food-bank-network/

3 This would involve some combination of available data from States, ERAs, and, perhaps, the Feeding America database.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	Supporting Statement for OMB No
Author	USDA
File Modified	0000-00-00
File Created	2021-01-21

Part B - SNP QRS Revised Oct 2017