SUPPORTING STATEMENT - PART B for
OMB Control Number 0584-NEW:
Generic Clearance for Stakeholder Feedback and Surveys as Part of FNS Planning: Regulatory Actions, Semi-annual Regulatory Agenda, Program Changes, Research Studies, Outreach, Training and/or Development of Guidance
Maureen Lydon, Chief of Planning and Regulatory Affairs
Jamia Franklin, Information Collection Clearance Officer
Planning and Regulatory Affairs Office
Office of Policy Support
Food and Nutrition Service
United States Department of Agriculture
1320 Braddock Place
Alexandria, Virginia 22314
Email: [email protected] and [email protected]
TABLE OF CONTENTS
B1. Respondent Universe and Sampling Methods 2
B2. Procedures for the Collection of Information 6
B3. Methods to Maximize the Response Rates and to Deal with Nonresponse 8
B4. Test of Procedures or Methods to be Undertaken 10
B5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 12
TABLES
Table B1. Summary of Respondent Universe and Expected Response Rates 4
Table B2. Individuals consulted on statistical aspects of study design 12
Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.
Surveys will likely be conducted throughout the life of this generic information collection for the following Food and Nutrition Service (FNS) programs or a subset thereof: the National School Lunch Program (NSLP), School Breakfast Program (SBP), Fresh Fruit and Vegetable Program (FFVP), Summer Food Service Program (SFSP), Summer Electronic Benefits Transfer (EBT), Child and Adult Care Food Program (CACFP), Food Distribution Program on Indian Reservations (FDPIR), Commodity Supplemental Food Program (CSFP), and The Emergency Food Assistance Program (TEFAP), other child nutrition and food distribution activities, the Farm to School program, the Supplemental Nutrition Assistance Program (SNAP), the Special Supplemental Nutrition Program for Women, Infants and Children (WIC), the WIC Farmers Market Nutrition Program (FMNP), the Senior Farmers Market Nutrition Program (SFMNP), Child Nutrition (CN) program, nutrition and policy promotion, and any associated challenges in implementing programs or subsets of programs. The potential respondent universe would vary dependent on the specific program and would be discussed in the memo to be submitted to OIRA on a case-by-case basis for each information collection under this generic information collection request (ICR).
Information gathered could potentially yield qualitative and/or quantitative information and, when appropriate, may be shared outside the Agency (along with the context and parameters for the subject information collection and excluding personally identifiable information).
The potential surveys to be implemented by FNS could be used as part of discussion groups, focus groups, stakeholder meetings, interviews and surveys that involve other than routine customer feedback and may involve use of standardized questions such as focus group discussion guides. These focus groups, meetings, other stakeholder interactions, and surveys would involve stakeholders, program participants, and state, local and/or Tribal agencies who would potentially provide valuable input necessary for planning: a) research studies, including program evaluations such as impact or process evaluations; b) regulatory actions; c) program changes; d) outreach, promotion activities and/or training for stakeholders and program participants; e) guidance, technical assistance documents, questions and answers, to be issued by FNS; and/or f) other necessary program or policy analyses. The surveys would include both regular surveys and pulse-type surveys. The pulse-type surveys differ in that they would provide a real-time and reoccurring snapshot on key issues and give FNS an immediate stream of data to understand current program needs.
As referenced above, the exact surveys to be used on a case-by-case basis will be identified in the memorandum to be submitted to OIRA for each usage of this generic information collection. For now, FNS has inserted in each section an example of pulse surveys that could be carried out under this generic information collection for any of the FNS programs. The Child Nutrition Program is used as one example with regard to pulse surveys.
Pulse Surveys for Child Nutrition Program
The respondent universe for a Child Nutrition (CN) information collection could include the 67 State Agency (SA) Child Nutrition (CN) directors that administer the CN Programs in the 50 States, District of Columbia, Guam, Puerto Rico, and U.S. Virgin Islands as well as the approximate 19,050 school food authorities (SFAs), public and private, that administer the CN Programs at the local level. No sampling or weighting is required as such a collection will be a census of both SAs and SFAs. Each SA will be asked to provide contact information for the SFAs under its jurisdiction, which each SA is uniquely positioned to provide; thus, a sample of SAs is not sufficient for the purposes of this collection.
Based on previous experience with State-level collections, the timeliness of the survey, and the minimal amount of burden involved on the part of State agencies for this collection, a 100 percent response rate is expected for the state-level SFA contact information request. Additionally, if the topic which is the focus of the census survey is an important issue affecting school districts nationwide and is a short FNS survey presents an opportunity for SFAs to communicate their challenges and/or other experiences directly to FNS, FNS expects all 19,050 SFAs to respond.
Table B1. Summary of Respondent Universe and Expected Response Rates if Issues Are of Concern to Respondents
Respondent |
Universe |
Target completed cases |
Expected Response Rates |
|
|
|
|
State CN Agencies |
67 |
67 |
100% |
SFAs |
19,050 |
19,050 |
100% |
Total |
19,117 |
19,117 |
100% |
The approach to achieving high response rates builds on prior FNS survey experience as well as FNS’s relationships with its partners at the State and local levels. Those experiences will come into play for each collection of information under this generic clearance.
FNS will build support for each survey through engaging with regional and State partners to promote awareness and the importance of responding for any given survey. Because we know that SFAs consistently look to their State Agencies (SAs) for guidance related to the CN Programs, FNS will request State agency assistance to build support for any study to be done in the future among SFAs by asking all SAs to distribute the Survey Support Email that would be prepared per survey, as appropriate, to let SFAs know that the survey is coming and to express their support for the collection. The period of time over which FNS would collect data would be identified in each memo to OIRA for information collections under this generic clearance. SFAs that have not yet completed their surveys will receive a reminder email on a periodic basis as appropriate. Based on past experience, FNS expects that each SFA could potentially receive 2 reminder emails, on average, for a survey.
To further promote high response rates, the web survey will likely allow respondents to save and exit the survey at any point, and then return to access and complete the survey later. In the past, FNS has also created a dedicated email address through which SFAs may request assistance or ask questions. These tools will help SFAs to complete the survey whenever it is convenient for them to do so and will help FNS to track and respond to SFA requests for assistance in a timely manner. Additionally, States and SFAs will be informed whether or not each collection is mandatory or voluntary. Some surveys may be a mandatory collection under the provisions of the National School Lunch Act, as one example.
Describe the procedures for the collection of information including:
Statistical methodology for stratification and sample selection;
Estimation procedure;
Degree of accuracy needed for the purpose described in the justification;
Unusual problems requiring specialized sampling procedures; and
Any use of periodic (less frequent than annual) data collection cycles to reduce burden.
The procedures for the collection of the information would vary dependent on the specific program and purpose of each specific collection. The procedures for collection would be discussed in the memo to be submitted to OIRA on a case-by-case basis for each information collection under this generic ICR. For now, an example is included below for pulse surveys which could be conducted for the Child Nutrition Program.
One example - Pulse Surveys Under the Child Nutrition Program
Because this would be census and a 100 percent response rate would be expected, there is no need for sampling, weighting or nonresponse adjustments.
To facilitate such a data collection, FNS will first obtain SFA contact information from the 56 State CN agencies that administer the CN Programs by sending State Agencies (SAs) an SFA Contact Information Request email with an attached spreadsheet to complete and return to FNS. SAs may also submit the information in other forms depending on the specific systems they use and their ability to export existing information. With the SFA Contact Information Request email, FNS will attach a Survey Support Email and ask all SAs to distribute it to the SFAs in their respective States to let SFAs know that the survey is coming and to express their support for the collection. Shortly thereafter, FNS will email the specific SFA Survey, depending on the focus of the collection, to each SFA. One potential option for sharing the survey is via the Qualtrics Survey Software platform.
FNS intends to collect data over a time period to be specified per collection. SFAs that have not yet completed their surveys will receive a reminder email from experienced survey support personnel at FNS each week or over another appropriate time period to be identified. FNS expects each SFA to receive 2 reminder emails, on average. Upon completion of the survey, FNS will send each SFA a thank you email. Respondents may also email the survey team at FNS to request help in completing their survey or with technical issues.
Pulse Survey & Statistical Methodology for Stratification and Sample Selection. Because the collection will be administered via a census of State CN agencies and SFAs, there will be no sample stratification or selection.
Pulse Survey & Estimation Procedure. Because data will be collected from a census of State CN agencies and SFAs, no estimation procedures are necessary for this collection.
Pulse Survey and Degree of Accuracy Needed for the Purpose Described in the Justification: Because data will be collected from a census of State CN agencies and SFAs, this question is not relevant to this submission.
Pulse Survey and Unusual Problems Requiring Specialized Sampling Procedures: No specialized sampling procedures are planned.
Pulse Survey and Use of Periodic (Less Frequent than Annual) Data Collection Cycles to Reduce Burden. The data collection procedures will be conducted once. Concern regarding the periodicity of data collection cycles is not applicable.
Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.
The methods to maximize response rates for each individual collection of information would vary dependent on the specific program and purpose of each specific collection. The methods to maximize response rates would be discussed in the memo to be submitted to OIRA on a case-by-case basis for each information collection under this generic ICR. For now, an example is included below for pulse surveys which could be conducted for the Child Nutrition Program.
One FNS Example - Pulse surveys
The study is expected to achieve a 100 percent response rate for both the state-level collection of SFA contact information and the SFA Survey on each topic to be identified. This means that the data collected will represent the entire universe of State CN Directors and SFAs, and rather than providing estimates to answer the research questions, we will be able to provide actual population totals. Thus, these data will provide reliable information on the extent and impacts of school food supply chain disruption that represent the full population.
Achieving the specified response rate involves contacting the States, securing their support for the study, and then contacting SFAs and offering support and completion reminders. In such a situation, the study team will likely use the following methods to maximize participation and reduce nonresponse:
FNS will inform States and SFAs that their participation in a collection is mandatory per the provisions of the National School Lunch Act or voluntary depending on the specifics of each situation.
FNS will request State agency assistance to build support for the study among SFAs by asking all SAs to distribute the Survey Support Email to let SFAs know that the survey is coming and to express their support for the collection. When appropriate, FNS may also use intermediaries (e.g., associations, etc.) who have regular connections with agencies to support participation.
The web survey will allow respondents to save and exit the survey at any point, and then return to access and complete the survey later, allowing respondents to complete the survey at their convenience.
SFAs that have not yet completed their surveys will receive a reminder email likely each week or at other intervals to be identified for OIRA on a case-by-case basis. FNS expects each SFA to receive 2 reminder emails, on average.
Recruiting materials would be carefully developed to emphasize points such as the following, as applicable, depending on the unique circumstances of each information collection. The following are examples of points which may resonate with respondents:
State agency and SFA participation in the collection are vital to providing FNS with information we need to support school districts.
This is an opportunity to communicate the specific issues each SFA is experiencing directly to FNS.
This survey will assist FNS and its partners to enhance the toolbox for school nutrition professionals working hard to make sure students have reliable access to healthy meals.
The survey was crafted with input from SFAs and the amount of time it will take to complete each survey will be identified.
FNS could create a dedicated email address through which SFAs may request assistance or ask questions to enable FNS to track and respond to SFA requests for assistance in a timely manner. Staff will be readily available to clarify survey questions and work with participants to resolve technical issues, such as difficulty logging on or advancing past pages. Personalized assistance bolsters the perceived legitimacy of each collection and will encourage respondents to persist in completing each survey.
If the response rate is lower than expected after reminder emails have been sent, FNS will compare the characteristics of the non-responding SFAs to those of all SFAs to determine if there are patterns in non-response across key characteristics such as SFA size, geographic location (State or region), or urbanicity (urban vs. rural). If there are identifiable patterns in nonresponse, FNS will consider providing more targeted outreach to the non-responding SFAs to encourage their participation. If necessary, FNS will also prepare suitable nonresponse adjustment weights to be used for estimates and will prepare suitable estimates of standard errors on critical outcomes prior to presenting results of the survey.
Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.
The tests for procedures or methods to be undertaken would vary dependent on the specific collection and approaches to be used for each information collection to be conducted under this generic. The tests for procedures or methods would be discussed in the memo to be submitted to OIRA on a case-by-case basis for each information collection under this generic ICR. For now, an example is included below for pulse surveys which could be conducted for the Child Nutrition Program.
Pulse Surveys
The study team could potentially pretest any SFA Survey with nine SFAs or utilize this generic information collection for pretests. Through the regional offices, FNS could ask State agencies to recommend SFAs for the pretest with consideration for a set of identified criteria. SFA respondents could be selected based on their availability to participate within the pre-test timeframe and to ensure variation in participation across characteristics such as SFA size in terms of student enrollment, geographic location, urbanicity, percentage of students certified for free or reduced-price meals, and use of a food service management company, in order to gather different perspectives on the survey.
FNS could schedule a 30-minute debriefing conversation with each of the 9 participants and send each a link to complete the web survey via the Qualtrics Survey Software platform, as one example. In the invitation email, FNS could request feedback from the pretest SFAs on question wording, response options, and burden. Prior to each debriefing phone call, FNS could send the respondent a copy of his or her completed survey responses to facilitate the discussion. The pretested version of the survey and the debriefing protocol would be made available to OIRA on a case-by-case basis.
In response to participant feedback, FNS could make appropriate changes to each survey.
Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.
Table B2 lists FNS managers who would likely be consulted on statistical aspects of the design of each survey, as appropriate, dependent on the specific program which would be the focus on each survey. Staff who reported to these managers would likely be responsible for collecting and/or overseeing the collection and analysis of the study data for specific FNS program areas. As appropriate, names of such staff would be identified in each memorandum to be submitted to OIRA on a case-by-case basis for each information collection under this generic ICR.
Table B2. Individuals consulted on statistical aspects of study design per FNS program
Staff |
Title |
|
|
Melissa Abelev |
Director of Research & Analysis Division, SNAS, Office of Policy Support, FNS |
|
703-305-2209 |
Kathryn Law |
Director of SNAP Research & Analysis Division, Office of Policy Support, FNS |
|
703-305-2138 |
Barbara Murphy |
Director of Child Nutrition Research & Analysis Division, Office of Policy Support, FNS
|
|
703-305-2532
|
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Mathematica Standard Report Template |
| Author | MATHEMATICA |
| File Modified | 0000-00-00 |
| File Created | 2023-12-12 |