Sodium OMB Package Part B Revised 10-24-16

Supporting Statement for OMB Clearance for
Successful Approaches to Reduce Sodium in School Meals

Part B: Collections of Information Employing
Statistical Methods

September 8, 2016

Alice Ann Gola

Social Science Research Analyst

Office of Policy Support

Food and Nutrition Service

United States Department of Agriculture

3101 Park Center Drive

Alexandria, Virginia 22302

Telephone: 703-305-4347

Email: [email protected]

Table of Contents

Part B. Collections of Information Employing Statistical Methods B-1

B.1 Respondent Universe and Sampling Methods B-1

B.2 Procedures for the Collection of Information B-7

B.3 Methods to Maximize the Response Rates and to Deal with Non-Response B-10

B.4 Test of Procedures or Methods to be Undertaken B-12

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting
and/or Analyzing Data B-12

Tables

Table B1. Estimates of the Percentage of SFAs Meeting Target 1 During SY 2013–14 B-2

Table B2. Estimated Number of SFAs Qualified for Brief Site Visit Selection Interviews B-3

Table B3. SFA Sampling Plan B-4

Table B4. Research Team Contact Information B-13

Appendices

Appendix B1. Prescreening Web Survey Questionnaire

Appendix B2a. Brief Site Visit Selection Interview Invitation/Reminder Phone Script

Appendix B2b. Brief Site Visit Selection Interview Guide

Appendix B3. SFA Director Interview Guide

Appendix B4. School Administrator In-Depth Interview Guide

Appendix B5. Food Supplier In-Depth Interview Guide

Appendix B6. Community-Based Stakeholder In-Depth Interview Guide

Appendix B7. SFA Director On-Site In-Depth Interview Observational Instrument

Appendix B8. School Administrator On-Site In-Depth Interview Observational Instrument

Appendix B9. Food Supplier On-Site In-Depth Interview Observational Instrument

Appendix B10. Community-Based Stakeholder On-Site In-Depth Interview Observational Instrument

Appendix C1a. Notification Email to Regional Office Directors

Appendix C1b. Notification Email to State Agency CN Directors

Appendix C1c. Attachment Notification Email to SFA Directors

Appendix C1. Notification Email to Regional and State Offices

Appendix C2. Prescreening Web Survey Invitation Email

Appendix C3. Prescreening Web Survey Reminder Email

Appendix C4. Prescreening Web Survey Reminder Phone Script

Appendix C5. Brief Site Visit Selection Interview Invitation and Preparation Email

Appendix C6a. SFA Director In-Depth Interview Invitation Phone Script

Appendix C6b. In-Depth Interview Invitation Phone Script

Appendix C7a. SFA Director In-Depth Interview Invitation Email

Appendix C7b. School Administrator In-Depth Interview Invitation Email

Appendix C7c. Food Supplier In-Depth Interview Invitation Email

Appendix C7d. Community-Based Stakeholder In-Depth Interview Invitation Email

Appendix C8a. SFA Director In-Depth Interview Confirmation and Preparation Email

Appendix C8b. School Administrator In-Depth Interview Confirmation and Preparation Email

Appendix C8c. Food Supplier In-Depth Interview Confirmation and Preparation Email

Appendix C8d. Community-Based Stakeholder In-Depth Interview Confirmation and Preparation Email

Appendix C9. In-Depth Interview Thank You Letter

Appendix C10. Participant Consent Script

Appendix C11. Brief Site Visit Selection Follow-Up Email

2. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

B.1 Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

B.1.a Sampling

This study will focus on understanding the successful efforts and approaches to reduce sodium in school meals used by key individuals in School Food Authorities (SFAs). An SFA may encompass a school district, several school districts, or an individual school, and serves as the governing body responsible for the administration and operation of Child Nutrition Programs therein, as approved by U.S. Department of Agriculture (USDA). Case studies will be conducted with selected individuals in SFAs successfully meeting Target 1, and close to meeting Target 2, sodium standards. The SFAs will be asked to discuss the range of issues encountered, and methods of success, while working towards meeting the sodium targets. The purpose of this study is not to produce a generalizable estimate, or catalogue descriptions of all SFAs meeting sodium targets, but rather, to identify the best practices and innovative approaches from select SFAs that can be used to help other SFAs meet future sodium targets. Therefore, this study does not require a nationally representative sample.

This study includes four groups of respondents within each SFA: (1) SFA directors, (2) school administrators,¹ (3) food suppliers,² and (4) community-based stakeholders.³ SFA directors may be administered up to one Prescreening Web Survey (Appendix B1); one Brief Site-Visit Selection Interview (Appendix B2b); and one In-Depth Interview (Appendix B3) that will either be administered by telephone or in person. School administrators, food suppliers, and community-based stakeholders will participate in In-depth Interviews by telephone or in person (see Appendices B4, B5, and B6 for the respective In-Depth Interview guides). Four observational tools (Appendices B7, B8, B9, and B10) will be used for SFA directors, school administrators, food suppliers, and community-based stakeholders that participate in the on-site In-Depth Interviews.

This study does not require a nationally representative sample to identify the best practices and innovative approaches used to meet the sodium targets. The potential respondent universe is estimated to be 14,836 SFAs (see Table B1).⁴

Table B1. Estimates of the Percentage of SFAs Meeting Target 1 During SY 2013–14

SFA Characteristics	SY 2013–14
	Percent	Total SFAs
		Weighted n	Unweighted n
All SFAs	13.6	14,836	1,572
SFA Size (number of students)
Small (1–999)	15.5	7,674	372
Medium (1,000–4,999)	9.4	5,246	601
Large (5,000–24,999)	16.3	1,626	407
Very Large (25,000+)	20.3	291	192
Source: Special Nutrition Operations Study Year 3, SFA Director Survey SY 2013–14, question 5.16.

SFAs in the respondent universe will be stratified into small (< 1,000), medium (1,000-4,999), large (5,000-24,999), and very large (> 25,000). A total of 625 SFAs will be randomly sampled by strata so that sample sizes of 190 small, 285 medium, 88 large, and 62 very large SFAs are selected. Assuming that 80 percent of SFA directors respond to the Prescreening Web Survey, the resulting respondent sample will include approximately 500 SFA directors (152 small, 228 medium, 70 large, and 50 very large SFAs). The 500 SFAs selected for the initial sample will be administered a short (lasting less than 15 minutes) web-based screening survey. Using estimates of the percentage of SFAs meeting Target 1 sodium standards reported in the Special Nutrition Operations Study Year 3 SFA Director Survey during SY 2013–14, at least 68 SFAs (24 small, 22 medium, 12 large, and 10 very large SFAs) are expected to be qualified for Brief Site Visit Selection Interviews (see Table B2).⁵

Table B2. Estimated Number of SFAs Qualified for Brief Site Visit Selection Interviews

SFAs Size	Sample Size	Estimated Number of Respondents	Percent Meeting Target 1 in SY 2013–20144	Minimum Estimate of Qualified SFAs
Small (1–999)	190	152	15.5	24
Medium (1,000–4,999)	285	228	9.4	22
Large (5,000–24,999)	88	70	16.3	12
Very Large (25,000+)	62	50	20.3	10
Total	625	500	13.6	68

Using responses from the Prescreening Web Survey, a sample size of 56 SFAs (20 small, 18 medium, 10 large, and 8 very large SFAs), from the expected 68 SFAs qualified, will be recruited for a brief telephone interview. Assuming an 80 percent response rate, 45 SFAs (16 small, 14 medium, 8 large, and 7 very large SFAs) are expected to respond to the Brief Site Visit Selection Interview. Based on the information obtained from the Prescreening Web Survey and the Brief Site Visit Selection Interview, 12 of the 45 SFAs will be recommended for on-site In-Depth Interviews (10 primaries, with 2 back-ups for potential non-response). The remaining 33 SFAs will be identified for telephone In-Depth Interviews (26 primaries, with 7 back-ups for potential non-response), resulting in a total of 36 completed In-Depth Interviews (estimated to include 13 small, 11 medium, 6 large, and 6 very large SFAs).

Table B3. SFA Sampling Plan

Estimated Number of Respondents, by SFA Size	Prescreening Web Survey		Brief Site Visit Selection Interview		In-Depth Interview
	Sample Size	Estimated Number of Respondents	Sample Size	Estimated Number of Respondents	Sample Size	Estimated Number of Respondents
Small (1–999)	190	152	20	16	16	13
Medium (1,000–4,999)	285	228	18	14	14	11
Large (5,000–24,999)	88	70	10	8	8	6
Very Large (25,000+)	62	50	8	7	7	6
Total	625	500	56	45	45	36

The main criteria for selecting SFAs to participate in either on-site or telephone In-Depth Interviews is whether SFAs have successfully met the Target 1 sodium requirements and are either meeting, close to meeting, or exceeding Target 2 in most or all of their schools, while demonstrating high student participation rates in school meals. Selection criteria for determining which of the selected SFAs would make the best on-site candidates will be based on a combination of their willingness to participate in a site visit and elements of their program that appear to be interesting or worthy of a site visit. SFA directors who are interested in participating in the study, and who have involved not only their own food service staff, but also teachers, administrators, parents, students, and/or the community in promoting sodium reduction efforts, will be identified. In addition, SFA sites where it may be possible to observe activities or evidence of activities of sodium reduction efforts will be identified. Examples of these activities might include signage or nutritional information in or around cafeterias, prominent placement of lower sodium options at point of purchase, school newsletters with messages about sodium reduction, taste testing events for students, and/or coordination with school teaching staff to incorporate sodium education into school curriculums. An observational tool will be used to capture this information during the on-site In-Depth Interviews.

After the targeted 36 SFA sites are selected, the remaining 32 sites in the larger set of 68 qualified sites will be retained as potential replacement sites in the event any of the 36 selected sites choose not to participate in the In-Depth Interviews. Some of these 32 sites may also be contacted if the Target 1 sodium levels have not been met by any of the 36 SFAs that were originally selected. However, FNS believes that this is a very unlikely scenario. If the 68 qualified sites are insufficient to gain cooperation from 36 sites, replacement SFA sites will be recommended from the remaining pool of 500 sites that completed the Prescreening Web Survey. FNS will inform and consult with the Contractor regarding any potential replacement SFAs before they are contacted.

In addition to the SFA directors, In-Depth Interviews will also be conducted with school administrators, food suppliers, and community-based stakeholders. The Brief Site Visit Selection Interview with SFA directors by telephone will verify each SFA’s status of meeting Target 1 and 2 sodium standards, and collect contact information for the respondent types (school administrators, food suppliers, and community-based stakeholders). In-Depth Interviews with SFA directors, school administrators, food suppliers, and community-based stakeholders will be conducted with a total of 36 SFA sites (10 site visits and 26 telephone interviews), resulting in a total of 144 In-Depth Interviews.

B.1.b Expected Response Rates

The total estimated number of respondents is 760. This figure includes 608 respondents and 152 non-respondents. The initial sample will consist of 625 SFA directors. Assuming that 80 percent respond to the Prescreening Web Survey, the resulting respondent sample will include approximately 500 SFA directors. This response rate is based upon the response rates from a similar FNS study, Special Nutrition Operations Study Year 3 SFA Director Survey, that achieved an 85 percent response rate among sampled SFAs using comparable data collection procedures (OMB No. 0584-0562, expiration date 04/30/2016).

Of the estimated 68 SFA directors identified as eligible from the Prescreening Web Survey results, 56 will be contacted for the Brief Site Visit Selection Interview, with an expected response rate of 80 percent (45 SFA director respondents and 11 non-respondents). Assuming a response rate of 80 percent, 36 of the 45 SFAs who participate in the Brief Site Visit Selection Interviews are expected to participate in the In-Depth Interviews. Within each of the participating 36 SFA sites, 45 school administrators, 45 food suppliers, and 45 community-based stakeholders (35 individual stakeholders and 10 not-for-profit institutions) will be contacted to participate in In-Depth Interviews. Assuming an 80 percent response rate, this will result in 36 responses from each respondent type. A high response rate is anticipated here given that the SFA Directors will have already agreed to participate in the study. FNS will ask the SFA directors to recommend and encourage others within their SFA to participate as well, which is expected to lead to higher levels of cooperation. The 152 non-respondents include 125 SFA directors who do not respond to the Prescreening Web Survey (or subsequent Brief Site Visit Selection Interviews and In-Depth Interviews) and a total of 27 individuals who do not respond to In-Depth Interviews (9 school administrators, 9 food suppliers, 7 individual community-based stakeholders, and 2 not-for-profit community-based stakeholders).

B.2 Describe the procedures for the collection of information including:

B.2.a Statistical methodology for stratification and sample selection

This is not a nationally representative study; therefore, concern regarding the statistical methodology for both stratification and sample selection is not applicable. However, the SFAs in the respondent universe will be stratified into small (< 1,000), medium (1,000-4,999), large (5,000-24,999), and very large (> 25,000). A total of 625 SFAs will be randomly sampled by strata so that sample sizes of 190 small, 285 medium, 88 large, and 62 very large SFAs are selected. See Table B3 for additional detail on stratification of SFAs by size.

B.2.b Estimation procedure

This is not a nationally representative study; therefore, concern regarding the estimation procedure is not applicable.

B.2.c Degree of accuracy needed for the purpose described in the justification

This is not a nationally representative study; therefore, concern regarding the degree of accuracy is not applicable.

B.2.d Unusual problems requiring specialized sampling procedures

FNS does not anticipate unusual problems that require the usage of specialized sampling procedures.

B.2.e Any use of periodic (less frequent than annual) data collection cycles to reduce burden

This is a one-time study; concern regarding the periodicity of data collection cycles is not applicable.

B.2.f General data collection procedures

Prescreening Web Survey

The SFAs will be stratified into small, medium, large, and very large. A total sample size of 625 SFAs will be randomly selected, including estimated sample sizes of 190 small, 285 medium, 88 large, and 62 very large SFAs. Regional FNS office directors, select State agency CN directors in the 48 contiguous States and District of Columbia, and SFA directors selected for participation in the Prescreening Web Survey will be notified of the study by email (Appendix C1a, C1b, and C1c, respectfully). Regional FNS offices will be asked to notify all Child Nutrition (CN) Directors and selected SFAs by email of the Prescreening Web Survey (Appendix C1). An email (Appendix C2) will be sent directly to the SFAs directors to provide login information necessary to complete the survey. The Prescreening Web Survey completed by SFA directors will identify which SFAs are currently meeting Target 1 sodium standards, and will collect preliminary information on approaches SFAs use to reduce sodium in school meals. Once the Prescreening Web Survey is sent, SFA directors may be contacted up to two times by email (Appendix C3) and up to four times by telephone (Appendix C4) to receive a reminder and encouragement to complete the Prescreening Web Survey.

Brief Site Visit Selection Interview

The Brief Site Visit Selection Interview by telephone with SFA directors will verify each SFA’s status of meeting Target 1 and 2 sodium standards and will collect contact information for the following respondent types: school administrators, food suppliers, and community-based stakeholders. The information collected from the Brief Site Visit Selection Interview will then be used to determine which of the eligible sites (SFAs that have met Target 1) will be selected for In-Depth Interviews, either on-site or by telephone. SFA directors may be contacted up to two times by telephone and once by email (Appendix C5) to encourage and confirm participation in the Brief Site Visit Selection Interview. A follow-up email (Appendix C11) will be sent to SFA directors following the Brief Site Visit Selection Interview to verify or obtain requested contact information.

In-Depth Interviews

Telephone and on-site In-Depth Interviews will be conducted with all four of the previously mentioned respondent types. SFA directors, school administrators, food suppliers, and community-based stakeholders may be contacted up to three times by telephone (Appendices C6a or C6b) and two times by email (Appendices C7a, C7b, C7c, or C7d, and C8a, C8b, C8c, or C8d) to encourage and confirm participation in the telephone and on-site In-Depth Interviews. In-Depth Interviews will collect extensive information on successes and challenges while working towards reducing sodium in school meals, along with strategies, best practices, tools, or resources that supported sodium reduction efforts.

Observational Instruments

The observational instrument will be used to collect additional information during the on-site visits by examining the cafeteria environment and communication materials such as posters, letters sent to parents, and news articles from each of the four respondent types (SFA directors, school administrators, food suppliers, and community-based stakeholders) who are interviewed on site during In-Depth Interviews. Each of the respondents will be sent a thank you letter (Appendix C9) at the end of their participation in the study to express appreciation for their contribution to the study.

B.3 Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Overall response rate projections are presented above in section B.1.b. Achieving the specified response rates involves using procedures described below to secure participation once the potential respondents have been identified. Based on a previous study with a similar respondent type and data collection protocol (OMB No.: 0584-0562, expiration date 04/30/2016), it is estimated that 80 percent of the sampled SFA directors will complete the Prescreening Web Survey; 80 percent of the prescreened SFA directors will complete the Brief Site Visit Selection Interview; and of those selected for the In-Depth Interviews, an 80 percent completion rate is expected. In the unlikely event that the SFA director scheduled for the on-site In-Depth Interview is unavailable during the site visit, an attempt will be made, while on site, to identify and interview an individual qualified to respond to the interview (e.g., SFA dietitian, SFA nutritionist). If no qualified individual is available, follow-up contact with the SFA director will be made in an attempt to conduct a telephone In-Depth Interview. Given the established rapport with the SFA directors at this point in the study, it is expected that 80 percent of school administrators, food suppliers, and community-based stakeholders (both individuals and not-for-profits) will complete their telephone or on site In-Depth Interviews.

The recruitment procedures designed to maximize the number of sampled SFA directors who complete the Prescreening Web Survey are described below:

• The letters inviting SFA directors to participate in the Prescreening Web Survey were carefully developed to emphasize the importance of this study and how the information will help the Food and Nutrition Service (FNS) identify successful approaches to reduce sodium in school meals.

• SFA directors participating in programs authorized under the Healthy, Hunger-Free Kids Act of 2010 (HHFKA) will be reminded that they are required to cooperate in USDA studies (Section 305).

• Designated FNS regional staff will be kept closely informed of the project so they will be able to answer questions from SFAs and States and encourage participation.

• A toll-free number and study email address will be provided so SFAs can receive assistance with the study.

• Sampled SFA directors will have the option of completing the web-based Prescreening Web Survey as a telephone survey.

• Periodic email reminders will be sent to sample members who have not yet completed the survey.

• Follow up attempts by telephone will be made with all sampled SFA directors who do not complete the survey between January 16 and February 3, 2017. The primary purpose of the call will be to urge them to complete the survey. At that point, if the directors prefer to complete the survey or remaining sections of the survey over the telephone, an interviewer will administer the full survey or any remaining parts of the survey over the telephone.

The following procedures will be used to maximize the completion rates for the Brief Site Visit Selection Interviews and In-Depth Interviews that will be administered by telephone and in person:

• Participating SFA directors will be asked to provide the contact information for school administrators, food suppliers, and community-based stakeholders for the In-Depth Interviews. Permission will be obtained from the SFA directors to mention their name when recruiting the school administrators, food suppliers, and community-based stakeholders they identified.

• The emailed letters inviting school administrators, food suppliers, and community-based stakeholders to participate in the In-Depth Interviews were very carefully developed to emphasize the importance of this study and how the information will help FNS identify successful approaches to reduce sodium in school meals.

• A toll-free number and email address will be provided to respondents. They will be encouraged to call if they have questions about the study.

• Follow-up attempts will be made by telephone with school administrators, food suppliers, and community-based stakeholders who do not respond between April 4 and May 26, 2017. The primary purpose of this call will be to urge them to participate in the In-Depth Interview.

• Call scheduling procedures that are designed to call numbers at different times of the day (between 8 a.m. and 6 p.m.) and days of the week (Monday through Friday) will be used to improve response rates.

• A core set of interviewers with experience conducting telephone interviews, particularly interviewers who have proven their ability to obtain cooperation from a high proportion of sample members, will be employed.

• A training for telephone and on-site interviewers will be conducted. The training, specific to this study, will include an overview of the project, a review of the research questions the study will address, a primer on interviewing practices and procedures, and techniques for encouraging respondent candor.

B.4 Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

Each of the data collection instruments were pretested with respondents of the appropriate respondent type, selected through recommendations from the Contractor and other pretest participants. Pretesting respondents evaluated assigned instruments for understandability (confusing wording or layout, failure to comprehend the question, etc.) and length of time to answer. All instruments were revised to incorporate pretest results.

B.5 Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The Contractor, 2M Research Services, and its Subcontractors, Abt Associates and Y-Pulse, will conduct this study. See Table B4 for contact information.
Table B4. Contact Information

Name	Affiliation	Telephone Number	E-mail
Paul Ruggiere	Project Director, 2M Research Services	(817) 856-0871	[email protected]
Molly Matthews-Ewald	Lead Project Manager, 2M Research Services	(469) 453-0888	[email protected]
Amy Wieczorek	Data Collector, 2M Research Services	(817) 666-3786	[email protected]
Ethan McGaffey	Data Collector, 2M Research Services	(817)-618-5515	[email protected]
Susan Ullrich	Data Collector, 2M Research Services	(817)-856-0866	[email protected]
Patty Connor	Project Director, Abt Associates	(617) 520-2907	[email protected]
Tara Wommack	Data Collector, Abt Associates	(617) 520-2940	[email protected]
Tami Cline	Project Director, Y-Pulse	(816) 569-0143	[email protected]
Prakash Adhikari	National Agricultural Statistics Service (NASS)	(202) 720 5467	[email protected]
Alice Ann Gola	FNS/USDA	(703) 305-4347	[email protected]

1 Defined as a school administrator or other employee who is knowledgeable about, or has been instrumental in promoting or working with students on the acceptance of, and changes to, nutrition in their school. These individuals could include a principal, assistant principal, nurse, an administrator, a staff member on a local school wellness committee, or teacher, for example.

2 Defined as an individual or part of an organization that delivers meals, food, or ingredients to an SFA or school(s) for use in school meals. These individuals could be local manufacturers, distributors, vendors, local farmers, or local food hubs.

3 Defined as someone who has a strong interest, even enthusiasm, for improving the school food environment and is someone who has been instrumental in efforts to improve the school food environment. These individuals are not employed by the schools, but are aware of, or attuned to, child nutrition. This individual may be a parent, community member, school board member, or a non-school employee member of a school wellness committee.

4 The respondent universe was estimated using data provided by Special Nutrition Operations Study Year (SN-OPS) Year 3 SFA Director Survey SY 2013–14 question 5.16. The actual respondent universe will include all SFAs who are required to submit FNS-742 and will be updated once access to the most current data is obtained.

5 Percentage of SFAs meeting sodium Target 1 in SY 2013–14, by SFA size (based on estimates provided by SN-OPS Year 3 SFA Director Survey SY 2013–14 question 5.16), are as follows: 15.5 percent of small (1–999), 9.4 percent of medium (1,000–4,999), 16.3 percent of large (5,000–24,999), and 20.3 percent of very large (25,000+). The rates of all SFAs meeting sodium targets in SY 2013–14 was 13.6 percent. The expectation is that the rates have improved, making the estimated number of qualified SFAs a minimum estimate. However, additions were made to the minimum sample calculation to cover non-responses (a response rate of 80 percent based on SN-OPS Year 3 is expected). These two conditions should render a sample size of 500 sufficient. The assumptions described above will be monitored to determine if sample size adjustments are necessary. Regardless of the number of qualified SFA sites, 56 qualified SFAs exhibiting practices meriting further exploration will be selected for the Brief Site Visit Selection Interviews.