OACS Feasibility Reassessment Memo Revised

MEMORANDUM

Date: December 19, 2018

To: Stephanie Tatham, Office of Management and Budget Desk Officer

Through: Christina Sandberg C.S.

Food and Nutrition Service, Information Collection Officer, Planning & Regulatory Affairs Office

Ruth Brown

United States Department of Agriculture, Departmental Information Clearance Officer, Office of Chief Information Collection Officer

From: John Endahl

Senior Program Analyst, Food and Nutrition Service

Office of Policy Support

Re: Generic OMB Clearance No. 0584-0606—School Nutrition and Meal Cost Study-II, Outlying Areas Cost Study Feasibility Reassessment

The U.S. Department of Agriculture’s Food and Nutrition Service requests approval to conduct a second round of exploratory interviews for the referenced study under approved Generic OMB Clearance No. 0584-0606 FNS Generic Clearance for Pre-Testing, Pilot, and Field Test Studies. The following information is provided for review:

1. Title of the Project: School Nutrition and Meal Cost Study-II, Outlying Areas Cost Study Feasibility Reassessment

2. Control Number: 0584-0606, expires 03/31/2019

3. Public Affected by this Project: 56

1. State, Local, Tribal Governments: 56

State and Local Governments:

• 4 State agency officials

• 10 school food authority (SFA) directors

• 10 district business managers

• 16 school nutrition managers (SNMs)

• 16 school principals

2. Businesses/Private Nonprofit Organizations: 0

3. Individuals/Households: 0

4. Number of Respondents and Research Activity

Respondents	Research activity	Number of respondents
State official	Notification and recruitment	2
	Interview, document submission, and thank-you letter	4
School food authority director	Notification, recruitment, interview, document submission, and thank-you letter	10
District business manager	Interview, document submission, and thank-you letter	10
School nutrition manager	Recruitment, interview, document submission, and thank-you letter	16
School principal	Recruitment, interview, document submission, and thank-you letter	16

5. Time Needed per Response

Respondents	Research activity	Time (hours)a
State official	Notification and recruitment	0.50
	Interview	1.00
	Document submission	0.25b
	Thank-you letter	0.05
School food authority director	Notification and recruitment	0.77
	Interview	2.30c
	Document submission	1.15d
	Thank-you letter	0.05
District business manager	Recruitment	0.05
	Interview	1.50
	Document submission	0.50
	Thank-you letter	0.05
School nutrition manager	Recruitment	0.27
	Interview	2.00
	Document submission	0.50
	Thank-you letter	0.05
School principal	Recruitment	0.27
	Interview	0.50
	Document submission	0.50
	Thank-you letter	0.05

^a The time is an average response per respondent.

^b The time per document submission is 0.50 hours for the Puerto Rico State Child Nutrition agency director, 0.25 hours each for Virgin Islands State Child Nutrition agency director and the Virgin Islands Division of Finance Director of Accounting and Financial Reporting, and 0.00 hours for the Virgin Islands Department of Education Chief of Staff. The average time per document submission is therefore 0.25 hours.

^c The time per interview is 3.00 hours for the central SFA director in Puerto Rico and the two SFA directors in the United States Virgin Islands, and 2.00 hours for the seven regional SFA directors in Puerto Rico. The average time per interview is therefore 2.30 hours.

^d The time per document submission is 1.50 hours for the central SFA director in Puerto Rico and the two SFA directors in the United States Virgin Islands, and 1.00 hour for the seven regional SFA directors in Puerto Rico. The average time per document submission is therefore 1.15 hours.

6. Total Burden Hours on Public: 136.20 hours and 268 responses

See the enclosed Microsoft Excel burden table for the detailed burden estimates broken down by the affected public (Appendix P, OACS Feasibility Reassessment Burden Table). According to OMB Form 83-I, a small entity may be “a small government jurisdiction which is a government of a city, county, town, township, school district, or special district with a population of less than 50,000.” Two of the nine SFAs included in the study, or 22 percent, are small entities. According to U.S. Department of Education statistics, the total enrollment of public school students in the United States Virgin Islands (USVI) is less than 14,000 students.¹ Both SFAs are therefore considered small entities. The total enrollment of public school students in Puerto Rico is 379,818. As described below, the single SFA in Puerto Rico is restructuring into seven SFAs. If the student population will be divided evenly among the SFAs, then none of the new SFAs will be considered small entities.

Although small SFAs are involved in this study, they deliver the same program benefits and perform the same functions as larger SFAs. The information being requested is the minimum required for the intended use, and data collection procedures were designed to minimize burden on all of the SFAs.

7. Project Purpose, Methodology, and Formative Research Design

Background

School food authorities (SFAs) that participate in the National School Lunch Program (NSLP) and the School Breakfast Program (SBP) receive cash reimbursements for meals served to students. Section 12(f) of the Richard B. Russell National School Lunch Act (NSLA) allows the Food and Nutrition Service (FNS) of the U.S. Department of Agriculture (USDA), which administers the NSLP and SBP, to adjust the reimbursement for SFAs in outlying areas, including Alaska, Guam, Hawaii, Puerto Rico, and the United States Virgin Islands (USVI). Adjusted reimbursements are based on the difference between the costs of providing reimbursable meals in the outlying areas compared to in the contiguous 48 States and the District of Columbia (DC).

Currently, reimbursement rates are 62 percent higher for SFAs in Alaska and 17 percent higher for SFAs in Hawaii and Puerto Rico. The higher reimbursements in Alaska and Hawaii are long-standing—established in 1979 and 1981, respectively. SFAs in Puerto Rico became eligible for higher reimbursements in July 2016. SFAs in Guam and USVI receive the same reimbursements as SFAs in the 48 States and DC. The adjusted reimbursement rates for Alaska and Hawaii are based on an internal FNS analysis conducted in 1979 that compared food and labor costs in these areas with costs in the contiguous 48 States. The analysis attempted to examine costs in the territories as well, but the necessary data were not available and there was some indication that higher food costs in these areas might be offset by lower labor costs.

In the intervening years, Guam, Hawaii, and Puerto Rico have commissioned studies to document the need for higher reimbursement rates (Falcon-Sanchez Consulting Group 2015; University of Guam Cooperative Extension Service 2014; Hawaii Appleseed Center for Law and Economic Justice 2015). Puerto Rico sponsored two studies—the first in 2008 and the second in 2015. At the time of the 2008 study, FNS concluded that operating costs in Puerto Rico were comparable to costs in the contiguous States and, therefore, higher reimbursement rates were not warranted (Falcon-Sanchez Consulting Group 2015). In reviewing the 2015 report, FNS noted concerns about the rigor and reliability of the data presented but concluded, based on comparisons with the most recent study of meal costs in the 48 States and DC, that a 15 to 17 percent increase was appropriate. FNS concluded that the studies commissioned by Guam and Hawaii did not provide an adequate basis for assessing the costs of producing individual lunch and breakfast meals.

USDA periodically assesses the cost of producing reimbursable school meals in the contiguous 48 States and DC using a rigorous, standardized methodology that has historically relied upon on-site data collection. Given the length of time since the initial analysis to support adjustments to reimbursement rates in Alaska and Hawaii and the challenges outlying areas have faced in assessing meal costs on their own, FNS contracted with Mathematica Policy Research to use a standardized methodology to assess meal costs in the outlying areas. The first step, the Outlying Areas Cost Study (OACS) Feasibility Assessment, was conducted in early 2018 (OMB Control Number 0584-0606, Expires 3/31/2019). It assessed the quality and completeness of SFAs’ and schools’ financial and administrative data needed to support a rigorous cost study in Alaska, Guam, Hawaii, Puerto Rico, and USVI that minimizes or eliminates on-site data collection (Forrestal et al. 2018). The assessment included both exploratory interviews and document reviews (such as revenue and expenditure statements, production records, standardized recipes, and union contracts) to determine the feasibility of adapting the cost data collection instruments and procedures for use in the outlying areas. The assessment concluded that Alaska, Guam, and Hawaii can support the rigorous cost study, and plans for data collection for these States and Territories will be included in the OMB Information Collection Request (ICR) for the School Nutrition and Meal Cost Study-II (SNMCS-II).

Findings from the feasibility assessment in Puerto Rico and USVI were less conclusive. The State Child Nutrition (CN) and SFA directors in the two Territories were difficult to contact because of limited telephone service, lack of responsiveness to telephone and email requests to schedule interviews, and in USVI, lack of availability of one of the respondents during the data collection period. As a result, no SNMs or principals were interviewed and no documents were obtained from these school staff in either Territory, and no data were collected from one of the two SFAs in USVI. This precluded our ability to determine whether it would be feasible to support the rigorous cost study in either Territory. In addition, both Territories are undergoing operational changes that could affect the cost study design. USVI’s two SFAs are establishing accounts to receive Federal funding via the State CN agency. The SFAs currently do not have accounts and instead receive an allowance from the State CN agency. Puerto Rico is in the process of restructuring by dividing its central SFA into seven regional SFAs. The central SFA functions will be delegated to the current seven SFA regions. At the time of the feasibility assessment, the central SFA director explained that the goal is to have the structural changes in place and operating by the 2019–2020 school year. The central SFA director is now the acting State CN director, and he recently informed FNS that the restructuring will be completed by November 2020. Previously, he had stated that the restructuring would be completed by February 2019, the new SFAs would be phased in over time, and that regional staff are aware of the restructuring.

Based on this information from Puerto Rico’s acting State CN director, we anticipate that at least one of the seven regional SFAs will be operational before this feasibility reassessment data collection, and will therefore have an SFA director and district business manager. However, in other regions, the new SFAs will not be operational but staff will be involved in the restructuring. The data collection team will pursue interviews with these staff. For simplicity, these staff are referred to as “SFA directors,” and “district business managers” in this memorandum because they are anticipated to have equivalent operational knowledge and expertise as the staff in the new SFAs.

Purpose

The purpose of the OACS Feasibility Reassessment is to (1) assess the quality and completeness of the meal cost data available in Puerto Rico and USVI; (2) obtain updated information on the status of operational changes; and (3) develop a standardized methodology to estimate the costs of producing reimbursable meals in the Territories. The feasibility reassessment will not collect all of the data needed to estimate the costs of producing reimbursable meals. Rather, it will obtain the information necessary to determine if it is possible to estimate the costs. FNS is requesting clearance for the cost data collection in the SNMCS-II ICR with the expectation that it will be feasible to adapt the cost study instruments and procedures for use in Puerto Rico and USVI.² As described below, the reassessment will address the challenges encountered in the first feasibility assessment to enable firm conclusions to be reached and will also collect updated information about the status of restructuring in Puerto Rico. First, we anticipate the staff in USVI who were unavailable in spring 2018 to be available in spring 2019. Second, based on recent recommendations from FNS staff, we plan to interview additional State officials in USVI to help clarify financial operations in the Territory. Third, we plan onsite data collection to conduct interviews face-to-face to overcome the challenges encountered with communicating with respondents.

Methodology and Research Design

The feasibility reassessment includes collecting data and documents from State, SFA, district, and school staff, and synthesizing the information collected about each Territory to assess the feasibility of estimating its full or reported costs.³ The activities include face-to-face, exploratory interviews with staff. Although on-site data collection with face-to-face interviews does not guarantee complete participation among interviewees, it offers advantages over remote data collection. First, it overcomes the challenges associated with limited communications infrastructure that were encountered in the first feasibility assessment. Overcoming these challenges could in turn lead to efficiencies in data collection (for example, avoiding extensive outreach and follow-up contact attempts through phone and email). Second, it will enable the data collectors to establish greater rapport with respondents, which is particularly valuable when collecting data in these Territories. Building rapport during the feasibility reassessment could additionally make recruiting and data collection activities easier if the rigorous cost study methodology is implemented in the future.

Document collection. The study team will request several documents from SFA directors: the most recent audited and unaudited annual revenue and expense statements for the SFA; completed meal production records; indirect cost documentation; cycle menus; standardized recipes; food price lists; non-reimbursable foods sales records; district, SFA, and school personnel salary information; and copies of food service management company and union contracts. The study team will inquire about the availability of any documents from State staff if documents are unavailable from the SFA directors. In Puerto Rico we will also ask about any documents that are unique to the regional SFAs to avoid requesting identical documents from both the central SFA director and regional SFA directors.

We will request examples of completed meal production records from SNMs and may also request examples of standardized recipes and food price lists if these documents are not available at the SFA or State level. We will request school personnel information from principals if the documents are not available at the SFA or State level.

We will request documents from sample members during recruiting to allow for sufficient review of these documents before the interviews. If any documents are not obtained before the interviews, we will request the documents while on-site.

Exploratory interviews with State, SFA, and district staff. The study team will conduct semistructured, exploratory, face-to-face interviews with SFA directors, district business managers, and State officials to determine the quality and completeness of the available data, and to update the information about food service operations obtained during the prior feasibility assessment in 2018. If staff availability and travel logistics allow, in Puerto Rico we will interview SFA directors and district business managers first (Appendix A, SFA-Level Interview Guide)⁴, followed by the State CN agency director (Appendix B, State-Level Interview Guide). District business managers will participate with SFA directors to answer questions about staff salaries that are not paid out of the food service account, food service indirect costs, and SFA financial statements. Interviews with State CN agency staff will omit questions that were answered adequately at the SFA level.

In USVI, we plan to interview the State CN agency director together with the Virgin Islands Division of Finance Director of Accounting and Financial Reporting and the Virgin Islands Department of Education Chief of Staff (Appendix B, State-Level Interview Guide). FNS staff recommended interviewing these staff together to help provide a complete picture of food service financial operations in the Territory. This interview will precede interviews with the SFA directors and district business managers (Appendix A, SFA-Level Interview Guide).

Exploratory interviews with SNMs and school principals. The study team will conduct semistructured, exploratory, face-to-face interviews with SNMs and school principals to determine the quality and completeness of available data at the school level, such as data on producing school meals, food prices, non-reimbursable food sales, and staff time and salary information.

Information obtained from State and SFA level interviews will be used to tailor the questions in the SNM and principal interviews (Appendix C, School Nutrition Manager Interview Guide, and Appendix D, Principal Interview Guide, respectively). We will determine during the screening process, described below, whether principals must be interviewed. If a principal confirms the school does not have staff who work on food service activities and are paid from a non-food service account, no interview will be necessary. If any of the SFAs use a central kitchen to produce reimbursable meals served in schools, one of the SNM interviews will be with the manager of the central kitchen using the interview protocol in Appendix C.

Design and Sampling Procedures

The State-level interviews will include the State CN agency directors in Puerto Rico and USVI, plus the Virgin Islands Division of Finance Director of Accounting and Financial Reporting and the Virgin Islands Department of Education Chief of Staff. The SFA-level interviews will include the central SFA in Puerto Rico; the seven regional SFAs that are being formed in Puerto Rico; and the two SFAs in USVI.

We will purposively select up to 10 schools across Puerto Rico (including at least one school per region), and up to three schools per SFA in USVI (six schools total). Schools will be selected based on the willingness of the school staff to participate in the interviews and proximity to State- and SFA-level interviewees to minimize travel. During the recruitment process (described below), SFA directors will assist in identifying schools based on characteristics that might influence the availability and quality of the data needed for the cost study—for example, whether staff who are not part of the food service department work on food service activities; whether a school purchases its own food; and whether a school prepares its own food or receives food from a central or production kitchen. We anticipate that all sample members will participate in the data collection because of the relevance of the study topic, purposive selection, and because State, SFA, and school participation in the evaluation of the NSLP and SBP is mandatory under Section 28 of the NSLA.

Recruitment and Consent

Notification. Before data collection begins, the Regional Special Nutrition Directors in the Mid-Atlantic Regional Office will first notify the State CN agency directors of the upcoming study, and request that the State CN agency directors notify the SFA directors (Appendix E, State CN Agency Director Notification Letter). This notification letter will also explain that the study team will contact the State CN agency directors to schedule an interview. The State CN agency directors will notify SFA directors of the upcoming study and inform them that contractor staff will contact the SFA directors (Appendix F, SFA Director Notification Letter).

Recruitment in Puerto Rico. The study team will send a follow-up email to central SFA director (Appendix G, SFA Director Recruiting Email). This email will ask the SFA director to provide contact information for regional SFA and school staff and gather the documentation to be submitted. The email will inform the SFA director that the study team will follow up by telephone to review the request, answer questions, confirm which documents will be provided, and share document submission instructions, and begin planning the visit (Appendix H, SFA Director Recruiting Telephone Script). The study team will determine with the SFA director which party should take the lead arranging the visit with regional SFA staff and school staff.

The study team will also send a follow-up email to the State CN agency director to schedule an interview, determine the role of the agency in food service operations, and ask the director to gather the documentation to be submitted (Appendix I, State CN Agency Director Recruiting Email). The documents requested of the State CN agency director will be limited to only the items that were not available from the SFA director. The study team will follow up by telephone to answer questions, schedule the interview, confirm which documents will be provided, and share document submission instructions (Appendix J, State CN Agency Director Recruiting Telephone Script).

After the central SFA director provides contact information for the regional SFA and school staff, either the central SFA director or the study team will email these staff (Appendix G, SFA Director Recruiting Email and Appendix K, School Nutrition Manager/Principal Recruiting Email, respectively). If SNMs or principals do not have an email address, Appendix K will be sent as a letter instead. We will call regional SFA directors, SNMs, and principals to answer questions, schedule the interviews, confirm which documents will be provided, and share document submission instructions (Appendix H, SFA Director Recruiting Telephone Script; Appendix L, School Nutrition Manager Recruiting Telephone Script; and Appendix M, Principal Recruiting Telephone Script, respectively). During calls with principals, the study team will determine whether an interview is needed. An interview will not be scheduled if the school has no staff who work on food service activities and are paid from a non-food service account (that is, no staff who provide “off-budget” labor).

After all of the interviews have been scheduled, the study team will send a confirmation email to each recruited sample member to communicate the itinerary for the face-to-face interviews and remind them of the document submission instructions (Appendix N, Visit Confirmation Email). If any SNMs or principals do not have an email address, Appendix N will be sent as a letter instead. Following completion of the data collection, participants will be formally thanked (Appendix O, Thank-You Letter).

Recruitment in USVI. The procedures for USVI are similar to those described for Puerto Rico, except no regional SFA directors are present in the Territory, and the interview with the State CN agency director will be scheduled to also include the Virgin Islands Division of Finance Director of Accounting and Financial Reporting and the Virgin Islands Department of Education Chief of Staff. Following notification, recruiting steps include:

• Contacting the SFA directors by email and telephone (Appendix G, SFA Director Recruiting Email and Appendix H, SFA Director Recruiting Telephone Script)

• Contacting the State CN agency director by email and telephone (Appendix I, State CN Agency Director Recruiting Email and Appendix J, State CN Agency Director Recruiting Telephone Script)

• Contacting SNMs and principals by email and telephone (Appendix K, School Nutrition Manager/Principal Recruiting Email; Appendix L, School Nutrition Manager Recruiting Telephone Script; and Appendix M, Principal Recruiting Telephone Script)

• Confirming visit details with all recruited sample members (Appendix N, Visit Confirmation Email)

• Thanking participants following the data collection (Appendix O, Thank-You Letter)

Consent. At the start of each interview, the interviewer will explain the purpose of the interview, how the data will be stored and used, the measures taken to protect privacy, and that participation is mandatory. The interviewer will ask the respondent to consent to recording of the interview. The interviewer will answer any clarification questions before proceeding with the interview.

Incentives

No incentives will be provided to respondents.

Data Analysis

Because of the nature of the data to be collected, the study team will use qualitative analysis methods to assess feasibility. We will synthesize interview notes with an assessment of the provided documentation to determine the completeness and accuracy of the available data in each Territory. We will summarize in a table the findings for the critical data elements needed to estimate meal costs, and the respondents needed to provide the data. Because it might be possible to estimate reported costs but not full costs, the table will be organized to classify the data needed to estimate reported costs from the additional data needed to estimate full costs.

8. Confidentiality

State, SFA, and school participation in the evaluation of the NSLP and SBP is mandatory under Section 28 of the NSLA. The privacy of all collected data will be held in accordance with a system of record notice titled FNS-8, “FNS Studies and Reports” published in the Federal Register on April 25, 1991, Volume 56, pages 19078-19080. Respondents will not be identified and information will not be shared outside of the study team. Some of the documents requested in this study, such as personnel records or contracts, may be considered sensitive. Respondents will be instructed to redact personally identifiable information (PII) from personnel records submitted to the study team. If respondents submit personnel records containing PII, the study team will redact the PII. All electronic documents, recordings, and interview notes will remain in password-protected, secure folders to which only project staff with appropriate clearance will have access. All hard-copy documents will be stored in locked filing cabinets. Records will be destroyed after the project ends. No PII will be collected about students or parents during this study.

9. Federal costs

The total cost to the Federal government is $270,051.22, which includes the total cost for contractor and Federal staff costs for the abbreviated supporting statement, developing the study instruments, collecting and analyzing the pre-test information, and fully loaded wage rates. The contractor cost is estimated at $263,223.00. This is based on an estimate of 1,470 hours, with fully loaded wage rates of $76.00 to $250.00 per hour, and includes overhead costs. The Federal staff cost is estimated at $6,828.22, assuming a total of 85 hours of Federal employee time. Federal employee time includes 80 hours for a GS-13, Step 10 program analyst at $60.40 per hour ($4,832.00 total), and 5 hours for a GS-14, Step 4 branch chief at $60.40 per hour for supervisory oversight ($302.00 total), for a total of $5,134.00 on an annual basis. To account for a fully loaded wage rate, an additional 33 percent, or $1,694.22 ($5,134.00 X 0.33), was added to the total Federal hourly wages. Federal employee hourly rates are based on the General Schedule and locality payment for the Washington D.C. Metropolitan Area provided by Office of Personnel Management (OPM) for 2018 (https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2018/DCB_h.pdf).

10. Study Instruments and other Appendices

Appendix A. SFA-Level Interview Guide

Appendix B. State-Level Interview Guide

Appendix C. School Nutrition Manager Interview Guide

Appendix D. Principal Interview Guide

Appendix E. State CN Agency Director Notification Letter

Appendix F. SFA Director Notification Letter

Appendix G. SFA Director Recruiting Email

Appendix H. SFA Director Recruiting Telephone Script

Appendix I. State CN Agency Director Recruiting Email

Appendix J. State CN Agency Director Recruiting Telephone Script

Appendix K. School Nutrition Manager/Principal Recruiting Email

Appendix L. School Nutrition Manager Recruiting Telephone Script

Appendix M. Principal Recruiting Telephone Script

Appendix N. Visit Confirmation Email

Appendix O. Thank-You Letter

Appendix P. OACS Feasibility Reassessment Burden Table

References

Falcon-Sanchez Consulting Group. “Puerto Rico School Food Authority: Lunch, Breakfast, and Snack Cost Study-II, Final Report.” Report submitted to the Puerto Rico School Food Authority. San Juan, PR: Falcon-Sanchez Consulting Group, May 2015.

Forrestal, Sarah, Brittany McGill, Rikki Welch, Andre Williamson, and Kathy Wroblewska. “Outlying Areas Cost Study Feasibility Assessment.” Memorandum submitted to the United States Department of Agriculture, Food and Nutrition Service, Office of Policy Support. Chicago, IL: Mathematica Policy Research, July 2018.

Hawaii Appleseed Center for Law and Economic Justice. “Report on the Need for Increased Meal Reimbursement Rates for Hawaii.” Report submitted to the Office of Hawaii Child Nutrition Programs. Honolulu, HI: Hawaii Appleseed Center for Law and Economic Justice, May 2015.

University of Guam Cooperative Extension Service. “Report to the Guam Department of Education: GDOE School Lunch and Breakfast Cost Study.” Report submitted to the Guam Department of Education. Mangilao, GU: University of Guam Cooperative Extension Service, November 2014.

1 U.S. Department of Education, National Center for Education Statistics, Common Core of Data (CCD), "State Nonfiscal Public Elementary/Secondary Education Survey Membership Data," 2015-16, v.1a.

2 The SNMCS-II ICR, including the data collection to estimate meal costs, is anticipated to be cleared in summer 2019. The SNMCS-II ICR will be submitted to OMB review before the feasibility reassessment described in this generic clearance will be completed. Therefore, Puerto Rico and USVI must be included in the SNMCS-II ICR so that data collection to estimate meal costs can be completed in SY 2019-2020. This will allow for cost estimates to be compared between the outlying areas and SFAs in the continental U.S. in the same year.

3 Reported costs include only those charged to the SFA or school budgets that the SFA is expected to cover with the school food service account. Full costs include costs incurred by the school district in support of SFA operations that are not charged to the school food service account, such as school staff labor that is paid for out of the district account.

4 The burden associated with gathering and submitting requested documents is included in the disclosure statements on Appendices A, B, C, and D.

1