State Agency Directors/SFA Directors (State, Local, and Tribal Government)

Attachment B.9 – Paper State Agency Survey

The purpose of this State School Nutrition Management Information System (MIS) Survey is to assess your agency’s school nutrition electronic management information system (MIS). In addition to a general description of the features of your MIS, FNS is interested in what data are available in your MIS, the level of aggregation of such data, and how you use such data to manage your school nutrition program operations. The information that you provide will be used only for the purposes of this study.

Definitions:

For this survey, the term school nutrition includes only the National School Lunch Program (NSLP) and School Breakfast Program (SBP).

School nutrition MIS means computer software and electronic systems used to record, manage and track major program functions such as claims for reimbursement, direct verification and certification, monitoring and administrative reviews, sponsor applications and record management, and others. We do not consider information captured, stored, and analyzed in spreadsheet applications such as Microsoft Excel to be an electronic MIS. Subsequent mentions of “MIS” refer to your school nutrition MIS.

Structure of the Survey:

This survey is organized into three main sections, each covering a major topic. These sections are:

1. Description of your current electronic school nutrition MIS

2. Information on data elements and reporting available in your school nutrition MIS

3. Technical features of your current school nutrition MIS

The questions in each of these three sections are organized to allow for different individuals to provide answers, if needed. Ideally, the Director or a designated representative who is familiar with the school nutrition MIS would answer the questions in sections 1 and 2. Section 3 is best answered by an individual with technical understanding of the school nutrition MIS. These are just suggestions; any staff member who is knowledgeable about the MIS may respond.

According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is OMB-NEW.  The time required to complete this information collection is estimated to average 60 minutes or 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information.

Section 1: Description of State School Nutrition MIS

In this section, we ask some questions about the background of your current school nutrition MIS and the costs associated with acquiring and maintaining it.

The questions are grouped into three areas:

1. Overview –the general description of the type of software/systems you have, including how you acquired them, the functional modules available, and how long it took to conceive, develop and implement your school nutrition MIS.

2. Costs –the expenditures associated with the acquisition and ongoing operations of your school nutrition MIS and any planned upgrades.

3. Challenges and Future Plans – the extent to which your current MIS meets your needs and plans to upgrade or replace it.

Overview

We will start by asking you some general questions about your school nutrition state agency and the tools you use to manage your operations.

1. In general, how do you manage your state agency’s school nutrition program data? Remember that use of spreadsheet applications such as Microsoft Excel is not considered an electronic MIS for our purposes. Check all that apply.

	Custom software developed in-house by our state or acquired from another state	Q3
	Custom software developed by a vendor (not including commercial off-the-shelf [COTS] software that is later customized)	Q3
	Software purchased or licensed from a vendor (including commercial off-the-shelf [COTS] software that is later customized)	Q3
	We use basic spreadsheets such as Microsoft Excel	Q2
	We collect and store the data manually on paper	Q2

2. Why do you not use an electronic MIS to manage at least part of your school nutrition program data? Check all that apply.

	The available software is too expensive	END
	We cannot find a product that will work for us	END
	We do not have the staff to develop one	END
	We do not have staff trained to use the software	END
	We tried in the past, but it didn’t work for us	END
	Other reason (please specify) ____________________________________	END

3. How many individuals employed in your state school nutrition operations are users of your school nutrition MIS? (Please include all individuals who enter, view, download, or upload data and/or reports.)

Number of users: _______________________________________

4. Which of the following modules do you have in your current school nutrition MIS? Please indicate the vendor of the modules you have and approximately how long you have had them.

Module	Have	Vendor/Developer	No. of Years
Administrative Review	 Yes  No	{Select from dropdown list} Ardent Technologies, Inc. ArnAmy, Inc. Colyar Technology Solutions Cybersoft Deloitte Dynamic Internet Solutions (DIS) Estrada Consulting, Inc. Heartland (Nutrikids) Hupp Information Technologies Image API Informatica Corporation Information Research Group, Inc. ISF Kyran Northrop Grumman/Metis Peoplesoft Financials The Canton Group Vision Software Other (please specify)_____________	{Select from dropdown list}  Less than 1 year  1 year to less than 4 years  4 years to less than 7 years  7 years to less than 11 years  11 years to less than 15 years  15 years or more
Sponsor Application and Record Management	 Yes  No
Financial Management – School Food Authority (SFA) School Nutrition Revenues and Expenditures	 Yes  No
Financial Management - State Administrative Expense (SAE) Funds	 Yes  No
SFA Claims and Reimbursement	 Yes  No
Direct Certification (matching)	 Yes  No
Online Applications (free/reduced price)	 Yes  No
Menu Planning	 Yes  No
Nutrient Analysis	 Yes  No
USDA Foods Utilization Tracking	 Yes  No
Professional Standards Training	 Yes  No
Federal Reporting	 Yes  No
Other (please specify)________________	 Yes  No

5. Please tell us if your needs for the following functions are fully met, partially met, or not met at all by your current school nutrition MIS.

Module {Populate with Modules Selected in Q4}	Fully met	Partially met	Not met at all
Administrative Review			
Sponsor Application and Record Management			
Financial Management - SFA School Nutrition Revenues and Expenditures			
Financial Management - State Administrative Expense (SAE) Funds			
SFA Claims and Reimbursement			
Direct Certification (matching)			
Online Applications (free/reduced price)			
Menu Planning			
Nutrient Analysis			
USDA Foods Utilization Tracking			
Professional Standards Training			
Federal Reporting			
Other (please specify)_____________________________			

6. What department within your state primarily manages the day-to-day operations of your school nutrition MIS? By primarily manage we mean responsible for arranging staff access to the system and overseeing contracts with vendor(s) (if applicable). Check only one.

	School Nutrition Department	Q7
	Dedicated IT Department within the State Government	Q7
	Other (please specify) ____________________________________	Q7

Costs

Next, we will ask you questions about the costs of your current school nutrition MIS, including cost to develop, implement, upgrade, and operate your school nutrition MIS and the sources of funds used to finance it. Please estimate costs related to NSLP/SBP operations only.

Initial Development and Upgrade Costs for Current Modules

The first questions focus on the costs your state incurred in procuring or developing your current school nutrition MIS and for any subsequent upgrades. For initial development, please include costs of direct development, software development tools, hardware and any other resources expended in initially developing or purchasing each module. For upgrades, please include costs for changes, other than routine updates, made to the software by the software vendor or your in-house IT staff to either enhance existing functions or add new functions. These are sometimes called “add-on” enhancements and should only include changes procured outside of any maintenance contract(s).

7. Please indicate the cost to initially develop and implement each of the modules in your current school nutrition MIS and for any subsequent upgrades. Please check one response in each column for each module.

Module {Populate with Modules Selected in Q4}	Initial Development and Implementation Costs	Upgrade Costs (If non-routine upgrade(s) occurred after initial implementation)
Administrative Review	 No direct cost to state school nutrition agency (upgrades are made by state IT staff)  Less than $50,000  $50,000 - $99,000  $100,000 - $299,999  $300,000 - $499,999  $500,000 - $699,999  $700,000 - $999,999  $1,000,000 -$4,999,999  $5,000,000 or more  Don’t know	 No upgrades purchased  No direct cost to state school nutrition agency (upgrades are made by state IT staff)  Less than $50,000  $50,000 - $99,000  $100,000 - $299,999  $300,000 - $499,999  $500,000 - $699,999  $700,000 - $999,999  $1,000,000 -$4,999,999  $5,000,000 or more  Don’t know
Sponsor Application and Record Management
Financial Management – SFA School Nutrition Revenues and Expenditures
Financial Management - State Administrative Expense (SAE) Funds
SFA Claims and Reimbursement
Direct Certification (Matching)
Online Applications (free/reduced price)
Menu Planning
Nutrient Analysis
USDA Foods Utilization Tracking
Professional Standards Training
Federal Reporting
Other (please specify)________________________

8. How did you fund the initial development and implementation of your current school nutrition MIS? Check all that apply.

	State agency general funds	Q9
	Federal State Administrative Expense (SAE) funds – annual allocation	Q9
	Federal State Administrative Expense (SAE) funds – reallocation request	Q9
	Federal education funds	Q9
	Other federal funds (please specify) _______________________________	Q9
	Other grant monies (please specify) _______________________________	Q9
	Other source (please specify) _______________________________	Q9
	Don’t know	Q9

9. Generally, how often do you upgrade your school nutrition MIS modules (both in-house and/or via software purchased from a vendor)? Check all that apply.

	On an as needed basis	Q10
	Quarterly	Q10
	Semi-annually	Q10
	Annually	Q10
	Other (please specify) ________________________________________________	Q10
	Don’t know	Q10
	Never upgraded	Q10

10. For the components of your current school nutrition MIS that are provided by a vendor, do you own or lease (pay regular license fees for) them? Check all that apply.

	We own the software (i.e., own the copyright or equivalent)	Q11
	We have a renewable license to use the software	Q11
	We have no MIS components provided by a vendor	Q11
	We use open source software (software available for free to anyone)	Q11
	Other (please specify) __________________________________________	Q11

Operating Costs

Next, we will ask you about the costs of operating and maintaining your school nutrition MIS. ”Operating” refers to the regular use of your MIS to manage your school nutrition activities. “Maintaining” refers to scheduled or ad-hoc activities you or your vendor perform on your school nutrition MIS to ensure optimal performance. Please estimate costs related to NSLP/SBP operations only.

11. You indicated that you purchased all or some of your current school nutrition MIS from a vendor. Do you have a maintenance contract with your vendor(s)? Check only one.

	Yes, our state has maintenance contract(s) with all of our vendor(s)	Q13
	Yes, our state has maintenance contract(s) with some of our vendor(s)	Q12
	No, we have no maintenance contracts with any of our vendors	Q12

12. You indicated that you do NOT have a maintenance contract with at least one of your current school nutrition MIS vendor(s). How do you update that vendor’s MIS? Check all that apply.

	We make changes using in-house IT staff	Q13
	We make changes using contracted software developers	Q13
	We do not make any changes to the school nutrition MIS	Q13
	Other (please specify) _________________________________________________	Q13

13. What support functions does your maintenance contract(s) cover? Check all that apply.

	Regular updates to conform with FNS policies	Q14
	Other updates to system functionality or features	Q14
	Patches and fixes to problems	Q14
	Training (remote, on- or off-site)	Q14
	Backup and recovery	Q14
	In-person software support	Q14
	In-person hardware support	Q14
	Remote software support	Q14
	Technical support via telephone or online	Q14
	Other (please specify) ______________________________	Q14
	Don’t know	Q14

14. Turning now to annual school nutrition MIS operational costs, for each of the following, indicate the cost to operate your current school nutrition MIS in the 2015-16 school year. Please enter $0 if there was no cost for a particular item.

Operational cost item	Annual cost ($)
Maintenance contract fees
Hardware replacement (If you update hardware less frequently than once per year, please estimate the per year cost of those expenses)
License fees to use school nutrition software	{If Q10 answer choice 2 selected, this should not equal $0}
Other (please specify) _____________________________________

Future Plans

Next, we will ask you about plans and expectations you may have for replacing your school nutrition MIS in the next 12 months. When answering questions on plans over the next 12 months, please exclude updates made to your school nutrition MIS to comply with policy updates by FNS. Policy updates refer to the regulations or policy memoranda that FNS occasionally issues to change requirements or provide guidance on the administration of the school nutrition programs.

15. Are you expecting to make non-routine upgrades or expansions of your current school nutrition MIS in the next 12 months? Check all that apply.

	Purchase new modules	Q16
	Perform software upgrades	Q16
	Perform hardware upgrades	Q16
	Add new sites to the system	Q16
	Other changes (please specify) __________________________________	Q16
	No upgrades or expansions expected	Q16

16. When do you expect to replace all or part of your current school nutrition MIS? Check only one.

	In less than 1 year	Q17
	1 year to less than 3 years	Q17
	3 years to less than 6 years	Q17
	6 years or more	Q17
	No current replacement plans	Q19
	Don’t know	Q19

17. You indicated that you expect to replace all or part of your current school nutrition MIS in the future. How do you anticipate acquiring a new MIS or a module? Check all that apply.

	Develop customized MIS using in-house resources	Q18
	Develop customized MIS using a vendor	Q18
	Purchase a new MIS product from a vendor	Q18
	We have not yet made a decision	Q18
	Other (please specify) _________________________________________	Q18
	Don’t know	Q18

18. How much do you anticipate it will cost to replace all or part of your current school nutrition MIS? Include all costs such as personnel, software license, and hardware. Check only one.

	Less than $50,000	Q19
	$50,000 - $99,000	Q19
	$100,000 - $249,000	Q19
	$250,000 – $499,999	Q19
	$500,000 - $999,999	Q19
	$1,000,000 - $2,999,999	Q19
	$3,000,000 or more	Q19
	Don’t know	Q19

Section 2: Data Elements and Reporting

Section 2 of the survey focuses on the types of data or data “elements” that are available in your current school nutrition MIS. Some of these are data you report out to FNS but others may not be.

We have grouped the data into five categories.

1. Reimbursement claims

2. Free and Reduced Price Eligibility Determination and Verification

3. Meal Service and Nutrition Information

4. Financial Management

5. Other Data Elements

In addition to asking whether the data are in your school nutrition MIS, we also want to know how frequently you collect them, the level of detail (either SFA or school/site) you have, and whether you generate reports with the data.

Claims for Reimbursement

First, we will ask you about the capabilities of your current school nutrition MIS in helping you manage reimbursement claims.

19. Does your current school nutrition MIS collect the following data elements reported to FNS on the Form FNS-10, Report of School Program Operations? (References below are to the cells of the Form FNS-10). Please choose Frequency of collection and Finest level of detail for all data available in your school nutrition MIS. Please check all that apply.

Data	Data are available in MIS	Frequency of collection (Please select most frequent level for entering data)	Finest level of detail available (Choose one)
Meal counts for free, reduced price, paid, and total lunches, breakfasts and afterschool snacks (Part A, 5a, A-D)	 Yes  No  Not Applicable	 Daily  Monthly  Other	 Student  Site or School  SFA  Other ___
Average Daily Meals counts for lunches, breakfasts, severe need breakfasts, and afterschool snacks (Part A, 5a-E, 5c-E, 6E, 7E)	 Yes  No  Not Applicable	 Daily  Monthly  Other	 Student  Site or School  SFA  Other ___
Number of lunches served in school food authorities (SFAs) that qualify for additional payment (two-cent differential) (Part A, 5b1, D)	 Yes  No  Not Applicable	 Daily  Monthly  Other	 Student  Site or School  SFA  Other ___
Number of lunches served in school food authorities (SFAs) certified for performance based reimbursement (eligible for $0.06) (Part A, 5b2, D)	 Yes  No  Not Applicable	 Daily  Monthly  Other	 Student  Site or School  SFA  Other ___
Number of public school sites operating NSLP, SBP, and afterschool snacks and their enrollment (October report, Part B, 12a+b, F-J)	 Yes  No  Not Applicable	 Daily  Monthly  Annual  Other	 Student  Site or School  SFA  Other ___
Number of private school sites operating NSLP, SBP and afterschool snacks and their enrollment (October report, Part B, 13a+b, F-J)	 Yes  No  Not Applicable	 Daily  Monthly  Annual  Other	 Student  Site or School  SFA  Other ___
Number of residential child care institution (RCCI) sites operating NSLP, SBP and afterschool snacks and their enrollment (October report, Part B, 14a+b, F-J)	 Yes  No  Not Applicable	 Daily  Monthly  Annual  Other	 Student  Site or School  SFA Student  Other ___
Total number of children approved for free lunches and reduced-price lunches (October report, Part B, 15a+b, F)	 Yes  No  Not Applicable	 Daily  Monthly  Annual  Other	 Student  Site or School  SFA  Other ___

Free and Reduced Price Eligibility Determinations and Verification

Next, we will ask you some questions about the capabilities of your current school nutrition MIS in helping your state manage free and reduced price eligibility determinations and verification.

20. Does your school nutrition MIS collect FNS-742 (School Food Authority Verification Collection Report) data at the site/school level?

	Yes	Q21
	No	Q21

21. Does your school nutrition MIS collect FNS-874 (formerly 742a) (Local Educational Agency Second Review of Applications Report) data at the site/school level?

	Yes	Q22
	No	Q22

22. Does your current school nutrition MIS collect and maintain a list of schools/group of schools/SFAs operating the Community Eligibility Provision (CEP) and their Identified Student Percentages (ISP) or claiming percentages, and enrollment? Check all that apply for each item.

Operating Provision	MIS collects this data	Available for reports	Not applicable, no CEP schools in our state
List of schools operating CEP			
Claiming percentages or identified student percentages (ISP) for schools operating CEP			
Enrollment for schools operating CEP			

23. Does your current school nutrition MIS collect and maintain a list of schools operating Provision 2 and schools operating Provision 3 and their claiming percentages and enrollment? Check all that apply for each item.

Operating Provision	We collect this data	Available for reports	Not applicable, no Provision 2 or 3 schools in our state
List of schools operating Provision 2			
Claiming percentages for schools operating Provision 2			
Enrollment for schools operating Provision 2			
List of schools operating Provision 3			
Claiming percentages for schools operating Provision 3			
Enrollment for schools operating Provision 3			

Meal Service and Nutrition Information

The next set of questions focuses on the data in your school nutrition MIS related to meal service and nutrition. We are interested in how you collect and store information on meal pattern component compliance and dietary specification compliance by your school nutrition sites.

24. Does your current school nutrition MIS maintain data on SFA meal pattern component compliance? This includes documentation that shows meal components and quantities by grade level, such as detailed menus and/or a completed FNS-approved menu planning tool. Check all that apply.

	Yes, SFAs upload or enter this data electronically into the state MIS	Q25
	Yes, SFAs manually enter this data into an online form	Q25
	No, we do not collect information on meal pattern component compliance in our MIS	Q26

25. What is the frequency of data collection on SFAs’ meal pattern component compliance? Check all that apply.

	Annually	Q26
	During Administrative Review	Q26
	On an as needed basis	Q26
	Other (please specify) __________________________________________	Q26

26. Does your current school nutrition MIS maintain data on SFA dietary specification compliance? These data include documentation that shows results from a nutrient analysis or simplified nutrient assessment for calories, saturated fat, sodium and trans fat using USDA approved software. Check all that apply.

	Yes, SFAs upload or enter this data electronically into the state MIS	Q27
	Yes, SFAs manually enter this data into an online form	Q27
	Yes, we enter or collect this information during administrative review	Q27
	No, we do not collect information on dietary specification compliance in our MIS	Q28

27. What is the frequency of collection of data on SFA dietary specification compliance? Check all that apply.

	Annually	Q28
	During administrative review	Q28
	On an as needed basis	Q28
	Other (please specify) __________________________________________	Q28

28. Do you have an automated report in your school nutrition MIS for SFA compliance with meal components and dietary specifications? At a minimum, this includes a list of SFAs and schools receiving the additional 6 cent reimbursement.

	Yes	Q29
	No	Q29

Financial Management

Next, we will ask you about the types of financial data your school nutrition MIS collects and generates. Many of these questions focus on data that you collect from SFAs in your state.

29. Does your current school nutrition MIS collect an Annual Financial Report from each SFA in the state? This report is also called a Balance Sheet or Statement of Revenues and Expenditures and contains information on the SFA’s revenues, expenditures and ending net cash resources (account balance) for a school year.

	Yes	Q30
	No	Q30

30. For each of the following types of SFA revenue data, please indicate whether the data element is available in your school nutrition MIS and, if so, the frequency with which it is collected from SFAs, level of detail, and whether reports are generated from the data.

Types of SFA revenue data	Data are available in MIS	Frequency of collection (Please select most frequent level for entering data)	Finest level of detail available	Internal management reports available from these data
USDA Meal Reimbursements	 Yes  No	{Choose one from dropdown box} Daily Weekly Monthly Quarterly Annually Other___	{Choose one from dropdown box} Site/School SFA	 Yes  No
Student Meal Payments	 Yes  No			 Yes  No
Adult Meal Revenues	 Yes  No			 Yes  No
State/local Meal Reimbursements	 Yes  No			 Yes  No
USDA Foods Monetary Value	 Yes  No			 Yes  No
Non-Program Food Sales, including A La Carte Sales and Vending	 Yes  No			 Yes  No
Contracted/Catering Sales	 Yes  No			 Yes  No
Other State Funding	 Yes  No			 Yes  No
Funds Transferred in	 Yes  No			 Yes  No
Grants (state or federal)	 Yes  No			 Yes  No
Interest	 Yes  No			 Yes  No
Other (please specify) ______________	 Yes  No			 Yes  No
Other (please specify) ______________	 Yes  No			 Yes  No
Other (please specify) ______________	 Yes  No			 Yes  No

31. For each of the following types of SFA expenditure data, please indicate whether the data element is available in your school nutrition MIS and, if so, the frequency with which the data that is entered is collected from SFAs, level of detail, and whether reports are generated from the data.

Types of SFA expenditure data	Data are available in MIS	Frequency of collection (Please select most frequent level for entering data)	Finest level of detail available	Internal Management reports available from these data
Purchased Food	 Yes  No	{Choose one from dropdown box} Daily Weekly Monthly Quarterly Annually Other ___	{Choose one from dropdown box} Site/School SFA	 Yes  No
Food production and operating supplies	 Yes  No			 Yes  No
USDA Foods Miscellaneous Charges	 Yes  No			 Yes  No
Labor Costs (salaries and benefits)	 Yes  No			 Yes  No
Equipment Purchases (>$5000)	 Yes  No			 Yes  No
Purchased Services	 Yes  No			 Yes  No
Indirect Cost Charges	 Yes  No			 Yes  No
Funds Transferred out	 Yes  No			 Yes  No
Other (please specify) _____________	 Yes  No			 Yes  No
Other (please specify) _____________	 Yes  No			 Yes  No

32. Does your current school nutrition MIS collect SFA meal prices charged? Some meal price information is reported annually on the FNS School Food Authority Paid Lunch Report Form FNS-828.

	Yes	Q33
	No	Q34

33. How does your current school nutrition MIS collect SFA meal prices charged?

	Most frequently charged price by SFA elementary, middle and high school levels	Q34
	Price by individual school	Q34

34. What types of performance and operating efficiency measures does your current school nutrition MIS collect from SFAs? Please choose frequency of collection and finest level of detail for all data available in your school nutrition MIS.

Performance and operating efficiency measures	Data to calculate these measures are available in MIS	Calculated measure is available in MIS	Frequency of collection (Please select most frequent level for entering data)	Finest level of detail available
Meals per labor hour	 Yes  No	 Yes  No	{Choose one from dropdown box} Monthly Quarterly Annually Other	{Choose one from dropdown box} Site/School SFA
Meal equivalents (e.g., ratio of # breakfasts to lunch)	 Yes  No	 Yes  No
Percentage of certified students served by meal type (Free/Reduced Price/Paid)	 Yes  No	 Yes  No
Percentage of Meals Reimbursed by Category (Free/Reduced Price/Paid)	 Yes  No	 Yes  No
Targeted maximum cost per meal (“plate costs”)	 Yes  No	 Yes  No
Other (please specify) ____________	 Yes  No	 Yes  No
Other (please specify) _____________	 Yes  No	 Yes  No

Other Data Elements

The next set of questions on data elements do not fall into any of the previous categories. They focus on data available on other administrative activities.

35. How does your state use school nutrition MIS to conduct Administrative Reviews of SFAs? Please indicate to what extent your current school nutrition MIS captures each of the following administrative review tools and forms. Check only one per row.

Administrative Review tool	MIS FULLY automates data collection and processing	MIS PARTIALLY automates data collection and processing	Forms from FNS website are used (entered manually outside of MIS)	Not applicable to our state
Eligibility Certification and Benefit Issuance Error Worksheet (SFA-1)				
Special Provisions Non-Base Year and CEP Claiming Percentage/Funding Level Summary Form (SFA-1A)				
Other Eligibility Certification and Benefit Issuance Errors Worksheet (SFA-2)				
Community Eligibility Provision ISP Validation Form (SFA-2A)				
School Data and Meal Pattern Error Form (S-1)				
Meal Compliance Risk Assessment Tool				
Non-reimbursable Meal Allocation Form				
Nutrient Analysis Protocols				
Nutrient Analysis and Validation Checklist				
Dietary Specifications Assessment Tool				
Off-site Assessment Tool				
On-site Assessment Tool				
Resource Management Comprehensive Review Form				
Resource Management Risk Indicator Tool				
Fiscal Action Workbook				
General Areas (i.e., Professional Standards, Civil Rights, Local Wellness, etc.)
Supplemental SFAs Contracting with Food Service Management Companies Review Form				

36. What types of professional standards and training data does your current school nutrition MIS collect? Please choose Frequency, Finest level of detail and if you have internal management reports for all data available in your school nutrition MIS.

Professional standards and training data	Data are available in MIS	Frequency of collection (Please select most frequent level for entering data)	Finest level of detail available	Internal management reports available from these data
Training hours offered	 Yes  No	{Choose one from dropdown box} Daily Monthly Quarterly Annually Other	{Choose one from dropdown box} Individual SFA State	 Yes  No
Individual state employee training hours tracking	 Yes  No			 Yes  No
Number of state employees meeting professional standards	 Yes  No			 Yes  No
SFA employee training hours tracking	 Yes  No			 Yes  No
Number of SFA employees meeting professional standards	 Yes  No			 Yes  No
Education/experience level for SFA directors	 Yes  No			 Yes  No
Other (please specify) ______________	 Yes  No			 Yes  No
Other (please specify) _______________	 Yes  No			 Yes  No
Other (please specify) _______________	 Yes  No			 Yes  No

37. What types of school food safety inspection data does your current school nutrition MIS collect? Please choose frequency of collection, finest level of detail, and if you have internal management reports for all data available in your school nutrition MIS.

    School food safety inspection data	Data are available in MIS	Frequency of collection (Please select most frequent level for entering data)	Finest level of detail available	Internal Management reports available from these data
    Number of inspections	 Yes  No	{Choose one from dropdown box} Monthly Quarterly Annually	{Choose one from dropdown box} Site or School SFA State	 Yes  No
    Results of food safety inspections (e.g., scores or pass/fail)	 Yes  No			 Yes  No
    Other (please specify) ___________	 Yes  No			 Yes  No
    Other (please specify) ___________	 Yes  No			 Yes  No

38. Do any SFAs in your state have contracts with food service management companies (FSMCs)?

	Yes	Q39
	No	Q40

39. What types of FSMC contracts data does your current school nutrition MIS collect? Please choose frequency of collection, finest level of detail, and if you have internal management reports for all data available in your school nutrition MIS.

FSMC contracts data	Data are available in MIS	Frequency of collection (Please select most frequent level for entering data)	Finest level of detail available	Internal management reports available from these data
Data on FSMC contract details (e.g., name of FSMC, year of contract, fixed price vs. cost reimbursable)	 Yes  No	{Choose one from dropdown box} Monthly Quarterly Annually Other	{Choose one from dropdown box} Site or School SFA State	 Yes  No
Copies of contracts between SFAs and FSMCs	 Yes  No			 Yes  No
Data to support monitoring of SFA contracts (e.g. tracking receipt of rebates, discounts and credits)	 Yes  No			 Yes  No
Data and logs to support bids/procurement of FSMC services	 Yes  No			 Yes  No
Other (please specify) ________________	 Yes  No			 Yes  No
Other (please specify) ________________	 Yes  No			 Yes  No

Reporting Capabilities and Data Quality

The next questions focus on the reporting capabilities of your current school nutrition MIS and the quality of the data in the MIS. We also ask about the processes you have within your school nutrition MIS to ensure that data is accurate (e.g., edit checks) and has not been tampered with or manipulated (data integrity).

40. Which of the following types of analysis and tracking of data are you able to perform using your current school nutrition MIS? Check all that apply.

	Standard, summary reports that came with the software	Q41
	Standard, detailed reports that came with the software	Q41
	Customized reports by choosing data categories	Q41
	Customized reports by writing code such as structured query language (SQL), Crystal Reports, etc.	Q41
	Management dashboards or interfaces that provide visual analysis of data or key indicators	Q41
	Scorecards that show progress of specified measures against their targets	Q41
	Other reports (please specify) ________________________________________________	Q41
	Our school nutrition MIS does not perform analysis and tracking of data	Q41

41. What data quality and integrity checks are built into your current school nutrition MIS? Check all that apply.

	Incomplete applications	Q43
	Over-claims (for example, number of meals claimed greater than student enrollment)	Q43
	Duplicate students (for example, same student claimed more than once for same meal service)	Q43
	Required claim edit checks	Q43
	Reasonableness of data (for example, system only accepts values within an anticipated range)	Q43
	Audit logs, which record all activities of users	Q43
	Reports with missing data are rejected	Q43
	Checks to ensure data are not manipulated outside of the software	Q43
	Other (please specify) ______________________________________________________	Q43
	Other (please specify) ______________________________________________________	Q43
	Other (please specify) ______________________________________________________	Q43
	Don’t know/Unsure	Q42
	Our school nutrition MIS does not provide data quality and integrity checks	Q42

42. You indicated that your current school nutrition MIS does not provide data quality and integrity checks or you are unsure if it does. How do you ensure that your data is accurate? Check all that apply.

	Manually check all reports	Q43
	Manually check randomly selected reports	Q43
	We don’t check data accuracy	Q43
	Other (please specify) _________________________________________________	Q43
	Don’t know	Q43

43. How does your current school nutrition MIS support oversight, monitoring and accountability? Check all that apply.

	Real-time access to SFA claiming systems	Q44
	Custom reports on missing data	Q44
	Custom reports on late data	Q44
	Integrated address book/email system or messaging system to communicate with SFAs on data issues	Q44
	Ability to design and test new components to improve agency protocols	Q44
	System of logs and checks to identify error-prone SFAs	Q44
	Criteria and reports to identify high-performing SFAs	Q44
	Edit checks integrated into SFA data entry forms	Q44
	Edit checks integrated into State reporting forms	Q44
	Trend analyses to compare data received to historical trends.	Q44
	Other (please specify) _________________________________________________	Q44

44. How would you rate the quality of the following types of data in your current school nutrition MIS? For each data type, please select one of the four quality categories that best describes your quality rating of the data available in your school nutrition MIS.

Data type	Quality
Administrative Review Preparation	[Excellent] Meets or exceeds all our needs [Good] Meets most of our needs [Adequate] Meets some of our needs [Poor] Meets few, if any, of our needs Not Applicable
Meal Claiming
Menu Planning
Nutrient Analysis
Order Processing
Recipes
FNS 10 Report of School Program Operations
FNS 742 Report (Verification Collection Report)
FNS 834 Report (State Agency (NSLP/SNAP) Direct Certification Rate Data Element Report)
FNS -640 Report (Administrative Review Data Report)
FNS-874 Report (Local Education Agency Second Review of Applications)
Other

45. In using your school nutrition MIS, do you face any of the following challenges concerning data collection? Check all that apply.

	The level of detail does not meet our needs	Q46
	The data is not adequately secured or protected from external manipulation	Q46
	The level of security makes it difficult to use the system	Q46
	Not all users are adequately trained to use the system	Q46
	Data is collected to fulfill an old FNS report form and there is too much lag time to reprogram the system to meet the requirements of the new FNS report form	Q46
	Other (please specify) _______________________________________	Q46
	I do not face any challenges with data collection	Q46

46. In using your school nutrition MIS, do you face any of the following challenges concerning data aggregation? Check all that apply.

	Our data are spread over disparate systems, making data aggregation time consuming	Q47
	The data are too fine-grained, making it difficult to aggregate them and produce reports	Q47
	Compiling all or some of our data requires manual data re-entry tasks	Q47
	Other (please specify) _______________________________________	Q47
	I do not face any challenges with data aggregation	Q47

47. In using your school nutrition MIS, do you face any of the following challenges concerning reporting? Check all that apply.

	Reports do not cover all topics relevant to management of the program	Q48
	Reports are not available at the level of detail needed to support program management	Q48
	All or some of the reports to the FNS involve manual compilation	Q48
	Data is reported in a format to fulfill an old FNS report form and there is too much lag time to reprogram the system to meet the requirements of the new FNS report form	Q48
	Other (please specify) _______________________________________	Q48
	I do not face any challenges with reporting	Q48

Section 3: Technical Features of School Nutrition MIS

This section contains questions about the technical features of your current school nutrition MIS. In particular, we will cover the following topics:

1. Copyright Protection - Copyright protection for your custom-built software

2. Interoperability – How your system connects and shares data with other systems

3. Architecture and Deployment – How you access and store data within your system

4. Data Security – How your system performs backups and secures sensitive student data

Copyright Protection

The following questions focus on copyright protection for your in-house or custom developed software.

48. You indicated that a component of your school nutrition MIS was custom-built. Who owns the copyright? Check all that apply.

	The state	Q49
	The developer/vendor	Q49
	Other (please specify) _________________________________	Q49
	No copyright	Q49
	Don’t know	Q49

49. Can your current school nutrition MIS be transferred with no or minimal costs to other state school nutrition agencies? This may be possible if your MIS was purchased using federal funds and your agency owns the copyright.

	Yes	Q50
	No	Q50
	Don’t know	Q50

Interoperability

We will now ask about interoperability, which is how your MIS connects to other data systems within your state.

50. Do you share system login capability with SFAs?

    	Yes, SFAs can login to the State system	Q51
    	Yes, the State agency can login to SFA systems	Q51
    	No	Q51
    	Don’t know	Q51

51. How do you share data with your SFAs/sites? An example of data sharing would be how direct certification match files are shared with or incorporated by the appropriate LEAs. Check all that apply.

	Manually key in data to SFA systems	Q52
	File import/export stored on a secure server	Q52
	File import/export using removable media such as flash drive, CD, DVD, etc.	Q52
	SFAs connect directly to the state system to retrieve or upload data	Q52
	Other collaborative platforms such as SharePoint	Q52
	Some other means (please specify) _______________________________	Q52

52. Is your school nutrition MIS part of a larger state MIS?

	Yes	Q53
	No	Q54
	Don’t know	Q54

53. Does your state MIS offer single sign-on capability for your school nutrition MIS? Single sign-on allows users to access multiple systems with one username and password.

	Yes	Q54
	No	Q54
	Don’t know	Q54

54. Do you share data (send to or receive from) with any of the following programs or agencies? Check all that apply.

	SNAP State or Local Agencies	Q55
	TANF State or Local Agencies	Q55
	Foster Care/Department of School Children and Family Services	Q55
	State Medicaid agency	Q55
	Other (please specify) ____________________________________________	Q55
	We do not share data with any other agency	Q56

55. How do you share data with the programs/agencies you selected earlier?

	Manually key in data to other program/agency systems	Q56
	File import/export stored on a secure server	Q56
	File import/export using removable media such as flash drive, CD, DVD, etc.	Q56
	Connect directly to download or upload data	Q56
	Other collaborative platforms such as SharePoint	Q56
	Other means (please specify) _______________________________	Q56

56. Do you or your software vendor implement any interoperability standards within your school nutrition MIS? “Interoperability standards” refers to following third-party defined protocols to support efficient interactions between software/data systems. Check all that apply.

	School Interoperability Framework (SIF)	Q57
	Ed-Fi Alliance	Q57
	Common Education Data Standards (CEDS)	Q57
	Other standards _________________________	Q57
	Part of a standard but do not know the name	Q57
	Not part of any standards	Q57
	Don’t know	Q57

Architecture and Deployment

The next questions focus on how your school nutrition MIS is physically deployed and operated. In particular, FNS is interested in whether your data are stored locally or in the cloud.

57. Where are the data from your current school nutrition MIS stored? Check all that apply.

	At a central state location	Q58
	In the cloud, hosted by the vendor	Q58
	Other (please specify) ______________________________________________________	Q58
	Don’t know	Q58

58. What type of database management system does your school nutrition MIS use to store data? Check all that apply.

• Oracle Q59

• Microsoft SQL Q59

• MySQL Q59

• PostgresSQL Q59

• IMB DB2 Q59

• IBM Informix Q59

• SAP Sybase Q59

• Teradata Q59

• Non-relational database (e.g. MonoDB, Cassandra, HBase, Neo4j) Q59

• Other (please specify) _______________________________ Q59

• Don’t know Q59

Data Security

The next questions focus on your backup and data security systems and protocols. By protocols, we mean the set of rules and procedures that govern backup and security of your school nutrition MIS.

59. How long do you retain data in your current school nutrition MIS? By retain we mean that the data are electronically available in your system without the need to restore the data from an archive. Check only one.

	Just the current school year	Q60
	1 year to less than 4 years	Q60
	4 years to less than 7 years	Q60
	7 years to less than 10 years	Q60
	Indefinitely	Q60
	Other time period (please specify)________________________________	Q60

60. Do you have any of the following security protocols to protect the data in your current school nutrition MIS, specifically those related to personally identifiable information? Check all that apply.

	Login requirements	END
	Data encryption	END
	Confidential data maintained in a secure environment	END
	User profiles with access to only specific data	END
	Virus protection	END
	Security penetration testing	END
	Virtual private network (VPN)	END
	Training for users to protect from phishing (i.e. email scams)	END
	Other (please specify) __________________	END
	Other (please specify) __________________	END
	Other (please specify) __________________	END

Thank you for taking the time to complete this survey. As we noted at the beginning, the information you provided will be used only for the purposes of this study.

Please provide the following information about any staff members who have assisted in completing this survey.

	Job Title		# of Years at State agency
1.
2.
3.
4.

If you have any questions about completion of this survey please contact Madeleine Levin, at 202-774-1982 or [email protected].