NASS Comments

Responses to Comments from NASS

Special Nutrition Program Operations Study (SN-OPS)

We have carefully reviewed the comments and suggestions. Editorial comments not specific to the content of the study were taken into consideration and incorporated where necessary. Below we provide a responses to each comment and note changes to the surveys.

Part A

1. What is FNS 742?

FNS-742 refers to the data submitted by all SFAs each year on Verification Summary Report.

2. What does FRN stand for? What publication are you citing here?

FRN stands for Federal Register Notice. This sentence has been deleted.

3. Did you publish this [SNDA Study]? If so, update the language and footnotes to reflect that. If not, give the updated target publication date.

Yes, the results of the SNDA-IV were published in November 2012. We have updated the text to reflect this.

4. Do have an alternate start date if OMB approval does not come through before March 1?

Yes, we have revised the schedule and text to reflect an April 15, 2013 start date.

5. Please include a copy of the reminder email. How do you plan to contact respondents for whom you have no valid email address?

We have included a copy of the reminder email which will be sent to State Child Nutrition Directors. It appears as Appendix D2.

6. What happens in Week 1?

During Week 1 the invitation letters are mailed to all sampled SFA Directors. We respond to all questions received via the hotline or by email regarding the survey.

7. Are these the same emails and postcards that are sent in Week 2? This is the third week you say you are sending out post cards and emails, not the second. Are these the same as those sent out in Week 2 or Week 3?

We have clarified the schedule. One week after the initial mailing, during Week 2 of the data collection, we will send out reminders either by email or regular mail depending on whether we have valid email addresses for the SFA Director. During Week 3 we will we will send another round of reminder emails and postcards. During Week 4 we will contact SFA Directors by telephone to answer any questions and remind them about the deadline for completing the survey. During Weeks 5 and 6 we will send reminder emails and postcards to all non-respondents. Additional non-response follow-up procedures will be added as needed.

8. Possibly out of scope for this survey, but have you considered also collecting price data on these food sources?

Thank you for your suggestion. This is out of scope for this study but we will take under consideration for future studies.

9. Can you be more specific about who these people are?

Initial contact for the on-site data collection activities will be with the SFA Director or their designee. We have updated the text to reflect this clarification.

10. Who will be interviewed during these site visits?

During the on-site visits we plan to conduct interviews with the School Cafeteria Managers. We have updated the text to reflect this clarification.

11. What publication does this reference? [5 CFR 1320.5]

The referenced publication is the Code of Federal Regulations, 5 CFR 1320.5. We have updated the text to reflect this clarification.

12. Does the pretest really take an hour longer to complete than the survey?

The additional time reflects followup discussions with pretest respondents to debrief about the survey and their responses and comments.

13. Given that this survey is ~4x as long as the State CN Director survey, I would expect this survey to take longer to complete.

We have revised the time to complete the SFA Director survey based on the results of the pretest which were not previously available.

14. In the previous table, you said this would take 1.75 hours to complete.

Table A2 has been corrected.

15. Do you intend for the on-site data collection to end on a different day?

We anticipate completing all data collection activities at the same time. The on-site visits must be completed prior to the end of the school year and the survey data must be completed by the end of the fiscal year.

16. Why would you send a reminder notice before you sent out notices to verify if they ever received the survey?

The schedule has been corrected.

17. Is this referring to total number of students? Please be specific.

Yes, the SFA size variable is based on the number of students served by the SFA. We have clarified this in the text.

18. What does CCD stand for?

CCD stands for the Common Core of Data which are data collected by the National Center for Education Statistics (NCES). We have clarified this in the text.

19. What adult meals? The programs in question only serve children.

While it is correct that the NSLP and SBP programs only serve children the SFAs also provide meals to adults. It is important for FNS to understand the prices charged for non-student meals since these meals are not subsidized and should not be less than the cost to produce the meal.

Part B

1. Please define what states are included in each region.

We have added a list of the seven regions and the states within each region.

The seven regions (and states) are: Northeast (CT, ME, MA, NH, NY, RI, VT), Mid-Atlantic (DE, DC, MD, NJ, PA, PR, VA, VI, WV), Southeast (AL, FL, GA, KY, MS, NC, SC, TN), Midwest (IL, IN, MI, MN, OH, WI), Southwest (AR, LA, NM, OK, TX), Mountain Plains (CO, IA, KS, MO, MT, NE, ND, SD, UT, WY), and Western (AK, AZ, CA, GU, HI, ID, NV, OR, WA).

2. How do you define elementary and secondary schools?

We have added the criteria used to define elementary and secondary schools.

Elementary school is defined as any school with any span of grades from kindergarten through grade 6. Middle or junior high school is defined as any school that has no grade lower than grade 6 and no grade higher than grade 9. High school is defined as any school that has no grade lower than grade 9 and continues through grade 12. Schools that do not fit these definitions are categorized as “other.”

3. Please recheck all numbers in this table and in the above paragraph. There seem to be several addition errors.

We have reviewed the table and adjusted the numbers to eliminate the apparent addition errors which were due to rounding.

4. How do you define these school types?

5. Can you be more specific? [about call attempts]

We have clarified that we will make 6 call attempts before we consider whether to treat a case as “unable to contact.”

Appendix A

1. Please include a copy of this questionnaire with your docket.

We have not included a copy of the 2011-12 State Child Nutrition Director Survey because it is already available on-line.

Appendix B

1. Please include a copy of this questionnaire with your docket.

We have not included a copy of the 2011-12 SFA Director Survey because it is already on-line.

2. Isn’t 2.1a included in 2.1(a-e)? Is one of these question numbers a typo?

We apologize for the confusion. Q2.1a was an open-ended question that appeared after Q2.1(a) in the table.

3. Similar to Q5.1 from the 2012-13 questionnaire or from the 2011-2012 questionnaire?

Q6.4 from the 2012-13 questionnaire is a new question although it is similar to Q5.1 from the 2011-12 questionnaire. We have updated the table to reflect this clarification.

Appendix C

1. Does the [local wellness] policy dictate any specific foods cannot be ordered?

We are interested in collecting information about the foods that can be ordered.

2. If they don’t know the current average sodium content, why would they work to reduce it? They don’t know if it’s too high.

While the SFA may not know the exact sodium content most are aware that sodium contents are too high and will work to reduce while also working to determine the exact sodium contents.

3. I thought the programs involved only served meals to children.

Please see response to Q19 under Part A.

4. Why are you asking about this school year? Everything else is addressing either 2011-2012 or 2012-2013.

This research question has been revised to say “Did any schools receive equipment assistance during FY 2009 through FY 2011?” During FY 2009 SFAs may have purchased equipment with American Recovery and Reinvestment Act (ARRA) funds and during FY 2010 or FY 2011 they may have received NSLP Equipment Assistance Grant Funding.

5. Is a district the same as an SFA?

Over 90 percent of the SFAs are synonymous with a school district.

Appendix D

1. Do you want to include the number of schools sampled for on-site visits in this answer?

We have included information about the on-site visits in this response.

2. Please include this in your docket.

We have added the Web Information Sheet: Getting Started with Your SFA Director Survey to the invitation letter packet.

3. Nowhere in this script is the option to complete the survey over the phone. In Part A.2.c you state State Child Nutrition Directors will be offered the option to complete the survey over the phone.

It is possible for the State Child Nutrition Director Survey to be completed by telephone. We have revised the script to offer the option to complete the survey over the telephone. A separate script will be used with SFA Directors since it is not possible for them to answer many of the questions during a telephone interview.

Appendix E: State Child Nutrition Director Survey

The table below summarizes the changes made to the State Child Nutrition Director Survey.

Question #	Comment	Changes or modifications made
Intro	Should “Survey 2012” be “Survey 2012-2013” or “Survey 2011-2012”?	Title page was changed from “Survey 2012” to “Survey SY 2012-2013.”
Intro	In Part A, Table A1, you state the response burden is 1.75 hours, not 30 minutes.	Changed the response burden from 30 minutes to 2 hours.
Intro	Why have a direct line to a Westat employee? Why not provide the toll free number?	Changed the Westat phone number from the direct line of a Westat employee to the toll free number.
A10	What if respondents cannot identify the impact of wellness policies?	Added a “Don’t know” column.
B1a – B1	“Does your state provide” while B1a asks “Did you provide.” Please make the wording consistent.	For consistency, changed from “Does your state provide” in B1 and “Did you provide” in B1a to “Does your state provide” for both B1 and B1a.
B2		Clarified by adding “at the SFA level” to the question stem.
B5	What if the state has been able to fully use funds for one program and not the other?	Added columns, NSLP and SBP, so respondent’s answer whether they were able to fully-use Federal funds for the NSLP and SBP separately
C2	The wording of the question does not match the wording of C1 and does not match the answer chart provided. Delete the stray 1 after the question mark. Is it possible for a school to operate one program under one provision and the other program under a different provision? If so, how would a respondent be expected to fill out this chart?	Changed the wording to match the wording of C1. The wording was changed to, “How many schools in the state are operating the NSLP and/or SBP under each of the following provisions? The stray 1 after the question mark was deleted.
C4a		Clarified that the term charter schools refers to all charter schools including those independent or part of a SFA.
C5a	For line b, would an advisory council consist of SFAs or SFA directors? For line c, do you mean SFA directors?	Changed the wording from “SFAs” to “SFA directors” for response categories a through c.
D1a	What does HACCP stand for?	Clarified by adding “Hazard Analysis and Critical Control Point (HACCP)” in place of HACCP.
D2c	What does NFSMI stand for?	Clarified by adding “National Food Service Management Institute (NFSMI)” in place of NFSMI.
D5a D6a	This question seems more suited to a check all that apply answer style than yes/no.	Based on Westat’s past experience, respondents give less ambiguous answers when the questions force them to answer each item rather than check items that apply.

Appendix F: SFA Director Survey

The table below summarizes the changes made to the SFA Director Survey.

Question #	Comment	Changes or modifications made
Introduction	Title – Should “Survey 2012” be “Survey 2012-2013” or “Survey 2011-2012”?	Changed to Survey SY 2012-2013
Section 1 1.1f	Do you want the grades of each school listed separately or a list of any grade that is served by one of the ‘other schools’?	We want the grades of each school listed separately
1.3	In 1.4 you define a high need school. Do you need to define a severe need school in this question?	Added definition for severe need school
Section 2 2.1a	The wording of the footnote is somewhat confusing. Do you want to include these students in your counts or not?	Revised the footnote. We want to include these students in the counts
2.1d	The wording of the footnote is somewhat confusing. Do you want to include these students in your counts or not?	Revised the footnote. We want to include these students in the counts
2.3d	There is an asterisk but no footnote.	Removed asterisk
2.5	Change “Number of students” to “Number of Reimbursable Breakfasts Claimed” to match the other years’ tables.	Changed.
Section 3	Please specify if you are talking about the current school year.	Added the 2012-2013 school year to the description of the questions.
3.2 3.4	This question seems more suited to a check all that apply answer style than yes/no.	Based on Westat’s past experience, respondents give less ambiguous answers when the questions force them to answer each item rather than check items that apply.
3.5c	Other text field in 3.5.d should be moved here.	Corrected.
3.5d	This option should be deleted.	Deleted option.
3.7	What if some elementary schools have water available and some don’t? How should a respondent complete this chart?	The SFA Directors are asked to respond to questions such as this in general by thinking about all the schools under their SFA
3.7	What should a respondent do if DON’T KNOW to all of 3.7? What is their skip pattern?	Respondent is asked to skip to 3.11
3.8	This question seems more suited to a check all that apply answer style than yes/no.	Based on Westat’s past experience, respondents give less ambiguous answers when the questions force them to answer each item rather than check items that apply
3.12	I think larger response blocks (in the style of 2.9) are needed in this chart to give the respondent room to write numbers.	The response blocks were made bigger so that respondents could write numbers.
	Is it possible for a school to operate one program under one provision and the other program under a different provision? If so, how would a respondent be expected to fill out this chart?	The respondent is able to give the number of schools operating under each provision
3.13	I do not understand how a respondent is supposed to answer this question using this chart. What if, out of 10 schools, 2 were Very Likely, 3 were Very Unlikely, 2 were Don’t Know, and 3 were Likely?	Respondents will be allowed to mark several options for this question.
3.16	What do HHS and CN stand for?	Department of Health and Human Services and Child Nutrition. Spelled out these acronyms in the questionnaire
Section 4	Is it possible for different schools in the same SFA to have different local wellness policies? If so, please consider how this would effect the respondent’s answers for all questions in this section.	Respondents are expected to respond in general for all the schools under their SFAs
4.6	This question seems more suited to a check all that apply answer style than yes/no.	Based on Westat’s past experience, respondents give less ambiguous answers when the questions force them to answer each item rather than check items that apply
4.7	The skip pattern directs respondents to return to Q4.6. What question should they be directed to?	Changed the skip to go to Q4.8
4.12 4.14 4.17	This question seems more suited to a check all that apply answer style than yes/no.	Based on Westat’s past experience, respondents give less ambiguous answers when the questions force them to answer each item rather than check items that apply
Section 5	For the General section, how should a respondent answer if schools reacted in different ways for the given topic?	Respondent will provide an overall response for the district/SFA.
5.3	Change “how students are” to “how students were”.	Changed.
5.6	It seems like the respondent should skip to Q5.8 regardless of their answer.	Added a skip to send those who respond Yes to skip to Q5.8
5.11	It seems like the skip pattern should direct respondents to Q5.13, not Q5.16.	Done- Directed respondents to Q5.13
5.14	It seems like the skip pattern should direct respondents to Q5.16, not Q5.18.	Done- Directed respondents to Q5.16
5.16	The table headers to do match Q5.13. Please make them consistent.	Corrected.
5.26	I don’t understand the skip pattern. If a respondent does not review a label for transfats, isn’t it possible they review a label for other reasons? Thus you would want their answers to Q5.27 and Q5.28.	Took out the skip pattern so that all respondents go to Q 5.27 and 5.28
5.29 5.35 5.42	This question seems more suited to a check all that apply answer style than yes/no.	Based on Westat’s past experience, respondents give less ambiguous answers when the questions force them to answer each item rather than check items that apply
5.42	Given that there are breakout for 5.42.b and 5.42.c, do you want respondents to give an overall answer for Vegetables and Grains or only answer for the breakouts?	We want the respondents to only answer for the breakouts.
5.45	This question seems more suited to a check all that apply answer style than yes/no.	Based on Westat’s past experience, respondents give less ambiguous answers when the questions force them to answer each item rather than check items that apply
Section 6	Instructions – would the person answering this questionnaire necessarily know whether or not their SFA participated in the previous study? Maybe these questions should only appear on surveys sent to SFAs in the new sample and left off the remaining surveys.	Since this is a web survey only the new sample of respondents will get the questions for meal prices for the past 3 years
6.1 through 6.4	Why are you asking about adult breakfasts? Doesn’t the program in question only serve children?	The program serves breakfast to adults as well
6.1	The skip pattern does not seem right. Is it possible an SFA previously had a breakfast program and has since suspended it? Could they have had a program in 2011-2012 and eliminated it in 2012-1013? Wouldn’t you still want their price data for 2011-2012?	We would want the prices for the previous years. However, it is very uncommon for an SFA that previously had a breakfast program to have since suspended it
6.5	Change “which of the following” to “did any of the following”.	Changed.
6.6 through 6.9	Why are you asking about adult lunches? Doesn’t the program in question only serve children?	The program serves lunch to adults as well
6.10	Change “which of the following” to “did any of the following”.	Changed.
6.14.d	Change “did not increase” to “would not have increased”.	Changed.
Section 7 7.1	If they need to report the total of a and b together, should they report the total only in a, only in b, or in both a and b?	They should report the total in a only. Changed the instructions to ask them to respond to a only.
7.1.1.b	What do FFVP and SMP stand for?	Fresh Food & Vegetable Program and School Milk Program. Spelled out these acronyms in the text.
Section 8 8.1		Changed the instruction to align with instruction to question 7.1.
Section 9 9.2 9.3	This question seems more suited to a check all that apply answer style than yes/no.	Based on Westat’s past experience, respondents give less ambiguous answers when the questions force them to answer each item rather than check items that apply
Section 11 11.1	Adjust headers, so all words are visible.	Corrected.
11.2	This question seems more suited to a check all that apply answer style than yes/no.	Based on Westat’s past experience, respondents give less ambiguous answers when the questions force them to answer each item rather than check items that apply
Section 12 12.1	Does it matter what their degree is in?	FNS did not want to know what their degree was in
12.4	Does is matter what their degree is in?	FNS did not want to know what their degree was in

Appendix G: On Site Data Collection

Food Service Manager Interview Form

1. Are you interested in how long each lunch period lasts?

From the information provided we will be able to calculate the length of each lunch period. In addition, FNS collected this information from SFA Directors during the 2011-12 data collection.

Cafeteria Observation Guide

2. Does it matter if students are allowed to eat breakfast somewhere other than where it is served?

We are observing the location that serves the most students and where most of the students eat their breakfast.

3. If students are only served in classrooms, do you want the observer in one of those rooms?

We are only observing the space where the most students eat their breakfast.

4. What if students are allowed to eat in more than one space?

We will only observe one location where most of the students eat their breakfast.

5. That is not the same question as # of tables/benches/chairs. Students are just as likely to sit on the ground as in a chair. What about spaces with no tables/benches/chairs?

We agree students may sit on the ground. We have revised the question.

6. What about students served only in classrooms?

Students may eat in their classrooms but still typically go through a food service line.

7. Fix your question numbering. Also how are you observing students served only in classrooms?

We will observe students eating in their classroom if students eat in more than one location and the classroom serves the most students.

8. What if a food type is not served at breakfast?

We added a “Not Served” column.

9. Does it matter if students are allowed to eat lunch somewhere other than where it is served?

We will observe the location where most students eat their lunch.

10. If students are only served in classrooms, do you want the observer in one of those rooms?

We are only observing the space where the most students eat their lunch.

11. What if students are allowed to eat in more than one space?

Q5 in Section II A will be collected about the space observed.

12. That is not the same question as # of tables/benches/chairs. Students are just as likely to sit on the ground as in a chair. What about spaces with no tables/benches/chairs?

We agree students are likely to sit on the ground and have revised the question.

13. What about students served only in classrooms?

Students may eat in their classrooms but still typically go through a food service line.

14. Do you want an answer option for Only student that bring food from home can use the microwave?

Yes.

15. What about schools that have classes running throughout the time lunch is served?

We have added a clarifying note to the question that the question should be answered relating to the last lunch period.

16. What if all students are served in classrooms?

If all students are served in classrooms and they are roughly of equal size we will randomly select one of the classrooms for the observation.

Other Food Sources Checklist and Vending Machine Checklist

17. Wouldn’t this be included in Juice Drinks?

Juice drinks may contain preservatives, added sugar and water while water with juice does not.

18. This is included in the Other Bread list.

Other breads category only includes hot pretzels. The pretzel category will capture pre-packaged products.

19. Do you mean Canned or Cooked fruit or only fruit that is both Canned and Cooked?

We mean canned or cooked.

The table below summarizes the changes made to the On-Site Data Collection instruments.

Question #	Comment	Changes or modifications made
Food Service Manager Interview
Q13	Q14?	Corrected skip in instruction box to Q14.
Cafeteria Observation Guide
I.A. Q3	If students are only served in classrooms, do you want the observer in one of those rooms?	Revised text to read “If there is more than one location (cafeteria, classroom, etc) in use for breakfast, observe the one that serves the most students. Describe the location observed.”
I.A. Q7	That is not the same question as # of tables/benches/chairs. Students are just as likely to sit on the ground as in a chair. What about spaces with no tables/benches/chairs?	Revised the question to read “How many tables and seats are available?” Added a category for other seating and a followup question about whether students are sitting on the ground.
I.C. Q1	What if a food type is not served at breakfast?	Added a “Not Served” column to the grid.
II.A Q4	If students are only served in classrooms, do you want the observer in one of those rooms?	Revised text to read “If there is more than one location (cafeteria, classroom, etc) in use for breakfast, observe the one that serves the most students. Describe the location observed.”
II.A Q8	That is not the same question as # of tables/benches/chairs. Students are just as likely to sit on the ground as in a chair. What about spaces with no tables/benches/chairs?	Revised the question to read “How many tables and seats are available?” Added a category for other seating and a followup question about whether students are sitting on the ground.
II.B. Q23	Do you want an answer option for Only student that bring food from home can use the microwave? Suggest adding Skip to Q25.	Added a response category to capture when microwaves can only be used by students who bring their lunches from home. We added a skip to Q24 for the response “No microwave.”
II.B. Q28	What about schools that have classes running throughout the time lunch is served?	Revised the question to read “How many minutes prior to the next lunch period does the cafeteria stop serving lunch?”
III. Intro	What if all students are served in classrooms?	Revised text to read “If there is more than one location (cafeteria, classroom, etc) serves reimbursable lunches answer this section about the one that serves the most students.”
Other Food Sources Checklist and Vending Machine Checklist
Q3F	Do you mean Canned or Cooked fruit or only fruit that is both Canned and Cooked?	Changed “Canned, Cooked” to “Canned or Cooked”

20