State CN Agencies (Directors and Key Staff), SFSP Sponsors (Directors and Key Staff), and SFSP Sites (Supervisors) - SLT

APPENDIX C-2. STATE DIRECTOR WEB SURVEY

OMB Control Number: 0584-XXXX

Expiration Date: XX/XX/20XX

INTRODUCTION

[PROGRAMMER: SHOW THIS INTRODUCTION EVERY TIME A USER LOGS IN]

The U.S. Department of Agriculture’s Food and Nutrition Service (FNS) is conducting a study to obtain information about the administration and oversight of the Summer Food Service Program (SFSP) and to identify potential barriers to ensuring the integrity and effective management of the program. FNS hired Westat to conduct the SFSP Integrity Study to describe how different entities (States, sponsors, and sites) administer and oversee the SFSP, any program integrity challenges, and any resources or training that could better support the effective administration and monitoring of the SFSP.

The first part of the SFSP Integrity Study involves a survey of all State agencies that administer SFSP. The survey will answer questions like:

• How do States train sponsors?

• How do States monitor sponsors and sites?

• What integrity challenges do States face?

Your answers are important, and will help FNS understand more about how the SFSP is administered. There are no right or wrong answers. If you are unsure of how to answer a question, please give the best answer you can.

The survey link may be shared, and other staff in your agency may login to complete the survey. However, the State Child Nutrition Director must approve and submit the completed survey. This survey should take no more than 20 minutes to complete.

Participation in this study is voluntary, and the information collected will be used to determine resources, develop training, and provide technical assistance. Refusal to participate will not have any impact on your position, your State agency, or child nutrition programs. You may also skip questions that you do not wish to answer.

Under the Privacy Act of 1974 and the System of Record Notice FNS-8 USDA/FNS Studies and Reports, any personally identifying information obtained will be kept private to the extent provided by law. We will use all data we collect only for the purposes we describe. In the final report we will present the aggregated survey data, and will not link individual States to their responses. The raw survey data will be submitted to FNS at the end of the study for research purposes, and your responses will not impact your agency, the SFSP, or you or your staff in any way.

You may login to the survey link as many times as you wish in order to complete the survey between now and [DATE]. Simply click “Save and Continue Later” at the bottom of the screen before logging out to save your answers. The survey will close on [DATE].

Please answer the questions in the survey based on your State’s current policies or procedures.

If you need additional information, please call [study phone number] or email us at [study email].

Thank you.

SECTION A: STATE AGENCY ADMINISTRATION

A1. Please provide the total number of full-time equivalent (FTE) State-level staff who work on the Child Nutrition Programs, including the SFSP? Use decimals if needed (e.g., 1.5 FTEs).

Note: If your State’s SAE plan is current, please provide the total FTEs listed in the approved SAE plan.

|___| Total FTEs for all Child Nutrition Programs, including SFSP (programmer--allow decimal points)

A2. Please provide the total number of full-time equivalent (FTE) State-level staff who work on the SFSP. Use decimals if needed (e.g., 1.5 FTEs).

Note: If your State’s SAF plan is current, please provide the total FTEs listed in the approved SAF plan.

|___| Total FTEs working on the SFSP (programmer--allow decimal points)

SECTION B: OUTREACH

B1. In preparation for summer 2019, did the State agency attempt to actively recruit new sponsor organizations?

Yes

No  GO TO QUESTION B2

B1a. How did the State agency attempt to recruit sponsors?

SELECT ALL THAT APPLY.

Called potential sponsors

E-mailed potential sponsors

Mailed SFSP information in hard copy to potential sponsors

Collaborated with local advocacy organizations to recruit potential sponsors

Presented SFSP information at relevant conferences and meetings

Fielded Spanish TV and/or radio ads

Fielded English TV and/or radio ads

Other (PLEASE SPECIFY):

B2. In preparation for summer 2019, did the State agency assist sponsors in recruiting potential sites?

SELECT ONLY ONE.

Yes

No  GO TO QUESTION C1

B2a. How did the State agency assist in recruiting potential sites?

Notified area eligible schools

Disseminated information to at-risk afterschool program sites

Disseminated information to potential non-school sites

Worked with State or local advocacy organizations to target outreach

Utilized maps or mapping software to identify eligible areas where sites could be located

Presented SFSP information at relevant conferences and meetings

Fielded Spanish TV and/or radio ads

Fielded English TV and/or radio ads

Other (PLEASE SPECIFY):

SECTION C: TRAINING AND TECHNICAL ASSISTANCE

C1. In preparation for summer 2019, how many annual trainings did the State agency host for SFSP sponsors?

# OF TRAININGS: _______

(programmer: if respondent answers more than “1” to C1, show C1a)

C1a. Please briefly describe the rationale for hosting multiple trainings.

(Examples may include: the State hosts an annual training for returning sponsors and a separate training for first-time sponsors; or the State holds a separate training for each sponsor type (SFAs, private nonprofits, camps, etc.))

_______________________________________________________________________________________________________________________________________________________________________________________________________________________________________

C2. In preparation for summer 2019, when did the State agency provide the annual training(s) for SFSP sponsors?

SELECT ALL THAT APPLY.

(programmer: if respondent answers “1” to C1, they cannot select multiple months here)

January May September

February June October

March July November

April August December

C3. Did you require sponsors to participate in the annual training?

Yes, annual training is required for all sponsors

Yes, annual training is required but only for certain sponsors

Explain:______________________________________

No

C4. Besides sponsor personnel, who else participated in the 2019 annual training?

SELECT ALL THAT APPLY.

Site personnel

Food service management company/vendor representatives

Auditors who work with SFSP

Health inspectors who work with SFSP

Other (PLEASE SPECIFY):

No other entities attended the training

C5. In what format did the State agency provide the 2019 annual training(s) for SFSP sponsors?

SELECT ALL THAT APPLY.

Group training in person

Group online training (e.g., Skype or GoToMeeting)

Self-paced online training (sponsors can participate when it is convenient for them)

Conference calls

Resource materials available on the web

Other (PLEASE SPECIFY):

[If respondent selects multiple responses, go to C6, else skip to C7]

C6. In your opinion, which training format is most effective at preparing sponsors to operate the SFSP?

SELECT ONLY ONE.

Group training in person

Group online training (e.g., Skype or GoToMeeting)

Self-paced online training (sponsors can participate when it is convenient for them)

Conference calls

Resource materials available on the web

Other (PLEASE SPECIFY):

C7. Which of the following topics did the State agency cover in the 2019 annual training for SFSP sponsors?

SELECT ALL THAT APPLY.

Purpose of the SFSP

Sponsor application process

Eligibility requirements for sponsors and sites

Recordkeeping requirements

Meal patterns

Meal service requirements (e.g., congregate feeding, meal service times)

Meal preparation and ordering adjustments

Meal counting

Meal claim preparation

Documentation of nonprofit food service

Food specifications and meal quality standards

Procurement standards, bid requirements, vendor contract requirements

Food safety and health inspection requirements

Incorporating local foods and agriculture-based activities

Site visit requirements

Civil Rights racial/ethnic data collection and nondiscrimination

Guidance on how to train site-level staff

Other (PLEASE SPECIFY):

C8. Did any part of the 2019 annual training include a post-test or other method to verify sponsor comprehension of the information?

Yes

No

C9. Apart from the annual training, did the State agency provide any additional training(s) for SFSP sponsors during 2019?

Yes  GO TO QUESTION C9a

No  GO TO QUESTION C10

C9a. In what format did the State agency provide this additional training for sponsors?

SELECT ALL THAT APPLY.

Group training in person

Group online training (e.g., Skype or GoToMeeting)

Self-paced online training modules (sponsors can participate when it is convenient for them)

Conference calls

Other (PLEASE SPECIFY):

C9b. What was the target audience for those additional trainings?

SELECT ALL THAT APPLY.

First-time sponsors

Sponsors identified as seriously deficient

School food authority (SFA) sponsors

Private nonprofit sponsors

Camp sponsors

Government sponsors (local/municipal/county)

Other (PLEASE SPECIFY):

No target audience, open to all

C10. During summer 2019, which 3 topics did your sponsors have the most questions about for SFSP?

SELECT THE TOP 3 MOST COMMON TOPICS OF INQUIRY.

Sponsor application process

Documenting site eligibility (area eligibility, income applications)

Selection of site location

Completing claims for reimbursement

Menu planning and meal pattern requirements

Estimating yearly projected reimbursement

Recording meal counts

Improving meal quality

Advanced payments

Site outreach/advertising

Financial management and documentation of a nonprofit food service

Correcting program violations

Selecting food service management companies for vended meals

Procurement requirements

Transportation

Other (PLEASE SPECIFY):

SECTION D: PROGRAM RESOURCES

D1. Which of the following SFSP tools and resources do State staff use, provide to sponsors, or sites?

[Acceptable responses include marking any combination of the first three response options, or ONLY the final response option (not used or provided).]

Resource	Used by State Staff	Provided to Sponsors	Provided to Sites	Not Used or Provided
USDA Resources
USDA Summer Meals Toolkit
USDA Mapping Tools for Summer Meal Programs (e.g. Capacity Builder)
USDA State Agency Monitor Guide
USDA Administration Guide for Sponsors
USDA Sponsor Monitor’s Guide
USDA Nutrition Guide
USDA Site Supervisor’s Guide
USDA Sample Sponsor Administrative Review Report
USDA Sample Site Operational Review Report
Non-USDA Resources
Choice Plus: A Reference Guide for Foods and Ingredients (Institute of Child Nutrition)
Share Our Strength Collaborative Planning Toolkit
No Kid Hungry’s Text Service (Text “FOOD” to 877-877)
FRAC Guide on Purchasing High-Quality Summer Meals
Other (PLEASE SPECIFY):
Other (PLEASE SPECIFY):

[for all rows in D1 where respondent indicates “not used or provided,” populate D1a]

D1a. Explain why the following resources are neither used nor provided.

Tool or Resource	Reason Not Used or Provided
	Unfamiliar with this Resource	Not Found to be Useful	Distribute Similar Guidance Produced by the State
[fill using response to D1]
[fill using response to D1]
[fill using response to D1]
[fill using response to D1]

[for all rows in D1 where respondent indicates “provided to sponsors” OR “provided to sites”, populate D1b]

D1b. How does the State agency disseminate these resources?

Tool or Resource	How Disseminated to Sponsors and/or Sites
	Posted on State agency’s public website	Posted on private State portal that sponsors and sites can access	Hard copies distributed at trainings	Shared via Email	Other
[fill using response to D1]
[fill using response to D1]
[fill using response to D1]
[fill using response to D1]

D2. Do you have any State-developed tools and resources that you make available to sponsors or sites in your State?

Yes, provided to sponsors only  GO TO QUESTION D2a

Yes, provided to sites only  GO TO QUESTION D2a

Yes, provided to sponsors and sites  GO TO QUESTION D2a

No  GO TO QUESTION E1

D2a. What topics do the State-developed tools and resources address?

Meal patterns

Monitoring

Mapping of eligible sites

Procurement and purchasing

Meal counting

Meal claiming

Site eligibility

Budgets and allowable costs

Recordkeeping

Food safety

Other (PLEASE SPECIFY):

SECTION E: SPONSOR AND SITE APPROVAL PROCESS

E1. What is the State agency’s deadline for accepting sponsor applications to participate in SFSP?

_____[Dropdown menus for month and day]______

E2. Does the State agency accept applications after the deadline?

No

Yes, under the following circumstances (PLEASE SPECIFY) ____________

E3. What are the top three challenges for the State agency in the sponsor application and approval process?

SELECT THE TOP 3 CHALLENGES

Late applications

Incomplete applications

Documenting sponsor eligibility

Documenting site eligibility

Reviewing sponsor budgets

Timeline for review and approval

Conducting pre-approval visits

Finding enough eligible sponsors to apply

Other (PLEASE SPECIFY):

Not applicable; the process is not challenging

E4. What is the most common reason for disapproving a sponsor’s application?

SELECT ONLY ONE.

Sponsor does not meet eligibility criteria

Sponsor has uncorrected problems from the prior year

Sponsor is not financially viable and/or does not have a reasonable budget for SFSP

Other sponsors or sites cover the same service area

Sponsor does not have adequate qualified staff to oversee program

Other (PLEASE SPECIFY):

Not applicable

E5. How does the State agency verify each proposed site’s area eligibility?

SELECT ONLY ONE.

Sponsor uploads area eligibility documentation and State staff review and confirm

Sponsor submits hard copy (paper) area eligibility documentation and State staff review and confirm

State system automatically confirms area eligibility based on address of site

Other (PLEASE SPECIFY):

E6. How does the State agency verify each closed enrolled site’s eligibility?

SELECT ONLY ONE.

Sponsor uploads free and reduced applications and other documentation and State staff review and confirm

Sponsor uploads eligibility information from local schools and State staff review and confirm

State reviews free and reduced applications and other documentation on monitoring visits

Other (PLEASE SPECIFY):

E7. How does the State agency verify each camp’s eligibility?

SELECT ONLY ONE.

Sponsor uploads documentation, including free and reduced applications, and State staff review and confirm

State reviews free and reduced applications and other documentation during on-site monitoring visits

Sponsor mails in documentation to the State and State staff upload

Other (PLEASE SPECIFY):

E8. How does the State agency verify that a site is rural?

SELECT ONLY ONE.

Sponsor uploads documentation and State staff review and confirm

State system automatically confirms rural status based on address of site

State checks the site location against census map

Other (PLEASE SPECIFY):

E9. How does the State agency track compliance with health and sanitation inspections for sites?

SELECT ALL THAT APPLY.

State or local health agencies provide reports directly to the State agency

Sponsors or sites maintain documentation that is checked during State monitoring visits

Sponsors upload information to the State

Other (PLEASE SPECIFY):

No specific process

E10. Does the State agency have additional FNS-approved requirements for sponsors beyond what SFSP regulations require?

Yes

No

E11. Does the State agency have additional FNS-approved requirements for sites beyond what SFSP regulations require?

Yes

No

SECTION F: PROGRAM MONITORING

F1. To what extent does the State agency conduct unannounced reviews of SFSP sponsors?

SELECT ONLY ONE.

More than 75% of sponsor reviews are unannounced (all or almost all)

25-75% of sponsor reviews are unannounced (some)

Less than 25% of sponsor reviews are unannounced (a few)

No sponsor reviews are unannounced

F2. To what extent does the State agency conduct unannounced reviews of sites?

Note: by unannounced we mean that neither a site nor its sponsor is aware of the review in advance.

SELECT ONLY ONE.

More than 75% of site reviews are unannounced (all or almost all)

25-75% of site reviews are unannounced (some)

Less than 25% of site reviews are unannounced (a few)

No site reviews are unannounced

F3. How does the State agency verify use of reimbursement funds on allowable costs?

SELECT ALL THAT APPLY.

Review of proposed budget prior to program operations

On-site review of sponsor or site documentation

Review of documentation submitted by sponsors

Other (PLEASE SPECIFY):

F4. How does the State agency verify that corrective action is implemented by sponsors/sites?

SELECT ALL THAT APPLY.

Follow-up visits

Desk review of documentation

Other (PLEASE SPECIFY):

F5. For summer 2019, which of the following expenditures of SFSP funds did the State agency most frequently deny?

SELECT ALL THAT APPLY.

Meals served over the approved site cap

Organized sports activities

Transportation

Fundraising costs

Items not included in budget approved by State agency

Excessive salary

Other (PLEASE SPECIFY):

Not applicable

F6. Which of the following areas does the State agency most closely monitor among new sponsors in their first year of operating the SFSP?

SELECT ONLY ONE.

Budgets and allowable costs

Meal patterns

Meal counting and claiming

Site training

Health and sanitation inspections

Meal preparation/ordering

Procurement

Other (PLEASE SPECIFY):

There is not a specific area that the State more closely monitors for new sponsors

F7. Which of the following areas does the State agency most closely monitor among experienced sponsors, who have operated the SFSP for more than two years?

SELECT ONLY ONE.

Budget and allowable costs

Meal patterns

Meal counting

Meal claiming

Site training

Health and sanitation inspections

Meal preparation/ordering

Procurement

Other (PLEASE SPECIFY):

There is not a specific area that the State more closely monitors for experienced sponsors

SECTION G: PAYMENTS TO SPONSORS

G1. Does the State agency’s meal claiming system track meal counts by site?

Yes

No

G2. Which of the following edit checks does the State’s meal claiming system have for sponsor claims for reimbursement?

SELECT ALL THAT APPLY.

Total number of meals that can be claimed (e.g., # operating days times average daily participation)

Second meal limit

Block claims (i.e., claiming the exact same number of meals for consecutive time periods)

Approved meals (breakfast, lunch/supper, snack)

Other (PLEASE SPECIFY):

None

SECTION H: MEAL DISALLOWANCES

The next questions ask about meal disallowances that resulted from State monitoring visits.

H1. For how many sites were meals disallowed in summer 2019?

_______[NUMBER]__________ (AN ESTIMATE IS FINE)

(programmer: if respondent answers zero, skip to H3)

H2. For the following meal types, how many meals were claimed and disallowed by the State in summer 2019?

For the number of meals disallowed, an estimate is fine.

(programmer, column 3 cannot be greater than column 2)

Meal Type	# Claimed in Summer 2019	# Disallowed in Summer 2019
Breakfast
Lunch
Supper
Snack

H3. According to the State monitoring staff, what are the three most common reasons that the State agency disallowed meals in summer 2019?

SELECT THE TOP 3 REASONS.

Insufficient or incomplete meal count records

Insufficient or incomplete menus, food service records and/or meal delivery receipts

Site exceeded the meal cap set by the State agency

State reviewer observed fewer meals served to children than sponsor/site claimed

State reviewer observed meals served that did not meet meal patterns

Meals were served outside of approved times

Meals were taken offsite for consumption

Meals claimed were served to adults

Number of second meals claimed was too high

Other (PLEASE SPECIFY):

No meals were disallowed in 2019

SECTION I: PROGRAM TERMINATIONS AND TURNOVER

I1. How many of the sponsors and sites that participated in summer 2018 did not return to operate the SFSP in summer 2019?

	Number	Don’t Know	Not Applicable
SFSP Sponsors in good standing from 2018 that did not return in 2019
SFSP Sponsors flagged as “seriously deficient” from 2018 that did not return in 2019
SFSP Sites in good standing from 2018 that did not return in 2019
SFSP Sites flagged as “seriously deficient” from 2018 that did not return in 2019

I2. How many sponsors and sites participated in the SFSP in summer 2019?

	Number
SFSP Sponsors in summer 2019
SFSP Sites in summer 2019

I3. During or after summer 2019, how many SFSP sponsors and sites were either terminated for cause or self-terminated?

Termination for cause refers to the termination of a sponsor’s or site’s SFSP agreement by the State agency due to the sponsor’s or site’s violation of the agreement.

A sponsor or site that self-terminates is one that elects to stop participating in the SFSP.

	Number	Don’t Know
SFSP Sponsors terminated for cause
SFSP Sponsors that self-terminated
SFSP Sites terminated for cause
SFSP Sites that self-terminated

I4. In general, what is the most common reason sponsors are terminated for cause? SELECT ONLY ONE.

Repeat or uncorrected violations from a prior review

Sponsor is found seriously deficient in another federal child nutrition program

Sponsor does not repay over-claims

Sponsor does not maintain tax exempt status documentation

The sponsor’s sites do not meet eligibility requirements

Other (PLEASE SPECIFY):

Not applicable, no sponsor has been terminated for cause in the last 5 years

I5. In general, what is the most common reason sites are terminated for cause?

SELECT ONLY ONE.

Health or safety violations

Repeat or uncorrected violations from a prior review

Sites do not meet eligibility requirements

Other (PLEASE SPECIFY):

Not applicable, no site has been terminated for cause in the last 5 years

I6. Does the State agency maintain a State Disqualified List for SFSP?

Yes

No

I7. What is the most common reason that sponsors self-terminate and do not return from one year to the next?

SELECT ONLY ONE.

Another sponsor serves the same area or children

Losing money on SFSP

Site locations are no longer available for meal service

Not enough children attend the program

Too much paperwork

Not enough staff or volunteers to run the program

Meal vendors are not available in the area

Other (PLEASE SPECIFY):

Unknown

Not applicable, no sponsor has self-terminated in the last 5 years

I8. What is the most common reason that sites self-terminate and do not return from one year to the next?

SELECT ONLY ONE.

Another site serves the same area or children

Losing money on SFSP

Site locations are no longer available for meal service

Not enough children attend the program

Too much paperwork

Not enough staff or volunteers to run the program

Meal vendors are not available in the area

Other (PLEASE SPECIFY):

Unknown

Not applicable, no site has self-terminated in the last 5 years

SECTION J: CHALLENGES

J1. To what extent are each of the following issues a challenge faced by the State agency in administering and overseeing the SFSP in rural areas?

	Not a challenge	Moderate challenge	Significant challenge	Not sure
Identifying capable sponsors
Providing effective oversight of sponsors
Providing effective oversight of sites
Ensuring reimbursement funds are used for allowable costs
Staff turnover at sponsor organizations
Staff turnover at sites
Site recordkeeping
Sponsor recordkeeping
Distance to sites for monitoring
Other, please specify:

J2. To what extent are each of the following issues a challenge faced by the State agency in administering and overseeing the SFSP in urban and suburban areas?

	Not a challenge	Moderate challenge	Significant challenge	Not sure
Identifying capable sponsors
Providing effective oversight of sponsors
Providing effective oversight of sites
Ensuring reimbursement funds are used for allowable costs
Staff turnover at sponsors
Staff turnover at sites
Site recordkeeping
Sponsor recordkeeping
Distance to sites for monitoring
Other, please specify:

SECTION K: FINAL REFLECTIONS

K1. Please describe the primary challenge that the State agency faces in ensuring program integrity in the SFSP.

K2. Please tell us one thing that the State agency does particularly well to ensure program integrity in the SFSP.

K3. Please note anything else you would like to tell us about your State’s SFSP.

SECTION L: SUBMIT SURVEY (State Child Nutrition Director only)

I, [State CN Director Name], have reviewed the information in this survey and confirm that it is an accurate accounting of the administration of SFSP in my State.

Thank you for participating in this USDA study.

FNS anticipates the study results will be published on the FNS website in [DATE 2021], located here: https://www.fns.usda.gov/ops/child-nutrition-programs.

The Food and Nutrition Service (FNS) is conducting this study to obtain information about the administration and oversight of the Summer Food Service Program and to identify potential barriers to ensuring the integrity and effective management of the program. Participation in this study is voluntary and the information collected will be used to determine resources, develop training, and provide technical assistance. Under the Privacy Act of 1974 and the System of Record Notice FNS-8 USDA/FNS Studies and Reports, any personally identifying information obtained will be kept private to the extent provided by law. According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0584-XXX. The time required to complete this information collection is estimated to average 20 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: U.S. Department of Agriculture, Food and Nutrition Service, Office of Policy Support, 3101 Park Center Drive, Room 1014, Alexandria, VA 22302 ATTN: PRA (0584-xxxx).