SFSP Generic Clearance Package_revised_6.29

MEMORANDUM

Date: June 28, 2017

To: Steph Tathum, OMB Desk Officer

Through: Rachelle Ragland-Greene

Food and Nutrition Service, Information Collection Officer, Planning & Regulatory Affairs

Ruth Brown, United States Department of Agriculture, Office of Chief Information Collection Officer

From: Alice Ann Gola, PhD

Social Science Analyst, Food and Nutrition Service

Special Nutrition Evaluation Branch, Office of Policy Support

Re: Generic OMB Clearance No. 0584-0606 – Pretesting Instruments for the Summer Meals Study

The U.S. Department of Agriculture’s Food and Nutrition Service is requesting approval for pretesting of data collection instruments for the referenced study under approved Generic OMB Clearance No. 0584-0606. The following information is provided for review:

1. Title of the Project: Pretesting Instruments for the Summer Meals Study

2. Control Number: 0584-0606, Expires 03/31/2019

3. Public Affected by this Project: 87

1. State, Local, Tribal Governments: (12)

Local, Tribal Governments:

• 8 CN SFSP/SSO Sponsors

• 4 CN SFSP/SSO Site Supervisors

2. Businesses /Private Nonprofit Organization (12):

• 8 CN SFSP Sponsors

• 4 CN SFSP Site Supervisors

3. Individuals/Households: (63)

• 40 Parents/caregivers of SFSP/SSO participants and SFSP/SSO-eligible nonparticipants

• 9 Children/teens 5 through 18 years of age who are SFSP/SSO participants

• 14 Teens 13 through 18 years of age who are SFSP/SSO-eligible nonparticipants

4. Number of Respondents and Research Activity:

Respondents	Research Activity		# of Respondents
CN SFSP and SSO Sponsors (State/Local/Tribal Government)		Pretest data collection instruments	4
CN SFSP Sponsors (Businesses and Nonprofit Organizations)		Pretest data collection instruments	4
CN SFSP and SSO Site Supervisors (State/Local/Tribal Government)		Pretest data collection instruments	4
CN SFSP Site Supervisors (Businesses and Nonprofit Organizations)		Pretest data collection instruments	4
Former CN SFSP and SSO Sponsors (State/Local/Tribal Government)		Pretest data collection instruments	4
Former CN SFSP Sponsors (Businesses and Nonprofit Organizations)		Pretest data collection instruments	4
Parents/caregivers of participants and nonparticipants		Pretest data collection instruments	40
Child/teen participants		Pretest data collection instruments	9
Teen nonparticipants		Pretest data collection instruments	14

5. Time Needed per Response:

Respondents	Research Activity	Time (hours)*
CN SFSP and SSO Sponsors (State/Local Government)	Pretest data collection instruments	2.1†
CN SFSP and SSO Sponsors (Businesses and Nonprofit Organizations)	Pretest data collection instruments	2.1†
CN SFSP and SSO Site Supervisors (State/Local/Tribal Government)	Pretest data collection instruments	2.1†
CN SFSP Site Supervisors (Businesses and Nonprofit Organizations)	Pretest data collection instruments	2.1†
Former CN SFSP and SSO Sponsors (State/Local Government)	Pretest data collection instruments	0.62
Former CN SFSP and SSO Sponsors (Businesses and Nonprofit Organizations)	Pretest data collection instruments	0.62
Parents/caregivers of participants and nonparticipants	Pretest data collection instruments	1.7
Child and teen participants	Pretest data collection instruments	0.58
Teen nonparticipants	Pretest data collection instruments	0.58

* Average time per response.

†Time includes burden hours for pretesting the QI, which only 2 of the 4 respondents will complete

Total Burden Hours on Public: 64 Burden hours and 258 Responses

See the enclosed Excel burden table for the detailed burden estimate for a breakdown by affected public.

6. Project Purpose, Methodology, and Formative Research Design:

Background

The Summer Food Service Program (SFSP) and the National School Lunch Program’s Seamless Summer Option (SSO) are programs administered by the U.S. Department of Agriculture’s Food and Nutrition Service (FNS) to improve food security of low-income families and children. These programs help ensure that children receiving free and reduced price meals during the school year have continual access to healthy meals, by providing free meals during the summer months when school is not in session.

State agencies enter into agreements with program sponsors, including school food authorities (SFAs), local and tribal government agencies, and Businesses - private nonprofit organizations.¹ Sponsors oversee one or more sites that serve free meals to children. Sites are located in low-income areas where at least half of the children in the area are eligible for free or reduced price meals (as demonstrated through school or Census data), or serve predominantly low-income children. Any child or teen through the age of 18 may attend an “open” area-eligible site to receive a free meal. “Enrolled” sites generally serve a specific, targeted low-income population of children and may or may not be in low-income areas.

The most recent study of summer meal programs -- Evaluation of the 2013 Summer Food Service Program Participant Characteristics²--involved data collection from States, sponsor, and site supervisors as well as a small number of participants (n=30) and nonparticipants (n=25). The findings from the 2013 study will provide valuable information on program operations, limited information on barriers and facilitators to parent and child participation, and no information on meal quality. To address these gaps, FNS contracted with Westat, Inc., to conduct the Summer Meals Study. Data collected through the national study will provide an understanding of facilitators and barriers to program implementation, nutritional quality of meals served, perceived benefits and challenges for sponsor and site participation, as well as awareness and factors influencing child participation. Both programs--SFSP and SSO--are included in the study. Data collection will take place in summer 2018.

To accomplish the study objectives, the following data collection tools are planned for the full data collection:

1. Sponsor Surveys

2. Menu Planning Surveys and Menu Survey Follow-Up Reports

3. Sponsor Qualitative Interviews

4. Former Sponsor Qualitative Interviews

5. Site Supervisor Surveys

6. Site Supervisor Qualitative Interviews

7. Household Eligibility Screeners for sampled site catchment areas

8. Participant Caregiver Surveys

9. Child Participant Surveys

10. Teen Participant Surveys

11. Participant Caregiver Qualitative Interviews

12. Nonparticipant Caregiver Surveys

13. Teen Nonparticipant Surveys

14. Nonparticipant Caregiver Qualitative Interviews

Note: See item 10 for Appendices references.

Recruitment

The activities to recruit respondents for the pretesting of the above instruments are listed below, organized by respondent type.

Sponsors and Sites (State, Local, Tribal Governments; Businesses)

The research team will use the USDA Summer Meal Site Finder to identify current sponsors in the Washington, D.C. metropolitan area to pretest the Sponsor Survey, Sponsor Qualitative Interview, Menu Planning Survey and Menu Survey Follow-Up Report. Recruiters will call organizations, explain the purpose of the study and the pretests, and request their participation (Appendix A-1, Sponsor/Former Sponsor/Site Supervisor Pretest Recruitment Script). Eight sponsors will be contacted in order to yield four sponsors to pretest the Sponsor Survey and Sponsor Qualitative Interview. Two of these four sponsors will also be asked to test the Menu Planning Survey and Menu Survey Follow-Up Report. These will be sponsors who have responsibility for menu planning.

The USDA Summer Meal Site Finder will also be used to identify former sponsors to pretest the Former Sponsor Qualitative Interview. The research team will cross-check data from the 2016 Site Finder with current data to identify and contact six former sponsors, in order to yield 4 former sponsors for the pretest. Sponsors who participated in 2016 but are not participating in 2017 will be identified as former sponsors. This will be confirmed at the time of the recruitment call (Appendix A-1, Sponsor/Former Sponsor/Site Supervisor Pretest Recruitment Script).

Current sponsors who participate in pretesting will be asked by the research team to identify at least two associated site supervisors who can be contacted for pretesting the site and menu instruments. Recruiters will contact site supervisors by telephone, explain the purpose of the study and the pretests, and request their participation (Appendix A-1, Sponsor/Former Sponsor/Site Supervisor Pretest Recruitment Script). Eight site supervisors will be contacted to reach four to pretest the Site Supervisor Survey and Site Supervisor Qualitative Interview guide. Two of these site supervisors with responsibility for menu planning will also be asked to test the Menu Planning Survey and Menu Survey Follow-Up Report.

Parents/Caregivers, Teens and Children (Individuals and Household)

Low-income households in the Washington, D.C. metro area, with at least one child or teenager (aged 5-18) will be invited to pretest the Household Screener, Caregiver Surveys, Caregiver Qualitative Interviews, and Child and Teen Surveys. We will identify a total of 9 parents/caregivers of participants, 9 parents/caregivers of nonparticipants, 5 child participants, 4 teen participants, and 9 teen nonparticipants for pretesting.

To identify participants, site supervisors who participate in pretesting will be asked to facilitate the identification of parents/caregivers of summer meal program participants interested in participating in pretesting. These parents/caregivers will be asked to call the study telephone line, where we will ask them to complete a brief telephone screening questionnaire to establish eligibility (Appendices E-1, Study voicemail message for inbound calls from caregivers in pretesting, and E-2, Script for outbound telephone calls to schedule cognitive testing interviews with caregivers). Parents/caregivers will be considered eligible if they had a child ages 5 through 18 who attended a summer meal site in 2016 or 2017. If eligible, parents/caregivers will be asked if their teen or child is willing to take part in the pretest. To the extent possible, we will attempt to interview children and teenagers of varying ages, races, and ethnicities.

To identify nonparticipants, we will make telephone calls to low-income households contained in Westat’s internal database. The database includes contact information for volunteers who participated or expressed interest in participating in testing data collection instruments. The household will be asked to complete a brief telephone-screening questionnaire to establish eligibility (Appendix E-2, Script for outbound telephone calls to schedule cognitive testing interviews with caregivers). Nonparticipants will be considered eligible if they had children ages 5 through 18 who did not attend a summer program offering free meals in 2016 or 2017.

Purpose and Procedure

The purpose of each data collection activity that will be undertaken through this generic clearance is described below.

1. Sponsor Survey

The Sponsor Survey (Appendix A-3, Sponsor Survey) obtains information on sponsor characteristics; sponsor policies for food safety and special dietary needs; and operations of the sponsored site. Within one week of OMB approval, selected sponsors will be called and asked to participate in a telephone pretest of the sponsor survey (Appendix A-1, Sponsor/Former Sponsor/Site Supervisor Pretest Recruitment Script). As part of this telephone recruitment, sponsors will be asked to provide the names of two site supervisors who could be contacted for pretesting of the site data collection instruments. They will also be asked to indicate if the menus for their sites are planned by them or the site staff. Finally, we will ask them if they are willing to participate in an additional pretest of the Sponsor Qualitative Interview Guide at a later date (Appendix A-8, Sponsor Qualitative Interview). Upon agreeing to participate in the telephone pretest, the sponsors will be emailed the instruments so they will have them for the pretest (Appendix A-2, Email Sponsor/Site Supervisor Surveys). During the telephone pretest, once the sponsor agrees to participate, they will sign a consent form (Appendix D-1, Sponsor and Site Consent to Answer Questions to be Used in the Summer Meals Study), which will also allow us to audio-record the interview. Within two days of the pretest, a thank you email with appointment time for the follow-up qualitative interview will be sent to all sponsors who take part (Appendix A-7, Email with QI Appointment Time).

2. Menu Planning Survey and Menu Survey Follow-Up Report

The Menu Planning Survey (Appendix A-5, Menu Planning Survey) obtains information on the role of sponsors/sites in menu planning and factors that influence that role, along with meal planning resources used and needed. The Menu Survey Follow-Up Report (Appendix A-6, Menu Survey Follow-Up Report) collects detailed information on two weeks of menus served at the sampled site to assess compliance with regulatory meal pattern requirements and to determine the nutritional content of summer meals served to children. Upon agreeing to participate in the telephone pretest, the sponsor or site staff identified as responsible for menu planning will be emailed the instruments, including Appendix A-4, Menu Survey Initial Letter, so they will have them for the pretest (Appendix A-2, Email Sponsor/Site Supervisor Surveys). During the telephone pretest, once the sponsor/sites agree to participate, they will sign a consent form (Appendix D-1, Sponsor and Site Consent to Answer Questions to be Used in the Summer Meals Study), which will also allow us to audio-record the interview.

3. Sponsor Qualitative Interview

The Sponsor Qualitative Interview guide (Appendix A-8, Sponsor Qualitative Interview) obtains more detailed information on decisions about roles and responsibilities in menu planning, and facilitators and barriers to serving summer meals. During the telephone pretest, once the sponsor agrees to participate, they will sign a consent form (Appendix D-1, Sponsor and Site Consent to Answer Questions to be Used in the Summer Meals Study), which will also allow us to audio-record the interview.

4. Former Sponsor Qualitative Interview

The Former Sponsor Qualitative Interview guide (Appendix B-1, Former Sponsor Qualitative Interview) obtains detailed information on challenges in operating the summer meal program, and reasons for no longer participating in the program. Within one week of OMB approval, selected former sponsors will be emailed an invitation to pretest (Appendix A-1, Sponsor/Former Sponsor/Site Supervisor Pretest Recruitment Script). If they agree to participate, they will be emailed an appointment time for the qualitative interview (Appendix A-7, Email with QI Appointment Time). During the telephone pretest, once the sponsor agrees to participate, they will sign a consent form (Appendix D-1, Sponsor and Site Consent to Answer Questions to be Used in the Summer Meals Study), which will also allow us to audio-record the interview.

5. Site Supervisor Survey

The Site Supervisor Survey (Appendix C-1, Site Supervisor Survey) obtains information on site and meal service characteristics; participant demographics; site purpose; length of operations; and site equipment and storage capacity. Within one week of OMB approval, selected site supervisors will be called and asked to participate in a telephone pretest of the site supervisor survey (Appendix A-1, Sponsor/Former Sponsor/Site Supervisor Pretest Recruitment Script). Upon agreeing to participate in the telephone pretest, the site supervisors will be emailed the instrument so they will have it for the pretest (Appendix A-2, Email Sponsor/Site Supervisor Surveys). During the telephone pretest, once the site supervisor agrees to participate, they will sign a consent form (Appendix D-1, Sponsor and Site Consent to Answer Questions to be Used in the Summer Meals Study), which will also allow us to audio-record the interview. Within two days of the pretest, a thank you email with appointment time for the follow-up qualitative interview will be sent to all site supervisors who take part (Appendix A-7, Email with QI Appointment Time).

6. Site Supervisor Qualitative Interview

The Site Supervisor Qualitative Interview guide (Appendix C-2, Site Supervisor Qualitative Interview) obtains more detailed information on decisions about roles and responsibilities in menu planning, and facilitators and barriers to serving summer meals. Selected site supervisors who complete the Site Supervisor Survey pretest and indicate they are willing to participate in an additional telephone pretest will be emailed the instrument so they have it for the pretest (Appendix A-2, Email Sponsor/Site Supervisor Surveys). During the telephone pretest, once the site supervisor agrees to participate, they will sign a consent form (Appendix D-1, Sponsor and Site Consent to Answer Questions to be Used in the Summer Meals Study), which will also allow us to audio-record the interview.

7. Household Eligibility Screener

The Household Eligibility Screener (Appendix E-4, Household Eligibility Screener) asks whether children 18 and under reside in the household, and whether the children attend the sampled site or another site that serves free meals in the summer. Screeners completed for the full study will be used to develop the sampling frame for study participants and nonparticipants. Selected parents/caregivers (of participants and nonparticipants) will be asked to complete a hard-copy of the screener at the in-person pretest. At the in-person pretest, once the caregiver agrees to participate, they will sign a consent form (Appendix E-5, Parent/Caregiver (and 18-year-old Teen) Consent to Answer Questions to be used in the Summer Meals Study), which will also allow us to audio-record the interview.

8. Participant Caregiver Survey

The Participant Caregiver Survey (Appendix E-6, Participant Caregiver Survey) assesses why children participate in summer meal programs; frequency of participation; overall satisfaction with the program; participation in other federal programs; and household food security status and demographics. Selected parents/caregivers will be provided a hard-copy of the survey at the in-person pretest. Interested caregivers can call the Summer Meals Study telephone line and hear a message (Appendix E-1, Study voicemail message for inbound calls from caregivers in pretesting). The message clearly explains the information they will need to provide so that we can contact them at a later date (using Appendix E-2, Script for outbound telephone calls to schedule cognitive testing interviews with caregivers) to determine their eligibility to participate in pretesting and schedule the cognitive testing interviews. Once the interviews are scheduled, we will send an email to caregivers containing the details of the location site, the date, and time of their cognitive testing interview (Appendix E-3, Email Caregivers Details for Cognitive Testing). At the in-person pretest, once the caregiver agrees to participate, they will sign a consent form (Appendix E-5, Parent/Caregiver (and 18-year-old Teen) Consent to Answer Questions to be used in the Summer Meals Study), which will also allow us to audio-record the interview.

9. Child Participant Survey

The Child Participant Survey (Appendix E-13, Child Participant Survey) assesses the overall satisfaction with the summer meal program and its meals of children ages 5 through 12. Selected children, who will be accompanied by their parent/caregiver, will be given a hard-copy of the survey at the in-person pretest. At the in-person pretest, once the child’s parent/caregiver agrees that the child may participate, they will sign a consent form (Appendix E-10, Parental Permission Form: Consent for Child (5 to 12 years) to Participate in the Summer Meals Study Pretest), which will also allow us to audio-record the interview. As part of the consent, the parent/caregiver will be asked to assist the child in completing the survey, as needed. At the beginning of the child interview, the child will be asked to verbally consent to participation (Appendix E-11, Child (5 to 12 years) Assent to Participate in the Summer Meals Study Pretest).

10. Teen Participant Survey

The Teen Participant Survey (Appendix E-14, Teen Participant Survey) assesses why teens aged 13 through 18 participate in the summer meal program, and their overall satisfaction with the program and its meals. Selected teens will be given a hard-copy of the survey at the in-person pretest. At the in-person pretest, once the teen agrees to participate, they will sign a consent form (Appendix E-12, Teen (13 to 17 years) Assent to Participate in the Summer Meals Study Pretest and Appendix E-5, Parent/Caregiver (and 18-year-old Teen) Consent to Answer Questions to be used in the Summer Meals Study), which will also allow us to audio-record the interview.

11. Participant Caregiver Qualitative Interview

The Participant Caregiver Qualitative Interview guide (Appendix E-8, Participant Caregiver Qualitative Interview) obtains more detailed information on decisions related to a child’s participation in the summer meal programs. Parents/caregivers who complete the Participant Caregiver Survey pretest will complete the qualitative interview pretest at the same interview, immediately following completion of the survey pretest.

12. Nonparticipant Caregiver Survey

The Nonparticipant Caregiver Survey (Appendix E-7, Nonparticipant Caregiver Survey) assesses why children do not participate in the summer meal program; summer child care and other arrangements; available transportation; participation in other federal programs; and household food security status and demographics. Selected parents/caregivers will be given a hard-copy of the survey at the in-person pretest. Interested caregivers can call the Summer Meals Study telephone line and hear a message (Appendix E-1, Study voicemail message for inbound calls from caregivers in pretesting). The message clearly explains the information they will need to provide so that we can contact them at a later date (using Appendix E-2, Script for outbound telephone calls to schedule cognitive testing interviews with caregivers) to determine their eligibility to participate in pretesting and schedule the cognitive testing interviews. Once the interviews are scheduled, we will send an email to caregivers containing the details of the location site, the date, and time of their cognitive testing interview (Appendix E-3, Email Caregivers Details for Cognitive Testing). At the in-person pretest, once the caregiver agrees to participate, they will sign a consent form (Appendix E-5, Parent/Caregiver (and 18-year-old Teen) Consent to Answer Questions to be used in the Summer Meals Study), which will also allow us to audio-record the interview.

13. Teen Nonparticipant Survey

The Teen Nonparticipant Survey (Appendix E-15, Teen Nonparticipant Survey) assesses why teens ages 13 through 18 do not participate in summer meal programs; whether they are aware of the program; and what could get them to attend. Selected teens will be given a hard-copy of the survey at the in-person pretest. At the in-person pretest, once the teen agrees to participate, they will sign a consent form (Appendix E-12, Teen (13 to 17 years) Assent to Participate in the Summer Meals Study Pretest or Appendix E-5, Parent/Caregiver (and 18-year-old Teen) Consent to Answer Questions to be used in the Summer Meals Study), which will also allow us to audio-record the interview.

14. Nonparticipant Caregiver Qualitative Interview

The Nonparticipant Caregiver Qualitative Interview guide (Appendix E-9, Nonparticipant Caregiver Qualitative Interview) obtains more detailed information on effective methods to advertise summer meal programs, and decisions related to their child’s summer activities. Parents/caregivers who complete the Nonparticipant Caregiver Survey pretest will complete the qualitative interview pretest at the same interview, immediately following completion of the survey pretest.

Methodology/Research Design

A semi-structured guide will be developed and used to conduct cognitive testing of surveys and qualitative interview guides. Interviews will be conducted by a team of trained and experienced qualitative data collectors. All interviews will be audio recorded. After the respondent has completed a section of the survey or a series of questions on the qualitative interview guide, interviewers will ask respondents to explain how they arrived at their answers to specific questions and probe to determine whether respondents comprehend the items, are able to recall the relevant information, and have the information they need to answer the questions.

Current sponsors and site supervisors will be asked to participate in an hour-long semi-structured telephone interview. The interview for former sponsors will be 45 minutes. Eligible parents/caregivers will be asked to participate in in-person cognitive testing for the recruitment materials and surveys. They also will be asked about their interest in participating in a follow-up telephone interview to test the qualitative interview guide.

Parents/caregivers and their child/teen will be invited to FNS’ contractor’s office Westat in Rockville, MD. For those who are unable to travel to Rockville, an alternate venue such as a private room or office at a community center or library will be identified. Parents/caregivers will be required to accompany children and teenagers to the interview site. Where both teenagers/children and parents/caregivers are to be interviewed, the teenager/child and the parent/caregiver will be interviewed at the same time, but separately.

Individual interviews with adult respondents will take no more than 60 minutes to complete. Interviews with teenagers and children will be restricted to 30 minutes as they generally have shorter concentration spans compared with adults (Borgers et al., 2000).

We will pretest the English versions of all data collection instruments. The caregiver surveys, teen/child surveys, and site supervisor survey will be translated to Spanish after the English versions are finalized and submitted to OMB for approval. A team-based approach to translation will be adopted based on the Translate, Review, and Adjudicate framework (Harness, 2003), generally considered best practice in the field. During the translation process, translation issues are considered that might affect the questioning and translators take into account any adaptions that might be needed to the Spanish versions to minimize linguistic or cultural differences. The goal is to translate the surveys and discussion guides accurately, to reflect the intention of the English-language documents and maintain functional equivalence.

Informed Consent

Sponsors and site supervisors will be asked for their verbal consent (confirmed on the recording of the interview) before participating in any interview (Appendix D-1, Sponsor and Site Consent to Answer Questions to be Used in the Summer Meals Study). This consent will include permission to record the interview. The cognitive interviewer will explain the purpose of the interview, the risks and benefits, if any, to the respondent, and the privacy and voluntary nature of the study. The trained interviewer will answer any clarification questions before proceeding.

Parents/caregivers, and teens age 18 who will be interviewed in person will be asked for their written consent before participating in any interview and for audio recording the interview (Appendix E-5, Parent/Caregiver (and 18-year-old Teen) Consent to Answer Questions to be used in the Summer Meals Study). The interviewer also will answer any questions and review the consent form. All participants will be asked to sign the form prior to starting the cognitive interview.

For teenagers and children ages 5-17, written parental consent to interview a teenager or child and audio record the interview will be obtained prior to conducting the interview (Appendix E-10, Parental Permission Form: Consent for Child to Participate in the Summer Meals Study Pretest). Verbal assent to participate will be obtained from the teenagers/children before conducting the interview and confirmed on the audio recording (Appendices E-11, Child (5 to 12 yrs) Assent to Participate in the Summer Meals Study Pretest, and E-12, Teen (13 to 17 yrs) Assent to Participate in the Summer Meals Study Pretest).

Individuals refusing consent will be excused and thanked for their time.

Incentives

For the in-person parent/caregiver with child/teen interviews, parents/caregivers will be provided $40 (in the form of a prepaid gift card) as a token of our appreciation and for any expenses incurred in attending the interview such as transportation or childcare. We will also provide a $30 prepaid gift card for the child/teen interviews. For child/teen interviews, parents/caregivers will be given the option to receive the gift card on behalf of their children, or for us to provide it directly to the child/teen. Parents/caregivers will receive an additional $20 gift card after taking part in the telephone follow-up interview.

Prior experience with this population and these instruments and data collection procedures suggest success with this approach to the incentives.

Data Analysis

Cognitive Interviews. Because of the small numbers involved, qualitative methods will be used to analyze the cognitive interview data from the pretests. These data will be analyzed using an approach based on the Framework Method (Spencer et al., 2003). Framework is a matrix-based approach for managing qualitative data that allows for both case- and theme-based analysis of the data set. The recordings will be considered “the data” from which an item-by-item summary of problems identified will be made, forming the basis for making any recommendations for changes or amendments to the data collection instruments. The tested survey items and associated probes will provide the framework for written interview summaries. Interviewers will prepare summaries of each interview based on the completed survey modules and associated audio recordings. Summaries are entered into a series of grids, with each row representing a single participant, and each column an area of investigation. This analysis will guide recommendations for the final question wording, item by item.

Data collection instruments will be revised based on this input.

8. Confidentiality

Participation in the cognitive testing study is voluntary. FNS published a system of record notice (SORN) titled FNS-8 USDA/FNS Studies and Reports in the Federal Register on April 25, 1991, volume 56, pages 19078-19080, that discusses the terms of protections that will be provided to respondents. Personally identifiable information (PII), including names and contact information (telephone number and/or email address), will be collected for recruiting and scheduling proposes only. Participants will be informed that their PII will be kept private. These data will be securely stored in password-protected files to which only project staff will have access, and will be destroyed after the project is finished. Names provided by participants on consent and incentive receipt forms will be stored in locked cabinets, separate from the data. Participant PII will never be associated with the data collected during the interview. PII for individuals not selected for interview will be destroyed immediately.

9. Federal Costs

The total cost to the Federal government is $125,655.60, which includes the total cost for Contractor and Federal staff costs for the abbreviated supporting statement, developing the study instruments, and collection of the pretesting information. The Contractor cost is estimated at $123,778. This is based on an estimate of 859 hours, with a salary range of $93 – $368/hour, and includes overhead costs. This information collection also assumes a total of 40 hours of Federal employee time, for GS-13, Step 2 at $46.94 per hour, for a total of $1,877.60 on an annual basis. Federal employee pay rates are based on the General Schedule of Office of Personnel Management (OPM) for 2017.

10. Study Instruments

Sponsors, Former Sponsors, Site Supervisors

Appendix A-1 Sponsor/Former Sponsor/Site Supervisor Pretest Recruitment Script

Appendix A-2. Email <Sponsor/Site Supervisor> Surveys

Appendix A-3. Sponsor Survey

Appendix A-4. Menu Survey Initial Letter

Appendix A-5. Menu Planning Survey

Appendix A-6. Menu Survey Follow-up Report

Appendix A-7. Email with QI Appointment Time

Appendix A-8. Sponsor Qualitative Interview

Appendix B-1. Former Sponsor Qualitative Interview

Appendix C-1. Site Supervisor Survey

Appendix C-2. Site Supervisor Qualitative Interview

Appendix D-1. Sponsor and Site Consent to Answer Questions to be Used in the Summer Meals Study

Parents/Caregivers, Teens, Children

Appendix E-1. Study voicemail message for inbound calls from caregivers in pretesting

Appendix E-2. Script for outbound telephone calls to schedule cognitive testing interviews with caregivers

Appendix E-3. Email Caregivers Details for Cognitive Testing

Appendix E-4. Household Eligibility Screener

Appendix E-5. Parent/Caregiver (and 18-year-old Teen) Consent to Answer Questions to be used in the Summer Meals Study

Appendix E-6. Participant Caregiver Survey

Appendix E-7. Nonparticipant Caregiver Survey

Appendix E-8. Participant Caregiver Qualitative Interview

Appendix E-9. Nonparticipant Caregiver Qualitative Interview

Appendix E-10. Parental Permission Form: Consent for Child to Participate in the Summer Meals Study Pretest

Appendix E-11. Child (5 to 12 yrs) Assent to Participate in the Summer Meals Study Pretest

Appendix E-12. Teen (13 to 17 yrs) Assent to Participate in the Summer Meals Study Pretest

Appendix E-13. Child Participant Survey

Appendix E-14. Teen Participant Survey

Appendix E-15. Teen Nonparticipant Survey

References

Borgers, N., de Leeuw, E. and Hox, J. (2000). Children as Respondents in Survey Research: Cognitive Development and Response Quality. Bulletin of Sociological Methodology vol. 66 no. 1 60-75.

Cole, C. and Loftus, E. (1987). The memory of children. In Children’s eyewitness memory (pp.178-208). Springer US.

Scheffer, F. (2013) Cognitive Aspects of Surveys on Children. Presented at QUEST meeting, Washington D.C. http://wwwn.cdc.gov/qbank/QUest/2013/Scheffer.pdf

Spencer, L., Ritchie, J. and O’Connor, W. (2003). Analysis practices, principles and processes. In Ritchie, J, and Lewis, J. (Eds), Qualitative Research Practice (1^st edition). London: Sage Publications. Pp. 199-218.

Willis, G. (2005). Cognitive interviewing: A tool for improving questionnaire design. Thousand

Oaks, CA: Sage.

1 SSO is operated exclusively by SFAs in the National School Lunch Program. All sponsor types indicated, including SFAs, may operate SFSP.

2 OMB Control No. 0584-0594, Expiration Date 09/30/2019

1