Memorandum
Date: 25 June 2020
To: James Crowe, OMB Desk Officer
Through: Ruth Brown, United States Department of Agriculture, Office of the Chief
Information Officer
From: Christina Sandberg C.S.
Food and Nutrition Service, Information Collection Officer, Planning & Regulatory Affairs
Re: Under Approved Generic OMB Clearance No.0584-0524 Request for Approval for Evaluation of FY 2019 Summer Electronic Benefit Transfer (EBT) Demonstrations
The Food and Nutrition Service (FNS) of the United States Department of Agriculture (USDA) is requesting approval for research under Approved Generic OMB Control No. 0584-0524 Generic Clearance to Conduct Formative Research or Development of Nutrition Education and Promotion Materials and Related Tools and Grants for FNS Population Groups.
This request is to acquire clearance to conduct a formative evaluation of the FY 2019 Summer Electronic Benefit Transfer for Children (Summer EBT) Demonstrations. This research will document grantee implementation processes, challenges, and successes and will describe participant and retailer experiences with Summer EBT. The study consists of a series of site visits, completed on an annual basis. Annual data collection activities during the period of 2020-2022 will include semi-structured interviews with key grantee staff; school food authorities (SFAs); retailers who participate in the Summer EBT project; and focus groups with parents/caregivers of children who receive these benefits. Recruitment of interviewees and participants in early summer of 2020 for site visits (while Summer EBT projects are operating) is important for the success of the data collection activities.
The following information is provided for your review:
Evaluation of FY 2019 Summer Electronic Benefit Transfer (EBT) Demonstrations
0584-0524, Expires 12/31/2022
State, Local and Tribal Government:
Summer EBT project administrators for Grantees and School Food Authorities (SFAs).
Individuals:
Parents/caregivers of children who participate in the project.
Business or Other for Profit:
Retailers who participate in the project.
See section 7, Project Purpose, Methodology & Formative Research Design, for a description of the number of participants for each audience and by research methodology.
Table 4.1 – Consent Forms for Focus Group Participation (Attachment F.2)
Audience |
Annual # of Participants |
Parents/caregivers who participate in focus groups |
|
Annual Total |
160 |
|
|
Table 4.2 – Research Activities by Audience
Target Audience |
Research Activity |
Annual # of Participants |
State and Tribal Grantees |
Grantee interview |
4 |
SFAs |
SFA interview |
8 |
Parents/caregivers |
Parent/caregiver focus group |
160 |
Retailers |
Retailer interview |
20 |
Table 5.1 – Time Needed per Initial Recruitment Annually*
Target Audience |
Time (minutes) |
Time (hours) |
State and Tribal Grantees a |
121 |
2.02 |
SFAsb |
549 |
9.15 |
Parents/caregivers |
9 |
.15 |
Retailers |
8 |
.13 |
*The time is an average response per respondent. a. The state and tribal recruitment activities includes burden for 2 hours of providing the study team with suggestions for SFAs and Retailers to contact (Attachment C.1). It also includes a one minute burden for a confirmation email template from the study team (Attachment C.2). b. The burden here includes 9 minutes’ worth of recruitment activities (Attachments E.1 and E.3 through E.6) and 9 hours of burden associated with the SFAs helping recruit participants for the focus group (Attachment E.2).
|
||
Table 5.2 – Time Needed for Research Activities by Audience
Target Audience |
Research Activity |
Time (minutes) |
Time (hours) |
State and Tribal Grantees |
Most years, grantees will be interviewed using Attachment B1.Grantee Interview Protocol |
90 |
1.5 |
State and Tribal Grantees |
One grantee (WI) will be interviewed using a shortened version of the interview guide (Attachment B.2) during the first year of the study. |
30 |
.5 |
SFAs |
SFA interview protocol |
60 |
1 |
Parents/caregivers c |
Parent/caregiver focus group interview protocol |
68 |
1.13 |
Retailers |
Retailer interview protocol |
30 |
.5 |
c. This includes the burden for signing-in (.05), consenting (.08) and participating in the focus group (1.0) itself.
Respondent Type |
Respondent Description |
Attachment |
Instrument |
Original Sample Size |
Responsive |
Non-Responsive |
Total Annual Estimated Burden Hours |
||||||||
Number of respondents |
Frequency of Responses |
Total Annual Responses |
Avg. Burden per Response (in hours) |
Annual Estimated Burden Hours |
Estimated number of non-respondents |
Frequency of Responses |
Total Annual non-Responses |
Avg. Burden per non-Response (in hours) |
Annual Estimated Burden Hours |
||||||
State, Local, or Tribal Governments |
State and Tribal Grantees |
B.1 |
Grantee Interview Protocol |
4 |
4 |
0.75 |
3.00 |
1.5 |
4.50 |
0 |
0 |
0 |
0 |
0 |
4.50 |
B.2 |
Shortened Grantee Interview Protocol |
1 |
1 |
0.33 |
0.33 |
0.5 |
0.17 |
0 |
0 |
0 |
0 |
0 |
0.17 |
||
C.1 |
Study Team Introductory Email to Grantees |
4 |
4 |
0.75 |
3.00 |
2.00 |
6.00 |
0 |
0 |
0 |
0 |
0 |
6.00 |
||
C.2 |
Interview Confirmation Email Template for Grantees |
4 |
4 |
0.75 |
3.00 |
0.02 |
0.05 |
0 |
0 |
0 |
0 |
0 |
0.05 |
||
Subtotal for state and tribal grantees |
4 |
4 |
2.33 |
9.33 |
1.15 |
10.72 |
0 |
0 |
0 |
0 |
0 |
10.72 |
|||
School Food Authorities (SFAs) |
D |
SFA Interview Protocol |
8 |
8 |
0.75 |
6.00 |
1 |
6.00 |
0 |
0 |
0 |
0 |
0 |
6.00 |
|
E.1 |
Grantee Introductory Email to SFAs |
8 |
8 |
0.75 |
6.00 |
0.02 |
0.10 |
0 |
0 |
0 |
0 |
0 |
0.10 |
||
E.2 |
Study Team Introductory Email to SFAs |
8 |
8 |
0.75 |
6.00 |
9.00 |
54.00 |
0 |
0 |
0 |
0 |
0 |
54.00 |
||
E.3 |
Recruitment FAQ Sheet for SFAs |
8 |
8 |
0.75 |
6.00 |
0.02 |
0.10 |
0 |
0 |
0 |
0 |
0 |
0.10 |
||
E.4 |
Study Overview Sheet for SFAs |
8 |
8 |
0.75 |
6.00 |
0.02 |
0.10 |
0 |
0 |
0 |
0 |
0 |
0.10 |
||
E.5 |
Telephone Script for SFA Recruitment |
8 |
8 |
0.75 |
6.00 |
0.08 |
0.50 |
0 |
0 |
0 |
0 |
0 |
0.50 |
||
E.6 |
Interview Confirmation Email Template for SFAs |
8 |
8 |
0.75 |
6.00 |
0.02 |
0.10 |
0 |
0 |
0 |
0 |
0 |
0.10 |
||
Subtotal for SFAs |
8 |
8 |
5.25 |
42.0 |
1.45 |
60.9 |
0 |
0 |
0 |
0 |
0 |
60.9 |
|||
Subtotal of all state, local, tribal governments |
12 |
12 |
4.28 |
51.33 |
1.40 |
71.62 |
0 |
0 |
0 |
0 |
0 |
71.62 |
|||
Individuals |
Parents/caregivers of Children in Program |
F.1 |
Focus Group Sign-In Sheet |
160 |
160 |
0.75 |
120.0 |
0.05 |
6.0 |
0 |
0 |
0 |
0 |
0 |
6.0 |
F.2 |
Focus Group Consent Form |
160 |
160 |
0.75 |
120.0 |
0.08 |
10.0 |
0 |
0 |
0 |
0 |
0 |
10.0 |
||
F.3 |
Parent/Caregiver Focus Group Interview Protocol |
160 |
160 |
0.75 |
120.0 |
1.00 |
120.0 |
0 |
0 |
0 |
0 |
0 |
120.0 |
||
G.1 |
Recruitment Flyer/Handout for Focus Group |
480 |
160 |
0.75 |
120.0 |
0.02 |
2.0 |
320 |
0.75 |
240 |
0.02 |
4.00 |
6.0 |
||
G.2 |
Recruitment FAQ Sheet for Focus Groups |
480 |
160 |
0.75 |
120.0 |
0.02 |
2.0 |
320 |
0.75 |
240 |
0.02 |
4.00 |
6.0 |
||
G.3 |
Study Overview Sheet for Focus Group Participants |
480 |
160 |
0.75 |
120.0 |
0.02 |
2.0 |
320 |
0.75 |
240 |
0.02 |
4.00 |
6.0 |
||
G.4 |
Telephone Script for Focus Group Confirmation |
160 |
160 |
0.75 |
120.0 |
0.08 |
10.0 |
0 |
0 |
0 |
0.00 |
0.00 |
10.0 |
||
G.5 |
Focus Group Confirmation Email Template for Focus Group Participants |
160 |
160 |
0.75 |
120.0 |
0.02 |
2.0 |
0 |
0 |
0 |
0.00 |
0.00 |
2.0 |
||
Subtotals for Individuals/Households |
480 |
160 |
6.00 |
960.00 |
0.16 |
154.00 |
320 |
2.25 |
720.0 |
0.02 |
12.00 |
166.0 |
|||
Businesses or Other for Profit |
Retailers Who Participate in Program |
H |
Retailer interview protocol |
20 |
20 |
0.75 |
15.0 |
0.5 |
8.00 |
0 |
0 |
0 |
0 |
0 |
8.00 |
I.1 |
Grantee introductory email to retailers |
32 |
20 |
0.75 |
15.0 |
0.02 |
0.25 |
12 |
0.75 |
9 |
0.02 |
0.15 |
0.40 |
||
I.2 |
Study team introductory email to retailers |
32 |
20 |
0.75 |
15.0 |
0.02 |
0.25 |
12 |
0.75 |
9 |
0.02 |
0.15 |
0.40 |
||
1.3 |
Telephone Script For Retailer Recruitment |
32 |
20 |
0.75 |
15.0 |
0.08 |
1.25 |
12 |
0.75 |
9 |
0.08 |
0.75 |
2.00 |
||
I.4 |
Interview Confirmation Email Template for Retailer |
20 |
20 |
0.75 |
15.0 |
0.02 |
0.25 |
0 |
0 |
0 |
0.00 |
0.00 |
0.25 |
||
Subtotals for Businesses |
32 |
20 |
3.75 |
75.0 |
0.13 |
10.00 |
12 |
2.25 |
27.0 |
0.04 |
1.05 |
11.05 |
|||
Total Reporting Burden |
524 |
192 |
5.66 |
1086.33 |
0.22 |
235.62 |
332 |
2.25 |
747.00 |
0.02 |
13.05 |
248.67 |
|||
USDA’s Food and Nutrition Service (USDA/FNS) developed the Summer Electronic Benefits Transfer (EBT) project to help address child hunger and food insecurity during the summer months. It is meant to help low-income families, who rely on school breakfasts and lunches during the school year, provide nutritious food to their children during the summer months. With funding from the 2010 Agriculture Appropriations Act (P.L. 111-80), USDA/FNS and State grantees piloted what was then called the Summer EBT for Children Demonstration (now called the Summer EBT Project) during the summers of 2011 to 2014 in 10 States and Indian Tribal Organizations. The Project provided low-income families with children an electronic benefit transfer (EBT) card to purchase food. A random assignment evaluation of this project found large positive impacts on food security and child nutrition. It also found that project models could be implemented successfully in a range of settings and with a range of implementation strategies.3
Building on the success of the Summer EBT Demonstrations, USDA/FNS previously funded existing grantees from 2015-2018 to continue their projects and to expand into additional rural areas or other areas of extreme need and to test new and innovative concepts. FNS also awarded funds to two new grantees to offer benefits for the summer of 2018. These grantees were involved in an Implementation and Benefit Use Study documenting how these grants were implemented and administered during the summers of 2015-2018 and analyzing EBT data from grantees and retailers to describe participant benefit use. This study was exempt from OMB requirements because it interviewed nine grantees.
In 2019, USDA/FNS competitively selected two new grantees and two of the existing grantees to demonstrate how Summer EBT can best be deployed as a complement to other Federal, State, and local efforts to feed eligible children in the summer, and operated in a way that makes efficient use of taxpayer dollars. FNS also funded the evaluation of these projects, which is the current evaluation under consideration for this OMB submission, including 1) a Benefits Use Study that uses existing administrative, cost and household project data, and 2) a Process Study consisting of primary data collection activities included in this OMB request.
The formative Evaluation of FY 2019 Summer Electronic Benefits Transfer (EBT) Demonstrations builds on the findings of previous Summer EBT evaluations while expanding the current base of knowledge about this innovative solution to children’s food insecurity. It will document how the demonstration projects were implemented and administered in 2019 through 2022 by 4 grantees, including two new grantees, and will describe participants’ and retailers’ experiences with Summer EBT in these States. The qualitative data collection will complement the Benefits Use study that will collect and analyze EBT data from the grantees to describe and assess participant benefit use across the three summers when each project offers benefits.
The Process Study aims to 1) document grantee implementation processes, challenges, and successes and 2) describe participant and retailer experiences with Summer EBT over a three-year period (summers 2020 and 2021 for Chickasaw Nation and Michigan and summers 2021-2022 for Wisconsin, and summers 2020-2022 for Inter Tribal Council of Arizona). Results from this evaluation will help FNS understand how Summer EBT projects can be implemented in new and varying settings. For example, this evaluation will describe the dynamics of Summer EBT implementation as agencies expand the geographic scope of their service areas. Other findings will document start-up and implementation processes among agencies without prior experience implementing Summer EBT.
To obtain information from grantees, retailers, SFAs and participants while the project is active, FNS aims to conduct annual in-person primary data collection (interviews and focus groups) during the summer months (July and August). To achieve this, the study team must begin respondent recruitment in late spring/early summer 2020 to be able to recruit active parents/caregivers of children for focus group participation. Conducting data collection while the project is active is essential for the success of the evaluation; respondents and participants will be able to provide better, more accurate information regarding successes, challenges, and desired project changes or improvements, while the project is operational.
Methodology/Research Design
Site visits will be conducted each summer to three grantees for the first year of data collection (2020), to all four grantees for the second year of data collection (2021) and to two grantees in the final year (2022). Wisconsin declined to participate in Summer EBT for the summer of 2020, so the study team will not collect data from retailers, schools or participants from Wisconsin for 2020. The team will conduct a shortened grantee interview with Wisconsin to capture lessons learned and challenges from preparing to implement Summer EBT for the first time.
Each site visit will include:
Semi-Structured Interviews with Key Grantee Staff (Attachments B.1 and B.2). For each site visit, the study team will conduct one interview with key grantee staff. The 1.5 hour protocol includes the following topics: background on respondent; implementation processes, challenges, and successes; experience with partnership; allocation of resources; implementation costs; beneficiaries’ use of benefits; and risk of benefit issuance errors. One of the grantees, Wisconsin, will not be participating in Summer EBT in the summer of 2020, but the study team has developed a shortened grantee interview protocol to still capture information from Wisconsin about challenges and lessons learned regarding preparation for implementation the summer of 2020. The interviews will take place at the interviewees’ office. Please see Attachment B.1 for the grantee interview protocol, Attachment B.2 for the shortened grantee interview protocol, and Attachment C.1 for the grantee recruitment letter, which requests that, after reading this introductory email, the grantee will assist with scheduling and, providing recommendations for SFAs and retailers, and connecting the study team with SFAs and retailers. We have included the 2 hours of burden under Attachment C.1 to account for all of the activities related to this interview and assume that this communication work will be done via email. We have also included C.2, which is a confirmation email from the study team to confirm the date, time and location of the interview.
The study team expects that all grantee staff approached about the study will agree to participate because the Request for Applications (RFA) for the FY 2019 Summer Electronic Benefits Transfer (EBT) Demonstrations (CFDA 10.59) established that all grantees shall participate in any evaluation initiated by the department.
Semi-Structured Interviews with SFAs (Attachment D). Each site visit will include hour-long interviews with representatives from two SFAs. SFAs will be selected based on grantee input, with the aim of including one urban and one rural (or one large and one small) community per grantee. A lead staff member involved in implementing Summer EBT at each SFA will be asked about their experiences with the Summer EBT project and its implementation processes, challenges, and successes. The study team expects to travel to each SFA’s office to conduct the interview in-person. Please see Attachment D for the complete protocol.
Given SFAs’ prior relationships with grantees and the fact that they are purposefully selected, we expect that all SFAs approached about the study will participate in interviews. The study team has also developed email correspondence for the grantee to introduce the study team to their SFA contacts (Attachment E.1), as well as a study team introductory email, recruitment FAQs, and study overview sheet (Attachment E.2). The study team will also work with the participating SFAs to identify a lead staff member to serve as the focus group liaison.
The liaison role is described in the recruitment FAQ and includes recruiting parents of participating children for the focus groups. In recognition of the importance and breadth of the activities carried out by the liaison, the study team proposes to offer a $250 honorarium in the form of a Visa gift card, or a donation to the SFA. Completing these study-related responsibilities are in addition to normal job responsibilities and will require up to nine hours of time outside of normal working hours. Please see details regarding the honorarium amount in the section on incentives. The estimated nine hours of burden associated with this assistance is noted in Attachment E.2. We have also included a recruitment FAQ sheet for SFAs (Attachment E.3); a study overview sheet (E.4); a telephone script, if needed, for follow-up by the study team (E.5); and an email confirmation template that outlines the time, date, and location of the interview (E.6).
Focus groups with parents/caregivers (Attachment F). The study team will conduct four focus groups per site visit with parents/caregivers of children who are participating in the Summer EBT project implemented by the four grantees. Focus group protocols will ask participants about their perceptions of and experience with the project; support for participants; benefit use; and recommendations for future changes. The proposed parent/caregiver focus groups will last one hour and will be held within the school district area where participant children attend school. Please see Attachment F.3 for the complete protocol. We have also included F.1, which is the sign-in sheet (F.1) and the formal consent form (F.2).
The focus group liaison will approach 30 individuals for each focus group, with the goal of having 8-10 participants (a 33 percent response rate) in each group. The study team will provide recruitment materials for distribution to the target population(s). Materials will include details regarding study purpose, scheduling and logistics (Attachment G). Focus group participants will also receive a $25 Visa gift card for their participation, which will help to cover the costs of transportation and childcare, which may be necessary in order for participants to come to the group. Attachment G.1 is the recruitment flyer, Attachment G.2 is the recruitment FAQ sheet, and Attachment G. 3 is the study overview sheet for recruitment. We have also included a telephone script to confirm with individuals that they are signed-up for the focus group (G.4) and an email version of this confirmation (G.5).
Pandemic Note: Participants will have the option of participating with appropriate social distancing protocols in place, or virtually.
Semi-structured Interviews with Retailer Key Informants (Attachment H). Using the discussion guide developed for retailers, the study team will interview up to five retailers per site visit. These retailers will be small, local retailers (less than 50,000 customers). Smaller retailers will be more likely to be familiar with Summer EBT, and more likely to observe changes resulting from project implementation and ongoing operations in their community, than large corporate retailers. Retailer interview protocols will ask about challenges in adapting systems for Summer EBT cards, changes made to inventories or other practices in response to Summer EBT, training for staff on how to accept Summer EBT benefits, retailer satisfaction with the Summer EBT project, how the project could be improved, and guidance or technical assistance received from the State. When possible, the study team will conduct the 30 minute interviews in-person at the retailers’ offices.
The study team will contact eight retailers in each grantee service area to participate in the study with the goal of recruiting and conducting five retailer interviews (63 percent response rate). The study team expects that this is reasonable because this approach will utilize prior relationships the grantee may have with retailers. Please see Attachment I.1 to view the introductory email from grantees to retailers and Attachment I.2 for an introductory email from the study team to retailers. We have also included Attachment I.3 which is a telephone script for the study team to follow-up with retailers if needed and an email confirmation noting the date, time and location of the interview (Attachment I.4).
All data collection will be performed by skilled qualitative data collectors—many of whom worked on previous or ongoing studies of Summer EBT projects. One data collector will lead the interviews and focus groups while the other takes notes. The interviews and focus group will be recorded and transcribed with respondents’ permission. After the interviews, the note-taker will develop a summary of the interviews and enter data by grantee into summary tables.
While the selection procedures for each respondent group varies, the aim is to gather information from a diverse set of respondents based on geography, size and representation from urban and rural communities (or large communities versus smaller communities), benefit redemption, and degree of participation in the Summer EBT projects.
Key Grantee Staff. A knowledgeable staff member from each grantee will be asked to participate in an interview each year. However, the study team will tailor the selection of respondents to invite programmatic or technical staff based on the stage of project implementation and operations. These include the Summer EBT Managers from both Chickasaw Nation and the Inter Tribal Council of Arizona, the Research and Technology Manager from Michigan and Project Manager from Wisconsin.
SFAs. SFA selection will be informed by grantee input, and will aim to include SFAs that vary in the following way: serve families with high and low Summer EBT benefit use; and urban and rural areas, or small and large designations if the grantee’s service area does not have urban and rural areas. A lead staff member from each SFA who is involved in Summer EBT implementation will be asked to participate in one interview per year.
Parents/caregivers. Parent/caregiver sampling will be informed by grantee input, and will aim to include parents/caregivers that vary in the following way and are from: targeted grantee areas, ZIP codes with highest and lowest prior year redemption rates; and areas within a reasonable distance from the lead grantee organization. The study team and grantee will work closely to provide the liaison with this information to inform parent/caregiver recruitment.
Retailers. Retailer sampling will be informed by grantee input and benefit redemption data, and will include retailers that vary in the following way: size – small (less than 50,000 customers); and lowest and highest benefit redemption rates.
Recruitment and Consent
This section describes how the study team will recruit participants for the focus groups and retailer interviews. The other interviews will not require formal recruitment efforts.
Parent/Caregiver Focus Groups. Recruitment materials can be found in the Attachments that begin with G. Attachment G.1 is the flyer for the focus group, Attachment G.2 is the Recruitment FAQ sheet for the focus group, and Attachment G.3 is the Study Overview Sheet for Focus Group Participants. The materials include details regarding study purpose, scheduling and logistics. The study team will also work with local SFAs to assist with focus group recruitment. The study team estimates that for each focus group 30 individuals will be approached, and a maximum of 10 participants will participate in the focus group per grantee each year. The study team will conduct four focus groups per grantee, per year.
Retailer Interviews. The grantee will send retailers an advance mailing/email that explains the study and encourages participation (Attachment I.1). The study team will follow the mailing with an email (Attachment I.2) and telephone calls (Attachment I.3) to retailers during which they will explain the study, confirm the retailers’ agreement to participate, and schedule the interview (Attachment I.4). We estimate that eight retailers will be approached and five of them will agree to participate per grantee.
All interviewees and focus group participants will be consented. Interviewers will provide consent language verbally in the introduction of all interviews. This verbal consent can be found as a part of the interview protocol in Attachments B.1, B.2, D, and H. For focus groups, consent forms will be signed by participants prior to beginning of the focus group. The consent form is included in Attachment F.2.
Focus group participants will receive a $25 Visa gift card as a token of appreciation. Two factors helped to determine the incentive amounts for the focus group: 1) costs of childcare; and 2) transportation costs. We have assumed that participants will need assistance with childcare and transportation in order to be in the focus group, and have provided an estimate of $25 to cover the partial costs of both childcare and transportation. The proposed parent/caregiver focus groups will last 60 minutes. We estimate two hours (this allows for the time to get to and from the focus group venue) of childcare. Based on the median wage rate for childcare workers from the U.S. Department of Labor, Bureau of Labor Statistics Occupational Employment and Wages May 2019, we use the median hourly wage of $11.65 for childcare workers (39-9011) (https://www.bls.gov/oes/current/oes399011.htm ). We have estimated that a participant will drive no more than 5 miles to the location of the focus group or 10 miles roundtrip. Using the 2020 GSA rate of .575 per mile for private vehicles, this would be $5.75 in transportation costs. The total of $29.05 includes $23.30 for childcare and $5.75 in transportation. We rounded down to $25 to determine the gift card amount.
As described earlier, SFA liaisons will receive a $250 honorarium for their work in recruiting focus group participants for the study. The work will include: advertising the focus groups, recruiting and screening potential focus group participants, coordinating location and time of focus group meetings and communicating with participants via email. The $250 amount is based on the BLS wage rate for “food service managers” at schools. The median rate for this position is $31.12, multiplied by the nine hours of time estimated for this work, comes to $280.08. We have rounded this amount down to $250 to determine the honorarium amount. The rates can be found via the U.S. Department of Labor, Bureau of Labor Statistics, May 2019 National Occupational and Wage Statistics, Occupational Group ( 11-9051 ) (this would be for the food service managers) (https://www.bls.gov/oes/current/naics3_611000.htm).
Qualitative Data. After site visits are concluded, data collectors will develop clean notes and review for accuracy. All focus group interviews will be audio-recorded (with respondent permission) and transcribed. This text will become the data for qualitative analysis.
The transcribed interview notes from interviews and focus groups will be uploaded into NVivo and the study team will develop a codebook to guide the coding of notes in NVivo. This coding will organize the data collected through interviews into “nodes,” or categorical themes. A lead analyst will conduct an in-depth training for the NVivo coders to ensure consistency across coded data from the interviews and focus groups. Please see Section 8, Confidentiality for data security measures.
Findings will be considered descriptive and directional, but not definitive. No attempt will be made to generalize findings as nationally representative.
Research findings will not include information that personally identifies any of the research participants. Results will be synthesized into two interim reports and a final report, all of which will be made public on the USDA-FNS website: http://www.fns.usda.gov/ops/research-and-analysis.
FNS complies with the Privacy Act of 1974 (5 USC 552a). The information gathered in this study will be kept private to the full extent permitted by law. The study will not collect personally identifiable information (PII). Consent language will be provided in the body of all interviews (grantee interview, SFA interview and retailer Interview), in verbal format, and as a separate written consent form for focus groups. All respondents’ information will be kept private and not disclosed to anyone but the analysts conducting this evaluation, except as otherwise required by law. Respondents to key informant interviews and focus groups will be informed of all planned uses of data, and that their information will be kept private to the extent permitted by law. System of Record FNS-8, FNS Studies and Reports, published in the Federal Register on 4/25/1991 at 56 FR 19078, covers personal information collected under this study and identifies safeguards for the information collected.
This study is voluntary for all potential participants and respondents, and is not required to obtain or retain benefits for the SFAs, State Agencies, retailers or parents/caregivers. Comments from interviews and focus groups will be used in a summary report for FNS. No names or other personally identifiable information will be shared in the report.
During the life of the project, any hard-copy documents containing personal identifiers will be stored in secured file cabinets and rooms, and electronic data will be maintained on secured, password-protected computer servers. Interviewers will keep notes and documents secure at all times while in the field. Sources of data will be accessible only by approved contractor staff. At the close of the evaluation, all hard-copy documents will be sent to secure storage and shredded as permitted under our contract.
For the grantee and SFA interviews, the study team will prepare a Restricted Use File that includes de-identified interview data for internal FNS use. There will be no Public Use File that includes any interview or focus group data, given that the relatively small number of respondents would make it easy to identify respondents in the data. In preparing restricted use files, the study team will remove respondents’ names and reserve the right to remove comments about specific people or grantee service areas that may negatively affect those people or service areas, but will not redact all information (e.g., job role) that could be used to identify an individual respondent.
All electronic data for the interviews and focus groups will be transferred using a secure portal website that uses FIPS 140-2 compliant encryption SSL/TLS protocols, and will be stored on secure drives with access restricted to staff associated with the project. The study team will destroy all data for the focus groups and interviews at the end of the contract.
The study has been reviewed by Abt’s Internal Review Board (IRB) and granted an approved exemption, the IRB approval can be found in Attachment J.
Federal Costs: The total annualized cost for the data collection activities under this current request will be $1,040,579.95. The largest cost to the Federal Government is to pay the contractor ($1,012,745.60 per year for five years, with fully loaded wages) to conduct this study and deliver reports and data files. The information collection also assumes a total of 424 hours of two federal employees’ time per year for a total cost of $20,928.08 per year with the following breakdown: 416 hours per year for a GS-13, Step 1 in the Washington, DC area at $49.19 and a GS-14, Step 1 at $58.13 per hour for a total of 8 hours per year. With fully loaded wages (adding $6,906.27, which is 33 percent of the initial total cost of $20,928.08) the total would be $27,834.35. The total cost of federal employees’ time over five years is: $139,171.73.
Attachment A: Burden Table
C.1: Study Team Introductory Email to Grantees
C.2: Interview Confirmation Email Template for Grantees
E.1: Grantee Introductory Email to SFAs
E.2: Study Team Introductory Email to SFAs
E.3: Recruitment FAQ Sheet for SFAs
E.4: Study Overview Sheet for SFAs
E.5: Telephone Script for SFA Recruitment
E.6 Interview Confirmation Email Template for SFAs
F.1 Focus Group Sign-In Sheet
F.2 Focus Group Consent Form
F. 3 Parent/Caregiver Focus Group Interview Protocol
G.1 Recruitment Flyer/Handout for Focus Group
G.2 Recruitment FAQ Sheet for Focus Groups
G.3 Study Overview Sheet for Focus Group Participants
G.4 Telephone Script for Focus Group Confirmation
G.5 Focus Group Confirmation Email Template for Focus Group Participants
Attachment I.1: Grantee Introductory Email to Retailers
Attachment I.2 Study Team Introductory Email to Retailers
Attachment I.3: Telephone Script for Retailer Recruitment
Attachment I.4: Interview Confirmation Email Template for Retailers
1 One grantee will receive 3 site visits over the data collection period. The other three grantees will receive only two visits over the three year data collection period. We have calculated the frequency for the individual activities (with the exception of Attachment B.2) to accommodate this. The calculation is as follows
3 grantees visited 2 times = 6 visits
1 grantee visited 3 times = 3 visits
= 9 visits across 4 grantees (6 +3)
= 2.25 visits per grantee (9/4)
=0.75 visits per grantee per year (2.25/ 3)
Additionally, Attachment B.2 will only be used once with one grantee over the three year data collection period.
2 We have estimated there are one small grantee, 4 small SFAs, and 20 small retailers involved in the evaluation. This is from a total responding sample of 4 grantees, 8 SFAs, and 20 retailers (32). Therefore, 25 small entities out of 32 responding entities (78.12%). This percentage is only for entities involved in the evaluation and does not include individuals.
3 Collins, A. M., Briefel, R., Klerman, J. A., Wolf, A., Rowe, G., Logan, C., … & Lyskawa, J. (2016). Summer Electronic Benefit Transfer for Children (SEBTC) Demonstration: Summary report 2011-2014 (Summary). Alexandria, VA: U.S. Department of Agriculture, Food and Nutrition Service, Office of Policy Support. Project Officer: Chanchalat Chanhatasilpa, May 2016. Contract #AG-3198-C-11-00024.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Tresa Kappil |
| File Modified | 0000-00-00 |
| File Created | 2022-09-14 |