Appendix G. NSWP-III General Data Collection Procedures

Appendix G

NSWP-III General Data Collection Procedures

General data collection procedures

All data collection activities are estimated to take place over a 5-month period. Prior to the data collection activities, FNS will send a notification email to the Regional Director to alert them about the study and to request that they notify their State WIC Directors about the study and encourage their assistance with any study requests (Appendix C1).

State Agency Survey. Prior to administration of the State Agency Survey, the Regional FNS Offices will notify all SA directors of the State Agency Survey (Appendix C2). SA directors will be sent an invitation email (Appendix C3) to request their participation in the study and remind them of their obligation to participate in these types of studies.¹ The invitation email will let the SA directors know that an invitation letter (Appendix C4) with a paper copy of the State Agency Survey will be mailed to them within a week. SA directors will be encouraged to complete the State Agency Survey (Appendix B1) via the web using a link provided in the invitation letter and emails. A simple URL with a secure login and password will be provided. In addition to the invitation letter, SA directors will be provided with the Study Description for State and Local WIC Agencies (Appendix D1). Beginning 2 weeks after the invitation email is sent, SA directors may be contacted up to seven times by email (Appendix C5) and up to seven times by telephone (Appendix C6) to receive a reminder and encouragement to complete the State Agency Survey. Each of the SA directors will be sent a thank you letter (Appendix D2) to show appreciation for their contribution to the study.

Local Agency Survey. LA directors will be sent an invitation email (Appendix C7) to request their participation in the study and remind them of their obligation to participate in these types of studies. The invitation email will let the LA directors know that an invitation letter (Appendix C8) with a paper questionnaire will be mailed to them within a week. LA directors will be encouraged to complete the Local Agency Survey (Appendix B2) via the web using a link provided in the invitation letter and emails. In addition to the invitation, LA directors will be provided with an informational summary (Appendix D1) describing the study. Beginning 2 weeks after the invitation email is sent, LA directors may be contacted up to seven times by email (Appendix C9) and up to seven times by telephone (Appendix C10) to receive a reminder and encouragement to complete the Local Agency Survey.

If there is an error with the expired certification date in the redemption data from respondents (“recently certified” participants) to the Certification Survey, the study team will confirm the end date of that participant’s certification period with the participant’s LA. Some of the LAs participating in the Local Agency Survey will be contacted, when necessary, to verify the certification end date of apparent expired certifications. These LA directors will be emailed (Appendix D3) and called (Appendix D4) up to five times to verify this essential information. Each of the LA directors will be sent a thank you letter (Appendix D5) to show appreciation for their contribution to the study.

Certification Survey. In contrast to procedures used in NSWP–II, in which the in-home participant data collection began only after a participant had completed a telephone interview, NSWP-III includes sampling of participants for in-home data collection and proceeds to the field within 1 week of selecting the sample. The sampling approach for NSWP-III is designed to assure that the Certification Survey sample is substantially less subject to nonresponse bias than the NSWP-II sample. To achieve the desired sample estimates, there will be two rounds of sample selection. Within each round, interviews will take place over 8 weeks, and the total field period will be 16 weeks.

Trained, experienced field staff will conduct the Certification Survey (Appendices B3.a, B3.b, B3.c, or B3.d) in person using the Computer Assisted Personal Interviewing (CAPI) system, either in the respondent’s home or at an alternate location, if preferred. Using the contact information provided by SA directors, participants will be recruited and scheduled by telephone (Appendices C11.a or C11.b). Several attempts (up to seven) will be made to reach the participant and schedule the survey by telephone before making one “door knock” attempt (Appendices C12.a or C12.b) to locate the participant at home.

Once the Certification Survey is scheduled, if a participant gives permission to send text messages (Appendices C13.a or C13.b), then text messages will be used to confirm a scheduled survey. The participant will also receive a reminder phone call (Appendices C14.a or C14.b) 1 day before the survey.

Prior to administering the Certification Survey, the participant will receive an informational letter (Appendices D6.a and D6.b) describing the study, and consent will be obtained in writing (Appendices D7.a and D7.b).

Denied Applicant Survey. Denied Applicant Survey respondents will be selected and interviewed during the 16-week field period in which Certification Surveys are conducted. Once the sample of the denied applicants has been identified, the data collection procedure will follow the protocol for the Certification Survey. Field interviewers (FIs) will schedule and conduct the Denied Applicant Survey (Appendices B4.a, B4.b, B4.c, or B4.d) in person, following the procedures used for the Certification Survey. Using the contact information provided by SA directors, participants will be recruited and scheduled by telephone (Appendices C15.a or C15.b). Several attempts (up to seven) will be made to reach the participant and schedule the survey by telephone before making up to two “door knock” attempts (Appendices C16.a or C16.b) to locate the participant at home.

Once the Denied Applicant Survey is scheduled, if a participant gives permission to send text messages (Appendices C17.a or C17.b), then text messages will be used to confirm a scheduled survey. The participant will also receive a reminder phone call (Appendices C18.a or C18.b) 1 day before the survey.

Prior to administering the Denied Applicant Survey, the participant will receive an informational letter (Appendices D8.a or D8.b) describing the study, and consent will be obtained in writing (Appendices D9.a and D9.b).

Program Experiences Survey. Some participants who complete the Certification Survey will also be asked in person (‘End of Survey” transition in Appendices B3.a, B3.b, B3.c, or B3.d) to complete the Program Experiences Survey (Appendices B5.a, B5.b, B5.c, or B5.d). Participants who agree to complete the Program Experiences Survey immediately following the completion of the Certification Survey will be surveyed in the same location as the Certification Survey (respondent’s home or specified alternate location).

Using the contact information provided by SA directors, participants will be recruited (and scheduled, if needed) by telephone (Appendices C19.a or C19.b). Several attempts (up to eight) will be made to reach the participant and complete the survey by telephone.² All potential participants who either have a non-working phone number or do not result in a successful survey after eight attempts will be mailed an invitation letter (Appendices C20.a or C20.b) that explains the study and requests participation in the study. A similar message will be emailed (Appendices C21.a or C21.b). For disconnected numbers, this mail and email procedure will begin immediately. For disconnected numbers, a postage-paid postcard (Appendices D10.a or D10.b) will be provided which asks the person to update their contact information by calling a toll-free number or by mailing back the information on a postcard. If there is no response (approximately 10 days after mailing the postcard³), the participant record will be coded for one in-person attempt or “door knock” (Appendices C22.a or C22.b) to recruit and complete the survey. The same protocol outlined for the Certification Survey will be used.

Prior to administering the in-person Program Experiences Survey, the participant will receive an informational brochure (Appendices D11.a or D11.b) describing the study, and verbal consent will be obtained (during recruitment using Appendices C19.a, C19.b, C22.a, or C22.b). Trained, experienced interviewers will conduct the Program Experiences Survey by telephone using CATI, or in person using CAPI. All respondents who complete the survey by telephone will be mailed a thank you letter (Appendices D12.a or D12.b).

Former WIC Participants Case Study. Qualitative researchers will conduct telephone interviews with former WIC participants, using a semi-structured interview guide (Appendices B6.a or B6.b). Using the contact information provided by LA directors, participants will be recruited and scheduled by telephone (Appendices C23.a or C23.b). Several attempts (up to five) will be made to reach the participant and complete the interview by telephone. Prior to administering the Former WIC Participants Case Study interviews, verbal consent will be obtained (during recruitment using Appendices C23.a or C23.b). All respondents who complete the survey by telephone will be mailed a thank you letter (Appendices D13.a or D13.b).

Pilot of Alternative Methodology. Once the sample of the “recently certified” participants and denied applicants has been identified, the data collection protocol for the Pilot of Alternative Methodology is identical to that for the Certification Survey and Denied Applicant Survey. FIs will schedule and conduct the pilot study in person, following the procedures used for the Certification Survey and Denied Applicant Survey.

Data collection instruments and recruiting materials, developed for the 2018 Certification Survey (Appendices B3.a, B3.b, B3.c, or B3.d) and 2018 Denied Applicants Survey (Appendices B4.a, B4.b, B4.c, or B4.d), will be fielded in 2019 and 2020. Select SA directors will be provided a description of the study (Appendix D1). LA directors will also be contacted by email (Appendix D3) and telephone (Appendix D4), as needed, to verify certification end dates needed for the sample of participants for the Certification Survey and Denied Applicant Survey.

Participant recruitment efforts for both surveys will include telephone calls (Appendices C11.a, C11.b, C14.a, C14.b, C15.a, C15.b, C18.a, or C18.b), text messages (Appendices C13.a, C13.b, C17.a, or C17.b), and in-person contacts (Appendices C12.a, C12.b, C16.a, or C16.b) to complete the surveys. Prior to administering the surveys, the respondents will receive an informational letter (Appendices D6.a, D6.b, D8.a, or D8.b) describing the study, and consent will be obtained in writing (Appendices D7.a, D7.b, D9.a, or D9.b).

Obtaining Extant Data. The study will require five types of extant data, both from study States and from FNS:

• The most recently available WIC PC data (currently WIC PC 2014)

• The most recently available WICLAD (currently WICLAD 2017)

• Study States’ contact data for denied WIC applicants, former WIC participants, and current WIC participants certified within specific reference period (up to 2 weeks prior to sample selection for the Certification Survey; after 2 weeks prior to sample selection for the Program Experiences Survey among current participants)

• Study States’ certification data

• Study States’ redemption/EBT transaction and rebate data

Procedures for obtaining these data are discussed below.

WIC Participant and Program Characteristics (WIC PC) and WIC Local Agency Directory (WICLAD). The most recently available WIC PC and WICLAD data will be used to construct the sampling frames for the Certification Survey, Denied Applicant Survey, Program Experiences Survey, and Local Agency Survey. These data contain information on LA size (i.e., in terms of WIC population served in a given month) and location, needed to construct the sampling frames and select the samples. Other characteristics of LAs from WIC PC data may also be used in analyses of study data—for example, to examine potential relationships between LA characteristics and erroneous payments and/or participant experiences.

Study States’ Participant and Certification Data. To conduct the fourth stage of sampling once States, LAs, and clinics have been recruited, the researchers will request State administrative data for WIC participants certified within a given reference period; their LAs (and the identity of their local clinic within the LA, if clinics will be sampled within LAs); and their most recent certification dates and participant categories, along with address, telephone, and other contact information. The data elements will be used for final sample selection as well as contact information for selected participants.

Study States’ Redemption and Rebate Data. Consistent with the approach used in NSWP–II, the researchers will provide estimates for both pre- and post-infant formula rebate certification and dollar error rates. State redemption and rebate data will be used for this purpose. The State Agency Survey will include a section that examines rebates for infant foods (such as infant cereal). Data collected from these sources will be used to calculate a national estimate of annual improper payments, including specific data on items that are not infant formula rebates.

The FNS Regional Offices will send an initial overview of the administrative data requests to the selected States within their Region (Appendix C24). Then, approximately one week later, the contractor will send more detailed instructions regarding the data request (Appendix B8). Beginning 2 weeks after the email is sent, SA directors may be contacted up to seven times by email (Appendix C25) and up to seven times by telephone (Appendix C26) to receive a reminder and encouragement to provide the requested administrative data.

1 Section 28 of the Richard B Russel National School Lunch Act as amended by the Healthy, Hunger-Free Kids Act of 2010 (Appendix A9)

2 The Certification Survey and Denied Applicant Survey reminder telephone calls are intended to arrange in-person interviews. The time to schedule these by telephone is limited because of the design which requires interviews to be completed within six weeks of certification. There is less time to try repeated phone calls before proceeding to contact the sampled participant in person. Therefore, the Certification Survey and Denied Applicant Survey only have time for 7 follow-up attempts compared to the 8 follow-up attempts for the Participant Experiences Survey.

3 The 10 days includes approximately five days to the delivery postcard and an additional five days for the participant to respond to the survey once they received the postcard.

3