APPENDIX_F._Data_Collection_Methods_for_Impact_Evaluation-SNAP-Ed

Appendix F.

Data Collection Methods for the Impact Evaluation

This appendix provides a detailed description of the data collection methods that will be used by the contractor, RTI International, to collect data for the impact evaluations of the four demonstration projects.

CNNS Impact Evaluation

Table F.1 summarizes the data collection methods for the Eagle Adventure Diabetes Prevention Program evaluation. As described below, we will work closely with the schools to make multiple attempts by mail and telephone to maximize the response rate for the survey.

Table F.1. Data Collection Methods for the Eagle Adventure Diabetes Prevention Program Evaluation

	Pre-survey	Post-survey
Mode	Mail/telephone	Mail/telephone
Data collection period	2-month period prior to intervention	Begin conducting survey 1 week after conclusion of intervention
Questionnaire length	15 minutes	18 minutes
Incentive amount	$10	$15

We will work with CNNS and the selected intervention and control schools to coordinate the recruitment of parents/caregivers for the pre- and post-surveys. CNNS will make the initial contact with treatment and control schools to encourage their cooperation in the evaluation study. An RTI field interviewer will also meet in person with the school principal and teachers, as appropriate, to discuss our study recruitment procedures and obtain their buy-in. Further, to acknowledge their assistance in distributing and collecting study enrollment materials, we will provide incentives to the school principal and classroom teachers. First, we will ask teachers to send home with the child (via their weekly folders) a pre-assembled informational packet with invitation for caregivers to participate in the study. Caregivers will be asked to return a consent/contact form indicating whether or not they are willing to participate and, if so, providing contact information (i.e., name, mailing address, phone number or contact number) sealed in the return envelope provided for this purpose. Children will receive a token incentive (e.g., a friendship bracelet, pen) for returning their caregiver’s form whether or not they agreed to participate. Teachers will be asked to track the return of these envelopes and distribute the token to children who do so. The teacher incentive is based upon the percentage of students returning the envelope, not the number of caregivers who enroll. School principals will be asked to designate someone to coordinate the daily collection of these returned sealed envelopes for pick up by a field interviewer on a daily basis during the enrollment period. Caregivers’ decision to participate will remain confidential. Contact information for cooperating parents/caregivers will be entered into an electronic template, encrypted, and emailed to RTI by the field interviewer on a nightly basis. Once received, the data collection process will begin. Field interviewers will ship the hard copy forms to RTI’s main office via Federal Express on a weekly basis.

For caregivers for whom we receive contact information, we will first mail a hard copy of the pre-survey with a cover letter again describing the study and offering a monetary incentive for completing the pre-survey. A self-addressed postage paid return envelope will be included. Five days later a follow-up postcard will be sent reminding participants to complete the survey and/or thanking them for their participation if they already did so. One week after the mailing nonrespondents will be contacted by telephone interviewers to see if they would prefer to complete the survey over the phone. A minimum of five call attempts on various days and times will be made to each working phone number. For the post-survey, the same process will be repeated except that a second mailing of the survey will be sent one week after the reminder/thank you postcard is sent. Telephone contact of nonrespondents will begin a week after the second mailing with a minimum of five call attempts made to each working phone number). The incentive will be mailed to respondents in a thank you letter following completion of each survey.

University of Nevada Impact Evaluation

Table F.2 summarizes the data collection methods for the All 4 Kids Program evaluation. As described below, we will work with the University of Nevada to implement the pre-survey data collection, and make multiple attempts by mail and telephone to maximize the response rate for the post-survey.

Table F.2 Data Collection Methods for the All 4 Kids Program Evaluation

	Pre-survey	Post-survey
Mode	In-person	Mail/telephone
Data collection period	2-week period prior to intervention	Begin conducting survey 1 week after conclusion of intervention
Questionnaire length	15 minutes	18 minutes
Incentive amount	$10	$15

We will work with the University of Nevada to coordinate the recruitment of parents/caregivers for the pre-survey at the intervention and control centers. The University of Nevada will make the initial contact with intervention and control centers to encourage their cooperation in the study. Sign-up sheets will be posted so caregivers can schedule a 45-minute time slot to complete the pre-survey in-person at the child care center on a predetermined date. During the interviews, caregivers will read and sign an informed consent form and will be asked to complete a questionnaire administered by University of Nevada staff and a separate questionnaire administered by an RTI field interviewer. We will randomize the order of administration of the University of Nevada and FNS questionnaires. The RTI interviewer will collect contact information so that we can contact study participants by mail/telephone for the follow-up survey. Because the University of Nevada and FNS instruments both include the Townsend Fruit and Vegetable Checklist, we plan to only collect this information in the FNS survey and share this information with the University of Nevada.

For respondents for whom we receive contact information, we will mail a hard copy of the post-survey with a cover letter describing the study and offering a monetary incentive for completing the post-survey. A self-addressed postage paid return envelope will be included. Five days later a follow-up postcard will be sent reminding them to complete the survey and/or thanking them for their participation if they already did so. One week later, nonrespondents will be contacted by telephone interviewers to see if they would prefer to complete the survey over the phone. A minimum of five call attempts will be made to each working phone number. A thank you letter with incentive enclosed will be mailed to survey respondents.

NYSDOH Impact Evaluation

Table F.3 summarizes the data collection methods for the EWPHCCS Program evaluation. As described below, we will work closely with center staff and make multiple attempts by mail and telephone to maximize the response rate for the survey.

Table F.3. Data Collection Methods for the EWPHCCS Program Evaluation

	Pre-survey	Post-survey
Mode	Mail/telephone	Mail/telephone
Data collection period	2-month period prior to intervention	Begin conducting survey 1 week after conclusion of intervention
Survey length	15 minutes	18 minutes
Incentive amount	$10	$15

We will work with NYSDOH and the centers to coordinate the recruitment for the pre- and post-surveys at the intervention and control centers. NYSDOH will make the initial contact with treatment and control centers to encourage their cooperation in the study. RTI field interviewers will also meet in person with the center directors to discuss our study procedures and obtain their buy-in. Further, we will provide monetary incentives to center and teachers in appreciation for their help in recruiting caregivers to complete the survey. First, we will ask teachers to send home with the child (via their cubbies in the center or other standard mode of communication with the home) a pre-assembled informational packet with an invitation in English and/or Spanish for them to participate in the study. Caregivers will be asked to return a consent/contact form indicating whether or not they are willing to participate and, if so, providing contact information (i.e., name, mailing address, phone number or contact number) sealed in the return envelope provided for this purpose. Children will receive a token incentive (e.g., a friendship bracelet) for returning the form whether or not their caregiver agreed to participate. Teachers will be asked to track the return of these envelopes and distribute the token to children who do so. A classroom sticker chart might be provided to assist in this tracking. The teacher incentive is based upon the percentage of children returning the envelope, not the number of caregivers who enroll. The center director will be asked to designate a person to coordinate the daily collection of the returned sealed envelopes and ship them daily via Federal Express next day service to RTI. Preprinted shipping labels with account information will be provided to each center for this purpose along with FedEx shipping envelopes. Daily pick-up by a FedEx driver will be arranged if needed. Caregivers’ decision to participate will remain confidential. Contact information for cooperating parents/caregivers will be entered into the study database once the forms arrive at RTI’s main office. Once entered the data collection process will begin.

For caregivers for whom we receive contact information, we will first mail a hard copy of the pre-survey with a cover letter reiterating the study description and offering a monetary incentive for completing the pre-survey. A self-addressed postage paid return envelope will be included. Five days later a follow-up postcard will be sent reminding them to complete the survey and/or thanking them for their participation if they have already done so. One week later, nonrespondents will be contacted by telephone interviewers to see if they would prefer to complete the survey over the phone. A minimum of five call attempts will be made to each working phone number at various times and days. For the post-survey, the same process will be repeated except that a second mailing of the survey will be sent one week after the reminder/postcard is sent. Telephone contact of nonrespondents will begin a week after the second mailing with a minimum of five call attempts made to each working phone number. A thank you letter with incentive enclosed will be mailed to respondents of each survey.

PSU Impact Evaluation

Table F.4 provides information on the data collection procedures for the About Eating program evaluation. We will employ a mixed mode survey approach which includes an Internet survey with contacts by mail/telephone for nonrespondents and individuals who drop out of the program.

We will subcontract with PSU to program and administer the FNS pre- and post-Internet surveys. We will use a combined informed consent for the PSU and FNS surveys and randomize the order of the PSU survey questions and FNS survey questions. We will collect contact information (telephone and mailing address), so that we can contact program dropouts and nonrespondents by mail/telephone for the post survey.

Table F.4. Data Collection Methods for the About Eating Program Evaluation

	Pre-survey	Post-survey
Mode	Internet	Internet with mail/telephone survey for nonrespondents
Data collection period	Immediately before starting the intervention	Immediately following completion of the intervention
Survey length	10 minutes	15 minutes
Incentive amount	$10	$15

For nonrespondents to the post-survey and individuals who drop out of the intervention, we will mail a hard copy of the survey with a cover letter and self-addressed postage paid return envelope. Five days later a follow-up letter will be sent reminding them to complete the survey and/or thanking them for their participation if they have already done so. If we do not receive a response by mail, we will then contact nonrespondents by telephone (making a minimum of five call attempts to each working phone number) and attempt to complete the survey over the phone.

Using a combined mail/telephone survey approach will allow us to reach individuals who do not have a working telephone number. To encourage response, we will offer monetary incentives for completing each survey as shown in Table F.4. A thank you letter with incentive enclosed will be mailed to respondents of each survey regardless of completion mode.

0