OMB Supporting Statement Part B HSIS Tracking

1. The Head Start Impact Study (HSIS) sampling universe was all Head Start grantees/delegate agencies operating in all 50 states, the Commonwealth of Puerto Rico, and the district of Columbia that do not specifically target the following special populations: grantees/delegate agencies serving migrant children; Head Start programs operated by Tribal organizations; children enrolled in Early Head Start; new grantees/delegate agencies (i.e., grantees/delegate agencies that are in operation for less than two years); and programs involved in FACES 2000. The starting point for creating this initial population of Head Start grantees/delegate agencies was the 1999-2000 Program Information Report (PIR) database maintained by ACYF. Migrant and Tribal Organization grantees/delegate agencies can be readily identified from this database, and “new” programs were identified as those grantees/delegate agencies that were listed in the 1999-2000 PIR but which were not listed in the 1998-1999 PIR (i.e., eliminating grantees/delegate agencies that were in operation for approximately less than two years). Early Head Start children were identified and excluded once the sample of grantees/delegate agencies had been selected. In fall 2002, a sample comprising 4,667 children and their families was randomly selected, from the applicant lists to a sample of Head Start centers. This sample was randomly assigned within centers (in a few cases, within programs) to either a treatment or a control group in a sample size ratio of 1.5. (See Section A. 2 for details of numbers of three- and four- year olds randomly assigned to the Treatment and Control groups.) The complete HSIS sampling plan is available in the original OMB package (OMB #0970-0229, Expiration Date: 09/30/2005).

2. The Third Grade Follow-up to the Head Start Impact Study (HSIS) sample continued to use only the 4,667 children and their families selected for the HSIS.

3. The Tracking of Former Participants in the Head Start Impact Study (HSIS) continued contact with 4,243 children and their families who had participated in the HSIS and Third Grade Follow-up studies in spring 2009, 2010 and 2011. This sample excluded, with ACF agreement, 230 hard refusals (those who refused if we continued to contact them) and 194 families that never participated during HSIS or the Third Grade Follow-up.

4. The Head Start Impact Study (HSIS) Participants Beyond 8^th Grade consisted of contacting 4,235 children and their families who participated in the HSIS, Third Grade Follow-up and/or the Tracking of Former HSIS Participants, excluding with ACF agreement, only 8 additional hard refusals identified during the spring 2009, 2010 and/or 2011 data collections. Data collection is scheduled for spring 2012 through spring 2016. An 85 percent response rate was achieved for spring 2012 and 2013. The 2014 data collection is ongoing with an expected 85 percent response rate. The 85 percent response rate is based on 4,235 children and their families, with the hard refusals (N=238) and families that have not participated in the study during any data collection period (n=194) removed from the original sample of 4,667.

The respondent universe and sampling methods for the current study for which we are seeking OMB approval is:

The last two years’ data collection for the study, Head Start Impact Study (HSIS) Participants Beyond 8^th Grade, which will be conducted in spring 2015 and 2016. The sample will consist of 4,235 children and their families who participated in the HSIS, Third Grade Follow-up, Tracking of Former HSIS Participants, and /or the first three years’ data collection for the HSIS Participants Beyond 8^th Grade. The expected response rate is 85 percent and little or no item non-response is anticipated.

B.2 Procedures for Collection of Information

Table 4 below provides the sample sizes for the original HSIS, broken out by age cohort and treatment condition (Section B.1 above describes the sampling procedures), with the anticipated sample sizes for the current study, HSIS Participants Beyond 8^th Grade. An attempt will be made to interview a parent of each of the 4, 235 children during each data collection period, with the expectation of reaching the 85 percent response rates achieved in 2012 and 2013.

Table 4. Expected Sample Sizes for the Study: Head Start Impact Study Participants Beyond 8^th Grade

	Fall 2002 sample size	Expected sample size for continual follow-up
Parent Interview
Head Start Treatment Group	2,783	2,589
age 3	1,530	1,441
age 4	1,253	1,148
Control Group	1,884	1,646
age 3	1,029	903
age 4	855	743
Combined Treatment & Control	4,667	4,235

Field staff will be used to conduct the telephone parent interviews (and in-person when necessary) for the HSIS Participants Beyond 8^th Grade as they have been used in each round of the HSIS and follow-up contacts to date. When cases are fielded for the parent interview, Westat prepares child profile reports, which include pertinent information on the families’ current telephone number, home address, work number, as well as telephone numbers of alternate contacts from the most recent locating information collected. In many cases this information is sufficient to easily reach the family. In cases where the telephone number is disconnected, Westat interviewers use the methods listed below to trace respondents. Tracing proceeds by following leads in a specific order, narrowly focused at first and then expanding as needed. Site coordinators are instructed to first use resources that are available from their homes before going into the field to physically reach the respondent. These resources, presented in the order to be implemented, include the following:

Calling alternate contacts. Telephone numbers and addresses of alternate contacts have been collected during each fall and spring data collection for both the HSIS and the Third Grade Follow-up, and then each spring for the Tracking of Former Participants in the HSIS through spring 2011 and the HSIS Participants Beyond 8^th Grade through spring 2014. Parent respondents identified these contacts during the full parent interviews and the shorter contact interviews with the understanding that these people would be contacted if we have difficulty reaching the respondent. If the alternate contact cannot provide a new telephone number or address for the respondent, interviewers ask if they know of anyone else who might know how to reach the respondent, if they know where the respondent works or is attending school, if they know the school the child attends, and whether the respondent might have married and taken a new name.
Telephone Directory. Local telephone directories can be excellent resources. Interviewers can first see if the respondent is listed at another address in or near the city. If calls to those numbers are unsuccessful and the surname is unusual, we call all of the others with the same last name and ask if they know the respondent. If the last name is common (e.g., Jones or Smith), interviewers look for a corroborating piece of information (e.g., the respondent’s middle name or a relative’s name) and call those numbers. They also try others with the same last name who live on the same street or very close to the respondent’s last address as family members often live in the same area.
Directory Assistance. Directory assistance often provides a new telephone number for families who have moved. Interviewers start with the respondent’s name (or in some cases a contact’s name) or relative’s name or branch out to only the last name. Some telephone numbers are listed under the child’s name so this is also a recommended name to try.
Sending letters to respondents in care of alternate contacts. When we have only an address for an alternate contact but no working telephone number, site coordinators may send a letter to the respondent in care of the alternate contact, informing the family that we are trying to reach them for the current round of data collection and providing a local or toll-free number the respondent can call to schedule an appointment.
Internet directories. Field staff can access search engines such as www.theultimates.com, which searches a number of directory databases simultaneously. Westat also subscribes to Lexis-Nexis, a very powerful Internet tool for locating and following difficult cases. It was used very effectively in the Third Grade Follow, Tracking of Former Participants in the HSIS, and HSIS Participants Beyond 8^th Grade through spring 2014.
Contacting neighbors via telephone. We also use Internet search engines to look up the telephone numbers of families who live on the same street as the respondent. Interviewers can call these neighbors to determine whether the family still lives at that address or to find out where the family moved. We have found that neighbors are often willing to provide this information over the telephone.

After lower cost at-home tracing methods have been exhausted, interviewers are instructed to conduct field tracing. Primary sources of information are the people who may know the respondent and who are most likely to have personal knowledge of the parent/primary caregiver’s whereabouts; they are therefore the most significant and productive tracing sources. Even if they are unable to give the interviewer an exact address, they will often provide clues that will eventually lead to a more knowledgeable source. They may also be able to provide the names, and possibly telephone numbers, of family members or friends of the family. Primary sources include the following:

1. Current residents at the respondent’s last known address. While interviewers are at the family’s last known address, they may ask the current resident if he or she knows the family’s current location. If the current resident bought the home from the respondent, the current resident may be able to provide the name of a lawyer or real estate agent who might know the respondent’s new location. If the original address is a rental property, the landlord may be able to provide the family’s current address.

2. Other neighbors. If other neighbors living within a few houses or apartments of the original address are available, interviewers may also contact them regarding the respondent’s new whereabouts.

3. Postal Service. Local post offices can be good sources of tracing information. If the person at the front counter is busy with customers, interviewers may ask to speak with someone in the back who has more contact with the carriers and may know the customers on various routes. Our experience has found that post offices in rural areas are often most helpful in locating respondents.

4. Schools. Schools may also provide information on a family’s current address or telephone number. Contacts with the schools are typically made by the site coordinator. Even when schools are not willing to release this information, they will often allow the site coordinator to write a note to the parent to be sent home with the child.

These procedures capitalize on the extensive experience, trust, and rapport achieved by field staff to form positive long-term relationships with the families and children selected for the study. Over the years this relationship has been important in obtaining parent cooperation and participation. Many study staff have worked with the same families since the beginning of the study, and have built trusted and comfortable relationships with them.

Retention rates for field supervisors are very high and generally are in the 90 to 100 percent range from one spring data collection to the next spring data collection with the retention rates for interviewers running about 85 to 90 percent. Highly experienced, skilled and trained field teams consisting of site coordinators and interviewers will be used under the supervision of the study’s operations director. The operations director will implement the data collection plan and ensure monitoring and quality control tasks are conducted in an efficient, organized, and timely manner to continue the collection of high quality data and achieve the high response rates consistently maintained through the various components of the HSIS. Site coordinators will serve in a pivotal role as the primary local contact, enlisting cooperation and maintaining participation of respondents; coordinating all data collection activities in the site; managing field interviewers; and ensuring quality control. They will report to the central office operations director and her staff. The number of field interviewers and the time allocated will be adjusted according to the number of cases and their location within the site.

B.3 Methods to Maximize Response Rates and Deal with Nonresponse

Expected Response Rates

The response rate for the parent interviews has been 85 percent since 2009 and is expected to remain the same for spring 2015 and 2016. Locating families and obtaining contact information is the primary goal of this data collection. We will employ several strategies to minimize sample attrition and obtain a high response rate. Minimizing attrition requires continuing to engage families who have participated in the past and convincing families who have not participated in recent rounds to rejoin the study. Both of these components, required to minimize attrition, are discussed below.

Dealing with Nonresponse

Converting Families Who Have Refused in the Past. Through every data collection of the HSIS, Third Grade Follow-up, Tracking Former Participants in the HSIS, and HSIS Participants Beyond 8^th Grade, the interviewers tried to contact each family who was randomly assigned to the study. The only exceptions were cases that had been determined to be “hard refusals.” These were individuals who told us, in no uncertain terms, that they did not wish to participate in the study and did not want to hear from a representative of the HSIS again. While respondents can be encouraged and often persuaded to participate, participation is voluntary. It is important for the integrity of the study that respondents participate at will. The HSIS started with 4,667 families with 4,235 remaining for the last two years’ data collection for the HSIS Participants Beyond 8^th Grade.

Field staff members also are trained in identifying, understanding, and responding appropriately and effectively to reasons why parents may be hesitant to cooperate. They are taught strategies to be empathetic with the parents, reassure them and ensure their cooperation including: (1) establishing the purpose of the call by identifying themselves as calling in connection with their child’s participation in a national study and referring to the child and the name of the child’s school to avoid respondents concluding that this is a telemarketing call; (2) verifying the legitimacy of the study and giving respondents key information such as the interviewer’s name, site coordinator’s name, Westat’s name, the study name, and the toll-free number; (3) establishing rapport and asking questions during the introduction that elicit a “yes” response, making it harder for the respondent who agrees with you to refuse later; (4) offering token agreement and understanding of the respondent’s viewpoint, e.g., “I can understand that” or “you certainly have the right to feel that way” to establish rapport and a sympathetic ear and disarm negativity; and (5) reassuring the respondent that the entire interview is private and individual responses will not be revealed. Field staff members are trained to try to find out why the respondent is reluctant and to address these concerns. Just as one respondent differs from another, the reasons for refusals are many and varied. Therefore, it is necessary to train field staff to become sensitive to how firm a “no” they are receiving and sense the reasons behind the hesitancy to develop ways to reassure the respondent and gain cooperation. If field staff still finds that they are not able to convince a parent to participate in the interview, they are trained to try and end on a friendly note and lay the groundwork and leave the door open for someone else to make another attempt in future, an approach that often has proven very effective.

Maximizing Response Rates

Engaging Current Participants. To keep respondents interested in continuing their participation, we will continue to emphasize that the HSIS was and continues to be a ground-breaking and important study. Advance letters will be mailed out that explain the study’s significance, the importance of continued participation, the fact that participation is voluntary, a cash thank you will be provided for participation, the assurance of privacy, and the fact that participation will not affect their current benefits. A copy of the advance letter is included in Appendix G. A cash thank you can motivate parents to participate by demonstrating the extent to which we value their time and insights. We will provide a cash thank you of $20 for each respondent to acknowledge the value of their continued participation as a respondent for this study.

Field interviewers and site coordinators are trained in strategies for gaining and maintaining cooperation. These include: (1) being prepared and confident in responding to questions regarding the study; (2) being enthusiastic about the study; (3) listening attentively to what the respondent is saying; (4) maintaining a pleasant, friendly, professional attitude and emphasizing the positive as opposed to arguing; (5) offering options and displaying flexibility in accommodating parents’ schedules; (6) letting the respondent know how important he or she is and how important the study is; and (7) emphasizing the privacy of responses.

B.4 Tests of Procedures or Methods to Be Undertaken

We are seeking OMB approval for the last two years’ data collection for the HSIS Participants Beyond 8^th Grade in spring 2015 and spring 2016. We will use the parent interview as used in previously approved OMB data collections with slight modifications. These modifications will include additional questions to get more complete contact information from the primary caregiver and for the child including cell phone number and email address for when the child reaches 18 years of age. Also, we have added a small set of questions on parents’ perceptions of children’s school safety and engagement as well as parent’s knowledge of children’s activities (parental monitoring). These questions assess important intermediary factors that may help with the interpretation of long-term outcomes findings in a potential HSIS follow-up study. See Appendix B for the justification and sources (when appropriate) for these questions.

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The contractors who worked on the initial design for the HSIS (28200-0022) and the Third Grade Follow-up (contracts 23302-008706TK001 and HHSP23320062929YC), include Westat (prime contractor), Chesapeake Research Associates, Abt Associates, Urban Institute, and the American Institutes for Research. For the Tracking of Former Participants in the HSIS and the first three years’ data collection for the HSIS Participants Beyond 8^th Grade, no statistical analysis of the data was required. It is not expected that statistical analysis will be required for the last two years’ data collection for the HSIS Participants Beyond 8^th Grade but should it be required, Westat statistical staff will be employed as necessary for this contract.

Project Staff

Frank Jenkins, Senior Statistician, Westat

Camilla Heid, Project Director, Westat

Janet Friedman, Operations Director, Westat

Federal Project Officer

Mary Bruce Webb, OPRE

Bibliography

Cantor, D., O’Hare, B. and O’Connor, K. (2007) “The Use of Monetary Incentives to Reduce Non-Response in Random Digit Dial Telephone Surveys” pp. 471-498 in J. M. Lepkowski, C. Tucker, J. M. Brick De Leeuw, E., Japec, L., Lavrakas, P. J., Link, M. W., & Sangster, R. L. (Eds.), Advances In Telephone Survey Methodology, New York: J.W. Wiley and Sons, Inc.

Laurie, H. and P. Lynn (2009) “The Use of Respondent Incentives on Longitudinal Surveys.” pp. 205 – 231, in P. Lynn (ed) Methodology of Longitudinal Surveys” John Wiley & Sons: Southern Gate.

Rodgers, Willard L. (2011) “Effects of Increasing the Incentive Size in a Longitudinal Study.” Journal of Official Statistics, Vol. 27, No. 2, 2011, pp. 279-299.

Schulman, K., Blank, H., and Ewen, D. (1999). Seeds of success: State prekindergarten initiatives 1998-1999. Washington, DC: Children’s Defense Fund.

U.S. General Accounting Office. (1997). Head Start: Research provides little information on impact of current program. Washington, DC: U.S. Government Printing Office.

U.S. General Accounting Office. (1998). Head Start: Challenges in monitoring program quality and demonstrating results. Washington, DC: U.S. Government Printing Office.

Zagorsky, Jay and Rhoton, Patricia (2008), “The Effects of Promised Monetary Incentives on Attrition in a Long-Term Panel Survey” (Fall 2008). Public Opinion Quarterly, Vol. 72, Issue 3, pp. 502-513.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Camilla Heid
File Modified	0000-00-00
File Created	2021-01-25

OMB Supporting Statement Part B HSIS Tracking