EHS-CCP SSB_Revised Draft_12 14 15

The Study of Early Head Start–Child Care Partnerships

OMB Information Collection Request

New Collection

Supporting Statement

Part B

September 2015

Updated December 2015

Submitted by:

Office of Planning, Research and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

7th Floor, West Aerospace Building

370 L’Enfant Promenade, SW

Washington, DC 20447

Project Officers:

Christine Fortunato and Amy Madigan

CONTENTS

B1. Respondent universe and sampling methods 1

Target population 1

Sampling frame and coverage of target population 1

Design of the sample 2

Size of the sample and precision needed for key estimates 3

Expected response rate 7

Expected item nonresponse rate for critical questions 8

B2. Procedures for collection of information 8

Partnership grantee and delegate agency director survey 8

Child care partner survey 8

Case studies 9

B3. Methods to maximize response rates and deal with nonresponse 10

Expected response rates 10

Dealing with nonresponse 10

Maximizing response rates 11

B4. Tests of procedures or methods to be undertaken 12

B5. Individuals consulted on statistical aspects and individuals collecting and/or analyzing data 12

References 13

B1. Respondent universe and sampling methods

The Office of Planning, Research, and Evaluation (OPRE) within the Administration for Child and Families (ACF) at the U.S. Department of Health and Human Services seeks approval to collect descriptive information for the new Early Head Start-child care partnership competitive grant opportunity. This information collection is being carried out as part of the Study of Early Head Start-Child Care Partnerships. The purpose of the study is to document the characteristics and features of Early Head Start and child care partnerships and activities that aim to improve professional development and the quality of services and better meet families’ needs. In March 2015, ACF awarded 275 Early Head Start expansion and Early Head Start-child care partnership grants in the 50 states; Washington, DC; Puerto Rico; and the Northern Mariana Islands. This study will focus on grantees that received grant funds for partnerships. Proposed data collection activities for the study are: a web-based survey of 311 partnership grantee and delegate agency directors, a web-based survey of a subset of 933 child care partners, semistructured interviews and focus groups with informants in 12 case study sites, and grantee and delegate agency director and child care partner questionnaires in the case study sites.

Target population

The target population for the surveys is the universe of grants focusing on Early Head Start and child care partnerships (referred to as “partnership grantees” herein). As described in more detail below, the partnerships selected for the case studies will be chosen to illustrate different partnership models and are not intended to be representative of the population of partnerships. For each case study, respondents will include the partnership grantee or delegate agency director; all child care partner managers/owners; and selected frontline staff, families, and other state and local stakeholders.

Sampling frame and coverage of target population

The sampling frame for the partnership grantee and delegate agency director survey is the list of Early Head Start–child care partnership grantees and their delegate agencies. Attempts will be made to survey all partnership grantee directors using a web-based survey. If states or other large grantees delegate part or all of their responsibilities for the operation of the partnership to delegate agencies, those delegate agency directors will be surveyed in addition to the grantee. We plan to survey delegate agencies because we anticipate that there may be variation in the partnership models implemented across delegate agencies within a single grantee. As described in Supporting Statement A, the partnership grantee and delegate agency director survey will document the perspectives of 311 partnership grantees and delegate agencies. There are 251 partnership grantees. Several of these are large grantees (for example, states or large cities) that ACF anticipates will contract with delegate agencies to carry out grant activities. Information gleaned from the grantee applications suggests that we can expect approximately 12 large grantees, each with an expected five delegate agencies. Since the study team plans to survey all delegate agency directors, there would be a total of 251 + (12 * 5) = 251 + 60 = 311 partnership grantee and delegate agency directors.

As part of the partnership grantee and delegate agency director survey, directors will be asked to list their child care partners and provide basic information about each partner, including contact information and partner type (child care partners can be child care centers or family child care providers). The director-provided list of child care partners will be the sampling frame for the child care partner survey. Given that we will attempt to survey all partnership grantee and delegate agency directors, the sampling frames for the child care partner survey corresponds to the universe of child care partners.

As described in more detail below, the information from the surveys will be used to purposively select case study sites that illustrate differing partnership models. Given that attempts will be made to survey all partnership grantee and delegate agency directors, the sampling frame for the partnerships selected for case studies corresponds to the universe of partnerships.

Design of the sample

Partnership grantee and delegate agency directors. We propose to survey all partnership grantee and delegate agency directors. We anticipate a great deal of variation in how partnerships implement the provisions of the grant. Due to grant negotiations with ACF, we expect that partnership grantees’ plans, including the number and type of child care partners, will change substantially from plans presented in grant proposals. Since these partnership grantees are new, there are no administrative data available on grantee characteristics and activities. These factors, along with the lack of research in the field at large, mean that we do not have sufficient information about Early Head Start-child care partnerships to know which features need to be considered to ensure appropriate representation of the variability in partnership characteristics and models in a sampling plan. Even if we did know something about the features to consider, we do not have actual data about these particular partnerships on which to stratify. Therefore, if we do not survey all grantees and delegate agency directors, we may miss important information on characteristics and practices of the partnerships that were not included in the sample.

Sampling child care partners. From the director-provided list of child care partners, we propose to select an explicitly stratified random sample of partners within the survey instrument. The partnership grantee and delegate agency directors will be asked to provide more detailed information on this selected subset of child care partners. Based on information gleaned from the partnership grantee applications, we expect that there will be a total of more than 4,000 child care partners. Therefore, we propose that sampling a subset of child care partners (described in more detail below) will be sufficient to capture variation in important child care partner and partnership characteristics.

This explicitly stratified random sampling approach will involve dividing the partners listed by the partnership grantee and delegate agency director into two groups: child care centers and family child care providers. Within each group, the web-based survey will be programmed to automatically select a random sample of at least 20 percent of the partners, with a minimum of one (see below for more details on sample size). For example, as shown in Figure B.1, the selection process for a partnership grantee with a total of five child care partners including both types of partners (family child care providers and child care centers), would result in the selection of one of each type of partner. We propose sampling partners of each type because we expect that the partnerships’ approaches to implementation and, in particular, supporting quality improvement and delivering comprehensive services will differ between partnerships with child care centers and those with family child care providers. In addition, the literature review (Del Grosso et al. 2014) highlighted the lack of information available about partnerships with family child care providers.

The sample design will ensure that adequate numbers of child care centers and family child care providers are captured. Selecting at least one partner of each type may result in a sample of partners that is not proportional to their representation in the population of partners; in particular, this method could oversample partners of the minority type. At this stage, the hypothesis of ACF and the study team is that there will be fewer family child care providers than there will be child care centers. Since the field of early care and education currently lacks information and understanding regarding characteristics and experiences of family child care partners, ACF places particular importance on understanding these issues; thus, oversampling family child care partners is desirable. As described in Section A16 of Supporting Statement Part A, sampling weights will be needed to weight responses on the child care partner survey so that we can generalize to the population of child care partners.

Figure B.1. Sampling plan example

Sampled child care partners

Selecting case study sites. Data from the partnership grantee and delegate agency director and child care partner surveys will be used to profile partnership models (that is, the configurations of grantees and partners and the activities they carry out to implement the partnership services). After developing these profiles, we will recommend a site recruitment plan that represents the range of partnership models identified and select a small number of partnerships for in-depth case studies. This qualitative data collection is preliminary and exploratory, and will provide foundational opportunities for exploring more quantitative approaches in future work.

Size of the sample and precision needed for key estimates

Sample size. ACF has awarded 275 Early Head Start expansion and Early Head Start-child care partnership grants. This study will focus on the 251 grantees that received partnership funds rather than those that chose to expand their current program. As mentioned previously, information gleaned from partnership grantee applications suggests that we can expect approximately 12 large grantees, each with an expected five delegate agencies. Since the study team plans to survey all delegate agency directors, we assume a respondent universe of 251 + (12 * 5) = 251 + 60 = 311 partnership grantee and delegate agency directors.¹

Based on information from the partnership grantee applications, we assume there will be, on average, 15 child care partners per grantee and delegate agency. Fifteen partners per grantee * 311 grantees and delegate agencies = 4,665 child care partners in the respondent universe. We plan to randomly select 20 percent of each grantee and delegate agency’s reported child care partners for a total of 933 child care partners in the sample (Figure B.2).² This sample size is small enough to avoid undue burden on survey respondents, but it is large enough to capture variation in important child care partner and partnership characteristics.

Figure B.2. Sample size calculations

311	*	15	=	4,665	*	0.20	=	933
Grantees and delegate agencies		Child care partners per grantee		Child care partners in universe		Sample 20 percent of child care partners		Sample of child care partners

The study team will use data from the partnership grantee and delegate agency director and child care partner surveys to identify partnership models using methods such as cluster analysis. Based on these models, 12 partnerships will be identified as case study sites. Partnerships that differ along important dimensions will be identified to illustrate a wide range of models (such as approaches to combining funding sources, allocating funding across partnering organizations, supporting quality improvement needs, and delivering comprehensive services to families and children). It is expected that 12 partnerships will be a large enough number to exemplify important differences. Since the study is exploratory in nature and not designed to be an impact study, we are not as concerned about having sufficient power to detect small difference that may be policy relevant.

For each case study, we plan to conduct one interview with the partnership grantee or delegate agency director (1 per site * 12 sites = 12 total); three staff interviews (3 per site * 12 sites = 36 total); four telephone interviews with state and local stakeholders (4 per site * 12 sites = 48 total); child care center director (8 participants per site * 12 sites = 96 total), child care center teacher (8 participants per site * 12 sites = 96 total), parent focus groups (8 participants per site * 12 sites = 96 total) at all sites; and family child care provider focus groups at six sites (8 per site * 6 site = 48 total). A self-administered paper questionnaire will be given to the partnership grantee or delegate agency director (1 per site * 12 sites = 12 total) and all child care partners (15 partners per partnership * 12 sites = 180 total).

Precision needed for key estimates. We anticipate reporting means, proportions, and standard deviations for items and constructs from the partnership grantee and delegate agency director and child care partner surveys. For example, key estimates from the partnership grantee and delegate agency director survey are the average number of partners per grantee and the proportion of partnerships with at least one family child care provider, and the proportion of partners that are child care centers or family child care providers. The proposed sample sizes, accounting for varying levels of nonresponse, are sufficiently large to estimate means with a reasonable degree of precision. Table B.1 provides 95 percent confidence intervals for estimated means for a range of responding sample sizes corresponding to different response rates (see below for additional details on expected response rates). The confidence intervals assume that continuous variables follow the standard normal distribution (mean = 0, standard deviation = 1). Some survey items will be proportions. Table B.1 also provides 95 percent confidence intervals for estimated proportions for these samples, assuming that the true proportion is 0.5.

Table B.1. Confidence intervals (95 percent) for means and proportions

		Means		Proportions
Response rate	Sample size	Lower bound of CI	Upper bound of CI	Lower bound of CI	Upper bound of CI
Partnership grantee and delegate agency director survey
100%	311	-0.112	0.112	0.444	0.556
95%	295	-0.115	0.115	0.442	0.558
90%	280	-0.118	0.118	0.441	0.559
85%	264	-0.122	0.122	0.439	0.561
80%	249	-0.125	0.125	0.437	0.563
75%	233	-0.130	0.130	0.435	0.565
70%	218	-0.134	0.134	0.433	0.567
Child care partner survey
100%	933	-0.067	0.067	0.466	0.534
95%	886	-0.069	0.069	0.465	0.535
90%	840	-0.071	0.071	0.465	0.535
85%	793	-0.073	0.073	0.463	0.537
80%	746	-0.075	0.075	0.462	0.538
75%	700	-0.078	0.078	0.461	0.539
70%	653	-0.080	0.080	0.460	0.540

Note: The design effect due to unequal weighting is 1.02 for grantee directors and 1.10 for partners.

CI = confidence interval.

We also anticipate testing for significant differences in means or proportions between groups. A key contrast of interest at the partnership grantee and delegate agency director level will be between partnerships that include at least one family child care provider and those that include child care centers only. For example, we might compare partnership quality between these two types of partnerships. The proposed sample sizes, allowing for different levels of nonresponse, are sufficiently large to detect moderate differences. Table B.2 provides minimum detectable effect sizes and differences in proportions for a range of responding sample sizes for the partnership grantee and delegate agency director sample. Based on input from project consultants and other technical experts, we assume that 60 percent of partnerships will partner with child care centers only, and that 40 percent will include partnerships with at least one family child care provider. We also set alpha = 0.05 (two-sided), and power equal to 80 percent. To calculate minimum detectable effect sizes, we assume that the variable of interest follows a standard normal distribution, with a mean of 0 for partnerships with child care centers only. To calculate minimum detectable differences in proportions, we assume that the proportion for partnerships with child care centers only equals 0.5.

Table B.2. Minimum detectable effect sizes and differences in proportions, partnership grantee and delegate agency director sample

Response rate	Total sample size	Number of partnerships with child care center partners only	Number of partnerships with family child care partners only	Minimum detectable effect size	Minimum detectable difference in proportions
100%	311	187	124	0.328	0.164
95%	295	177	118	0.336	0.169
90%	280	168	112	0.345	0.173
85%	264	159	106	0.355	0.178
80%	249	149	100	0.366	0.184
75%	233	140	93	0.378	0.190
70%	218	131	87	0.391	0.197

Note: We assume that 60 percent of partnerships will include partners with child care centers only, and that 40 percent will include family child care providers. Alpha = 0.05 (two-sided), and power is equal to 80 percent. To calculate minimum detectable effect sizes, we assume that the variable of interest follows a standard normal distribution, with a mean of 0 for partnerships with child care center partners only. To calculate minimum detectable differences in proportions, we assume that the proportion of interest for partnerships with child care center partners equals 0.5. We assume that the design effect due to unequal weighting is 1.02 for grantee directors.

Differences between child care centers and family child care providers on important constructs such as length of relationship with the grantee will be key contrasts of interest for the child care partner survey. The study is powered to detect small to moderate effect sizes or differences in proportions for the child care partner sample. Table B.3 provides minimum detectable effect sizes and differences in proportions for this sample. We assume that 60 percent of partners will be child care centers and 40 percent will be family child care providers. Alpha = 0.05 (two-sided) and power is equal to 80 percent. To calculate minimum detectable effect sizes, we assume that the variable of interest follows a standard normal distribution, with a mean of 0 for child care center partners. To calculate minimum detectable differences in proportions, we assume that the proportion for child care center partners equals 0.5.

Table B.3. Minimum detectable effect sizes and differences in proportions, child care partner sample

Response rate	Total sample size	Number of child care center partners	Number of family child care partners	Minimum detectable effect size	Minimum detectable difference in proportions
100%	933	560	373	0.196	0.098
95%	886	532	355	0.201	0.101
90%	840	504	336	0.207	0.104
85%	793	476	317	0.213	0.107
80%	746	448	299	0.220	0.110
75%	700	420	280	0.227	0.114
70%	653	392	261	0.235	0.118

Note: We assume that 60 percent of partners will be child care centers and 40 percent will be family child care providers. Alpha = 0.05 (two-sided), and power is equal to 80 percent. To calculate minimum detectable effect sizes, we assume that the variable of interest follows a standard normal distribution, with a mean of 0 for child care center partners. To calculate minimum detectable differences in proportions, we assume that the proportion for child care center partners equals 0.5. We assume that the design effect due to unequal weighting is 1.10 for partners.

Finally, to group characteristics and activities into models, we anticipate conducting a cluster analysis to distinguish different partnership models based on survey items or constructs. For example, as described in Section A2 of Supporting Statement Part A, partnership models can include one or both types of partners (child care centers and family child care providers), differing approaches to funding (such as using Early Head Start funds to fund comprehensive services and quality improvement activities and child care subsidies and other funds to cover the cost of care), and multiple approaches to dividing responsibility across partnering organizations (such as training staff from child care partners conduct child assessments or having staff from the partnership grantee conduct assessments). There are no generally accepted guidelines regarding minimum sample sizes for cluster analysis. Cluster analysis can yield a result regardless of sample size or the number of variables used (Mooi and Sarstedt 2011).

Precision of estimates is not an issue in analyzing qualitative data from the case studies. Since the case studies are exploratory, we do not plan to calculate means or proportions or conduct statistical hypothesis tests.

Expected response rate

We expect a response rate of 85 percent for the partnership grantee and delegate agency director survey, a response rate of 80 percent for the child care partner survey, and a response rate of between 85 and 100 percent for data collection activities conducted during the case studies (see Section B3 for details on the basis for these expected response rate estimates). Table B.4 provides expected response rates and expected number of responses for each study instrument.

Table B.4. Expected response rates and number of responses

Data source		Number of entities in universe	Number of entities in sample	Expected response rate	Expected number of responses
Partnership grantee and delegate agency director survey	311		311	85%	264
Child care partner survey	4,665		933	80%	746
Interview topic guide
Partnership grantee and delegate agency directors		311	12 (1 per site)	100%	12
Partnership staff		n.a.	36 (3 per site)	100%	36
State and local stakeholders		n.a.	48 (4 per site)	85%	41
Parent focus group guide		n.a.	96 (8 per site)	85%	82
Child care center director focus group guide		n.a.	96 (8 per site)	85%	82
Child care center teacher focus group guide		n.a.	96 (8 per site)	85%	82
Family child care provider focus group guide		n.a.	48 (8 per site, 6 sites)	85%	41
Partnership grantee and delegate agency director questionnaire		311	12 (1 per site)	100%	12
Child care partner questionnaire		4,665	180 (15 per site)	80%	144

Note: ACF has awarded 275 Early Head Start expansion and Early Head Start–child care partnership grants. We assume that approximately 87 percent of grantees are partnership grantees, yielding a total of 251 partnership grantees. We further assume that 12 grantees are states or other large grantees. We assume that each of these large grantees will have 5 delegate agencies. Thus, we assume a respondent universe of 251 + 60 = 311 grantee and delegate agency directors. We assume there will be, on average, 15 child care partners per grantee or delegate agency; 15 partners per grantee * 311 grantees and delegate agencies = 4,665 child care partners in the respondent universe. We plan to randomly select 20 percent of each grantee’s reported child care partners: 0.20 * 4,665 = 933 child care partners in the sample.

n.a. = not applicable.

Expected item nonresponse rate for critical questions

The most critical question for this data collection is included in the partnership grantee and delegate agency director survey, and asks grantee directors to list each of their child care partners and provide contact information. A very low item nonresponse rate (5 percent or less) is expected for this question. However, we also expect that some respondents may not have complete contact information for all their partners. In these cases, the study team will make attempts to find the missing contact information using the information the partnership grantee or delegate agency director is able to provide.

It may also be the case that partnership grantee and delegate agency directors may not have identified all partners at the time of the survey. For example, a grantee or delegate agency could have a child care partner vacancy because they have not found a suitable partner or because they need to replace a partner that is no longer engaged in the partnership. In this case, they will fill out information on all named partners only. In this way, the survey provides a snapshot of Early Head Start-child care partnerships at a specific point in time.

B2. Procedures for collection of information

Partnership grantee and delegate agency director survey

The partnership grantee and delegate agency director survey will be web-based, although we expect to conduct 20 percent of partnership grantee and delegate agency director surveys by phone if a director is unwilling or unable to take the survey via the Internet (in this case, the telephone interviewer will enter the responses into the web-based survey on behalf of the respondent). This survey will take, on average, 60 minutes to complete. The partnership grantee director and delegate agency survey will be programmed to randomly select a subset of the child care partners listed by the director using explicitly stratified random sampling, as described above. The short timeline of the study necessitates sampling child care partners during the partnership grantee and delegate agency director survey; using a web-based survey with real-time sampling functionality is essential to the study design. Other advantages of the web-based survey include coverage (we expect that most, if not all, partnership grantee and delegate agency directors will have Internet access); convenience for respondents (for example, respondents may take the survey at a time suitable for them); and cost effectiveness.

Partnership grantee and delegate agency directors will be sent an advance email notification inviting them to take part in the study. Over the course of the data collection period, we will send up to six email reminders to nonresponders; we will also make up to two reminder calls at weeks 4 and 10. Supplemental materials for the partnership grantee and delegate agency director survey are included in Attachment M.

Child care partner survey

The child care partner survey will also be web-based, although we expect to conduct 30 percent of the child care partner surveys by phone. We anticipate that a slightly larger proportion of child care partner respondents will lack Internet access or otherwise be unable or unwilling to respond via the Internet; for example, because they will need to conduct the survey in a language other than English or Spanish. This survey will be, on average, 30 minutes long. Real-time sampling functionality is not necessary for the child care partner survey, but the other web-based advantages mentioned previously apply to this survey.

Child care partners will be sent an advance email notification inviting them to take part in the study. Over the course of the data collection period, we will send up to seven email reminders to nonresponders; we will make up to three reminder calls at weeks 3, 9, and 16. Since it may be the case that some child care partners lack Internet access, we will use the reminder calls to identify these cases. As needed, we will also send an email to the partnership grantee or and delegate agency to solicit their help with encouraging participation of their selected child care partners. Supplemental materials for the child care partner survey are included in Attachment N.

Case studies

We plan to conduct a 2.5-day site visit to up to 12 purposively selected partnerships. As discussed in Supporting Statement A, delegate agencies will be eligible as case study sites and, if selected, we will treat each as a separate partnership model. For each case study, we plan to conduct an interview with the partnership grantee or delegate agency director, three staff interviews, and four telephone interviews with state and local stakeholders. At each case study site, we plan to convene focus groups with parents, child care center directors, and child care center teachers. We will also conduct family child care provider focus groups when partnerships include family child care homes (we have planned for half of the case study sites to include this type of partner). We will administer a self-administered paper questionnaire to the partnership grantee or delegate agency director and all child care partners. Case studies involving in-person site visits will allow the study team to address the research questions in more depth than the web-based surveys and incorporate multiple perspectives: grantee directors, the partnership’s full set of child care partner managers/owners, frontline staff, families, and other state and local stakeholders.

Recruitment and supplemental materials for the case study data collection activities are included in Attachment O. The study team will send an advance email to each partnership grantee and delegate agency director to notify them that they have been selected to participate in the case studies and to coordinate a scheduling call with them. During a follow-up telephone call to each director, the study team will describe the purpose of the case studies, confirm the director’s interest in participating, provide an overview of the case study activities, and begin working with the director to schedule the visit. The study team will send a memo summarizing the call to the director or someone designated by the director to work with the study team to coordinate the visit. The memo will include information about the number and types of staff that we would like to recruit for participation in the case study activities and lay out a plan for confirming their participation.

The study team will recruit child care center directors, child care center teachers, family child care providers (if applicable), and parents for focus groups. The first step will involve working with the partnership grantee or delegate agency director to confirm their current list of partners. From the list of partners, we will select ten child care centers. From this list, the study team will recruit eight child care center directors and eight teachers. In the sites that partner with family child care providers, we will select ten providers and recruit eight. In selecting the child care centers and family child care providers, the study team will identify centers and providers located in close proximity to the partnership grantee or delegate agency (such as within 15 miles of a partnership grantee located in an urban or suburban area). Once the child care centers and family child care providers have been selected, the study team will request that the partnership grantee or delegate agency director distribute the advance letter about the study and ask parents to contact the study team if interested in participating. The study team will recruit eight parents from those who express interest for the parent focus groups to be convened at each case study site. Each selected child care partner will receive an advance letter describing the study, as well as a recruitment telephone call during which the study team will describe the study and identify and invite directors and teachers to the focus groups. Parents will receive an advance letter and recruitment telephone call (Attachment O).

The study team will use the list of child care partners gathered from the partnership grantee and delegate agency director survey and updated during the case study recruitment process to identify all the partners that will receive a child care partner questionnaire. For the partnership grantee and delegate agency director and child care partner questionnaires, we will send one reminder call to grantee and delegate agency directors; and one reminder call and two email reminders to child care partners (Attachment P).

B3. Methods to maximize response rates and deal with nonresponse

Expected response rates

As described in Section B1, we expect a response rate of 85 percent for the partnership grantee and delegate agency director survey and a response rate of 80 percent for the child care partner survey. The response rates for the case studies vary by activity and range from 100 percent for the director interviews and 85 percent for focus groups (see Table B.4). These expected response rates are based on those achieved in prior surveys with similar populations. In particular, 89 percent of programs completed the Survey of Early Head Start Programs (Vogel et al. 2006). On the Head Start/Child Care Partnership Study, the response rate was 65 percent for child care center providers (Schilder et al. 2005). We expect the response rate for the child care providers to be higher than those achieved in the Head Start/Child Care Partnership Study given the child care partners we will be surveying are part of a federally funded grant program; they will likely be motivated to participate because they are vested in the success of the Early Head Start–child care partnerships grant initiative.

Dealing with nonresponse

Nonresponse weights. As described in Section A16 of Supporting Statement Part A, we will produce analysis weights for both web-based surveys that account for selection probabilities (partners only) and for differential nonresponse patterns (grantees and partners). These weights will be constructed in such a way as to mitigate the risk for nonresponse bias. Should response rates fall below 80 percent, we will conduct a nonresponse bias analysis, in accordance with OMB guidelines.

Case study site refusals. While we anticipate high levels of participation in the case studies, we will implement plans to mitigate any refusals by grantees or delegate agencies to participate in the case studies. When selecting partnership models for the case studies, we will identify two partnership grantees or delegate agencies representing each model (a primary site and an alternate site). If the first partnership grantee or delegate agency is unable to or refuses to participate in the case studies, we will approach the alternate site.

Maximizing response rates

Obtaining high response rates including 85 percent for the partnership grantee and delegate agency director survey and a response rate of 80 percent for the child care partner survey may be challenging. Therefore, the research team is planning to be persistent in e-mails and follow-up calls to get these surveys completed.  We also plan to place multiple calls at varying times of the day and days of the week and as needed request the assistance of regional program managers who work closely with grantee directors. To maximize response rates for this information collection, we will take the following steps:

• Advance email notification for the web-based surveys. Partnership grantee and delegate agency directors and selected child care partners will be sent an advance email notification inviting them to take part in the study (Attachments M and N, respectively). The advance email includes a brief overview of the purpose of the study, a description of the data collection activity in which we are asking them to participate, and an estimate of the amount of time required to complete the activity. It will also include information needed to complete the survey (such as log-in credentials). Respondents will also be provided with a number they can call should they have any questions about their participation in the study.

• Reminder notifications. Over the course of the data collection period, we will send up to six email reminders to partnership grantee and delegate agency directors who are invited to complete the survey; we will also make up to two reminder calls to nonresponders at weeks 4 and 10 (Attachment M). Similarly, up to seven email reminders will be sent to child care partners selected to complete the survey; we will make up to three reminder calls to nonresponders at weeks 3, 9, and 16. As needed, we will also mail a letter to the grantee or delegate agency to solicit their help with participation of their selected child care partners (Attachment N). We will implement a similar approach for the return of completed partnership grantee and delegate agency director and child care partners questionnaires that are part of the case studies—one reminder call to directors, and one reminder call and two email reminders to child care partners (Attachment P).

• Trained and experienced data collection staff. Reminder calls will be conducted by trained members of the data collection team, many with significant experience working on similar studies. Similarly, trained and experienced members of the study team will lead the case study activities. These staff will lead efforts to recruit grantees and delegate agencies to participate in the case studies and will lead recruitment for the focus groups. All staff assigned to the study will participate in both general training (if they are not already trained) and an extensive project-specific training. The project-specific training will include role playing with scenarios and other techniques to ensure that interviewers are ready to respond effectively to respondents’ questions. They will also focus on developing skills for securing respondents’ cooperation and averting and converting refusals.

• Flexibility in language of administration. Spanish versions of the child care partner survey and questionnaire will be available to Spanish-speaking respondents. During telephone contact, interviewers will identify Spanish-speaking respondents and connect them to speak with a certified Spanish-language interviewer. Mathematica employs staff who have experience conducting interviews in Spanish. If the study team determines that surveys or questionnaires will need to be conducted in languages other than English or Spanish, we will work to secure the commitment of interviewers with the necessary language skills. If needed, the data collection team will be prepared to conduct focus groups in Spanish with parents and family child care providers.

• Incorporating telephone administration into the study design. We expect to administer a portion of the web-based surveys and questionnaires by telephone: 20 percent of the partnership grantee and delegate agency director surveys, 30 percent of the child care partner surveys, and 20 percent of the child care partner questionnaires. We do not anticipate a need to complete the partnership grantee and delegate agency director questionnaires by telephone given these will be collected while on site visits.

• Gifts of appreciation. As described in Section A9 of Supporting Statement Part A, we plan to offer respondents a gift of appreciation for participation in the study activities.

B4. Tests of procedures or methods to be undertaken

As described in Supporting Statement Part A, many of the items included in the partnership grantee and delegate agency director and child care partner surveys were adapted from existing surveys (See Attachment B for additional information). The study team has also developed new items for measuring constructs for which existing measures are not currently available. These items have drawn ideas for phrasing and language from prior research on Early Head Start and child care.

We conducted cognitive interviews and telephone pretests of the surveys in summer 2015 to ensure that questions are understandable, use language familiar to respondents, and are consistent with the concepts they aim to measure; to identify typical instrumentation problems such as question wording and incomplete or inappropriate response categories; to measure the response burden; and to confirm that there are no unforeseen difficulties in administering the instrument via telephone. In addition, the study team will conduct a careful review of the web-based instruments to ensure that the flow through the instrument is working properly.

B5. Individuals consulted on statistical aspects and individuals collecting and/or analyzing data

Mathematica, subcontractor Dr. Margaret Burchinal of the University of North Carolina at Chapel Hill, and consultants Dr. Diane Horm of the University of Oklahoma at Tulsa and Dr. Jessica Sowa of the University of Colorado Denver are conducting this project under contract number HHSP23320095642WC. Mathematica developed the plans for statistical analyses for this study. The team is led by the following individuals:

Christine Fortunato, Ph.D. Project specialist Office of Planning, Research, and Evaluation	Amy Madigan, Ph.D. Project officer Office of Planning, Research, and Evaluation
Laura Hoard, Ph.D. Project officer Office of Planning, Research, and Evaluation	Diane Paulsell, M.P.A. Project director Mathematica Policy Research
Patricia Del Grosso, M.S. Deputy project director Mathematica Policy Research	Eileen Bandel, Ph.D. Survey director Mathematica Policy Research
Jaime Thomas, Ph.D. Senior researcher Mathematica Policy Research

In addition, Barbara Carlson, M.A. (senior statistician) and Cheri Vogel, Ph.D. (senior researcher) from Mathematica were consulted on statistical methods.

References

Del Grosso, P., L. Akers, A. Mraz Esposito, and D. Paulsell. “Early Childhood Education Partnerships: A Review of the Literature.” OPRE Report 2014-64. Washington, DC: U.S. Department of Health and Human Services, Administration for Children and Families, Office of Planning, Research and Evaluation, 2014.

Mooi, E., and M. Sarstedt. A Concise Guide to Market Research: The Process, Data, and Methods Using IBM SPSS Statistics. New York: Springer Science & Business Media, 2011.

Schilder, D., B. W. Chauncey, M. Broadstone, C. Miller, A. Smith, S. Skiffington, and K. Elliott. “Child Care/Head Start Partnership Study: Final Report.” Newton, MA: Education Development Center, 2005.

Vogel, C. A., N. Aikens, A. Burwick, L. Hawkinson, A. Richardson, and L. Mendenko. “Findings from the Survey of Early Head Start Programs: Communities, Programs, and Families.” Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, 2006.

1 A delegate agency is a local public or private entity to which the partnership grantee has delegated all or part of its responsibility for operation of the program.

2 The 933 sample is an upper bound for the number of child care partners to be surveyed. The expected response rate for the partnership grantee and delegate agency director survey is 85 percent, yielding an expected sample size of 264 partnership grantees and delegate agencies. Taking this into account, the expected number of child care partners to be sampled is 264 * 15 = 3,965; 20 percent of the expected number of child care partners is 0.20 * 3,965 = 793.