Q-CCIIT PDT SSB_REVISED_6-29-18_clean

Q-CCIIT PDT SSB_REVISED_6-29-18_clean.docx

Professional Development Tools to Improve the Quality of Infant and Toddler Care (Q-CCIIT PD Tools) Project

OMB: 0970-0513

Document [docx]
Download: docx | pdf


Professional Development Tools to Improve the Quality of Infant and Toddler Care (Q-CCIIT PD Tools) Project

OMB Information Collection Request

New Collection

Supporting Statement

Part B

February 2018

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services


4th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201


Project Officers:

Ann Rivera, Ph.D.

Amy Madigan, Ph.D.


CONTENTS

PART B. SUPPORTING STATEMENT FOR PAPERWORK REDUCTION ACT SUBMISSION 1

B1. Respondent Universe and Sampling Methods 1

B2. Procedures for Collection of Information 3

B3. Methods to Maximize Response Rates and Deal with Nonresponse 5

Expected Response Rates 5

Dealing with Nonresponse 5

Maximizing Response Rates 6

B4. Tests of Procedures or Methods to Be Undertaken 7

B5. Individual(s) Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 13





TABLES

B.1 Sample distribution by center-based classrooms and FCCs 3

B.2 Expected response rates and number of responses, by data source 5

B.3. Analytic methods used to inform research questions 10

B.4 Precision of estimates and minimum detectable correlations 13

B.5 Minimum detectable effect sizes (between subgroups) 13

B.6 OPRE project officers and study team leadership for the Q-CCIIT PD Tools project 14



Appendices

Appendix A. Research questions and sources

Appendix B. 60-Day Federal Register Notice and Comments

Appendix C. Mathematica Confidentiality Pledge

Appendix D. Study recruitment materials

Appendix E. Study participant consent form



Attachments (Study instruments)

Attachment 1. ECE setting eligibility screener

Attachment 2. PD provider training survey

Attachment 3a. Background survey – Caregiver

Attachment 3b. Background survey – PD Provider

Attachment 4a. Feedback survey – Caregiver

Attachment 4b. Feedback survey – PD Provider

Attachment 5. Website user pop-up questions

Attachment 6. Classroom information roster


PART B. SUPPORTING STATEMENT FOR PAPERWORK REDUCTION ACT SUBMISSION

The Administration for Children and Families (ACF) in the U.S. Department of Health and Human Services (HHS) seeks approval to conduct a field test of We Grow Together: The Q‑CCIIT Professional Development System. These professional development (PD) tools and the interactive website featuring the materials were developed to promote high-quality caregiver-child interactions in settings serving infants and toddlers. We Grow Together is based on the principles and practices emphasized by the Quality of Caregiver-Child Interactions for Infants and Toddlers (Q-CCIIT) observational measure. This field test aims to: (1) examine changes in caregiver practice that we expect to be associated with use of the We Grow Together system, and (2) examine implementation of We Grow Together. As a secondary goal, we will further evaluate the psychometrics of the Q-CCIIT measure. Ultimately, findings from this field test will inform refining We Grow Together to make evidence-based practices as user-friendly as possible for caregivers of infants and toddlers.

This request is for collecting data from field test participants from summer 2018 through spring 2019. First, we will screen ECE settings for eligibility in summer 2018. Second, we plan to collect a background survey with caregivers and PD providers in fall 2018.1 Third, in fall 2018 and spring 2019 we plan two rounds of classroom observations using the Q-CCIIT measure and classroom rosters.2 Fourth, we plan to collect data on the use and implementation of We Grow Together through the interactive website from fall 2018 through spring 2019. Finally, we plan to conduct a feedback survey with caregivers and PD providers in spring 2019.

B1. Respondent Universe and Sampling Methods

The goal of the field test is to evaluate use of the We Grow Together system by typical early childhood professionals working in a variety of care settings serving infants and toddlers. Therefore, we aim to have a diverse range of caregivers and PD providers participating in the study to understand whether caregivers are learning the principles and practices emphasized in We Grow Together. This study will include only centers and family child care (FCC) settings that already have local PD providers—the study team will exclude ECE settings without established PD providers. The field test will include a purposive sample of 175 center-based classrooms (approximately half will be affiliated with Early Head Start [EHS], and half will be community-based child care programs). The field test sample will also include 125 FCCs, including those operating under EHS and in community-based settings. The disadvantage of purposive sampling is that it will not allow the resulting estimates to be generalized to the population from which the study participants were selected. However, purposive sampling will allow us to include settings and participants that meet diverse criteria and still ensure efficiency in data collection activities at each stage, thereby helping to manage costs for this study.

The field test will take place in 10 geographic areas across the country. These areas will be chosen purposively based on a number of factors described below. To form catchment areas, these areas would branch out from the location of an EHS program, and their boundaries drawn to encompass a service area of sufficient size to contain enough of the various types of child care programs required for the study. These areas will vary in size depending on their population densities. The study team will use the Head Start Program Information Report (PIR) to identify all EHS programs and then form the geographic areas, some of which are likely to contain more than one EHS program. Based on the characteristics of the geographic areas, the EHS program(s) within them, and the types of services they provide, the study team will purposively select the 10 geographic areas for the study.

Additional criteria that can inform our choice of geographic areas will include achieving diversity by region of the country, state requirements or standards on early care and education (ECE) adult-child ratios and PD providers/coaches, states with PD provider registries, states with infant/toddler specialists or mental health specialists, states that have Educare schools,3 local racial, ethnic and linguistic composition of households, and median household income. Some of this information, as well as other demographic information, will come from the PIR, some from public sources containing information on state policies and systems and local demographic and economic characteristics. These geographic areas will be treated as sampling strata, not clusters, because we do not plan to generalize the study’s findings beyond these areas.

Within each area, the study team will use various sources, such as the PIR and lists of certified center- and family-based child care providers (using the local child care resource and referral [CCR&R] system, state quality rating and improvement system databases, approved PD provider lists, lists from the Office of Child Care and the State Capacity Building Center, and FCC provider networks), to begin the sampling process.4 For EHS-based centers and classrooms, the study team will select EHS programs from the PIR and then select centers and classrooms within the programs.5 For selected EHS programs that are part of EHS-Child Care Partnership [EHS-CCP] grants, the study team will also get lists of their partner community-based child care centers and FCCs. For community-based centers not partnered with EHS grantees, the study team will use web searches and the local CCR&R system to identify and select centers. The study team will begin recruiting centers and FCCs that meet the criteria for diverse settings in closest proximity to a central EHS center, then move out from there until recruitment targets are reached. The goal is to ensure that the study includes different types of settings while managing data collection costs. In center-based settings, the study team proposes to recruit two classrooms per center on average (one infant classroom and one toddler classroom, if available). For community-based FCCs not partnered with EHS grantees, the study team will use FCC provider networks and snowball sampling to identify and select PD providers and the FCCs to which they are providing their services. The study team expects to recruit one or two FCCs per FCC PD provider.

The study team will continue to recruit in each geographic area until we meet our target for a total of 175 center-based classrooms and 125 FCCs. These settings will be from a mix of providers with varied funding sources. Given the challenges of recruiting FCCs that are relatively unfamiliar with classroom observations and research, the study team will recruit a majority of the FCCs from those associated with EHS and EHS-CCP grantees or involved in the area PD provider networks. This approach will also maximize access to FCCs that have a relationship with PD providers.

As shown in Table B.1, the approximate sample target for each geographic area is 8.75 centers and 12.5 FCCs. Assuming 2 classrooms per center and 1 to 2 FCCs per FCC PD provider, the result would be 17.5 classrooms and 12.5 FCCs per area, distributed across both EHS and community-based settings. This distribution will help to provide variation in context and allow an adequate sample size to describe the range of different caregivers’ use of We Grow Together and PD providers’ support.

Table B.1. Sample distribution by center-based classrooms and FCCs


Per geographic area

Per PD provider

Study total

Centers

8.75

1

88

Classrooms

17.5

2

175

Infant or toddler classrooms

8.75

1

88

FCCs

12.5

1.5

125



B2. Procedures for Collection of Information

Following eligibility screening in summer 2018, baseline data collection is expected to take place during September and October 2018 and will include the background survey, on-site classroom observations with the Q-CCIIT measure, and classroom roster collection. PD implementation will take place from November 2018 through March 2019; during that time, the study team will collect implementation data and track usage of the PD materials through the We Grow Together website. Follow-up data collection is expected to take place from April 2019 into June 2019 and will include the feedback survey, on-site classroom observations with the Q-CCIIT measure, and classroom roster collection.

Working with EHS programs, FCC networks, and the Office of Child Care Capacity Building Network, as well as conducting website searches, the study team will obtain lists of centers that meet the selection criteria in each of the 10 geographic areas. The study team will prepare a comprehensive set of materials for informing program directors, center directors, FCC owners, PD providers, and caregivers about the study.6 These materials will be available both by email and in paper form. The study team will produce materials that are informative, friendly, and visually appealing. These recruitment materials will explain why participants are being contacted and the goals of the study (Appendix A). The study team will work to anticipate and resolve any misconceptions or reluctance to participate on the part of caregivers and PD providers (for example, by explaining that data will be kept private).

The study team will make individual 15-minute calls to program/center administrators, FCC providers, PD providers, and caregivers to share information about the study and to explain that we will request consent electronically (Attachment 1). The study team’s approach to recruitment will be flexible depending on our point of contact, but the general approach will be to first make contact by telephone with either the setting administrator or the PD provider. After the setting administrator has agreed and the PD provider is recruited, the study team will recruit the caregiver. If needed, the study team will ask for support from the Office of Head Start and/or local networks (for example, an FCC network) in gaining access to gatekeepers, caregivers, and PD providers.

A member of the study team is also prepared to visit each of the 10 geographic areas for several days to meet in person with staff at settings that indicated an interest, but have not yet provided consent. The member of the study team will provide information about the study, answer questions, and collect consent from participants electronically or on paper.

During recruitment, the team intends to collect email addresses from caregivers and PD providers. The study team plans to implement web versions of the consent form, background and feedback surveys, which will make it easier for respondents to complete them. The study team will invite participants to complete the surveys on the web and will provide them with a secure login ID and password to access the web surveys. Participants will also have the option of completing the surveys by paper and pencil if that is more convenient. After a PD provider and caregiver consent to participate, the following activities will take place:

  • At the completion of the We Grow Together training for PD Providers in summer 2018, PD providers will complete a 10-minute PD provider training survey online. This survey allows PD providers to provide feedback on training materials and experiences as well as PD provider materials. (Attachment 2).

  • The study team will conduct a 45-minute web-based background survey with caregivers and 30-minute web-based background survey with PD providers in fall 2018 (Attachments 3a and 3b).

  • We will conduct a 60-minute web-based feedback survey with caregivers and 45-minute web-based feedback survey with PD providers in spring 2019 (Attachments 4a and 4b).

  • We will collect implementation data through pop-up questions to web users every 30 days for ten minutes with caregivers and six minutes with PD providers at website login during implementation of We Grow Together (Attachment 5).

  • Certified Q-CCIIT field staff will visit each classroom to observe with the Q-CCIIT measure and to collect classroom roster information—for example, the number and age range of the children in the classroom—once in fall 2018 and once in spring 2019 (Attachment 6).

B3. Methods to Maximize Response Rates and Deal with Nonresponse

Expected Response Rates

Through the various data collection efforts and the provision of access to the professional development materials, the study team expects a 100 percent response rate for the background survey and an 80 percent response rate for the website user pop-up questions and feedback survey. For in-classroom data collections, we expect a 100 percent response rate in fall 2018, and we anticipate that rate will fall to 80 percent in spring 2019 because of caregiver attrition. Table B.2 shows the various data collection efforts, their target number of consented respondents, and the expected response rate for each.

Table B.2. Expected response rates and number of responses, by data source

Respondent

Data source

Number of consented respondents

Expected response rate (percentage)

Expected number of responses

Program Administrators

ECE setting eligibility screener

745

28

213

PD providers

PD provider training survey

175

100

175

Caregivers and PD Providers

Background survey

475

100

475

Caregivers only

Classroom information roster

300

100

600a

Caregivers and PD Providers

Feedback survey

475

80

325

Caregivers and PD Providers

Website pop-up questions

475

80

2,280b

a The classroom information rosters will be collected twice

b The website pop-up questions will be collected six times

The study team plans to obtain feedback surveys from all caregivers and PD providers in the spring. At the end of the background survey in the fall, the study team will collect contact information in case the participant leaves the current setting. The study team will ask respondents for information in the feedback surveys about the length of time they were participating in We Grow Together and also use information from our website pop-up questions supplemented with the web analytics. The study may have 20 percent attrition from fall to spring for the Q-CCIIT observations due to caregivers who leave the settings. .The study team will include an indicator for those caregivers who leave before the full five months and are missing data.

Dealing with Nonresponse

The study team will work closely with each early care and education (ECE) setting to maximize participation in the data collection activities. The study team will ask center administrators, site contacts, and PD providers to encourage caregivers to complete the background and feedback surveys. The team will follow up with nonresponders by email and regular mail to encourage them to complete the survey. The anticipated response rates are at or above those that OMB recommends to minimize nonresponse bias. The web-based survey will prompt respondents who enter out-of-range or inconsistent responses to review their response. Weekly reviews of web survey data will allow us to identify potential errors, review for nonresponse, and follow up with respondents before the end of data collection.

The research study team will construct scales based on available guidance and norms for existing measures whenever possible. For constructs without appropriate existing measures and norms, such as administrative and collegial support, the study team will use item response theory (IRT) confirmatory models to reduce the data for the analytic models. Use of IRT item and model fit statistics, and factor analyses of residuals will guide the development of measures that are reliable and valid. The hierarchy of items in the model provides initial evidence of validity when consistent with theoretical expectations. IRT estimates scores even when item level data are missing within a scale. This approach takes into account how difficult it is to respond correctly to or endorse items and provides a score even there is some missing item level response. For all measures, the study will require that at least 60 percent of the items in a measure have responses in order to estimate a score. We will use multiple imputation for missing data for variables that will be used in the analytic models. With 100 percent response rate expected for the fall Q-CCIIT observation and background survey data, the study will have strong sources of information for imputation. Given the constructs measured in this study, explicit models such as normal linear regression should be appropriate for most if not all of the missing data. As needed (when the distribution is not normal), implicit models will be used, such as closest neighbor.

Maximizing Response Rates

Mathematica has achieved high response rates in collecting data from staff in a variety of education, social services, and health programs. The study team recognizes that participating in the We Grow Together project will place some burden on the child care settings, PD providers, and caregivers. The study team will attempt to minimize this burden through our data collection procedures and use of carefully constructed instruments. These web-based surveys will enable respondents to complete the data collection instruments at a location and time of their choice. Nevertheless, the study team should acknowledge the burden that participation entails.

The study could have 20 percent attrition from fall to spring for the Q-CCIIT observations and surveys due to caregivers who leave the settings. Using contact information collected in the background survey, the study team will send the feedback survey to all participants who leave the current setting before the end of the study to try to obtain some information about these participants. Their feedback surveys will ask about the length of time that they participated in We Grow Together. In addition, information from our website pop-up questions and web analytics will provide information about involvement on the website and reported time spent outside the website, providing information about similarities to (and differences from) the group of participants who remained.

B4. Tests of Procedures or Methods to Be Undertaken

The instruments included in this OMB package will yield data that the study team will analyze using quantitative and qualitative methods. The study team will carefully link the research questions guiding the study with the data collected, constructs measured, and analyses undertaken. The study team will conduct several preliminary activities to prepare the data for analysis. For each of the data collection instruments (for example, background and feedback surveys and pop-up web surveys for caregivers and PD providers), the study team will assess the level of nonresponse overall, as well as nonresponse to specific items. The study team will also examine the quality of the data collected to look for outliers, unexpected responses, or inconsistencies. For the Q-CCIIT measure data, we will also examine the number and length of cycle observations to make sure the ratings are based on an adequate sample of time.

The scales and items in the proposed surveys were selected in part because they had been validated and shown to have good psychometric properties with caregivers and teachers in early care and education. The study team has also developed new items for measuring constructs for which existing measures are not currently available. These items have drawn ideas for phrasing and language from prior research on infant-toddler caregiving and early childhood professional development. The survey instruments (see Attachments) are annotated to identify sources of questions from existing studies as well as questions developed specifically for this study.

The purpose of the field test is to understand if the Q-CCIIT measure and the related We Grow Together system can be used by early childhood professionals to support improvement in caregivers’ interactions with infants and toddlers.

A pretest of the content of the written We Grow Together materials took place over four weeks in summer 2017 with eight pairs of caregivers and PD providers. The pretest was focused on gathering written and verbal feedback on the content, language, and written format of the materials. This pretest helped the study team refine the We Grow Together materials and approach and is informing development of the We Grow Together website. No questions were asked of more than nine people during the pretest.

A technology and process pretest with nine pairs of caregivers and PD providers is planned for testing the We Grow Together materials delivered through the website in early 2018. This pretest will allow for user testing of the website and will also allow the study team to gather feedback on We Grow Together materials. Additional goals of this pretest include allowing a pretest of the background survey with a sample of nine caregivers and nine PD providers and a feedback survey with a sample of nine selected caregivers or PD providers (these measures are described in Statement A), and to pretest the PD provider training. This pretest will help to (1) ensure that questions were understandable, used language familiar to respondents, and were consistent with the concepts they aimed to measure; (2) identify typical instrumentation problems such as question wording and incomplete or inappropriate response categories; (3) measure the response burden; and (4) confirm there were no unforeseen difficulties in administering the instruments. The same question will not be asked of more than 9 people. Any resulting updates to the instruments will be submitted to OMB as a non-substantive change request. If substantive changes result from pretesting, the study team will publish a 30-day Federal Register Notice allowing for public comment, and submit the revised instruments to OMB for review and approval.

In the field test, the study team will use varied analytic methods to examine associations and change from baseline to follow-up for a purposive sample of caregivers and PD providers who participate in We Grow Together. Ultimately, findings will provide the basis for refining the We Grow Together system. Although a design such as this cannot determine causality, it can answer the current study research questions and inform subsequent studies of infant and toddler caregiver professional development. Table B.4 provides a crosswalk of research questions, constructs of interest, and analytic methods. Table B.5 provides information on precision of estimates and minimum detectable correlations and Table B.6 shows minimum detectable effect sizes between subgroups.

We will use quantitative methods to analyze the data collected with the instruments in Attachments 1‑6. We will address the study research questions using three types of quantitative analyses:

A. Descriptive statistics (means and standard deviations). We will report data gathered from the caregiver and PD provider background surveys in the fall and feedback surveys in the spring about caregivers’ and providers’ knowledge and beliefs, and the data from the Q-CCIIT observation measure. Similarly, we will also estimate the frequency and amount of time spent working on the activities in the We Grow Together system, whether caregivers and PD providers perceive change in their practice by the end of 5 months of using the system, and their perceptions about the helpfulness of the materials and processes included in the system.

B. Associations with knowledge, beliefs, and practice. We will link the information gathered in the background and feedback surveys to observed quality in the classroom. Specifically, we will plan to use analysis models that examine associations between caregivers’ initial knowledge about child development and caregiving practice in the fall, PD provider characteristics, and the amount of time caregivers spent working on the activities in the We Grow Together system with both the quality of observed interaction with children and with the caregiver’s beliefs and knowledge in the spring. We will also look at whether these associations differ between caregivers in centers and caregivers in FCCs.

C. Validity of Q-CCIIT quality measure. We will examine agreement between raters observing the same classroom at the same time to look at how reliable the ratings of classroom quality are. We will use a confirmatory factor analysis to examine whether the items group into the same scales as in psychometric field test. We will estimate the fall-spring differences and variance in means on each of the scales, and examine correlations with caregiver’s report of change to look for evidence that the Q-CCIIT observation measure was able to detect change in the quality of the caregiver’s interactions and whether that change was correlated with the caregiver’s report of change in practice.



Table B.3. Analytic methods used to inform research questions

Research questions

Key outcomes constructsa

Analysesb

1a. Is five months’ implementation of We Grow Together associated with change in the quality of caregiver-child interactions, as measured by Q-CCIIT instrument scores (fall to spring)? [Primary]

  • Quality of caregiver-child interactions

  • Support for social-emotional development

  • Support for language and literacy development

  • Support for cognitive development

  • Areas of concern

  • Descriptive statistics (means, standard deviations, mean fall to spring difference) with t-tests to examine significance of differences

  • Associations of change from fall to spring with other characteristics will be analyzed with hierarchical linear models (HLM) (two levels with caregivers nested within PD provider)

1b. Is five months’ implementation of We Grow Together associated with change in caregivers’ and PD providers’ beliefs and knowledge about child development and caregiving (fall to spring)? [Primary]

  • Knowledge and beliefs about child development (Section B Background survey, Section A Feedback survey)

  • Knowledge of and beliefs about caregiving (Section D Background survey, Section C Feedback survey)

  • Descriptive statistics (means, standard deviations, mean fall to spring difference) and t tests

  • Associations of change from fall to spring with other characteristics will be analyzed with hierarchical linear models (HLM) (two levels with caregivers nested within PD provider)

1c. Does the PD Provider perceive change in his or her own practice after PD providers’ training and five months’ implementation of We Grow Together? [Primary]

  • Change in beliefs about PD and use of PD strategies (Section B2 and K Feedback survey, compared with H2 and I6 in Background survey)

  • Descriptive statistics (means, standard deviations, mean fall to spring difference) and t-tests

1d. Do answers to these primary questions differ by subgroups (caregivers and PD providers associated with FCCs versus center-based settings)? [Primary]

How are answers to these primary questions associated with characteristics of caregivers and providers (for example, demographicsa, FCC/classroom characteristicsb, caregiver mental healthc)? [Secondary]

  • Includes all constructs listed in this research question (RQ 1) analyzed by respondent type (e.g., caregivers and PD providers associated with FCCs or center-based settings)

  • Descriptive statistics (means, standard deviations, mean fall to spring difference)

  • Associations of change from fall to spring with other characteristics will be analyzed with hierarchical linear models (HLM) (two levels with caregivers nested within PD provider)

1e. Does the caregiver perceive change in his or her own practice after five months’ implementation of We Grow Together? [Secondary]

  • Self-reported change (Section E Feedback survey)

  • Descriptive statistics (means, standard deviations, mean fall to spring difference) and t tests

1f. Does the caregiver report other changes after participating in We Grow Together? [Secondary-exploratory]

  • Awareness of and access to resources (Section G Background survey; Section G Feedback survey

  • Administrative and collegial support (Section F, Background Survey, Section D, Feedback Survey)

  • Self-efficacy (Section H3, Background Survey, Section E1, Feedback Survey)

  • Beliefs about Professional Development (Section H, Background Survey; Section B, Feedback Survey

  • Descriptive statistics (means, standard deviations, mean fall to spring difference) and t tests

2a. How frequently do caregivers and PD providers make use of We Grow Together over the implementation period, and which tools did they access? [Primary]

  • Frequency of We Grow Together accessed (Website user data: Website analytics Section C; Pop-up web survey section B)

  • Average time spent with practices (Website user data: Website analytics: Section C; Pop-up web survey section B)

  • Number of times self-video-recorded (Website user data: Pop-up web survey section B)

  • Number of meetings (Feedback survey section I and Website user data: Pop-up web survey section B)

  • Length of meetings (Website user data: Pop-up web survey section B)

  • Dosage: average time spent per week working on PD in and outside of classroom (Website user data: Website analytics Section B; Pop-up web survey section B)

  • Descriptive statistics (means, standard deviations)

  • Cluster and factor analyses to identify patterns of use and reduce data

2b. How do caregivers and PD providers engage with the technological components of We Grow Together (that is, usability of the website, accessing the website and tools within it, using the tablets)? [Primary]

  • Access to and ease of use of website tools, website, video recording experience (Feedback survey section F; Website user data: Implementation data section D)

  • Descriptive statistics (means, standard deviations)

  • Factor analyses to reduce data

2c. Are participants satisfied with We Grow Together (tool types, content)? [Primary]

  • Satisfaction with website tools, process, and content (Feedback survey section F; Website user data: Implementation data section D)

  • Descriptive statistics (means, standard deviations)

  • Factor analyses to reduce data

2d. Are participants satisfied with the We Grow Together process (goal setting, action planning, practice and observation, reflection, feedback, trusting relationship)? [Primary]

  • Caregiver satisfaction with PD process/strategies (Feedback survey section F)

  • PD satisfaction with supports for PD strategies (Feedback survey section F)

  • Relationship satisfaction (Feedback survey section H)

  • Descriptive statistics (means, standard deviations)

  • Factor and reliability analyses to reduce data

2e. What are challenges and barriers to PD implementation in infant/toddler settings? [Primary]

  • Challenges and barriers encountered (Feedback survey section I; Website user data: Implementation data section D)

  • Descriptive statistics (means, standard deviations)

  • Factor and reliability analyses to reduce data

2f. Do answers to any of these questions differ by subgroups (caregivers and PD providers associated with FCC versus center-based settings)? [Primary]

How are answers to these primary questions associated with characteristics of caregivers and providers (for example, demographics, FCC/classroom characteristics, caregiver mental health, PD provider experience and beliefs?) [Secondary]

  • Includes all constructs in RQ2 analyzed by respondent type (e.g., caregivers and PD providers associated with FCCs or center-based settings)

  • Descriptive statistics (means, standard deviations)

  • Hierarchical Linear Models (HLMs)

  • Factor and reliability analyses to reduce data

3a. What is the inter-rater reliability of the Q-CCIIT measure? [Secondary]

  • Rater reliability

  • Correlations, mean rater differences

3b. Does the Q-CCIIT factor structure hold with a new sample? [Secondary]

  • Construct validity

  • Confirmatory factor analysis (CFA), Item Response Theory (IRT)

3c. Does the measure demonstrate sensitivity to intervention? [Secondary]

  • Change in observed interaction quality (Q-CCIIT observational measure)

  • Association with self-reported change (Feedback survey section E)

  • Descriptive statistics (means, standard deviations)

  • Bivariate correlations of subscale change scores with self-reported change in related practices

a Demographics includes cultural and linguistic diversity, education, experience in infant-toddler care, technology literacy

b FCC/Classroom characteristics include the ages of children, proportion of male children, group size, adult-child ratio, philosophy and curriculum, support from administrators and other staff

c Caregiver’s mental health includes the caregiver’s report of self-efficacy, symptoms of anxiety and depression, workplace stress, and openness to change

d When the data allow, we will use constructed variables in our primary models and exclude covariates that are not associated with the outcomes to work toward parsimony.



Table B.4. Precision of estimates and minimum detectable correlations


Sampled

Responding sample

Effective sample size

95 percent confidence intervals (half widths) for
Q-CCIIT

Minimum detectable correlations

All PD Providers

190

171

108

.189

.270

All caregivers

333

300

187

.143

.205

FCC caregivers

139

125

84

.214

.306

Center-based caregivers

194

175

104

.193

.275



Table B.5. Minimum detectable effect sizes (between subgroups)


Subgroup 1


Subgroup 2

Minimum
detectable
effect


Description

Proportion


Description

Proportion

Caregivers (N=300)

Center based

.58


FCC

.42

.390

Caregivers (N=300)

Experienced Infant-Toddler caregivers (> 3 years)

.80


New Infant-Toddler caregivers

.20

.512

Caregivers (N = 300)

Fall

1.00


Spring

1.00

.176

Note: Effect sizes are in standard deviation-sized units. The standard deviation for Q-CCIIT is approximately 1.

B5. Individual(s) Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Mathematica Policy Research is conducting this project under contract number HHSP233201500035I/HHSP23337007T. Mathematica developed the plans for statistical analyses for this study. Mathematica developed plans for this data collection and analysis and consulted with a technical expert panel (TEP). Leaders of the study team from the Office of Planning, Research, and Evaluation (OPRE) and from Mathematica (including consultants) are listed in Table B.7.



Table B.6. OPRE project officers and study team leadership for the Q-CCIIT PD Tools project

Name

Affiliation

Ann Rivera

Office of Planning, Research and Evaluation, ACF, COR

Amy Madigan

Office of Planning, Research and Evaluation, ACF, COR

Louisa Tarullo

Mathematica Policy Research, Project director

Shannon Monahan

Mathematica Policy Research, Deputy project director

Sally Atkins-Burnett

Mathematica Policy Research, Principal investigator

Timothy Bruursema

Mathematica Policy Research, Survey director

Jillian Stein

Mathematica Policy Research, Deputy survey director

Barbara Carlson

Mathematica Policy Research, Sampling statistician

Diane Horm

University of Oklahoma, Tulsa, consultant

Margaret Burchinal

University of North Carolina, Chapel Hill, consultant

Martha Zaslow

Society for Research in Child Development (SRCD) and Child Trends, consultant



1 Throughout this statement, “caregiver” is used to refer to teachers in infant and toddler classrooms in center-based settings and child care providers in FCC settings. “PD provider” is used to refer to the person providing professional development assistance, such as coaching or mentoring, designed to support or enhance the caregiver’s practice. These can represent a range of ECE staff, both those working within programs and those employed by outside entities, such as ECE setting managers and education directors, mentors, coaches, employees of technical assistance networks or centers, and master teachers.

2 Throughout this statement, “classroom” is used to refer to groups in both centers and family child care (FCC) settings. “ECE setting” is used to refer to a classroom or FCC.

3 Educare schools provide full-day, full-year, early education and family support to children from prenatal to age 5 who are at risk of failure in school. Educare schools have public-private partnerships with school districts, state and federal child care funders, Head Start, Early Head Start, private funders, and others to provide comprehensive services for young children and their families.

4 Child Care Resource & Referral (CCR&R) refers to an interrelated set of services to families, child care providers, employers, and communities that facilitate access to early care and education and school-age child care options for families; improve the quality of those child care options; and give the public and private sectors objective information for planning and policy development.

5 Home visiting services are excluded from this study of child care in group settings.

6 The study team anticipates collecting consent electronically from participating caregivers and their local PD providers. We will not collect consent from children, families, or other caregivers in the field test classrooms because the classroom observations will be of typical classroom practice. Video collected by caregivers during the PD process for improving their practice will be shared only between the caregiver and PD provider; it is not part of the data collection.


File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
File Modified0000-00-00
File Created2021-01-21

© 2024 OMB.report | Privacy Policy