QCCIIT Summary of changes_11-15-11_final

Summary of Non-substantive Changes to the OMB Package for

Measurement Development: Quality of Caregiver-Child Interactions for Infants and Toddlers (Q-CCIIT)

November 10, 2011

This memorandum summarizes the changes to the Q-CCIIT pilot and psychometric field tests based on recent project decisions. It provides supporting documentation for the submission of a revised OMB package for non-substantive changes to our September 2011 clearance approval (NOA 0970-0392). These changes include not conducting focus groups, and not collecting questionnaires from parents, reducing the number of observations that we conduct during the pilot test, and not video recording any observations. We have also added a consent form for the caregiver questionnaire, as required by the contractor’s IRB. Each of these changes is described below and reflected in the revised Supporting Statements A and B and Appendix E.

• No parent-report child competence questionnaire. Originally, parent self-administered questionnaires (SAQs) were to be collected to assess child competence. The SAQ would include: (a) select subscales from the Ages & Stages Questionnaires (ASQ3 communication, problem solving and personal-social subscales; Squires et al., 2009), (b) the MacArthur-Bates Communicative Development Inventories (CDI; Fenson et al., 2007), and (c) the Brief Infant-Toddler Social and Emotional Assessment (BITSEA; Briggs-Gowan et al., 2004). As raised in August with OMB, emerging evidence from other research studies demonstrates that the associations between measures of child care quality (for example, the ITERS-R or the CLASS-T) and parent-report using the ASQ and CDI are weak, with only a few significant zero-order correlations (r’s ranging from .10 to .15). In covariate-adjusted models, few associations were indicated (most consistent associations are noted with the language subscales of the ITERS-R). If we find weak associations with the Q-CCIIT measure, it will be difficult to determine whether the validity of the Q-CCIIT measure or measurement error in the validation instruments is the source. We explored the option of collecting child assessment data through direct assessment measures but found them to be cost prohibitive to conduct within the scope of the current contract. Validation using child assessment is a goal for ACF for the future development of the measure. However, given cost limitations, we will be validating the Q-CCIIT tool against established observation tools that have demonstrated relationships with direct child outcomes and are dropping the assessment of child outcomes from the plan for the current study. Instead, we will focus on expanding our validation plan to include observations with an established quality measure in 200 classrooms (rather than 100 as originally proposed). We discussed this and several other alternative options with our technical work group members, who ultimately supported this approach. This change results in a reduction in the total burden and changes the convergent validity analysis plan. The parent-report child competence questionnaire has been removed from Supporting Statement A, Table A3.

• No focus groups. As part of the pilot phase, we planned to conduct focus groups consisting of parents, caregivers, and training and technical assistance providers to obtain feedback from stakeholders about the new measure. We will no longer conduct focus groups. It was decided, however, that these groups are not the key stakeholders for the Q-CCIIT measure, given the purpose for which it is being developed. Because the measure is being developed primarily to fill a gap in measures available for accountability and performance measurement, we view the key stakeholders as the federal officials that will be using data from the measure for decision making. In place of the focus groups, we will be holding informal listening sessions with groups of federal staff on the Q-CCIIT measure. We will also obtain feedback on the measure from the Q-CCIIT expert panel (which includes experts in research, policy and practice). This change results in a reduction in the total burden given the decision to drop four instruments: (1) focus group interview guide, (2) parent focus group demographic questionnaire, (3) caregiver focus group demographic questionnaire, and (4) training and technical assistance provider focus group demographic questionnaire. These instruments have been removed from Supporting Statement A, Table A3. The analysis plan will also no longer include conceptual analyses.

• Changes in administration of particular caregiver questions. The Q-CCIIT measure-group activity and follow-up originally included four questions to be asked of the caregiver as follow-up to the observation. Three of these questions have been incorporated into the caregiver background questionnaire, leaving only one follow-up question in the Q-CCIIT measure-group activity and follow-up measure, addressing if anything unusual happened that day. Given the shift of three items from the follow-up form to the caregiver background questionnaire, we have changed the burden for each by five minutes, reducing the Q-CCIIT measure-group activity and follow-up from 15 to 10 minutes and increasing the caregiver background questionnaire from 15 to 20 minutes. These changes are reflected in Supporting Statement A, Table A3.

• Pilot sample size is reduced. Our OMB package described conducting 120 observations in four weeks in the two pretest locations as well as two additional locations. During the pretest, we discovered that site recruitment took significantly more time and resources than expected. Therefore, we plan to conduct the pilot in 60 classrooms from the two sites where we collected data during the pretest: (1) central New Jersey and (2) the Washington, DC metropolitan area. Our experience during the pretest suggests that these sites provide sufficient diversity of settings and children served to pilot our measure in a range of environments. Pilot recruitment will build on the relationships we established with these centers and family child care settings during the pretest. This change results in a reduction in the burden associated with three instruments: (1) child care setting recruitment form, (2) the Q-CCIIT measure-group activity and follow-up, and (3) the caregiver background questionnaire. This reduction is reflected in Supporting Statement A, Table A3. The sampling approach remains similar but is implemented in half as many sites.

• No video recording of observations. We will not video record any of the observations during the pilot as originally planned. We learned during the pretest that video recording was a significant barrier to site recruitment. We also found that the video recordings obtained during the pretest are sufficient to serve the purpose of additional item development, future training and certification, eliminating the need for additional video recordings during the pilot test. Therefore, written consent from adults and parents of children who might be video recorded during the observation will not be needed. However, all caregivers will receive documents explaining the project, how they were selected to participate, what their participation entails, and how the data will be used. Observers will be knowledgeable about privacy procedures and will be prepared to describe them in detail and to answer any related questions raised by participants. There are no burden-related changes associated with this change in procedure.

• Caregiver consent form added. OMB approved the Q-CCIIT package without requiring a consent form to be completed by caregivers prior to their completion of the self-administered questionnaires because completion of the questionnaire constituted implicit consent. However, the contractor’s IRB has required that written consent be obtained from caregivers. This form will be provided to caregivers at the time of distribution of the questionnaire to inform them of their rights as a study participant. The caregiver questionnaire consent form is now included in Appendix E.

1