NonSub Change Memo: Second round testing edits

MEMORANDUM

TO: Steph Tathum; Office Information and Regulatory Affairs (OIRA), Office of Management and Budget (OMB)

FROM: Laura Hoard; Office of Planning, Research and Evaluation (OPRE),
Administration for Children and Families (ACF) DATE: 03/10/2016

SUBJECT: “Assessing Early Childhood Teachers’ Use of Child Progress Monitoring to Individualize Teaching Practices” project – revised materials (0970-0355)

The Administration for Children and Families at the U.S. Department of Health and Human Services seeks approval for non-substantive changes to our revised OMB submission under generic clearance for pre-testing (OMB control 0970-0355) for the “Assessing Early Childhood Teachers’ Use of Child Progress Monitoring to Individualize Teaching Practices” project. As described in our original, approved, justification package the planned process is an iterative one, where changes will be incorporated, submitted to OMB as a non-substantive change and then administered again. Under this non-substantive change request, we are submitting revisions to the previously approved Examining Data Informing Teaching (EDIT) instrument and EDIT teacher pretest letter. We provide details about the revisions below.

The EDIT team is conducting data collection between December 2015 and April 2016 in nine classrooms to pretest the observational EDIT instrument (completed by EDIT team members) and teacher interview. The primary goals of this pretest are to implement and refine the newest EDIT instrument rubrics and ratings, and continue to assess the overall feasibility of the EDIT protocols, procedures, and materials.

Refinements to the EDIT instrument

The EDIT instrument is being refined through an iterative process. The EDIT development team conducts three classroom visits and then reviews and discusses field experiences in periodic team debriefings. The team has visited six classrooms thus far and recommends several refinements to the EDIT instrument based on the last three visits. The revised EDIT instrument in tracked changes is in Attachment A and a “clean” version is in Attachment B. Recommended refinements fall into these categories:

• Further differentiate levels of teacher implementation of assessment and individualized instruction. During pretesting, we realized that some items needed to be revised to better differentiate between the levels of quality with which teachers implement ongoing assessment and individualized instruction. Specifically, we revised items 2A (p. 3) and 6I (p. 13).

• Clarify prompts. We clarified the prompt “for video-based observations only” to read “for video-based assessment and instruction only” (items 2F–2H on p. 5 and item 2N on p. 6).

• Clarify terms. Terms are revised to specify them more clearly and/or consistently across items. For example, in items 2F and 2G (p. 5), we changed “observed tasks” and “observed assessments” to “observed assessment tasks.” Holistic rubric 7 (p. 14) was also edited for clarification.

• Introduce skip patterns. We reordered and/or grouped some items and introduced skip patterns to better specify items and make the rating process more efficient, including items 2I–2M (p. 6), item 3H (p. 8), and supplemental rubric items 12–17 (pp. 16–17).

• Eliminate redundancy. We eliminated item 3C (p. 7) because we found it to be redundant with the frequency ratings in items 3A and 3B.

• Specify “Not Applicable” options. Rating options for “Not Applicable” were added to items 4Q and 4R (p. 10). Item 4Q only applies when teachers use the assessment system to organize the data, and item 4R only applies when teachers use their own method for organizing data.

Refinement to the teacher contact letter

Our original EDIT OMB submission (OMB control 0970-0355) included a pretest letter sent to each lead teacher that explained project activities and the selection of focal children. We added a note (highlighted in green text in Attachment C) to clarify that because we do not conduct observations in-person on the day of our site visit, focal children need not be present in the classroom on the day of our visit. The revised pretest letter in tracked changes is in Attachment C and a “clean” version is in Attachment D.

An Affirmative Action/Equal Opportunity Employer