Attachment B - Pretesting Generic Usage Report

Status: Original and derived artifacts are available for this document.

Download: docx | pdf | html




Attachment b: use of pretesting generic CLEARANCE 
(0970-0355) From 2012-2015

























Office of Planning, Research and Evaluation
Administration for Children and Families 
U.S. Department of Health and Human Services

7th Floor, West Aerospace Building
370 L’Enfant Promenade, SW
Washington, D.C. 20447

In January 2012, OMB approved ACF’s request to renew the generic information collection for pre-testing of evaluation surveys (0970-0355). This report describes the use of the generic IC over the three years of approval, including the number of hours used, as well as the nature and results of the activities completed under this generic clearance. 

The renewal of the generic IC was approved for three years, during which time ACF requested nine generic ICs for pretesting. The use of the pre-testing generic IC has been beneficial to the development of instruments for these research studies. Over the three years, ACF was approved to collect information from 3,395 respondents, requiring 1,742 hours of burden. During the previous three years of clearance, information was collected from 3,605 respondents requiring 1,336 hours of burden. The significant continued use of the generic from 2012-2015 is indicative of the usefulness of pretesting our surveys and procedures. 

Date
Project
Respondents
Burden Hours
3/30/12
Quality of Family-Provider Relationships in Early Care and Education
162
235
10/10/12
Permanency Innovations Initiative - LA RISE
500
300
1/30/13
Permanency Innovations Initiative - AZ, CAPP, IL
954
521
Total Year 1
1616
1056
3/29/13
Head Start Family Voices Pilot Study
380
341
11/14/13
Measurement Development: Quality of Family-Provider Relationships in Early Care and Education
45
37
12/27/13
Redesign of the Head Start Family and Child Experiences Survey (FACES 2012)
958
185
01/09/14
Q-DOT Pilot Study
60
20
Total Year 2
1443
583
9/25/14
Measurement Development: Healthy Marriage and Relationship Education Models and Measures
36
38
11/20/14
Measurement Development: Quality Relationship of Family and Family Services Staff in Head Start/Early Head Start
300
65
Total Year 3
336
103
Total over 3 Years
3395
1742



Example uses of the Pre-testing Generic Clearance

Family-Teacher/Provider Relationship Quality measure development (FTPRQ) project 
The Family-Teacher/Provider Relationship Quality measure development (FTPRQ) project created a set of new measures to assess the quality of the relationship between families of young children (ages 0-5 years) and the child’s teacher or child care provider. These relationships are an important component of family engagement in early childhood programs, but no appropriate measure existed.  

Both the formative generic clearance (0970-0356) and the pre-testing generic clearance (0970-0355) were integral in developing these measures. Under the formative generic clearance, ACF/OPRE conducted focus groups with families and teachers/providers. These focus groups were essential for ensuring that our measures were based on a realistic conceptual model and that we were using terminology and language that families and teachers/providers would understand. After developing initial drafts of measures, which were based in part on the results of the focus groups as well as an extensive review of the literature and pre-existing related measures, ACF received permission conduct cognitive interviews under the pre-testing generic clearance. Three rounds of cognitive interviews were conducted, which further enabled us to test and refine our measures, so that they were ready for testing in pilot and field tests (conducted under OMB clearance 0970-0420). The thoroughly tested and final measures were released in November 2014 and have been very well-received by the field.

Head Start Family and Child Experiences Survey 
An expert panel convened as part of the Head Start Family and Child Experiences Survey (FACES) Redesign recommended that ACF consider optimal approaches to assessing DLL children in future rounds of FACES with the goal of expanding and improving measurement of dual language learner (DLL) children’s development. The panel recommended that ACF pilot additional measures suitable for assessing the vocabulary skills of DLL children more efficiently as well as measures that offer a broader picture of children’s language development. 

To accomplish these goals, ACF submitted a generic Information Collection Request (ICR) to pretest measures and procedures that would (1) be reliable and valid for assessing DLL children and (2) reduce the repetitive vocabulary assessment burden on DLL children in the study. Findings from the pretesting were used to inform the measurement battery for the FACES 2014-2018 child assessment (0970-0151).