Head Start Family Voices Pilot Study
OMB Information Collection Request
0970-0355
Supporting Statement
Part A
February 2013
Submitted By:
Office of Planning, Research and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
7th Floor, West Aerospace Building
370 L’Enfant Promenade, SW
Washington, D.C. 20447
Project Officers:
Mary Mueggenborg and Maria Woolverton
The Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services (DHHS) seeks approval for pilot data collection activities to support the Head Start Family Voices (HSFV) Pilot Study. ACF requests permission to (1) enroll programs and participants (parents and staff) into the pilot study and (2) pilot test qualitative interview protocols with families and staff. We will analyze and evaluate the performance of the qualitative interviews and develop a final set of qualitative instruments that can be used in future studies. Findings based on the information collected will be compiled in a report to be used for internal purposes only and will not be released to the public. Mathematica Policy Research is the contractor for the study.
A1. Necessity for the Data Collection
The purpose of HSFV Pilot Study is to develop and pilot test qualitative questionnaires with Head Start and Early Head Start families and staff that can be included in future large-scale studies. With the exception of a case study component in the 1997 cohort of the Head Start Family and Child Experiences Survey (FACES 1997), ACF does not have in-depth qualitative data on the experiences of families participating in Head Start and Early Head Start programs or the staff who provide family support services to them. FACES and the Early Head Start Family and Child Experiences Survey (Baby FACES), ACF’s two large-scale descriptive studies focused on the experiences of Head Start and Early Head Start families, instead provide a broader range of quantitative descriptive data on the population being served. The qualitative instruments developed and tested as part of the HSFV Pilot Study will, in their finalized state, provide a complement to quantitative data collection efforts such as FACES, Baby FACES, or other descriptive or experimental studies. They will also provide valuable information about Head Start and Early Head Start families and programs, including the best methods for gathering qualitative information with these participants.
The HSFV Pilot Study will help ACF develop better measures to understand the experiences and perspectives of families and staff participating in Head Start and Early Head Start, particularly around the topic of family engagement. As state Quality Rating and Improvement Systems (QRIS) adopt benchmarks related to family engagement, there is a growing recognition of the need to measure family engagement (Forry et al. 2011). While program performance standards and policies mandate the types of family engagement efforts that are initiated (the “what” of engagement), knowledge of the “how” – the ways in which staff perform the day-to-day work of engaging with families, their successes and challenges, and the ways in which they individualize their practices – is limited.
There are no legal or administrative requirements that necessitate the data collection activities.
A2. Purpose of Survey and Data Collection Procedures
Overview of Purpose and Approach
As part of the HSFV Pilot Study we will (1) pilot test qualitative interview protocols with families and staff; (2) evaluate the items and methods used for gathering qualitative information with these participants; and (3) develop a final set of qualitative instruments based on the performance of the piloted measures that can be used in future studies. We will also (4) describe the themes emerging from the data in order to assess how the interviews perform in response to the study’s research questions.
We will conduct interviews with Head Start and Early Head Start parents, home visitors, and family services staff. We will gather data using a dual-mode approach whereby half of the interviews will be conducted in-person and the other half by phone. This will provide the opportunity to assess the performance of the protocols using multiple modes of administration, including whether and how the mode of data collection impacts the information that is obtained. We will conduct interviews with parents in English and in Spanish, and interviewers will conclude each interview with a debriefing in which they probe respondents for more information about any questions that may have been unclear or difficult to respond to. We will audio record all interviews. The information from the audio recordings will assist interviewers in the compiling of summary notes in preparation for analysis.
We have developed separate interview protocols for parents and staff, and have created two versions of each interview protocol (Form A and Form B). Each version includes two overlapping modules and one distinct module such that each respondent will be asked to respond to questions in only three of the four available modules (See Table A.1). This approach will allow us to pilot test a larger number of questions without overly burdening individual respondents. Each module focuses on a specific topic. Topical modules for parents include: (1) choosing Head Start/Early Head Start, (2) relationships with programs and staff, (3) family engagement in the program and in children’s learning and development, and (4) components of community engagement. Modules for staff include: (1) family engagement in programs, (2) program supports for family engagement and service receipt, (3) working with families, and (4) components of community engagement. Together, the interviews address several research questions (described in the next section).
Table A.1. Modules Included in the Parent and Staff Qualitative Interviews
|
Parent Qualitative Interview |
Staff Qualitative Interview |
||
|
Form A |
Form B |
Form A |
Form B |
Module 1 |
x |
x |
x |
x |
Module 2 |
x |
x |
x |
x |
Module 3 |
x |
|
x |
|
Module 4 |
|
x |
|
x |
At the conclusion of the interview, all participating parents and staff will complete a brief questionnaire about their background. We will use this information to describe the characteristics of parents and staff who participated in the interviews, and we will also link the information to the interview data in order to evaluate the performance of the interviews by key characteristics of parents and staff (for example, Head Start versus Early Head Start). We are not testing the performance of the brief questionnaires as a part of this study; they are separate from the interviews to be piloted. In the questionnaires, staff will report on their characteristics (such as years of experience, education level, language, and race/ethnicity). Parents will provide information about themselves and characteristics of their child and household (for example, age and number of children enrolled in Head Start and Early Head Start, primary language spoken in the household, housing stability and homelessness, household income, and parent race/ethnicity and marital status).
Research Questions
The HSFV Pilot Study will develop and test instruments designed to address the following overarching research questions.
What does family engagement look like in Head Start and Early Head Start?
How do home visitors and family services staff work with families, and what support do they receive from the program to do so?
How are comprehensive services provided in Head Start and Early Head Start?
Do family engagement and/or service provision differ in high-risk families or by other family characteristics? Do they differ in Head Start versus Early Head Start?
What changes do families identify as resulting from their experiences with Head Start and Early Head Start?
The research questions have been informed by our knowledge of the research literature on family engagement; the Office of Head Start’s (OHS) Parent, Family, and Community Engagement Framework; and input from staff at ACF. The instruments designed to address the research questions also recognize the data already available to ACF from other information sources, seeking not to duplicate information that can be obtained from other sources.
Study Design
We will invite ten grantees (five Head Start and five Early Head Start) to participate in the pilot. Eight of the ten selected programs will be located within proximity to each of Mathematica’s five office locations (Princeton, NJ; Washington, DC; Cambridge, MA; Chicago, IL; and Oakland, CA), and two programs will be located in the south. Among the ten selected programs, we will recruit two in rural locations. Within each program, we will select 15 parents (totaling 150 parents) and 3 staff (totaling 30 staff), including representation of home visitors and family services staff. We will ask programs that agree to take part in the study to help us identify and recruit staff and a diverse sample of families. We provide more information on the approach to selecting programs, families, and staff in Supporting Statement B.
The qualitative interviews will be one-on-one and will last approximately one hour. We will complete half of the interviews in-person, and complete the other half by phone. We will conduct a subset of the parent interviews in Spanish. We will include representation of families who receive home- and center-based services and who experience different types of needs (including teen mothers, single-parent families, expectant mothers, families who live in transitional housing or may be homeless, and parents who may have mental health problems). Within each of these groups, we will balance the administration of interview forms. Following the interview, interviewers will complete a questionnaire with parents and staff about their background.
Universe of Data Collection Efforts
In addition to a number of supporting documents (elaborated on below), the HSFV Pilot Study includes the following data collection instruments:
Program Recruitment Script and Screener (Attachment A1)
Participant Recruitment Form (Attachment A2)
Parent Qualitative Interview, Forms A and B (Attachment A3)
Staff Qualitative Interview, Forms A and B (Attachment A4)
Parent Questionnaire (Attachment A5)
Staff Questionnaire (Attachment A6)
We include these data collection instruments in Attachments A1-A6 and the additional supporting documents (i.e., Program Recruitment Advance Letter and Consent Forms for Parents and Program Staff) in Attachments B1-B3. The Program Recruitment Advance Letter, which contains a brief overview of the study goals and activities, will serve as the initial mode of contact for inviting programs to take part in the study. Guided by the Program Recruitment Script, a team of two to three Mathematica recruiters will place a follow-up call during which programs will be more formally invited to participate in the study. During this call, recruiters will describe the study purpose, provide an overview of the study activities, confirm the program’s interest in participating, and administer the Program Recruitment Screener. The screener will provide program-level information on the families served by the program and will be used to guide recruitment of respondents for each program. It will also provide descriptive information about the programs participating in the pilot. Based on the information provided by programs, Mathematica staff will prepare and share the Participant Recruitment Form with each program. The Participant Recruitment Form contains pre-filled targets for respondent recruitment (e.g., the number of families served by the home-based option that programs should aim to recruit) and will be used by programs to provide basic information on the parents and staff who agree to participate in the pilot (including respondent names, dates/time of scheduled interviews, and contact information for those who will complete the interviews by phone). We will ask programs to designate an On-Site Coordinator who will serve as the key contact and study liaison for coordinating the recruitment of respondents and the scheduling of interviews. The On-Site Coordinator will either be the program director or someone appointed by the director. Prior to the scheduled interview date, the On-Site Coordinator will share information about the study with prospective respondents using the Consent Forms for Parents and Program Staff and will obtain written consent from all respondents who agree to take part in the study. All data collection instruments and supporting documents to be used with Spanish-speaking families – including the Consent Form for Parents, Parent Questionnaire, and Parent Qualitative Interview – will be translated by a certified Mathematica translator.
The Parent and Staff Qualitative Interviews will collect information on parent and staff experiences in the program, particularly as it relates to family engagement. Finally, as noted previously, the Parent and Staff Questionnaires will provide important information on the backgrounds of parents and staff participating in the interviews.
A3. Improved Information Technology to Reduce Burden
For a qualitative pilot interview, computer assisted interviewing is not appropriate. It would not be cost effective to use information technologies for the other instruments in this one-time set of pilot activities.
All interviews will be audio recorded to facilitate interviewers’ summary of information provided by respondents.
A4. Efforts to Identify Duplication
We have selected items for the Parent and Staff Qualitative Interviews that do not duplicate the information collected through other current and ongoing ACF studies.
A5. Involvement of Small Organizations
Not applicable. No small businesses are impacted by the data collection in this project.
A6. Consequences of Less Frequent Data Collection
Not applicable. This is a one-time data collection.
A7. Special Circumstances
There are no special circumstances for the proposed data collection efforts.
A8. Federal Register Notice and Consultation
Federal Register Notice and Comments
In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on June 10, 2011, Volume 76, Number 112, page 34077, and provided a sixty-day period for public comment. During the notice and comment period, no comments were received.
The research team for the HSFV Pilot Study includes senior consultant Judith Jerald, who serves on the Save the Children leadership team. Ms. Jerald helped to identify research questions for the pilot study and reviewed items included in the qualitative interviews. For the analysis and reporting tasks, she will review the team’s coding and interpretation through the lens of someone highly knowledgeable about family support, Early Head Start, and Head Start. She will also review and comment on drafts of the report.
A9. Incentives for Respondents
With OMB approval, we will offer participants a $20 gift card for taking part in the pilot interviews and for completing the brief questionnaire. We will distribute gift cards to participating staff or donate to program activities based on the program’s policies. We will offer programs a gift card valued at $200 as a gift of appreciation for their involvement in the study, including assisting us in recruiting families and staff to participate in the study activities. These amounts were determined based on the estimated burden to participants and are consistent with those offered in prior studies using similar methodologies and data collection instruments (such as FACES and Baby FACES).
A10. Privacy of Respondents
The study will comply with Government regulations for securing and protecting paper records, field notes, or other documents that contain sensitive or personally identifiable information. The study will not include personal identifiers on the brief questionnaires or on field notes prepared during the qualitative interviews with parents and staff. The study will assign a unique identification number to programs, parents, and staff to facilitate the linking of information across data sources.
Parents and staff will receive information about privacy protections when they consent to participate in the pilot. We have crafted carefully worded consent forms that explain in simple, direct language the steps we will take to protect the privacy of the information each sample member provides. The study will provide assurances of privacy to each parent and staff member as he or she is recruited for the pilot data collection. The consent form makes it clear that parents and staff may withdraw their consent at any time or refuse to answer any items in the questionnaire or interview.
As part of the recruitment process, program directors will provide the names, addresses, and phone numbers of centers that are willing to participate in the pilot. Based on information obtained on the Program Recruitment Screener, we will provide programs with information on recruitment targets using the Participant Recruitment Form. The On-Site Coordinator designated by the program will assist us in identifying families with specific characteristics for participation in the study. This includes families who receive home-based and center-based services, families from Spanish-speaking households, and expectant mothers (Early Head Start programs only). We will also provide targets for recruiting families who experience different types of needs (for example, teen mothers, single-parent families, families who live in transitional housing or may be homeless, and parents who may have mental health problems) depending on the prevalence of each of these types of families at the program, as identified in the Program Recruitment Screener. Based on these recruitment targets, programs will identify interested parents and staff and share Consent Forms with them. We will then ask programs to provide the names of consented parents, times when they are available for an interview, and names and available interview times for consented staff members. We will only receive participant phone numbers for those parents and staff scheduled to complete the interview by phone. Programs will also provide the service option received by the family, the primary household language of the family, and whether the parent is an expectant mother. We will use this information for scheduling bilingual interviewers, to identify in advance interview questions to be asked of families, and to ensure we are meeting targets for representing families in these groups.
To ensure the privacy of the information provided by programs, we will password encrypt the Participant Recruitment Form, and the study will electronically transmit forms from programs to the study’s project manager. In the field, interviewers will collect paper field notes and audio recordings. Interviewers will be instructed to keep these materials, along with the Participant Recruitment Form, on their person or in locked storage at all times. Interviewers will hand-carry all paper data (including field notes and completed questionnaires) back to Mathematica’s offices when field work is finished. Aside from the Participant Recruitment Form and signed Consent Forms, none of these data documents will include names or personal identifiers. Once at Mathematica, we will transcribe paper field notes, and we will save the transcriptions on the project’s secure study-specific electronic folder. We will store the paper field notes in a secure location, such as a locked file cabinet or locked drawer when not in use and when transcription is complete. We will upload audio recordings to a secure study-specific electronic folder and then delete them from the recorders. We will store the recorders in a secure location, such as a locked file cabinet or locked drawer when not in use and when transcription is complete. All hard copy documents containing personal identifiers will be maintained in a locked file cabinet by the study’s project manager.
We will maintain a data log, saved on the project’s secure study-specific electronic folder, to record the source, handling, receipt, location, and disposition of files that contain private data. Following the end of the project, when no longer required, we will destroy hardcopy materials and other physical media using a cross-cut shredder. We will wipe private electronic data using Eraser, a file deletion utility configured to use at least 3 passes to overwrite data. Eraser has been installed on all PCs at Mathematica for interviewers.
A11. Sensitive Questions
The Parent Questionnaire includes items about parent’s employment status, marital status, and depressive symptoms (Center for Epidemiologic Studies Depression Scale [CES-D]; Radloff 1977). Some parents may consider these questions to be somewhat sensitive in nature; all parents will be told that they can skip questions they prefer not to answer. Given established associations between these characteristics of parents and the ways that they engage in programs, we believe these items are important to include in the questionnaire for offering context to the qualitative interview findings. The Staff Questionnaire does not include any questions that might be perceived as sensitive in nature.
The Parent and Staff Qualitative Interviews include questions about parents’ and staff experiences with the Head Start or Early Head Start program. Although it is unlikely that participants will find any of these questions sensitive in nature, we will tell all participants that they can skip any items that they feel uncomfortable responding to. We will remind participants that their responses will be kept private, to encourage their candid responses. The consent forms also highlight this information for respondents.
A12. Estimation of Information Collection Burden
Newly Requested Information Collections
Total Annual Burden
The estimated burden for programs’ On-Site Coordinators, parents, and staff participating in the HSFV Pilot Study is listed in Table A.2. The total annual burden for the pilot activity is expected to be 339.6 hours. Response times were estimated based on prior experience with similar data collection instruments and materials.
Following the program’s receipt of the Program Recruitment Advance Letter, we will call the program director to more formally invite them to participate in the study. During this call, we will describe the study purpose, provide an overview of the study activities, confirm the program’s interest in participating, and complete the Program Recruitment Screener. These activities are estimated to take a combined 30 minutes to complete. We assume that it will take On-Site Coordinators approximately 8.5 hours to identify parents and staff for the HSFV Pilot Study based on our program-specific requests and to enter data for parent participants on the Participant Recruitment Form. Prior to conducting the interview, parents and staff will review and sign the Consent Form. Review of the consent form and completion of the Qualitative Interview should take approximately 69 minutes. The Parent Questionnaire will take approximately 15 minutes to complete, and the Staff Questionnaire will take about 10 minutes to complete.
To compute the total estimated annual cost, the total burden hours were multiplied by the average hourly wage for staff and parents. For parents, we used $22.88 per hour, which is the average hourly wage reported by the Bureau of Labor Statistics, Current Employment Statistics Survey, 2011. For program directors and On-Site Coordinators, we used 2011 data on the median weekly salary for full-time employees with a degree higher than a bachelor’s degree ($33.65 per hour assuming an average work week of 40 hours). For family service workers and home visitors, we used the mean salary for childcare workers ($10.15 per hour) reported in the May 2010 National Occupational Employment and Wage Estimates for the United States (the most recent year available).
Table A.2.Total Burden Requested Under this Information Collection
Instrument/Material |
Total Number of Respondents |
Number of Responses Per Respondent |
Average Burden Hours Per Response |
Total Burden Hours |
Annual Burden Hours |
Average Hourly Wage |
Total Annual Cost |
Program Recruitment Script and Screener a |
10 |
1 |
.50 a |
5 |
5 |
$33.65 |
$168.25 |
Participant Recruitment Form |
10 |
1 |
8.5 |
85 |
85 |
$33.65 |
$2860.25 |
Parent Qualitative Interview, Forms A and Bb |
150 |
1 |
1.15b |
173 |
173 |
$22.88 |
$3946.8 |
Staff Qualitative Interview, Forms A and Bb |
30 |
1 |
1.15b |
35 |
35 |
$10.15 |
$350.18 |
Parent Questionnaire |
150 |
1 |
.25 |
38 |
38 |
$22.88 |
$858.00 |
Staff Questionnaire |
30 |
1 |
.17 |
5 |
5 |
$10.15 |
$51.77 |
Estimated Annual Burden Subtotal |
341 |
341 |
|
|
|||
a Burden hours include time for completing the recruitment screener and script and reviewing the advance letter.
b Burden hours include time for reviewing the consent form (in advance of the interview) and completing the one-hour interview.
A13. Cost Burden to Respondents or Record Keepers
Not applicable. There are no additional costs to respondents; they spend only their time to participate in the study.
A14. Estimate of Cost to the Federal Government
The total cost to the federal government of contacting the 10 Head Start and Early Head Start programs, identifying and recruiting parents and staff, conducting qualitative interviews and completing questionnaires under the terms of the HSFV Pilot Study contract is estimated to be $73,131, including direct and indirect costs and fees.
The total cost to the federal government of analyzing the collected data, summarizing findings in response to the study’s research questions, and developing a final instrument package and training materials is estimated to be $122,949, including direct and indirect costs and fees.
A15. Change in Burden
This is an additional request under the pre-testing generic clearance (0970-0355).
A16. Plan and Time Schedule for Information Collection, Tabulation and Publication
There are no plans for publishing the data gathered from the HSFV Pilot Study. The data that are collected will be for internal use only. Findings from the coding, analysis, and tabulation of data will be shared only with ACF staff. The instrument package and training materials that are developed following the data collection will be designed to be shared with others. The information that is collected will be for internal use only; however, information might be included as a methodological appendix or footnote in a report containing data from a larger data collection effort.
The HSFV Pilot Study activities will take place over an 11-month period, commencing upon OMB approval and ending in March 2014. Recruitment and data collection activities are slated to occur prior to July 2013. Analysis activities will occur between July 2013 and September 2013. Reporting activities will take place between July 2013 and November 2013, with the draft report and instrument package complete by September 2013 and the final report and instrument package complete by late 2013. Briefing with ACF staff will take place in early 2014.
In order to develop the final report, the analysis process will include four steps: (1) writing up field notes based on the Parent and Staff Qualitative Interviews, (2) analyzing data from the Parent and Staff Questionnaires, (3) coding field notes, and (4) retrieving data on specific research questions and concepts to generate findings (Patton 2002; Ritchie and Spencer 2002). For coding of the field notes, we will develop a coding scheme that is based on the themes and concepts that emerged during the pre-pilot activities and the pilot interviews, grounded in the research literature, and closely tied to the research questions. The coding scheme will draw on common approaches for organizing large amounts of qualitative data and identifying patterns and themes (Miles and Huberman 1994).
Coding and analysis of the pilot data will serve two purposes: (1) to refine the qualitative interviews used in the HSFV Pilot Study for use in future studies and (2) describe the themes emerging from the data in response to the study’s research questions about family engagement. Based on the analysis, we will produce two primary documents: (1) a revised instrument package that includes qualitative protocols, training materials, and the coding scheme and (2) a report to ACF that summarizes the results of the pilot study. The final instrument package will be designed for use by multiple audiences, including ACF, other federal agencies, and researchers in the early care and education field. In addition, the final instrument package will include supportive training and analysis tools that can be used by program staff to help in gathering information from families and staff for continuous program improvement efforts.
The study’s final report will be designed as an internal document for ACF and will discuss the following:
Messages emerging from the interviews
How well the instruments and data collection methods worked, including whether collected information answered the research questions
Whether the instruments worked comparably across groups of interest
A summary of the revisions that were made to the instruments based on the pilot
Recommendations for revisions to instruments that might be needed to adapt them for large-scale data collection or for program and practitioner use
Considerations needed before moving forward with large-scale data collection
A17. Reasons Not to Display OMB Expiration Date
All instruments will display the OMB approval number and expiration date for OMB approval.
A18. Exceptions to Certification for Paperwork Reduction Act Submissions
No exceptions are necessary for this information collection.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | OPRE OMB Clearance Manual |
| Author | DHHS |
| File Modified | 0000-00-00 |
| File Created | 2021-01-28 |