MIHOPE Supporting Statement A_semi-structured interview change updated 2.25.19 clean

Mother and Infant Home Visiting Program Evaluation (MIHOPE):

Kindergarten Follow-Up

(MIHOPE-K)

OMB Information Collection Request

0970 - 0402

Supporting Statement

Part A

Original Document April 2018

Revised November 2018

Revised February 2019

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

Mary E. Switzer Building

330 C Street, SW, 4^th Floor

Washington, D.C. 20201

Project Officers:

Nancy Geyelin Margie

Laura Nerenberg

Part A. JUSTIFICATION

A1. Necessity for the Data Collection

The Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services (HHS) seeks approval to conduct a long-term follow-up of the families participating in the Mother and Infant Home Visiting Program Evaluation (MIHOPE).

Study Background

In 2011, the Administration for Children and Families (ACF) and the Health Resources and Services Administration (HRSA) within the U.S. Department of Health and Human Services (HHS) launched the Mother and Infant Home Visiting Program Evaluation (MIHOPE). MIHOPE is providing information about the effectiveness of the Maternal, Infant, and Early Childhood Home Visiting program (MIECHV) in its first few years of operation and providing information to help states and others develop and strengthen home visiting programs in the future. The goals of the study are:

1. to understand the effects of home visiting programs on parent and child outcomes, both overall and for key subgroups of families,

2. to understand how home visiting programs were implemented and how implementation varied across programs, and

3. to understand which features of local home visiting programs are associated with larger or smaller program impacts.

To estimate the effects of home visiting on family outcomes, MIHOPE enrolled over 4,200 families across 88 sites in 12 states. Families were eligible for the study if they included a pregnant woman or an infant under six months old and the mother was at least 15 years old at the time of study entry.

OMB has approved data collection packages (OMB Control Number 0970-0402) for three phases of MIHOPE:

• On July 12, 2012, OMB approved the data collection package for Phase 1 (MIHOPE 1), which covered the collection of data at baseline, when families were enrolled into the study.

• On June 26, 2013, OMB approved the data collection package for Phase 2 (MIHOPE 2), which covered the collection of follow-up data when the children in the study were 15 months old.

• On August 6, 2015, OMB approved the data collection package for MIHOPE Check-in to collect updated contact information and follow-up data on children and parents when the children in the study are 2½, 3½, and 4½ years old.

Because previous research on home visiting programs has found long-term impacts on child and family outcomes, ACF and HRSA would like to continue collecting follow-up information from families participating in the study. To design and conduct follow-up studies with MIHOPE families, OPRE awarded a contract to MDRC in September 2016 for the MIHOPE Long-Term Follow-Up project (MIHOPE-LT). Mathematica Policy Research is the subcontracted survey firm.

Legal or Administrative Requirements that Necessitate the Collection

There are no legal or administrative requirements that necessitate the collection. ACF and HRSA are undertaking the collection at the discretion of the agencies.

A2. Purpose of Survey and Data Collection Procedures

Overview of Purpose and Approach

The purpose of MIHOPE-LT is to study the long-term impacts of home visiting programs on child and family outcomes. The MIHOPE-LT design consists of a series of four studies, one each when the children from the MIHOPE sample are (1) kindergarten age, (2) in third grade, (3) early adolescents (approximately 12-13 years old), and (4) older adolescents (approximately 15-16 years old). The kindergarten follow-up (MIHOPE-K) is the focus of this OMB package, though semi-structured interviews described here are proposed for when children are approximately 3 ½ years old. Subsequent information collection requests will be submitted for the later follow-ups.

Research Questions

The four research questions MIHOPE-LT will try to answer are:

1. What are the long-term effects of home visiting overall for the MIHOPE sample?

2. Are the long-term effects of home visiting larger for some families than for others?

3. How do the benefits of home visiting compare to its costs?

4. What are the pathways through which home visiting affects families’ long-term outcomes?

Study Design

MIHOPE-LT will follow up with families in the original MIHOPE sample. Families who were recruited into MIHOPE were randomly assigned either to a MIECHV-funded local home visiting program or to a control group that could use other services available in the community.

The first phase of MIHOPE-LT data collection, the focus of this OMB package, will involve gathering follow-up data with families when the children are of kindergarten age (MIHOPE-K). The remainder of this supporting statement, as well as Supporting Statement B and all attachments, will refer specifically to MIHOPE-K, unless otherwise stated. MIHOPE-K includes six broad sets of data collection activities (more detailed information for each activity follows this list):

1. Structured interview with caregivers: Information on participating families will be collected through a structured interview with the focal child’s primary caregiver. The caregiver interview will provide information on several domains, including child health, child development and school performance, relationships and father involvement, maternal health and well-being, parenting practices, family economic self-sufficiency, intimate partner violence, child maltreatment, and the caregiver’s adverse childhood experiences.

2. Direct assessments of children: Direct child assessments such as Hearts & Flowers and the Woodcock Johnson IV Picture Vocabulary and Oral Comprehension subtests will be administered to assess the child’s receptive language skills, early numeracy, working memory, inhibitory control, and cognitive flexibility. Assessors will also observe and rate parental warmth and the child’s emotions, attention, and behavior.

3. Semi-structured interview with caregivers: Semi-structured interviews with parents who received home visiting services will be conducted to gather information that will enrich understanding of families’ past experiences in home visiting and their broader circumstances navigating parenthood with young children. The plan is to conduct these interviews as a separate effort from other MIHOPE data collection with families whose children are at the younger end of the MIHOPE sample (currently, these children are approximately 3 ½ years old).

4. Survey of the focal children’s teachers: Information from participating children’s teachers will be collected through a survey. The teacher survey will provide information on the children’s behavior, such as learning behaviors, which are best observed in a classroom setting.

5. Direct assessments of caregivers: Direct caregiver assessments (i.e. the Digit Span) will be administered to assess maternal self-regulation.

6. Videotaped caregiver-child interaction: Observations of caregiver-child interactions will be conducted using a videotaped interaction. Children’s behaviors towards the caregiver will be gathered in the context of caregiver-child interaction, including engagement with the caregiver and negativity toward the caregiver. Caregivers’ parenting behaviors, including supportiveness and respect for child’s autonomy, will also be assessed, as well as features of the caregiver-child dyad (e.g., affective mutuality).

Administrative data will also be collected but will be obtained directly from the agencies that hold the data, in its existing format, placing no extra burden on the families. Specifically, we plan to continue to collect child welfare, Medicaid, and National Dataset of New Hires data. We also plan to pursue the acquisition of school records, National Death Index records, and Social Security Administration data. Because child welfare and school records data will be obtained from state or local agencies (rather than federal entities), burden for these two data sources is included in the burden table and is discussed later in this section. The MIHOPE consent form allows the study to collect administrative data until the end of the study. If the study continues past the point at which children turn 18 years old, we will need to obtain the children’s consent when they are 18 years old in order to continue to collect administrative data about them. The section below provides more detail on the various data collection activities that will be included in the kindergarten follow-up.

Universe of Data Collection Efforts

Structured interview with caregivers

The structured interview with caregivers (Attachment 1) will include information on several domains: child health; child development and school performance; relationships and father involvement; maternal health and well-being; parenting practices; intimate partner violence; child maltreatment; family economic self-sufficiency, and the caregiver’s adverse childhood experiences. Interview questions primarily will focus on:

1. outcomes for which previous studies of home visiting have found effects,

2. outcomes with the greatest potential to be linked to long-term economic benefits

3. outcomes that are measures of adverse childhood experiences (ACEs), which are important predictors of poorer socioeconomic and health outcomes as children get older, and

4. measures that are key mediators of longer-term outcomes.

The information collected from this interview will not be available from other sources (such as administrative records).

Table A.1 lists the constructs that will be collected in these various domains and the proposed measures. The structured interview is designed to take 1 hour. It includes an introductory script, items to verify the participant’s current contact information and means of reaching them in the future, and items to collect information to assess the effects of home visiting. The structured interview attachment includes over one hour’s worth of questions, but we plan to use a technique called “planned missingness” to ensure that each respondent receives only 60 minutes of interview items. Specifically, for a construct with many items (such as social-emotional skills), each respondent will only be asked a subset of all of the items in the construct. In this way, groups of respondents will be assigned to answer only a portion of the items in this draft so that an individual’s total response time is 60 minutes or less.

Table A.1

Construct	Scale name (if applicable)
Child development and school performance
Child care setting before kindergarten	N/A
Social-emotional skills	Social Skills Improvement System (SSIS)
Behavior problems	Social Skills Improvement System (SSIS)
Early intervention services	N/A
Social support and relationships
Relationship and marital status	N/A
Relationship with child’s biological father	N/A
Biological father’s involvement	Maternal Social Support Index
Caregiver-child separations	N/A
Social support	Perceived Social Support Measure
Intimate partner violence
Women’s experience of battering	Women’s Experience with Battering Scale (WEB)
Physical assault: perpetration and victimization	Conflict Tactics Scale (CTS2)
Family conflict	Family Environment Scale
Parenting
Learning environment: Home literacy environment	N/A
Learning environment: Cognitive stimulation	N/A
Screen time	N/A
Parenting stress	Parenting Stress Index - Short Form
Household chaos	Chaos, Hubbub, and Order Scale
Mobilizing resources	Healthy Families Parenting Inventory
Parental warmth	Early Childhood HOME
Family economic self-sufficiency
Maternal education	N/A
Public assistance	N/A
Employment	N/A
Income	N/A
Housing	N/A
Food insecurity	USDA U.S. Household Food Security Survey Module - Short Form
Material hardship	N/A
Maternal health and well-being
Subsequent birth spacing and outcomes	N/A
Maternal depression	Center for Epidemiological Studies Depression Scale (CES-D)
Drug use	N/A
Alcohol use	CAGE Questionnaire
Mastery	Pearlin Mastery Scale
Child health
ED visits	N/A
Hospital admissions	N/A
Insurance coverage	N/A
Child maltreatment
Abuse: physical and psychological/emotional	Parent Child Conflict Tactics Scale (CTSPC)
Mother’s adverse childhood experiences
Mother’s adverse childhood experiences	N/A

Direct assessments of children

Maternal stimulation of children’s language development and cognitive functioning is a core component of many home visiting programs. Language development and cognitive functioning in the early years of life is a predictor of longer-term readiness and achievement. Furthermore, there is evidence of positive effects of home visiting programs in these areas. For these reasons, assessment of children’s language—particularly their receptive language—and their early numeracy and executive functioning are important outcomes. These outcomes are best measured via direct assessments of children.

A direct assessment of the child’s language development will be conducted using the Woodcock Johnson IV Picture Vocabulary and Oral Comprehension (WJPV) subtests, which are from the Woodcock Johnson IV: Tests of Oral Language (WJOL). The Picture Vocabulary subtest assesses receptive language by having the children point to pictures of objects or actions on an easel panel that are named by the assessor. The Oral Comprehension subtest assesses the children’s ability to understand a short passage by having them provide a missing word based on cues from the sentence (for example, “water looks blue and grass looks ______”). It takes about 5 minutes to administer each subtest. A Spanish version of the Woodcock Johnson subtests for bilingual Spanish-English speakers is available.

The Woodcock Johnson III Applied Problems subtest will be used to measure children’s early numeracy and math skills. This is a subtest from the Woodcock Johnson III: Test of Achievement and measures children’s ability to solve oral math problems (for example, “how many dogs are there in this picture?”). It takes approximately 5 minutes to administer this task. A Spanish version of the subtest is also available. Before we conduct the Woodcock Johnson subtests, we will also administer a preLAS language screener for children who may be bilingual in order to determine which versions they should be administered.

Children’s executive functioning, including their working memory, inhibitory control, and cognitive flexibility, will be assessed using a combination of the Digit Span, Hearts & Flowers, and Attention Sustained task:

• Digit Span, which is a measure of working memory, assesses the child’s ability to repeat an increasingly complex set of numbers first forward, and then backward. It takes about 2 to 3 minutes to administer.

• Hearts & Flowers is designed to capture inhibitory control and cognitive flexibility and is administered through an application on a tablet. The task includes three sets of trials: (1) 12 congruent “heart” trials, (2) 12 incongruent “flower” trials, and (3) 33 mixed “heart and flower” trials. Children are presented with an image of a red heart or flower on one side of the screen. For the congruent heart trials, the children are instructed to press the button on the same side as the presented heart. For incongruent flower trials, children are instructed to press the button on the opposite side of the presented flower. Accuracy scores are drawn from the incongruent block and mixed block. It takes approximately 5 minutes to administer this task.

• The Attention Sustained task is from the third edition of the Leiter Interactional Performance Scale. Children are shown a series of four pages, each with pictures of scattered objects that vary in their orientation, and are asked to cross out as many of the objects matching a target object shown at the top of the page as they can without accidentally crossing out any other objects. Children are given a limited amount of time to perform each trial (e.g. 30 seconds for the first three trials and 60 seconds for the fourth). It takes approximately 4 to 5 minutes to administer this task.

The total estimated burden for the direct child assessments is 90 minutes. The direct assessment of children protocols are in Attachment 2.

Along with direct assessments of the child’s language skills, early numeracy, and executive functioning, the assessor will also observe the families and their homes to assess parental warmth and the child’s emotion, attention, and behavior. Since it does not represent a burden to families, OMB is not being asked to approve the observational component follow-up. Specifically, it (1) does not require the family to provide any information, and (2) will be conducted at the same time as other in-home aspects of data collection. This is consistent with 44 USC, 5 CFR Ch. 11 (1-1-99 Edition), 1320.3, which indicates that “information” does not generally include facts or opinions obtained through direct observation by an employee or agent of the sponsoring agency or through nonstandardized oral communication in connection with such direct observations. The areas covered by the assessor observation component of the in-home visit are parental warmth and the child’s behavior, although the specific questions are not provided because they are proprietary.

Semi-structured interview with caregivers

Implementing a qualitative study as part of MIHOPE will enrich our understanding of families’ past experiences in home visiting and allow us to learn more about families’ broader circumstances navigating parenthood with young children and how home visiting programs could improve. This information cannot be gathered from other sources.

We will conduct in-person semi-structured interviews with 100 of the MIHOPE families who were assigned to the program group and whose children are on the younger end of the MIHOPE sample (who are currently approximately 3 ½ years of age). The interviews will focus on having the parents narrate and recall their experiences early in the child’s life, and the parents’ interactions with informal and more formal (programmatic) types of supports to navigate parenting of young children. Interviews will probe explicitly on why they became involved in home visiting programs, their expectations for the programs, their duration of participation, the role of the home visiting program in aiding the parent in the early years, their perceptions of sustained improvements (for example, through helping encourage the mom’s educational or employment goals or dealing with parenting stress), and their opinions on how the programs could be improved. Questions are tailored so that different questions will be asked depending on families’ length of participation in the home visiting program (for families who left the programs early – “early leavers” – versus families who stayed in the program for a moderate amount of time – “later leavers” – and families who participated long-term – “long-term participators”) in order to collect information on a wide range of experiences and develop a better understanding of why families may have left or stayed in a program.

These interviews will be conducted with MIHOPE families whose children are about three and a half years old. Of the families in the MIHOPE sample, these families will have participated most recently in home visiting and thus may be better able to recall their experiences with the home visiting program. The interview protocol is included as Attachment 3. The specific prompts align with the interview topics included in the original version of Attachment 3 included in the approved package. The semi-structured interview is designed to take two hours or less.

Survey of the focal children’s teachers

The teacher survey (Attachment 4) will collect information on behaviors that are more commonly demonstrated in a classroom setting rather  than the home. For these outcomes, such as learning behaviors and approaches to learning, teacher-reported measures have also shown better reliability than parent-reported measures.¹ Other outcomes (such as behavior problems and social-emotional skills) will be part of the teacher survey even though they will also be included on the family follow-up survey. For these outcomes, teachers can provide information about these outcomes in unique contexts (for example, parents usually reference their communities when assessing children while teachers use their school experiences). Further, teachers can provide a different, and presumably less biased, perspective since they were not the targets of the home visiting intervention.

The survey is designed to take 30 minutes or less. It includes items to verify that the participant is the focal child’s teacher and items to collect information to assess the effects of home visiting.

Direct assessments of caregivers

An emerging body of literature has indicated that mothers’ cognitive control capacities are particularly relevant for engaged and responsive caregiving; these skills support caregivers’ ability to be perceptive, responsive, and flexible.² Cognitive control capacities are especially important for mothers in stressful conditions including adverse life events and stress related to lower family socioeconomic status (e.g., poverty, unemployment).

Given that many families in MIHOPE experience a variety of risk factors and live in communities that are more disadvantaged than the national average,³ it is particularly important to directly assess mothers’ cognitive functioning to understand whether mothers possess skills that are theorized to enable them to resist environmental distractions, monitor children’s needs, and flexibly switch focus between competing contextual demands.⁴ The measure being used to assess maternal cognitive control (and specifically, working memory – an aspect of cognitive control) is the Digit Span. The burden estimated for this measure is fifteen minutes. This assessment protocol is included in Attachment 5.

Videotaped caregiver-child interactions

A caregiver-child interaction task will be administered in order to assess the behavior of the mother and of the child during a semi-structured play situation. The interaction task will be videotaped and viewed at a later date by trained coders, who will rate caregiver and child behavior to assess qualities of parenting (such as parental supportiveness, parental stimulation of cognitive development, parental intrusiveness, parental negative regard, and parental detachment) and the child’s behavior (such as child engagement of parent, child's quality of play, and child's negativity toward parent). These outcomes require independent assessments (as opposed to self-reports, which may be more likely to be influenced by home visiting programs through raising parents’ awareness of preferred or desired responses regarding various types of parenting behaviors).

The caregiver-child interaction task will involve semi-structured play and will consist of tasks that were used in previous longitudinal studies that measured child development outcomes (i.e., the NICHD Study of Early Child Care and Development, a longitudinal study that examined the relationship between child care experiences and characteristics and child development outcomes, and the Early Head Start Research and Evaluation Project, a longitudinal impact evaluation of the Early Head Start program). The activities involve having the parent and child play with toys such as an Etch-A-Sketch, wooden blocks, animal puppets, and/or Play Doh. With each toy, the pair is instructed to either complete a specific task or to play with them in whichever way they would like. The activities are fun and interesting for children to complete with their caregivers. The play time lasts approximately 15 minutes. The protocol is included in Attachment 6.

Three of the data collection components, the direct assessments of children (estimated burden 90 minutes), the direct assessments of caregivers (estimated burden 15 minutes), and the videotaped caregiver-child interactions (estimated burden 15 minutes), will occur during the in-home visit.

Caregiver Website

Caregivers will be provided with a website they can visit to confirm the focal child’s participation in kindergarten or first grade, update their contact information, schedule a time for the structured interview with caregivers, provide information about the child’s school and teacher, and give consent.

The website is organized so as to minimize burden for respondents and ensure that respondents are not being asked any unnecessary questions. For example, if caregivers do not provide consent for the teacher survey, we will not ask them about their child’s teacher.

Most of the questions on the website overlap with questions that will be asked during the structured interview with caregivers. As a result, the burden for the website fits into the current burden estimate (60 minutes) for the interview. If caregivers have already responded to these questions on the website, we will skip these questions during the interview.⁵ Additionally, if caregivers have already given consent for the teacher survey online, we will skip that step during the in-home visit.⁶

The only questions on the website that will not be asked during the structured interview are the questions around scheduling the structured interview. We have estimated that 25 percent of the sample will visit the website and that it will take about ten minutes for respondents to respond to questions on the website. We have therefore subtracted this estimated burden from the caregiver interview row in the burden table.

The text for the website is included as Attachment 12.

Administrative Data: Child welfare records and school records

As indicated earlier in this section, we plan to continue to obtain child welfare data from state agencies (the study obtained child welfare records from the 12 MIHOPE states at its 15-month follow-up) and plan to request school records data from state and local agencies. (For school records, we have assumed that we will obtain data from 12 states and 5 local education agencies.)

For both child welfare and school records, we plan to receive two data files from each agency. We have assumed a lower burden per state for child welfare records (as compared to school records). Requests to agencies are included in Attachments 13 and 14.

A3. Improved Information Technology to Reduce Burden

This study will use information technology, when possible, to minimize respondent burden and to collect data efficiently.

For the structured interview with caregivers, respondents will have the option to call a survey center and complete it using computer-assisted telephone interviewing (CATI). CATI reduces respondent burden by using skip logic to quickly move to the next appropriate question depending upon a respondent’s previous answer.

The semi-structured interviews with caregivers will be audio-recorded. This will reduce respondent burden by ensuring that the field staff are more focused on properly administering the protocol rather than other tasks (such as taking notes), thus preventing the interview from being delayed and preventing the respondent from having to repeat any points that the interviewer may have missed.

For the teacher survey, respondents will first be offered an opportunity to complete the survey via the Web. This will reduce respondent burden by using skip logic to ensure that only appropriate questions are asked of the respondent. It also will save project resources and may increase response rates by allowing respondents to complete it at a time of their choosing. Participants will receive information about how to complete the survey online shortly before they are eligible to complete the survey. Teachers who do not complete the survey online will have the option to complete the survey via CATI or by paper and pencil.

The caregiver-child interaction task will be video-recorded on a smart card, which allows for coding of the interaction to be done at a later date by trained coders. The use of electronic recording ensures that the field staff are more focused on proper administration of the task than on other tasks (such as coding), thus preventing the tasks from being delayed or prolonged and minimizing the chances of needing to re-do the tasks due to administration error, which reduces respondent burden.

Direct assessments of children and caregivers will be conducted using applications on tablets. The use of these applications will similarly prevent the field staff from focusing on other tasks instead of the task at-hand, therefore preventing the tasks from being delayed or prolonged and minimizing the chances of needing to re-do the assessments, which reduces respondent burden.

We will also create a web page that would allow respondents to update their contact information easily, efficiently, and at a time most convenient for them.

Electronic data collection will also allow the research team to track real-time response rates and to monitor data on a regular basis to ensure data quality in real time. The research team will receive weekly reports, which will allow them to monitor data collection by detailing who has completed the direct assessments. Given the study’s real-time access to the web-based data, research staff will be able to regularly review item frequencies and cross-tabulations to guard against inconsistent or incorrect values. In addition, the web-based system is designed such that invalid responses cannot be entered.

A4. Efforts to Identify Duplication

For outcomes included in the data instruments, there is no other source we can use to gather this information about MIHOPE participants.

A5. Involvement of Small Organizations

No small businesses are affected by the data collection in this project.

A6. Consequences to Less Frequent Data Collection

Data will be collected once for each family and from each teacher at the time of the kindergarten follow-up. Data include the structured interviews with caregivers, direct assessment of children, the assessor observation, semi-structured interviews with caregivers, survey of focal children’s teachers, direct assessment of caregivers, and video-taped caregiver-child interactions. Since these data are only collected once, reducing the frequency of the data collection would mean eliminating it, which would greatly limit the ability of the evaluation to answer questions about the long-term effectiveness of home visiting programs across a range of child and parent domains.

A7. Special Circumstances

There are no special circumstances requiring deviation from these guidelines.

A8. Federal Register Notice and Consultation

Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on January 30, 2018, Volume 83, Number 20, page 4208, and provided a sixty-day period for public comment. A copy of this notice is attached as Attachment 7. During the notice and comment period, we received one request for information and one comment. The comment and ACF’s response are attached (Attachment 8).

Consultation with Experts Outside of the Study

The following experts have provided guidance on the kindergarten follow-up:

• Mark Appelbaum (University of California San Diego)

• Elizabeth Doggett (Libby Doggett Consulting)

• Anne Duggan (Johns Hopkins University)

• Greg Duncan (University of California Irvine)

• Beth Green (Portland State University)

• Mark Greenberg (Pennsylvania State University)

• Rob Grunewald (Federal Reserve Bank of Minneapolis)

• Brenda Jones Harden (University of Maryland)

• Todd Little (Texas Tech University)

• Cynthia Minkovitz (Johns Hopkins University)

• Jelena Obradovic (Stanford University)

• Glenn Roisman (University of Minnesota)

A9. Incentives for Respondents

Incentives for caregivers

As discussed in Supporting Statement B, MIHOPE-K collects longitudinal data from young, low-income mothers. Combined with other administrative efforts intended to communicate the study’s relevance and salience to participants, incentives are an important means for improving participant engagement throughout the study; securing an adequate response rate to answer research questions; and reducing differential attrition of program and control groups and specific subgroups of interest.⁷ Based on MIHOPE data collection from the 2.5 year check in, we have concrete concerns about differential nonresponse for specific subgroups of interest highlighted below.

Incentive amounts proposed

We propose the following incentives during the multipart MIHOPE-K data collection:

• $25 for completing a 60-minute caregiver interview;

• $50 and a small book or toy for the child for completing in-home activities estimated to take up to 120 minutes;

• Two branded study reminders, intended to maintain the study’s salience for participants: a small gift, such as a lunch sack sent before the beginning of the kindergarten interview fielding period, and an additional physical reminder, such as a book of sticky notes with the study’s name and toll-free number, sent halfway between the kindergarten study and a potential third grade follow up. These items will be fully branded to show the MIHOPE logo, color scheme, and design, to be consistent with the study’s overall outreach and communication effort.⁸

Given the burden of the semi-structured interview, there is a need for separate incentives for this component of data collection. We propose offering a $50 incentive for the 2-hour semi-structured caregiver interview.

Using incentives to address study concerns

Incentives are intended to address the following concerns:

Reducing nonresponse bias, differential attrition, and overall attrition to ensure that the study has enough statistical power and a sufficiently representative sample to answer its key research questions. A high response rate makes it more likely that interview respondents are representative of the initial sample (including ensuring equal representation among the program and control groups), which is important when estimating effects of home visiting for the study population. As has previously been communicated to OMB, MIHOPE struggled with overall attrition and attrition across subgroups of families in the MIHOPE Check-in 2.5 year old survey.

In the 2.5 year sample, we conducted an experiment examining a pre-pay and an early bird incentive strategy (for additional information about the experiment and the results, please see the memorandum detailing this experiment). Overall attrition was 45.2% for the portion of the 2.5 year follow-up sample that participated in our incentive experiment (N = 1,705). As previously communicated to OMB, we also found some statistically significant differences in important baseline characteristics between respondents and nonrespondents of the 2.5 year old survey. For example, as shown in Table A.2, nonrespondents were:

• more likely to have entered the study while they were pregnant, which we expect to be an important predictor of the effectiveness of home visiting services

• more likely to have moved in the year prior to entering the study, so survey responses might not accurately represent the effects for the most mobile part of the sample

• less likely to live in a household with their child’s father figure, and are

• less likely to be married to the biological father of their child.

Table A.2: Differential response to the 2.5 year old survey: Significant differences at end of incentive experiment period^{9, 10}

Characteristics (at study entry)	Respondents (%)	Nonrespondents (%)	Difference
Pregnant	47.2	56.6	-9.4
Moved in the prior year	17.1	25.1	-8.0
Child’s father figure does not live in household	54.2	61.5	-7.3
Not married to biological father of child	77.9	85.3	-7.4

Differential attrition across key subgroups is also a major concern for the kindergarten time point of MIHOPE, as subgroup representativeness is necessary to address a primary research question: Are the long-term effects of home visiting larger for some types of families than for others?

Ensure the study meets quality rating standards set out by the What Works Clearinghouse (WWC) ¹¹ and by HHS’s Home Visiting Evidence of Effectiveness review (HomVEE).¹² It is especially important for MIHOPE to meet HomVEE’s standards, as this is the primary evidence review for the home visiting field. Study rating criteria include low overall attrition and low differential attrition. Meeting these standards is necessary for the study to be maximally useful for policymakers and practitioners.

Reduce the number of contact attempts and the length of the data collection window, to ensure that information on child development is collected at a similar time point for all families. Incentives can reduce the number of contact attempts needed to complete cases, which can both reduce the costs of data collection and improve data comparability. ¹³ In MIHOPE-K, minimizing the data collection window is important to address policy relevant questions about the effects of home visiting on aspects of child development and school readiness. Lengthening the data collection window to reduce nonresponse or nonresponse bias may mean that later respondents will provide information too late in the school year to offer a valid measure of their children’s school readiness.

Using incentives for semi-structured interviews:

Interview data are not intended to be representative in a statistical sense, in that they will not be used to make statements about the prevalence of experiences across all MIHOPE program group members. However, it is important to secure participants with a range of background characteristics, to capture a variety of possible experiences with home visiting services. Without offsetting the direct costs incurred by respondents for participating in the interviews, such as arranging child care or time off from paid work, the research team increases the risk that only those individuals able to overcome potential financial barriers will participate in the study. Participants will receive a $50 gift card to account for expenses such as time away from work and/or child care that may otherwise prevent their participation in the study. This amount is consistent with the $50 incentive that OMB approved for the MIHOPE-K in-home visit with a similar burden estimate. It is also consistent with incentive that OMB approved for other focus groups and interviews with a similar burden estimate. For example, participants were offered $50 for 90-minute case study interviews as part of the Evaluation of SNAP Employment and Training Pilots (OMB control number 0584-0604).

A10. Privacy of Respondents

Information collected will be kept private to the extent permitted by law. Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law.

When participants are contacted about their continued presence in the study, they will be reminded of the study goals, time required, and the nature of questions that will be asked. Parents will be assured that their responses will be shared only with researchers, will be reported only in the aggregate as part of statistical analyses, and will not affect their receipt of services. They will also be told that all data collection activities are voluntary, and they can refuse to answer any and all questions without penalty.

Due to the sensitive nature of this research (see A11 for more information), MIHOPE has obtained a Certificate of Confidentiality from HRSA, which we will extend to cover MIHOPE-LT and will provide it to OMB once it is received. The Certificate of Confidentiality helps to assure participants that their information will be kept private to the fullest extent permitted by law.

The study team is committed to protecting the privacy of participants and maintaining the privacy of the data that are entrusted to us and is experienced in implementing stringent security procedures. Every MDRC and Mathematica employee, including field staff employed for data collection, is required to sign a pledge to assure participants of nondisclosure of private information. Field staff will also be trained in maintaining respondent privacy and data security.

Mathematica’s Sample Management System (SMS), which has been used for all previous rounds of MIHOPE data collection, will continue to be the central clearinghouse for all contact information on MIHOPE families. Documents shipped from the field and the document transmittal form that accompanies them will contain only identification numbers so that data cannot be attributed to any particular individual. Security will be maintained on the complete set (and any deliverable backups) of all master survey/interview files and documentation, including sample information, tracking information, baseline, and follow-up data. Personally identifiable information will be removed from study files, which will contain a linking identification number that can be used to match records from one data file to another. Finally, data will be available only to staff associated with the project through password protection and encryption keys.

As specified in the evaluator’s contract, MDRC shall use Federal Information Processing Standard compliant encryption (Security Requirements for Cryptographic Module, as amended) to protect all instances of sensitive information during storage and transmission. MDRC shall securely generate and manage encryption keys to prevent unauthorized decryption of information, in accordance with the Federal Processing Standard. MDRC shall: ensure that this standard is incorporated into the MDRC’s property management/control system; establish a procedure to account for all laptop computers, desktop computers, and other mobile devices and portable media that store or process sensitive information. Any data stored electronically will be secured in accordance with the most current National Institute of Standards and Technology (NIST) requirements and other applicable Federal and Departmental regulations. In addition, MDRC shall minimize the inclusion of sensitive information on paper records and for the protection of any paper records, field notes, or other documents that contain sensitive or personally identifiable information that ensures secure storage and limits on access.

Information will not be maintained in a paper or electronic system from which data are actually or directly retrieved by an individuals’ personal identifier.

A11. Sensitive Questions

Questions in some components of the MIHOPE-K structured interview with caregivers are potentially sensitive for respondents. Parents are asked about personal topics, such as child and parental health, maternal depression, income, and intimate partner violence. To improve understanding of how home visiting programs affect families and children, it will be necessary to ask these types of sensitive questions. For example, maternal depression is a major risk factor for reduced family well-being and child development¹⁴ and an outcome that MIECHV-funded home visiting programs are encouraged to try to address.¹⁵ Similarly, there are MIECHV performance measures related to intimate partner violence, which has been shown to influence parenting distress, maternal behaviors such as substance use and engaging in unprotected sex,¹⁶ and child stress and externalizing behavior, all of which can have a negative impact on the family and child outcomes MIHOPE is examining.¹⁷

As noted in section A4, this information will not be available from other data sources. Respondents have been asked similar information at baseline and at the 15-month, 2.5-year, and 3.5-year follow-ups, so they are familiar with the types of questions that will be asked.

To reduce respondents’ potential discomfort about potentially sensitive questions, the MIHOPE-K structured interview with caregivers will remind participants that they may refuse to answer any question without penalty. Also, respondents will be informed by research staff prior to the start of the interviews or surveys that their answers will be kept private to the extent permitted by law, that results will only be reported in the aggregate, and that their responses will not affect any services or benefits they or their family members receive.

A12. Estimation of Information Collection Burden

Previously Approved Information Collections

Information collections were previously approved for: MIHOPE 1, which covered the collection of data at baseline; MIHOPE 2, which covered the collection of follow-up data when the children in the study were 15 months old; and MIHOPE Check-in, which covers the collection of updated contact information and follow-up data on children and parents when the children in the study are 2½, 3½, and 4½ years old. Data collection at 4½ years old is no longer planned.

Total Burden Previously Approved

Total burden previously approved under 0970-0402 is 13,311 hours. This includes 5,786 hours for MIHOPE 1, 5,375 hours for MIHOPE 2, and 2,150 hours for MIHOPE Check-in.

Burden Remaining from Previously Approved Information Collection

Data collection from MIHOPE 1 and MIHOPE 2 have ended, but collection for MIHOPE Check-in is ongoing at the time of this submission. 717 annual hours of burden from MIHOPE Check-in is unused and ongoing. Table A.3 shows the remaining burden for each activity.

Table A.3

Instrument	Total REMAINING Number Respondents	ANNUALIZED number of remaining respondents	Number Responses per Respondent	Average Burden Hour per Response	ANNUAL Estimated Burden Hours
Child and family outcome survey and updating contact information (2.5 and 3.5 year)	2150	717	2	0.5	717
Total annual burden:					717

Newly Requested Information Collections

Table A.4 shows the annual burden of the activities described in this supporting statement.

The team will try to collect follow-up information from all families in the initial MIHOPE study for most of the data collection components. One of the data collection activities will be done with a subsample of families. We currently project the study will include 4,115 families.¹⁸

Table A.4

Instrument	Total Number of Respondents	Annual Number of Respondents	Number of Responses Per Respondent	Average Burden Hours Per Response	Annual Burden Hours	Average Hourly Wage	Total Annual Cost
MIHOPE-LT Data Collection
Structured interview with caregivers	4115	1372	1	0.96	1314	$13.28	$17,449.92
Direct assessments of children	4115	1372	1	1.5	2058	$13.28	$27,330.24
Semi-structured interview with caregivers	100	33	1	2	66	$13.28	$876.48
Survey of the focal children’s teachers	4115	1372	1	0.5	686	$26.68	$18,302.48
Direct assessments of caregivers	4115	1372	1	0.25	343	$13.28	$4,555.04
Videotaped caregiver-child interactions	8230	2743	1	0.25	686	$13.28	$9,110.08
Caregiver website	1029	343	1	0.17	58	$13.28	$770.24
State child welfare records: data file submission	12	4	2	15	120	$39.87	$4,784.40
School records: data file submission	17	6	2	22.5	270	$39.87	$10764.90
Estimated Annual Burden Total					5,601		$93,943.78

Total Annual Cost

The requested annual burden for conducting the nine data collection activities is 5,601 hours. For collecting data from families, an hourly wage of $13.28 was assumed for mothers, which is the median wage for full-time workers 25 years old or older with less than a high school diploma.¹⁹ For collecting data from teachers, an hourly wage of $26.68 was assumed, which is the median wage for full-time kindergarten teachers.²⁰ For collecting data from states and local education agencies, an hourly wage of $39.87, which is the median for Computer and Information Analysts.²¹

A13. Cost Burden to Respondents or Record Keepers

For the survey of focal children’s teachers, we propose offering teachers a maximum $10 honoraria for providing their professional services and completing the survey, which has been designed to take 30 minutes or less. This is in line with the average hourly wage of a kindergarten teacher, which is about $26.68.²²

Respondent	Data collection instrument	Estimated time to complete	Proposed Honorarium	Hourly Wage Rate
Kindergarten teachers	Survey of the focal children’s teachers	0.5 hours	$10	$26.68

A14. Estimate of Cost to the Federal Government

The total cost for the data collection activities under this current request will be $24,841,914.49. This amount includes costs for new data collection activities under this request and the remaining costs from previously approved collections still in progress. Annual costs to the Federal government will be $8,280,638.16 for the proposed data collection.

A15. Change in Burden

This is a request to collect additional information from respondents participating in the MIHOPE Evaluation (0970-0402).

A16. Plan and Time Schedule for Information Collection, Tabulation, and Publication

We plan to collect information on the families starting in the fall of 2018. We plan to conduct semi-structured interviews starting in early 2019. This current information collection request is for three years. It is expected that data collection may last until 2022. If the data collection period needs to be extended past the three year expiration date in 2021, we will submit an additional information collection request to continue data collection. A report published by the federal government will show estimated long-term effects of home visiting programs through the kindergarten follow-up study.

A18. Reasons Not to Display OMB Expiration Date

All instruments will display the expiration date of OMB approval.

A19. Exceptions to Certification for Paperwork Reduction Act Submissions

No exceptions are necessary for this information collection.

REFERENCES

Barrett, J., & Fleming, A.S. (2011). Annual research review: All mothers are not created equal: Neural and psychobiological perspectives on mothering and the importance of individual differences. Journal of Child Psychology and Psychiatry 52(4), 368-397.

Chazan‐Cohen, R., Raikes, H.H., & VogelC. (201, 3). V. Program Subgroups: Patterns of Impacts for Home-Based, Center-Based, and Mixed-Approach Programs. Monographs of the Society for Research in Child Development 78(1), 93-109.

Crandall, A. A., Deater-Deckard, K., & Riley, A.W. (2015). Maternal emotion and cognitive control capacities and parenting: A conceptual framework. Developmental review 36, 105-126.

Cummings, M.E., & Davies, P.T. (1994). Maternal Depression and Child Development. Journal of Child Psychology and Psychiatry 35(1), 73-122.

Easterbrooks, M.A., Fauth, R.C. & Lamoreau, R. (2017). Effects of a Home Visiting Program on Parenting: Mediating Role of Intimate Partner Violence. Journal of Interpersonal Violence, 1-21.

James, T. (2001). Results of the Wave 1 incentive experiment in the 1996 survey of income and program participation. Paper presented at the Proceedings of the Section of Survey Research Methods, Alexandria, VA.

Laurie, H., & Lynn, P. (2008). The use of respondent incentives on longitudinal surveys. ISER Working Paper Series, No. 2008-42. Essex, UK: Institute for Social and Economic Research (ISER), University of Essex.

Mack, S., Huggins, V., Keathley, D., & Sundukchi, M. (1998). Do monetary incentives improve response rates in the survey of income and program participation? Paper presented at the Proceedings of the Section on Survey Research Methods, Alexandria, VA.

Markesich, J., & Kovac, M.D. (2003). The Effects of Differential Incentives on Completion Rates: A Telephone Survey Experiment with Low-Income Respondents. Presented at the Annual Conference of the American Association of Public Opinion Research. Nashville, TN.

Martin, E., Abreu, D., & Winters, F. (2001). Money and motive: Effects of incentives on panel attrition in the survey of income and program participation. Journal of Official Statistics, 17, 267-284.

Michalopoulos, C., Lee, H., Duggan, A., Lundquist, E., Tso, A., Crowne, S., Burrell, L., Somers, J., Filene, J.H., & Knox, V. (2015). The Mother and Infant Home Visiting Program Evaluation: Early Findings on the Maternal, Infant, and Early Childhood Home Visiting Program. OPRE Report 2015-11. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

Scribano, P.V., Stevens, J. & Kaizar, E. (2013). The Effects of Intimate Partner Violence Before, During, and After Pregnancy in Nurse Visited First Time Mothers. Maternal and Child Health Journal 17(2), 307-318.

Singer, E., & Ye, C. (2013). The Use and Effects of Incentives in Surveys. The Annals of the American Academy 645, 112-141.

Sternberg, K.J., Lamb. M.E., Greenbaum, C., Cicchetti, D., Dawud, S., Manela Cortes, R., Krispin, O., & Lorey, F. (1993). Effects of Domestic Violence on Children’s Behavior Problems and Depression. Developmental Psychology 29 (1), 44-52.

Wolfe, D.A., Crooks, C.V., Lee, V., McIntyre-Smith, A., & Jaffe, P.G. (2003). The Effects of Children’s Exposure to Domestic Violence: A Meta-Analysis and Critique. Clinical Child and Family Psychology Review 6(3), 171-187.

1 Chazan-Cohen et al., 2013

2 Crandall et al., 2015

3 Michalopoulos et al., 2015

4 Barrett & Fleming, 2011

5 The wording of some of the questions on the website and structured interview varies slightly because it is optimized for how each instrument will be administered (electronic vs. telephone). However, the information collected will be the same. Additionally, contact information will be re-confirmed, rather than skipped, during the interview, in order to ensure that the gift card incentive is sent to the correct address.

6 The consent form on the website will be the same version used during the in-home visit.

7 James, 2001; Mack et al., 1998; Martin et al., 2001

8 Dillman, 2007; Estrada, Woodcock, Schultz, 2014

9 Significant differences for these characteristics are also present for the full 2.5 year data collection sample. (Following the conclusion of the 2.5 year follow-up incentive experiment, we received OMB approval to use the early bird incentive structure for the remainder of the cases released for the 2.5 year data collection.)

10 The early bird incentive seemed to have a slightly larger effect for groups of participants defined by the characteristics listed below. In particular, we found that the subgroups with differential attrition in Table X had slightly higher response rates when they were offered the early bird incentive. For more information, please see [2.5 experiment OMB memo].

11 https://ies.ed.gov/ncee/wwc/Docs/ReferenceResources/wwc_attrition_v2.1.pdf

12 For study rating criteria, see http://homvee.acf.hhs.gov/Review-Process/4/Producing-Study-Ratings/19/5

13 For example, the DC TANF Caseload Survey, which has a similar sample to MIHOPE (young, low-income mothers), found that offering an incentive that was $15 higher significantly decreased the number of contact attempts. On average, the higher incentive group reached the target response rate with fewer than 20 attempts while the lower incentive group never reached the target response rate even after interviewers made over 31 contact attempts in some cases (Markesich & Kovac, 2003).

14 Cummings & Davies, 1994

15 MIECHV performance measures that all grantees are required to report on include: “percent of primary caregivers screened for depression” and “percent of primary caregivers referred to services for a positive screen for depression who receive one or more service contacts” (https://mchb.hrsa.gov/sites/default/files/mchb/MaternalChildHealthInitiatives/HomeVisiting/Federal_Home_Visiting_Program_Performance_Indicators_and_Systems_Outcomes_Summary.pdf).

16 Scribano, Stevens, & Kaizar, 2013; Easterbrooks, Fauth, & Lamoreau, 2017

17 Sternberg et al., 1993; Wolfe et al., 2003

18 The MIHOPE study randomized 4,229 families. However, 11 of these families withdrew from the study and the focal child passed away or was not born alive in 103 other families, leaving us with a sample of 4,115.

19 https://www.bls.gov/news.release/pdf/wkyeng.pdf

20 https://www.bls.gov/ooh/education-training-and-library/mobile/kindergarten-and-elementary-school-teachers.htm

21 https://www.bls.gov/oes/current/naics4_999300.htm#15-0000

22 https://www.bls.gov/ooh/education-training-and-library/mobile/kindergarten-and-elementary-school-teachers.htm

22