MIHOPE Check-in_Supporting Statement A REVISED 24jul15clean

Maternal and Infant Home Visiting Program Evaluation (MIHOPE):

MIHOPE Check-in

OMB Information Collection Request

0970 - 0402

Supporting Statement

Part A

June 2015

Submitted By:

Office of Planning, Research and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

7^th Floor, West Aerospace Building

370 L’Enfant Promenade, SW

Washington, D.C. 20447

Project Officer:

Nancy Geyelin Margie

Part A. JUSTIFICATION

A1. Circumstances necessitating data collection

In 2011, the Administration for Children and Families (ACF) and Health Resources and Services Administration (HRSA) within the U.S. Department of Health and Human Services (HHS) launched the Mother and Infant Home Visiting Program Evaluation (MIHOPE). MIHOPE, mandated by the Affordable Care Act (ACA), is providing information about the effectiveness of the Maternal, Infant, and Early Childhood Home Visiting program (MIECHV or the Home Visiting Program) in its first few years of operation, and providing information to help states and others develop and strengthen home visiting programs in the future. The goals of the study are: (1) to understand the effects of home visiting programs on parent and child outcomes, both overall and for key subgroups of families, (2) to understand how home visiting programs were implemented and how implementation varied across programs, and (3) to understand which features of local home visiting programs are associated with larger or smaller program impacts. To estimate the effects of home visiting on family outcomes, MIHOPE is enrolling 4,300 families across 88 sites in 12 states. Families are eligible for the study if they include a pregnant woman or an infant under six months old and the mother is at least 15 years old.

OMB approved data collection packages (OMB Control Number 0970-0402) for two phases of MIHOPE in July 2012 and June 2013. Phase 1 (MIHOPE1) covers the collection of data at baseline, when families are enrolled into the study (Attachment 1), while Phase 2 (MIHOPE2) covers the collection of follow-up data, when the children in the study are 15 months old (Attachment 2).

Because previous research on home visiting programs has found long-term impacts on child and family outcomes, ACF and HRSA want to continue collecting follow-up information from families participating in the study. To maintain contact with participants and to collect additional follow-up information, in September 2014, OPRE awarded a contract to MDRC to conduct MIHOPE Check-in.

A2. Purpose and use of the information collection: How, by whom, and for what purpose the information is to be used.

MIHOPE Check-in includes data collection to update information in two broad areas previously approved by OMB for MIHOPE. These are summarized below:

1. Collect updated contact information from families. This will permit researchers to maintain contact with the family and allow for longer-term follow-up, for example, as the child enters the school system. Similar collection of contact information was approved by OMB when families entered the study in MIHOPE1 and when they were followed up with in MIHOPE2.

2. Collect updated information on outcomes for parents and children via web or phone survey. This will allow the study to estimate ongoing effects of the Home Visiting Program on key family outcomes.

Data will be collected on an annual basis, when the child is approximately 2.5 years old, 3.5 years old, and 4.5 years old.

At each data collection timepoint, both updated contact information and family/child outcomes information will be collected with a single survey instrument. Administrative data, if it continues to be collected after MIHOPE 2, will be obtained directly from the agencies who hold the data, placing no extra burden on the families. The MIHOPE consent form allows the study to continue to collect these data, so the study does not expect to need additional consent related to administrative data.

Concerning collection of information on parent and child outcomes, over the course of MIHOPE Check-in, we propose to collect information related to all outcome domains the Home Visiting Program aims to address, such as child health, child development, parental health and well-being, parenting practices, family economic self-sufficiency, domestic violence, child maltreatment, and use of social services. To do this most efficiently and effectively, we are proposing to tailor each annual survey to collect only the pieces of information most pertinent or effectively captured at each age. For example, in the area of child development, we plan to ask about school readiness and behavior problems when children are 3.5 years old, even though these constructs will not be measured on the 2.5 year old survey. Survey questions will focus on outcomes for which previous studies of home visiting have found effects and on outcomes that would not be available from other sources (such as administrative records).

Attached is the survey we propose to use when the children are 2.5 years old (Attachment 3). The surveys proposed to be used with families when the children are 3.5 years old and 4.5 years old will be submitted to OMB as nonsubstantive changes prior to the start of data collection for each of those groups. All burden related to these surveys is included in section A.12 and was described in associated Federal Register Notices.

The survey is designed to take 30 minutes. It includes an introductory script, items to verify the participant’s current contact information and means of reaching them in the future, and items to collect information to assess the effects of home visiting on the outcomes outlined in the ACA.

A3. Use of information technology for data collection to reduce respondent burden

This study will use information technology, when possible, to minimize respondent burden and to collect data efficiently.

For example, survey respondents will first be offered an opportunity to complete the survey via the Web. This will save project resources and may increase response rates by allowing respondents to complete it at a time of their choosing. It also reduces respondent burden by using skip logic to ensure that only appropriate questions are asked of the respondent. Participants will receive information about how to complete the survey online shortly before they are eligible to complete the survey.

Families who do not complete the survey online will have the option to call a survey center and complete it using computer-assisted telephone interviewing (CATI). As with the Web survey, CATI reduces respondent burden by using skip logic to quickly move to the next appropriate question depending upon a respondent’s previous answer.

A4. Efforts to identify duplication and use of similar information

There is no other source of information on family outcomes included in the survey for participants of MIHOPE. Although electronic databases and other methods can be used to obtain some updated contact information, that information may be inaccurate without verifying it with the study participant.

A5. Burden on small business

No small businesses are affected by the data collection in this project.

A6. Consequences to collecting information less frequently

MIHOPE Check-in will survey each family once per year for three years. Information will be collected annually because frequent contact with families will make it easier to continue collecting information from families as their children age. Collecting contact information less frequently would make it more difficult to locate families in the future, which may reduce response rates overall or result in differences in response rates between the program and control groups. Collecting information less frequently might also result in the need for greater resources in order to find and gain cooperation from study participants at a later point in time. Finally, conducting fewer assessments of the family outcomes will reduce the ability of the study to understand how the effects of home visiting change over time and how home visiting affects mediators of longer-term effects on children and families.

A7. Special Data Collection Circumstances

There are no special circumstances requiring deviation from these guidelines.

A8. Form 5 CFR 1320.8 (d) and consultations prior to OMB Submission

The 60-day Federal Register notice soliciting comments for the MIHOPE Check-in data collection instruments was posted in the Federal Register, Volume 79 Number 161, page 49,325 on August 20, 2014. Two comments were received.

In one, the commentor was skeptical of the study and suggested the burden was too great on families and that there was some redundancy. The commentor questioned if the burden estimate for the survey was too high. The survey will be pretested with up to 9 people and, if found to exceed 30 minutes, will be edited so the burden does not exceed the estimated time. Because this survey is intended as a continued follow-up of the MIHOPE project, it is not expected to prove overly burdensome to families who completed 60-minute surveys at baseline and follow-up in addition to participating in 90-minute in-home assessments. In addition, families have consented to participate, and can decline to participate or answer any questions at any time. As with MIHOPE1 and MIHOPE2 (Attachments 1 and 2), MIHOPE Check-in will provide unique and significant information about the continued effects of the federal Home Visiting Program in a large, nationwide sample rather than merely being a replication of prior work.

The second commentor remarks that the “MIHOPE study has the potential to deepen our understanding of the impact of different home visiting programs for different types of families and communities” and offers a few specific suggestions for how to improve some of the questions in the survey. We have taken these under advisement, and changed the survey accordingly. In particular, we have added a question to learn about child care arrangements, which could be used to determine whether the child is enrolled in Head Start, and we have made sure that all questions are appropriate if the caregiver is the father or is in a two-parent family.

A9. Justification for Respondent Payments

Tokens of appreciation are important, especially in a longitudinal study, to gain respondents’ cooperation and ensure a high response rate and their participation throughout the study, both at the baseline and follow-up interviews (James, 2001; Mack, Huggins, Keathley, & Sundukchi, 1998; Martin, Abreu, & Winters, 2001).

As discussed in part B of this supporting statement, we propose testing the effects of an “early bird” incentive payment and a pre-paid incentive in the first round of data collection. Under these incentive approaches, families who complete the survey will receive either $15 or $25 in gift cards, depending on how soon they respond. We believe both of these amounts are sufficient to encourage families to continue their participation in the follow-up study but are not overly generous. Other studies that have provided similar incentives for commensurate levels of effort include two studies that conducted incentive experiments: the YouthBuild study, which offered $40 or $25 for a 40 minute interview, and the National Survey of Recent College Graduates, which offered $30 or $20 for a 25-minute interview (Stein and Schwartz, 2013; Mooney, et al., 2008). Offering lower amounts than the incentives proposed here could jeopardize the study and actually cost the government more because it could result in a lower retention of families into the study and more effort expended by the evaluation team to obtain information from families and to ensure that control group families are as likely to provide information as program group families. As part of our analysis of the experiment, we will examine the effect of incentive amount on nonresponse bias. Additional details about our plan to analyze nonresponse bias are included in Supporting Statement B.

A10. Privacy provided to respondents

The study team is committed to protecting the privacy of participants and maintaining the privacy of the data that are entrusted to us and is experienced in implementing stringent security procedures. Every MDRC and Mathematica employee, including field staff employed for data collection, is required to sign a pledge to assure participants of nondisclosure of private information. Field staff will also be trained in maintaining respondent privacy and data security.

When participants are contacted about their continued presence in the study, they will be reminded of the study goals, time required, and the nature of questions that will be asked. Parents will be assured that their responses will be shared only with researchers, will be reported only in the aggregate as part of statistical analyses, and will not affect their receipt of services. They will also be told that all data collection activities are voluntary, and they can refuse to answer any and all questions without penalty.

Due to the sensitive nature of this research (for example, questions about parental harshness, and depression) MIHOPE has obtained a Certificate of Confidentiality from HRSA, which we will extend to cover MIHOPE Check-in. The Certificate of Confidentiality helps to assure sites and participating mothers that their information will be kept private to the fullest extent permitted by law.

Documents shipped from the field and the document transmittal form that accompanies them will contain only identification numbers so that data cannot be attributed to any particular individual. Security will be maintained on the complete set (and any deliverable backups) of all master survey files and documentation, including sample information, tracking information, baseline, and follow-up data. Personally identifiable information will be removed from study files, which will contain a linking identification number that can be used to match records from one data file to another. Finally, data will be available only to staff associated with the project through password protection and encryption keys.

A11. Justification for sensitive questions

Questions in some components of the MIHOPE Check-in survey are potentially sensitive for respondents. Parents are asked about personal topics, such as child and parental health, maternal depression, and income. To improve understanding of how home visiting programs affect families and children, it will be necessary to ask these types of sensitive questions. For example, maternal depression is a major risk factor for reduced family well-being and child development. As noted in section A4, this information will not be available from other data sources. In addition, respondents will have been asked similar information at baseline and at the 15-month follow-up, so they are familiar with the types of questions that will be asked.

To reduce respondents’ potential discomfort about potentially sensitive questions, the follow-up survey for MIHOPE Check-in will remind participants that they may refuse to answer any question. Also, respondents will be informed by research staff prior to the start of the interviews or surveys that their answers will be kept private, that results will only be reported in the aggregate, and that their responses will not affect any services or benefits they or their family members receive.

A12. Estimate of the time burden of data collection to respondents

For information on previously approved burden under 0970-0402, see Attachment 1 (Phase 1) and Attachment 2 (Phase 2). Data collection from these phases is ongoing at the time of this submission.

Table A.1 shows the annual burden of the activities described in this supporting statement.

The team will try to collect follow-up information from all families in the initial MIHOPE study for each of the three follow up surveys. Participants are being recruited into MIHOPE through September 2015, but we currently project the study will include 4,300 families.

Table A.1

	Total number of Respondents	Annualized Number of Respondents	Number of Responses per Respondent	Average Burden Hours per Response	Total Annual Burden Hours	Average Hourly Wage	Total Annual Cost
	MIHOPE Check-in Data Collection (Under Review)
Child and Family Outcome Survey and Updating Contact Information	4,300	1,433	3	0.50	2,150	$11.48	$24,682
ESTIMATED TOTAL					2,150		$24,682

The requested annual burden for updating contact information and conducting the survey is 2,150 hours. For collecting data from families, an hourly wage of $11.48 was assumed for mothers, which is the median wage for full-time workers 25 years old or older with less than a high school diploma.

A13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

There is no additional burden to respondents or record keepers.

A14. Estimates of costs to federal government

ACF and HRSA are funding these activities. The estimated cost for activities covered in this submission is $2,400,000 over the three year data collection period and an annual cost of $800,000. This includes designing data collection instruments, collecting follow-up family information, and collecting all data on program impact.

A15. Changes in burden

This is a request to collect additional information from respondents participating in the MIHOPE Evaluation (0970-0402).

A16. Tabulation, analysis, and publication plans and schedule

Three publications will result from MIHOPE Check-in. The first publication will describe findings on family outcomes when their child is 2.5 years old. This publication may also provide information on the success of the study in contacting families at this follow-up point. The second and third publications will provide similar information when children are 3.5 years old and when they are 4.5 years old, respectively.

A18. Reasons for not displaying the OMB approval expiration date

All instruments will display the expiration date of OMB approval.

A19. Exceptions to Certification Statement

No exceptions are necessary for this information collection.

REFERENCES

James, T. (2001). Results of the Wave 1 incentive experiment in the 1996 survey of income and program participation. Paper presented at the Proceedings of the Section of Survey Research Methods, Alexandria, VA.

Mack, S., Huggins, V., Keathley, D., & Sundukchi, M. (1998). Do monetary incentives improve response rates in the survey of income and program participation? Paper presented at the Proceedings of the Section on Survey Research Methods, Alexandria, VA.

Martin, E., Abreu, D., & Winters, F. (2001). Money and motive: Effects of incentives on panel attrition in the survey of income and program participation. Journal of Official Statistics, 17, 267-284.

Baxter, G., Mooney, G., Heaviside, S., Herz, D., Jang, D., Sukasih, A., Barrett, K., & Raduzycki, A. (2010). “2008 NSRCG Methodology Report.” Draft Report submitted to National Science Foundation. Princeton, NJ: Mathematica Policy Research.

Stein, J. and Schwartz, L. (2013) Incentive Experiment Findings [Memorandum]. Princeton, NJ: Mathematica Policy Research.



10