MIHOPE_Phase_2_Supporting_Statement_B - revised March 2015_final

Maternal and Infant Home Visiting Program Evaluation (MIHOPE)

OMB Information Collection Request

0970 - 0402

Supporting Statement

Part B

January 2013

Revised June 2013

Revised March 2015

Submitted By:

Office of Planning, Research and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

7^th Floor, West Aerospace Building

370 L’Enfant Promenade, SW

Washington, D.C. 20447

Project Officer:

Nancy Geyelin Margie

Part B. COLLECTION OF INFORMATION USING STATISTICAL METHODS

B1. Sampling

The sampling plan for MIHOPE was described in the supporting statement for Phase 1 data collection activities. As described there, MIHOPE will seek to recruit approximately 5,100 families at baseline– divided between program and control groups – from approximately 85 local program sites in 12 states. Families are generally eligible for the study if the mother is pregnant or the family has a child under six months old, the mother is 15 years or older, and the mother is available to complete the baseline family survey. Local sites are being chosen to meet several criteria: (1) operating programs that have existed for at least two years, (2) evidence of enough demand for home visiting services that they could provide a control group, (3) no evidence of severe implementation problems that would interfere with the program’s ability to participate in the study, and (4) a contribution to the diversity of sites and families for purposes of estimating effects for important subgroups of families. Families are being recruited into the study by Mathematica’s survey research staff, who will visit families to obtain informed consent when home visitors determine whether a family is eligible or soon after that determination has been made.

The OMB supporting statement for Phase 1 data collection also indicated that the sample is adequate to detect policy relevant impacts of home visiting, both overall, for key subgroups, and for each of the four evidence-based models included in the evaluation. For example, for the pooled sample, the study is powered to detect impacts of about .06 standard deviations. In addition, the study is powered to be able to detect differences across subgroups of about .12 standard deviations. Further detail on the universe and sampling plan for MIHOPE, approved in July 2012, can be found in Appendix A.

As of May 8, 2013, 76 percent of families identified as eligible for MIHOPE have consented to be in the study, and 100 percent of families who have provided consent have completed the baseline interview. Materials approved by OMB in Phase 1 assumed that 90 percent of eligible families would consent to be in the study. This does not affect the study’s sample size, which is still expected to include 5,100 families who provide consent. One implication of the lower consent rate is that approximately 1,000 additional eligible families will be needed to obtain 5,100 study participants. We do not expect this to affect the burden to individuals, however, since most eligible families who have not provided consent have been unlocatable (for example, they have not answered the phone), so that the consent process has not used any of their time.

B2. Procedures for collection of information

This section describes the collection of follow up data for MIHOPE families when the focal child is 15 months old. Data collection with families at follow up will include conducting a family survey, a video-recorded parent-child interaction (three bags task plus a clean-up task), a child language assessment (PLS-5), and gathering mother and child’s height and weight informationand possibly saliva. Follow-up data collection will begin in summer 2013 when the focal children in the first families enrolled in the study are likely to be 15-months old. Best practices will be followed for conducting the Phase 2 data collection, including training and certifying staff on data collection procedures and monitoring data collection to ensure that high quality data are collected and high response rates are achieved. Our follow-up data collection methodology builds on the MIHOPE baseline methodology in several respects:

• The same field interviewer who met with a family at baseline will be assigned to the follow-up visit whenever possible to help maintain rapport with families and maximize response rates

• Computer assisted telephone interviewing (CATI) will be used for the follow-up parent interview to provide consistent monitoring of survey data collection, ensure high quality data are being collected, and provide improved data security (since no data need to be transmitted from laptops in the field).

• CATI allows for the development of a complex instrument with multiple pathways for different families and scenarios (for example, families in which the mother is no longer the child’s primary caregiver).

• The family survey will be conducted via telephone to ensure privacy for the parent (because no one else in the home can overhear the conversation).

• Tokens of appreciation will be provided to increase families’ willingness to respond to the follow up survey.

• Field staff will be trained and certified using standardized procedures will ensure high quality data collection.

Conducting the Follow-Up Family Survey. To collect follow-up data from MIHOPE participants, the study team will adopt the method that was successfully used on the FACES study. As soon as families become eligible for the 15-month follow up, telephone interviewers at Mathematica’s Survey Operations Center (SOC) will contact them via telephone and attempt to complete the follow-up family survey. Families will be reminded that they have already consented to the follow-up visit when they agreed to participate in the MIHOPE study at baseline.

Some families may be difficult to initially reach by telephone. For those families, field staff will go to the families’ home and help them initiate a call to the SOC via cellular phone to complete the family survey.

Conducting the Follow-Up In-Home Visit. If the follow-up interview is conducted without the field staff going to the families’ home, telephone interviewers will schedule a visit for field staff to the families’ home to conduct the in-home portion of data collection: video-recorded parent-child interaction, the child language assessment, the HOME observation and to collect the mother weight and child’s height and weight information. This approach— attempting to complete the follow-up surveys by telephone prior to sending field staff to the home—is efficient and cost-effective because it reduces the amount of time interviewers spend in some families’ homes.

Field interviewers will greet the family upon arrival. They will re-introduce the study, answer any questions the family has about continued participation in the study, and provide assurances of privacy. Field staff will also inform the family of the voluntary nature of their participation. As part of the follow up visit, field staff will (1) complete the follow-up family survey, if it has not already been completed by the SOC; (2) conduct a video-recorded parent-child interaction (three bags task); (3) conduct a child language assessment (PLS-5); (4) take the mother and the child’s height and weight information; and (5) possibly collect saliva from the mother and child.

Field staff will be trained to be flexible when approaching families for the follow up data collection. For example, it may be necessary to schedule more than one visit to complete all data collection pieces or it may be necessary to exercise flexibility in the order of the data collection pieces depending on the child’s alertness levels. For example, field staff may arrive at the home while the infant is sleeping and may start with the family survey and HOME observation measures first. Or if field staff arrive at the home and the infant is awake and alert, they may start with the Three-Bags or the PLS-5.

In order to better understand the relationship between incentive amounts and response rates, 1,200 families would be randomly assigned to different levels of incentives. For the survey, families would be randomly assigned to one of three groups: (1) a group that would be offered the current incentive ($25), (2) a group that would be offered a more generous incentive ($35), and (3) a group that would be offered a more generous incentive only if they respond to the survey by calling the Mathematica Survey Operations Center. For in-home data collection, families would be randomly divided so that half would be offered the current incentive ($20) and half would be offered a more generous incentive ($40). Results will be reviewed with OMB in deciding which version of the incentives should be used once the experiment ends.

Saliva Collection. If ACF decides to collect saliva, the field staff person will be collecting saliva from the mother and child. The mother will be able to opt out of providing saliva and can refuse to allow saliva to be collected from the child.

The field staff person will collect a saliva sample. Procedures for collecting saliva are based on discussions with Douglas Granger, Professor of Nursing, Public Health, and Medicine and Director of the Center for Interdisciplinary Bioscience Research at Johns Hopkins University (JHU) and will follow procedures he is designing to be implemented in the National Children’s Study.

Saliva will be collected from the mother by asking her to move her jaw as if she was chewing her favorite food in order to stimulate the production of saliva. She will then be asked to gently force saliva through a collection device into a 2 mL storage vial until 1 ml has been collected.  Collecting saliva from children under the age of six involves placing a foam rod-shaped swab under the children’s the tongue for 60-90 seconds.  The saliva saturated swabs are placed in a storage vial.   All samples will be immediately placed on ice.

To protect the family’s privacy, the vials will be labeled with a barcode. A second copy of the barcode will be attached by the field staff person to a paper form that contains the individual’s study id and other identifiers such as the woman’s name and contact information.

After leaving the house, the field staff person will ship the vials overnight to Mathematica’s SOC, where it will be logged in and kept in a locked laboratory freezer until it is ready for shipment. Samples will be shipped to the laboratory (e.g., at JHU) for analysis on a periodic basis. For example, samples might be shipped once 1,000 have been obtained or once every quarter.

A laboratory has not yet been chosen, but our current plan, depending on costs, is to use the JHU Center for Interdisciplinary Salivary Bioscience Research. That lab analyzes 60,000-90,000 saliva samples each year. If JHU is used, once the samples arrive at the lab in Baltimore, they will be stored in freezers that are located in a secure facility that is FISMA compliant.

All data files, including physiological data, will be marked with a research id. No identifiers will be maintained in these files. Names, contact information, case identification, and Social Security numbers will be excluded from these data files. Study data files will contain a linking identification number that can be used to match records from one data file to another, for example, linking the physiological information to the questionnaire responses.

Logs Maintained by Home Visitors. Data about service delivery will be collected through weekly web-based logs. For sites in which home visitors do not have regular access to the internet, paper versions of the logs will be offered. Home visitors can complete the paper forms and a support person in the site can enter these data using the site’s computers.

B3. Maximizing response rates

Minimizing sample attrition is of utmost importance to any longitudinal study. It is likely that many MIHOPE families will be highly mobile, and therefore there will be the risk of attrition at follow up. The target for the 15 month follow-up data collection for MIHOPE is an overall response rate of 85 percent, but the actual response rate achieved will likely be somewhere between 80 and 85 percent. Several strategies will be adopted to mitigate the risk of attrition at follow up:

• Under Phase 1 of MIHOPE, mail locating cards and welcome baby letters to families in the sample. These are initial attempts to obtain updated address and telephone information and maintain contact with families in preparation for follow up. These documents were reviewed and approved with the Phase 1 information collection request.

• Use the contact lists generated from the baseline data collection and employ Mathematica’s highly effective locating techniques.

• Train field staff in how to gain cooperation and avoid refusals.

• Provide tokens of appreciation at both baseline and follow up to encourage participation.

Tracking Participants. Mathematica’s Sample Management System (SMS) will be used to track sample recruitment, survey response rates and potential sample attrition. Tracking begins with the initial entry of a family into the SMS. Baseline data collection protocol gathers detailed information from families in order to find them at the time of the follow up interview; collecting names, dates of birth, Social Security numbers (if possible), addresses and phone numbers (home and work) for the family and for up to three relatives or friends who will know how to reach the family. As indicated in the informed consent form that participants sign, Social Security numbers are used both for tracking purposes and to link to state, county, and federal administrative data sources.

Between the baseline and follow up surveys, attempts will be made to reach each family by mail up to three times to request updated contact information for tracking purposes. As an added motivator, and as approved by OMB as part of the Phase 1 data collection package, families will receive $5 if they return a mailing with any updated contact information. The SMS will generate reports that list families who are due for their tracking letter and print the letter and address label for mailing. Letters that are returned as undeliverable will be sent to our tracing department for locating and will be re-mailed to the updated address. The SMS will generate reports that list families who are due for their 15-month follow up visit. Families that appear on the list will begin the locating process to verify their telephone and address information. These efforts will include a letter mailed to families reminding them that it is time for their follow up interview and to please call Mathematica’s toll-free number to complete the family survey as soon as possible (Attachment 4). Any letters that come back from the post office as undelivered will be sent to our tracing department for locating and then remailed to the updated address. Families who receive the letter but do not call in for the family survey will be contacted by telephone. The tracing department will attempt to contact the relatives and friends given at baseline for any families who we cannot reach by telephone in order to obtain the family’s current address and phone number.

Locating Participants. Although the outlined strategies to track participants between baseline and follow-up will likely result in lower attrition rates, additional techniques will be employed to ensure a high response rate is achieved at follow up from this mobile population. Mathematica has extensive experience conducting studies with mobile and hard-to-reach populations and has developed several techniques to locate these populations. Locating can be costly, depending on which methods are used. In general, mailing letters and receiving updated information via returned mail is less expensive than electronic database searches; electronic database searches are less expensive than locators calling neighbors or other contacts; and telephone tracing is less expensive than in-person field locating. The least expensive methods (mailing and electronic tracing) will be used before moving to more expensive methods (telephone tracing and in-person locating). As preparations to conduct the follow-up data collection get underway, the following process for locating participants will be employed: (1) pre-field mailing and electronic locating, (2) in-house electronic database searches and telephone tracing, and as needed, (3) field locating.

1. Pre-Field Locating. Letters (Attachment 4) will be mailed to all families who are due for their 15-month follow-up visit, to invite them to call Mathematica via a toll-free number to complete the follow-up family survey and schedule a visit for the in-person data collection. Any letters that are returned with updated information will be re-mailed to the new address.

National locating databases, such as Accurint and the National Change of Address Service (NCOA) are cost-effective methods for obtaining up-to-date contact information for sample members, and procedures have been developed that ensure the privacy of the data used to locate individuals. Each month, a locating file will be sent containing contact information from the baseline data or most recent update (including last known address, and date of birth) to Accurint or NCOA. These vendors will process the file through three steps. First, all addresses will be updated to the most recent address on record at the U.S. Post Office. Second, the file will be processed using ZIP+4, which cleans the address to match U.S. Postal Service formats and appends the four-digit ZIP extension. Third, the file will be matched with a telephone number database that adds the most recent telephone number to the file.

2. In-House Locating. Custom database searches and telephone contacts given at baseline by the family will be used when pre-field locating does not yield a valid telephone number or address for families. Mathematica’s specialized locating staff uses searchable databases, directory assistance services, reverse directories, and contacts with neighbors and community organizations to obtain current contact information.

3. Field locating. The remaining un-locatable families will be assigned to field staff that will employ proven techniques for finding hard-to-find populations. They may approach neighbors residing in close proximity to the families’ last known address or the contact persons given at baseline, and rely on neighborhood resources such as local post offices, churches, bars, homeless shelters, or community centers as sources of information. In particular, those doing in-person locating will be trained not to reveal any private information about the participant to any informants, including the study’s name or unique details about the study. All field staff will be equipped with cellular telephones so that families, once found, can conduct the follow up interview and complete the in-home activities on the spot.

Logs maintained by home visitors. Strategies for maximizing response rates are similar to those described above for the surveys of families at participating home visiting program sites. When the site enters the study, the research team will explain to program staff the importance of the logs for advancing the field of home visiting in general and the MIECHV program in particular. Research staff will closely monitor weekly log completion reports. They will send program staff two weekly messages (Attachment 5). The first message will remind staff to complete their logs. The second message will document the data that were entered in the previous log by that staff person, thank the staff member for the data provided, and remind those who have not yet completed the previous week’s log to do so.

Non-response bias analysis. Although all efforts will be made to obtain information on a high proportion of families, a non-response analysis will be conducted to determine whether the results may be biased by non-response. In particular, two types of bias will be assessed: (1) whether estimated effects among survey respondents apply to the full study sample, and (2) whether program group respondents are similar to control group respondents. The former type of bias affects whether results from the study can be generalized to the wider group of families involved in the study, while the second assesses whether the impacts of the programs are being confounded with pre-existing differences between program group and control group respondents.

To assess non-response bias, several tests will be conducted.

• The proportion of program group and control group respondents will be compared to make sure the response rate is not significantly higher for one research group.

• A logistic regression will be conducted among respondents. The “left hand side” variable will be their assignment (program group or control group) while the explanatory variables will include a range of baseline characteristics. An omnibus test such as a log-likelihood test will be used to test the hypothesis that the set of baseline characteristics are not significantly related to whether a respondent is in the program group. Not rejecting this null hypothesis will provide evidence that program group and control group respondents are similar.

• Impacts from administrative records sources – which are available for the full sample – will be compared for the full sample and for respondents to determine whether there are substantial differences between the two.

• Baseline characteristics of respondents will be compared to baseline characteristics of non-respondents. This will be done using a logistic regression where the outcome variable is whether someone is a respondent and the explanatory variables are baseline characteristics. An omnibus test such as a log-likelihood test will be used to test the hypothesis that the set of baseline characteristics are not significantly related to whether a respondent is in the program group. Not rejecting this null hypothesis will provide evidence that non-respondents and respondents are similar.

If any of these tests indicate that non-response is providing biased impact estimates, a standard techniq ue such as multiple imputation or weighting by the inverse probability of response will be used to determine the sensitivity of impact estimates to non-response.

B4. Pre-testing

As part of Phase 2, the study team will use pretesting to identify revisions to be made to materials, procedures, and instruments for follow-up data collection. We will reach out to home visiting programs in either the Washington, DC or New Jersey areas to identify 9 or fewer families (including both English and Spanish-speaking participants) with a child aged 15 months and recruit them to pretest the CATI follow-up survey and all in-home instruments and procedures. The study team will attempt to recruit participants that represent the diversity of the MIHOPE sample (including linguistic, ethnic, racial, and geographic diversity).

When pretesting the follow-up survey, the interviewer will begin by introducing the study, assuring privacy and reiterating that participation in the survey is voluntary. The interviewer will also ask for permission to audio-record the interview. The interviewer will ask the questions exactly as worded and will follow up with specific probes if any questions seem confusing. The pretest will include cognitive testing and debriefing with the parent and interviewer after the interview is completed. Cognitive interviews will investigate parents’ understanding of questions, and ease or difficulty of responding. As part of cognitive interviewing, an interviewer will administer the questions while a second team member listens to the interview and codes for parent hesitation and any indication of lack of understanding. After the interview, we will discuss with parents any questions or confusion they may have had. Debriefings with project staff and interviewers will investigate the ease of administration of the survey instrument. The telephone interviews and subsequent cognitive interviewing with respondents will be recorded so that survey management staff can hear the recordings and obtain accurate estimates of the length of the interview for OMB burden estimates.

Pretesting will also be conducted for the procedures and data collection instruments collected in the home. The purpose of the pretest will be to test the in-home protocols under realistic conditions. During the visit, staff will first introduce the study and inform families that participation in the pretest is voluntary and that the data collected will be kept private. Staff will then conduct the various in-home instruments taking into account the baby’s alertness or sleep state to dictate the order of items to be done. Each visit will end with a short debriefing to solicit feedback about the parent’s experience. The debriefings will assess the ease of administering the instruments, the handling of the equipment (video camera, weighing scale and cell phone), and identify any trouble spots and solutions for overcoming them.

Results of the pretests will be sent to OMB, and any resulting modifications to instruments will be submitted as nonsubstantive changes for OMB approval.

B5. Consultants on statistical aspects of the design

There are no consultants on the statistical aspects of Phase 2. We have drawn on the expertise of MIHOPE team members including Charles Michalopoulos and Howard Bloom of MDRC.

REFERENCES

Booth, A., Goslin, M., Johnson, D., & Granger, D. A. (under review). Cortisol concordance within families: Relationship quality, individual adjustment, and contextual strain moderators.

Brady-Smith, C., O'Brien, C., Berlin, L., Ware, A., Fauth, J., & Brooks-Gunn, J. (2000). Child-parent interaction rating scales for the Three-Bag assessment: 36-month wave. Teachers College, Columbia University. New York.

Bugental, D. B. S., Alex; Lynch, Colleen. (2010). Effects of an Early Family Intervention on Children's Memory: The Mediating Effects of Cortisol Levels. Mind, Brain, and Education, 4(4), 159-170.

Dietz, P. M., Homa, D., England, L. J., Burley, K., Tong, V. T., Dube, S. R., et al. (2011). Estimates of Nondisclosure of Cigarette Smoking Among Pregnant and Nonpregnant Women of Reproductive Age in the United States. American Journal of Epidemiology, 173(3), 355-359.

Dozier, M., Peloso, E., Lewis, E., Laurenceau, J.-P., & Levine, S. (2008). Effects of an attachment-based intervention on the cortisol production of infants and toddlers in foster care. Development and Psychopathology, 20(3), 845-859. doi: 10.1017/S0954579408000400

Dozier, M., Peloso, E., Lindhiem, O., Gordon, K., Manni, M., & Sepulveda, S. e. a. (2006). Developing evidence-based interventions for foster children: An example of a randomized clinical trial with infants and toddlers. Journal of Social Issues, 62, 767-785.

Harding, K., Galano, J., Martin, J., Huntington, L., & Schellenbach, C. J. (2007). Healthy Families America Effectiveness: A Comprehensive Review of Outcomes. Journal of Prevention & Intervention in the Community, 34(1/2), 149-179.

James, T. (2001). Results of the Wave 1 incentive experiment in the 1996 survey of income and program participation. Paper presented at the Proceedings of the Section of Survey Research Methods, Alexandria, VA.

Kendrick, J. S., Zahniser, S. C., Miller, N., Salas, N., Stine, J., Gargiullo, P. M., et al. (1995). Integrating smoking cessation into routine public prenatal care: the Smoking Cessation in Pregnancy project. American Journal of Public Health, 85(2), 217-222.

Mack, S., Huggins, V., Keathley, D., & Sundukchi, M. (1998). Do monetary incentives improve response rates in the survey of income and program participation? Paper presented at the Proceedings of the Section on Survey Research Methods, Alexandria, VA.

Martin, E., Abreu, D., & Winters, F. (2001). Money and motive: Effects of incentives on panel attrition in the survey of income and program participation. Journal of Official Statistics, 17, 267-284.

Olds, D. L., Robinson, J., O'Brien, R., Luckey, D. W., Pettitt, L. M., Henderson, C. R., et al. (2002). Home Visiting by Paraprofessionals and by Nurses: A Randomized, Controlled Trial. Pediatrics, 110(3), 486-496. doi: 10.1542/peds.110.3.486

Papp, L. M., Pendry, P., & Adam, E. K. (2009). Mother-adolescent physiological synchrony in naturalistic settings: Within-family cortisol associations and moderators. Journal of Family Psychology, 23(6), 882.

Patrick, D. L., Cheadle, A., Thompson, D. C., Diehr, P., Koepsell, T., & Kinne, S. (1994). The Validity of Self-Reported Smoking: A Review and Meta-Analysis. American Journal of Public Health, 84(7), 1086.

Russell, T., Crawford, M., & Woodby, L. (2004). Measurements for Active Cigarette Smoke Exposure in Prevalence and Cessation Studies: Why Simply Asking Pregnant Women Isn't Enough. Nicotine Tob Research, 5(Suppl 2), S141-151.

Ruttle, P. L., Serbin, L. A., Stack, D. M., Schwartzman, A. E., & Shirtcliff, E. A. (2011). Adrenocortical attunement in mother-child dyads: Importance of situational and behavioral characteristics. Biological Psychology, 88(1), 104-111.

Sethre-Hofstad, L., Stansbury, K., & Rice, M. A. (2002). Attunement of maternal and child adrenocortical response to child challenge. Psychoneuroendocrinology, 27, 731-747.

Studts, J. L., Ghate, S. R., Gill, J. L., Studts, C. R., Barnes, C. N., LaJoie, A. S., et al. (2006). Validity of self-reported smoking status among participants in a lung cancer screening trial. Cancer Epidemiology Biomarkers Prevention, 15(10), 1825-1828.

van Bakel, H. J. A., & Riksen-Walraven, J. M. (2008). Adrenocortical and Behavioral Attunement in Parents with 1-Year-Old Infants. Developmental Psychobiology, 50, 196-201.



9