BSF - OMB Package Part A - 3-19-08

Contract No.: 233-02-0056

Building Strong Families:

OMB Supporting Statement for the Second Follow-Up Data Collection – Part A (Justification)

March 19, 2008

Submitted by U.S. Department of Health and Human Services Administration for Children and Families Office of Planning, Research, and Evaluation 7th floor, West, Aerospace Center Building 370 L’Enfant Promenade, SW Washington, DC 20447 Project Officers: Nancye Campbell Seth Chamberlain

CONTENTS

Page

JUSTIFICATION 1

1. Circumstances Necessitating the Data Collection 1

2. How, by Whom, and for What Purpose Information Will Be Used 16

3. Use of Automated Electronic, Mechanical, and Other Technological Collection Techniques 17

4. Avoiding Duplication of Effort 18

5. Sensitivity to Burden of Small Entities 18

6. Consequences to Federal Program or Policy Activities if the Collection Is Not Conducted or Is Conducted Less Frequently than Proposed 18

7. Special Circumstances 20

8. Federal Register Announcement and Consultation 20

9. Payments and Gifts to Respondents 21

10. Confidentiality of the Data 23

11. Additional Justification for Sensitive Questions 25

12. Estimates of the Hour Burden of the Collection of Information 28

13. Estimate of Total Annual Cost Burden to Respondents or Record Keepers 30

14. Estimate of Annualized Costs to the Federal Government 30

15. Reasons for Program Changes or Adjustments 30

16. Plans for Tabulations and Publication and Schedule of Project 31

17. Reasons for Not Displaying Expiration Date for OMB Approval 35

18. Exception to the Certification Statement 35

References 36

TABLES

Tables Page

1 KEY FEATURES OF BSF SITES 5

2 Outcomes Measured at THE SECOND FOLLOW-UP 10

3 Outcomes MeasureD DURING the In-Home Direct Assessment 13

4 TECHNICAL WORK GROUP, CONSULTANTS, AND OTHER REVIEWERS 22

5 JUSTIFICATION FOR SENSITIVE QUESTIONS 25

6 ANNUAL BURDEN ESTIMATES FOR SECOND FOLLOW-UP

DATA COLLECTION 28

7 ANNUALIZED COSTS TO THE FEDERAL GOVERNMENT 30

FIGURES

Figures Page

1 BSF CONCEPTUAL FRAMEWORK 8

JUSTIFICATION

The Administration for Children and Families (ACF) of the U.S. Department of Health and Human Services (DHHS) is undertaking the Building Strong Families (BSF) project. We are requesting clearance for two data collection efforts. First, we are requesting clearance for a follow-up telephone survey of mothers and fathers at the time their child is 36 months old (see Appendix A, with supporting documentation in Appendix B). This is the second follow-up telephone survey of the BSF research sample. The longitudinal nature of the study requires that the second follow-up survey address the same key outcome areas covered in the first follow-up survey. Consequently, many of the questions are the same as or very similar to those on the survey administered at 15 months after random assignment (OMB clearance number 0970-0304). This longer-term follow-up will increase our knowledge of the effects of BSF over time. Second, we are requesting clearance for an in-home direct assessment of child outcomes and parent-child interactions conducted at about the same time as the second follow-up survey. Appendix C provides information on the protocols and activities to be conducted during the in-home assessment. This data collection effort will allow us to apply observational and direct assessment methods to further investigate the impacts of BSF on parent-child interactions and child well-being.

1. Circumstances Necessitating the Data Collection

The goal of the BSF project is to learn whether well-designed interventions can help interested and romantically involved unwed parents build stronger relationships and fulfill their aspirations for a healthy marriage if they choose to wed. The BSF programs target unwed parents before or around the time of their child’s birth and provide instruction and support to help couples develop the relationship skills that research has shown are associated with healthy marriages. Ultimately, healthy marriage between biological parents is expected to enhance child well-being. ACF has contracted with Mathematica Policy Research, Inc. (MPR) and its subcontractors¹ to support the development of these interventions and to determine their effectiveness.

a. Background on the Building Strong Families Project

The BSF project originated from three bodies of research. The first body of research relates to the influence of family structure on child outcomes. This research shows that, on average, children who grow up with two married biological parents do better than those growing up in single parent households on a wide range of outcomes, including academic and behavioral outcomes, the likelihood of growing up in poverty, and the likelihood of the children themselves becoming single parents (McLanahan and Sandefur 1994).

The second body of research comes from the landmark Fragile Families and Child Well-Being Study (http://crcw.princeton.edu/fragilefamilies) and relates to marriage among couples expecting a baby. The Fragile Families study follows approximately 3,700 unmarried couples who were recruited into the study shortly after the birth of their child between 1998 and 2000 in 20 large cities throughout the United States. On the positive side, results from this study show that most unwed parents are romantically involved around the time their child is born, and anticipate marrying each other. In addition, most unwed parents agree that it is better for children if their parents are married. Nevertheless, the study shows that a year later only 12 percent of couples who were romantically involved at the time of their child’s birth are married (McLanahan et al. 2001; Carlson 2002). The positive findings about the couples’ relationships and their aspirations for marriage suggest that there may be an opportunity for intervention around the time of the child’s birth.

The third body of research is on the effectiveness of programs that provide relationship education to married and engaged couples. Programs to prepare couples for marriage, strengthen the relationships of married couples, or prepare couples for the stresses on their relationships when they become parents have been the subject of rigorous evaluation. Multiple studies have shown these programs to be effective in improving couples’ marriages and reducing divorce rates (Markman et al. 1988; Markman et al. 1993; and Cowan and Cowan 2000).

There remain substantial gaps, however, in our understanding of how to strengthen the relationships of unwed parents and how to support those who choose to marry. Research on marriage and relationship skills programs has generally focused on applications for married or engaged couples. Further, the early marriage education field and research literature has focused on middle-class couples, rather than on the low-income unmarried couples who constitute the BSF target population. Moreover, much of the research undertaken in these programs has been constrained by small sample sizes, which limits the ability to detect statistically significantly impacts, if present.

b. Overview of the Building Strong Families Program

To address this knowledge gap, the BSF project implements and evaluates interventions with unwed parents, starting around the time of their child’s birth. One of the first tasks of the BSF project was to develop a program model, building on previous research. After developing a conceptual framework for determining whether and how to intervene with unmarried and romantically involved parents having a baby, we developed detailed guidelines for BSF programs.

As described in the program guidelines, BSF programs have three components:

1. Healthy Marriage and Relationship Skills Education. The core, distinctive component of BSF programs is the provision of information to enhance couples’ understanding of marriage and instruction in relationship skills identified in the research as essential to a healthy marriage. This instruction is provided in group sessions with BSF couples (usually weekly).

2. Family Support Services. This component includes services to address special issues that may be common among low-income parents and that are known to affect couple relationships and marriage. For example, these services might help improve parenting skills or assist with addressing problems with employment, physical and mental health, or substance abuse.

3. Family Coordinators. These program staff assess couples’ circumstances and needs, make referrals to other services when appropriate, reinforce relationship and marriage skills over time, provide ongoing emotional support, and promote sustained participation in program activities.

The BSF intervention is intensive. The core component of BSF, the group instruction related to relationship skills and healthy marriage, requires up to 44 hours of instruction. It is typically provided over a sustained period of time, as long as five or six months. Program sites differ in the frequency and duration of time that couples meet with the family coordinators, but it may extend beyond the group sessions for as long as three years.

Couples are recruited for BSF either during pregnancy or shortly after the birth of their baby. To be eligible for BSF, a mother and father must be:

• Expectant biological parents or the biological parents of a baby three months of age or younger

• Age 18 or older

• Unmarried (or married since conception of the baby)

• In a romantic relationship

• Not involved in domestic violence that could be aggravated by participation in BSF (the BSF programs, working with local domestic violence experts and with input from MPR and ACF, use program-specific screening approaches).

• Available to participate in BSF and able to speak and understand a language in which BSF is offered (specifically, English or Spanish).

c. The BSF Program Sites

The BSF programs are in: Atlanta, Georgia; Baton Rouge, Louisiana; Baltimore, Maryland; Orange and Broward counties, Florida; Marion, Allen, and Lake counties, Indiana; Oklahoma City, Oklahoma; and San Angelo and Houston, Texas. BSF programs differ in the type of organization hosting/operating the demonstration, the predominant population served, and whether they recruit primarily expectant couples, those with newborns, or both. Key features are presented in Table 1.

TABLE 1

KEY FEATURES OF BSF SITES

Pilot Site	Host Organization	Primary Recruitment Sources	Predominant Race/Ethnicity Served	Timing of Recruitment
Atlanta, Georgia	Georgia State University, Latin American Association	Large public hospital, public health clinics	African American and Hispanic	Primarily prenatal
Baltimore, Maryland	Center for Fathers, Families and Workforce Development	Local hospitals, prenatal clinics	African American	Prenatal and postnatal
Baton Rouge, Louisiana	Family Road of Greater Baton Rouge	Prenatal program for low-income women	African American	Prenatal
Florida: Orange and Broward counties	Healthy Families Florida	Birthing hospitals	African American and Hispanic	Postnatal
Indiana: Allen, Marion, Miami, and Lake counties	Healthy Families Indiana	Hospitals, prenatal clinics, WIC	African American and White	Prenatal and postnatal
Oklahoma City, Oklahoma	Public Strategies, Inc.	Hospitals, health care clinics, direct marketing	White	Prenatal
Texas: San Angelo and Houston	Healthy Families San Angelo and Healthy Families Houston	Hospitals, public health clinics	Hispanic and White	Prenatal and postnatal

d. Objectives and Overview of the BSF Evaluation

The goal of the BSF evaluation is to determine whether programs following the BSF model can help unwed parents develop stronger relationships and healthy marriages and enhance the well-being of their children. To meet this goal, the evaluation will estimate the impacts of BSF on a range of important adult and child outcomes over time.

The BSF impact analysis uses a rigorous experimental design with longitudinal data collection. When a couple is found to be eligible for BSF, the program services are explained to them. If they are interested in participating in the program, each partner completes a consent form and a baseline information form (OMB clearance number 0970-0273, expiration March 31, 2008). After the baseline forms have been completed, couples are randomly assigned to either the program (the BSF intervention) or the control group. Couples assigned to the program group are offered BSF services; couples assigned to the control group do not receive BSF services. The control group is eligible to receive other services available in the community.

The BSF evaluation design includes two follow-up surveys and an in-home direct assessment with members of both program and control groups. The first and second follow-up surveys are similar in content, allowing for assessment of the effects of the intervention on similar outcomes over time. The first follow-up survey is currently underway, occurring 15 months after random assignment (OMB clearance number 0970-0304, expiration July 31, 2009). Depending on when the couple was recruited, the BSF focal child (the child that the couple was expecting or that had just been born when they were recruited) is between 9 and 18 months of age at the time of the first follow-up survey. (The child is about 9 months old if the couple was recruited near the end of the first trimester of pregnancy and 18 months old if the couple was recruited when the child was 3 months old.)

We are now requesting OMB clearance for the second follow-up telephone survey and an in-home direct assessment, both of which are to occur when the child is about three years of age. We are structuring the timing of the second follow-up and in-home direct assessment based on the child’s age rather than on the date the couple was randomly assigned because several key measures being collected are measures of child outcomes. Tying the timing of the data collection to the birth date of the child ensures that these child outcome measures are: (1) appropriate to the age of the children being assessed, and (2) comparable across sample members regardless of whether they entered the study before or after the birth of their children.

The impact analysis addresses the following questions:

• Does BSF change family outcomes? What is the impact of BSF programs on a wide range of family outcomes, including marital and relationship status, couple relationship quality, parenting^, household structure, family self-sufficiency, parent well-being, the parent-child relationship, child social and emotional functioning, and child language development?

• Do BSF programs work better for some families than for others? Identifying the couples and families who benefit most will help programs improve and target services. We will examine whether program impacts vary by factors such as the demographic characteristics of couples (for example, age, race, and ethnicity), relationship quality at baseline, whether parents have children by other partners, and the “marriageability” of the parents (such as whether they are employed).

• What types of BSF programs work best? We will examine whether different program models have different impacts and whether program effectiveness depends on how they are implemented.

• How do the BSF programs work? If we find impacts of BSF on family outcomes, it will be important to identify the pathways by which BSF affects outcomes in estimating and interpreting the findings. For example, can changes in child outcomes be attributed to increases in marriage, improved relationship skills, better co-parenting, or other intermediate outcomes affected by BSF?

The outcomes that may be affected by BSF and the way they are expected to affect the couples and their children are illustrated in the conceptual framework for BSF (see Figure 1).

Outcomes of interest fall into four main categories:

• Services Received. While not common, other marriage and relationship skills education programs are available, as is marriage and relationship counseling. If BSF is effective, we would expect the members of the program group to receive more marriage and relationship education than members of the control group. Relative to control group members, we also expect BSF program group members to receive more home visits (in sites where BSF involves home visits) and other support services, such as employment and education.

• Parents’ Relationship. The status and quality of the BSF parents’ relationships are key outcomes for evaluation. Relationship status includes whether the parents marry, remain in a romantic relationship, and cohabit. The quality of the relationship includes happiness with the relationship, conflict management, friendship, supportiveness and intimacy, commitment and trust, fidelity, and domestic violence. It also includes how well the parents work together to “co-parent” their children. Some BSF couples will have split up by 36 months after entering BSF and may have formed new relationships. We will ask about the status and the quality of those relationships as well.

• Family. BSF may affect many aspects of the family. In particular, it may affect parenting behaviors (for example, how parents interact with one another while parenting, known as co-parenting), father involvement (such as the quantity and quality of time spent with the child), the parent-child relationship (for example, whether the child is securely attached to the parent), the structure of the family (such as whether the child lives with his or her mother, father, or both), the self-sufficiency of the family (such as employment and income), and parent well-being (such as mental health and substance use).

• Child Well-Being. Important child outcomes include the economic resources available to the child, the child’s physical health, his or her socio-emotional development and language development.

The individual-level outcomes to be measured in the second follow-up survey are provided in Table 2. Most of the outcomes to be measured in the second follow-up telephone survey were also measured during the first follow-up telephone survey. Including these items in the second follow-up survey will enable us to determine whether impacts present at the time of the first follow-up are sustained and whether new impacts emerge over time. Items collected in the first follow-up that would not elicit new information in a second follow-up (for example, “What country were you born in?”) will only be included if the sample member did not complete the item in the first follow-up survey.

New additions to this wave of data collection are marked in Table 2 with an asterisk. These new items include additional measures of parenting and measures of the parent-child relationship and child well-being that could be expected to be influenced by the BSF intervention. The new parenting measures include household routines and, in the fathers’ survey, the frequency of physical play activities conducted with the focal child. The second follow-up survey also includes items to elicit information about the warmth present in the parent-child relationship and items related to the child’s health (which are often correlated with family structure) —specifically, the general health of the child, the presence and severity of asthma and whether the child is covered by health insurance. In addition, the second follow-up survey includes items to measure the child’s internalizing and externalizing behavior problems, social competence, and emotional security amid parental conflict. The internalizing and externalizing behavior problems are measured by the Behavior Problems Index (Peterson and Zill 1986), the child’s social competence with items from the Social Interaction Subscale of the Preschool and Kindergarten

Table 2

Outcomes Measured at THE SECOND FOLLOW-UP

Services Received

Marriage and Relationship Skills Education

Whether attended groups, workshops, or classes

Parents’ Relationship

Marital/Relationship Status

Marital status of BSF parents at follow-up (married, separated, divorced, never married)

Whether still romantically involved

Whether cohabiting

Frequency of contact

If applicable, when the relationship ended and the reason relationship ended

If not married, whether engaged and have plans to marry

Chances of marrying the BSF partner in the future

Attitudes toward marriage

BSF Parent’s Relationship Quality

Overall happiness with relationship

Conflict management

Interactions, communication, and time spent together

Emotional and sexual intimacy and supportiveness

Commitment and trust

Fidelity

Domestic violence from BSF partner and other partners

Coparenting

Communication and problem solving between parents

Trust in other parent’s parenting skills and judgment

Work as a team for the child

Trust in commitment of other parent to the child

Satisfaction with responsibility (including financial) taken by other parent

Recognition of the importance of the other parent in the child’s life

Relationship With New Partner

Number of sexual relationships since random assignment

Whether currently in a new romantic relationship

Whether married to new partner

Whether cohabiting with new partner

Number and length of marriages since baseline

Overall quality of relationship with new partner

Family

Parenting/Family Involvement

Quantity and Quality of Time Spent with Child

Whether father has had contact with child in past year

Amount of time BSF parent spends with child

Frequency of activities conducted with child (e.g., play games, change diapers)

Frequency of physical play with child (e.g., chasing games, playing with a ball) *

Warmth present in the parent/child relationship*

Family

Material Support

Whether paternity has been established (if not established at time of the first follow-up survey)

Whether establishment was voluntary

Child support (whether legal order, amount of order, amount paid)

Informal child support (amount of cash and in-kind)

Contribution of each parent to cost of raising child

Household Routines

Whether child has meals with parent(s)*

Whether child follows a regular bedtime routine*

Stress in the Parenting Role

Whether respondent feels stress in his/her role as a parent

Discipline

Type and severity of discipline either BSF parent employs with child*

Type and severity of discipline new partner employs with child *

Family Structure

BSF Child’s Living Arrangements

Whether the child lives with mother, father, both parents, or someone else

Number of months child lived with each BSF parent since baseline

Number of months child lived with both BSF parents together since baseline

Fertility Decisions

Number of children born or conceived since BSF focal child

Number of children born or conceived with BSF parent

Household Structure

Number of children who live with BSF parent

Number of children who live with, and are the responsibility of, the BSF parent

Number of adults in the household

Family Self-Sufficiency

Employment and Earnings

Whether currently working

Number of months worked in the past year

Hours worked per week in past month

Earnings in past month/last month worked

Public Assistance

Amount of TANF received in previous month

Amount of food stamps received in previous month

Amount of SSI or SSDI received in previous month

Amount of Unemployment Insurance received in previous month

Family Income

Own earnings

Earnings from spouse or cohabiting partner

Amount of child support received

Amount of money received from friends and relatives

Extent to which earnings from spouse/cohabiting partner are available to child

Total family income

Family

Material Hardship

Whether during the past year was unable to pay rent, mortgage, or utility bills

Whether during the past year was evicted from residence

Whether during the past year had their electricity or water service cut off

Asset Accumulation

Whether respondent owns a car, truck, or van

Whether respondent owns his/her home

Child Wellbeing

BSF Child’s Social-Emotional Development

Child’s social competence (empathy)*

Presence of externalizing behaviors*

Presence of internalizing behaviors*

Emotional security amid parental conflict*

BSF Child’s Health

Child’s general health status*

Whether child has been sick in past month*

Asthma diagnosis and severity*

* Indicates outcomes that were not measured in the first follow-up survey

Behavior Scales (Merrell 2002), and the child’s emotional security with items from the Security in the Marital Subsystem-Parent Report Inventory (Davies et al. 2002).

During the in-home direct assessment, information will be collected on the child’s language development, self-regulation, and behavior. In addition, direct observations of mother-child and father-child interactions will occur. Observers will also record information about parenting behaviors and the physical home environment following the in-home visit. Table 3 lists the outcomes collected during the in-home direct assessment and observations and the measures of those outcomes.

e. Data Collection Activities Requiring OMB Clearance

Clearance is currently being requested for the evaluation’s second follow-up telephone survey and in-home direct assessments. As in the first telephone survey, to ensure privacy the father and the mother in the couple will be interviewed separately for the second telephone

TABLE 3

OUTCOMES MEASURED DURING THE IN-HOME DIRECT ASSESSMENT

Outcome Domain	Measures	Mode
CHILD DIRECT ASSESSMENT
Language Development
Receptive language	Peabody Picture Vocabulary Test 4 and Test de Vocabulario en Imágenes Peabody (PPVT-4, Dunn and Dunn 2006; TVIP, Dunn et al. 1986)	Direct assessment of child
Socio-Emotional Development
Self-regulation	Walk a Line Slowly (Murray and Kochanska 2002)	Direct assessment of child
QUALITY OF PARENT-CHILD INTERACTIONS
Mother-child interaction	Three-Bags Task (Love et al., 2002;Vandell 1979)	Videotape of interaction between parent and child
Father-child interaction	Three-Bags Task (Love et al., 2002; Vandell 1979)	Videotape of interaction between parent and child
QUALITY OF HOME ENVIRONMENT
Attachment	The Toddler Attachment Q-Sort 45 (TAS-45; Kirkland et al. 2004)	Observer completes following the home assessment
Home environment	HOME warmth, internal and external environment, and language and literacy subscales (Caldwell and Bradley 2003; Leventhal et al 2004)	Observer completes following the home assessment

survey. Interviewers will conduct interviews from MPR’s centralized telephone interviewing facility. In addition, field locating and interviewing using cellular phones will be used with sample members who initially cannot be contacted or successfully interviewed. For those located in the field, members of the couple will also be interviewed separately by telephone to ensure privacy. Based on our experience with the first follow-up, we estimate that 45 percent of cases will require field locating. Bilingual interviewers will complete interviews in Spanish when appropriate. In most cases, the telephone survey with both mother and father will be completed prior to the conduct of the in-home assessment.

The in-home assessment will typically take place at the child’s home. Assessments will be scheduled at a time that is convenient to the parent and appropriate for the child (taking into account the child’s eating and sleeping schedules). As illustrated in Table 3, during the in-home assessment, the child’s language skills and self-regulation will be assessed directly and interaction tasks with the parents will be videotaped for later coding by trained observers. Assessment tasks will be completed in a sequence that poses the least amount of burden on participants. Bilingual field staff will complete assessments in Spanish when necessary.

When possible, videotaping of the mother-child interaction task and the father-child interaction task will be scheduled for the same visit. If necessary or preferred by the parent, however, the interaction tasks can be completed at different times.

The protocols for each of the in-home assessment activities are included in Appendix C. The following assessments and tasks are proposed to be conducted with children and parent-child pairs:

• The Peabody Picture Vocabulary Test 4 (or the Spanish version) will be administered to obtain a measure of the child’s receptive language. An assessor will present a series of words, ranging from easy to difficult for children of about three years of age, each accompanied by a picture consisting of four drawings. The child is asked to indicate which drawing best represents the word. This test measures language development (Child measure).

• The Walk a Line Slowly Task is a motor inhibition task that provides information on the child’s self-regulation and executive control capabilities. The data collector asks the child to walk down a line taped to the floor. After the initial trial, there are two additional trials in which the child is asked to walk as slowly as he or she can. This task provides a measure of socio-emotional development (Child measure)

• The Three Bags Task, a semi-structured free play task for the child and one parent, is used to measure aspects of parent-child interaction. It involves the parent and child sitting on a mat and being presented with three bags of toys. They are asked to spend 15 minutes playing with the toys in the three bags, behaving in a way that feels natural to them. This activity will be conducted only with parents who are in regular contact with the child. The activity will be administered by a trained data collector and videotaped for later coding by trained coders. This task measures parental sensitivity, positive regard, and intrusiveness (Parent-Child Measure).

OUTLINE OF BUILDING STRONG FAMILIES IN-HOME ASSESSMENT ACTIVITIES

1. Arrival at the parent’s home.

2. Introductory comments will include a discussion of the following: purpose of the visit, outline of the activities for the visit, explanation that participation is voluntary, assurance of confidentiality, description of incentives, and an opportunity to answer any questions the participants might have.

3. Data collector will introduce self to child and ask warm-up questions (for example, asking the child’s name).

4. Data collector will administer the PPVT-4 or TVIP (Dunn and Dunn 2006; Dunn et al 1986) to the child.

5. Data collector will administer the Walk a Line Slowly Task (Murray and Kochanska 2002).

6. Data collector will set up the Three Bags Task for the child and primary caregiver and videotape the interaction.

7. If applicable, data collector will set up the Three Bags Task (Vandell 2979; Love et al. 2002) for the child and second parent and videotape the interaction.

8. Data collector will thank the participants and distribute incentives.

9. After leaving the parent’s home, the data collector will complete the observational items for the HOME (Caldwell and Bradley 2003; Leventhal et al. 2004), and the Toddler Attachment Q-Sort 45 (Kirkland et al. 2004).

The consent form completed by each parent during intake for the study included consent for the in-home assessment. When scheduling the in-home direct assessment, each parent will be informed of all planned activities for the data collection and any unusual circumstances will be taken into consideration. The parent-child interactions will not be conducted if the parent is not in regular contact with the child. If a parent raises objections about an in-home assessment being conducted with the other parent, these concerns will be taken into consideration in order to ensure that the child’s safety is not put at risk.

2. How, by Whom, and for What Purpose Information Will Be Used

The information obtained through the BSF evaluation and this specific information collection is critical to assess whether well-designed interventions can help interested and romantically involved unwed parents build stronger relationships and fulfill their aspirations for a healthy marriage if they choose to wed. Ultimately, healthy marriage between biological parents is expected to improve outcomes for children as well as for the adults. It is important to determine whether any shorter-term impacts observed via the 15-month survey are sustained and whether any new impacts emerge. The findings from the impact analysis will provide information on whether, for whom, and under what circumstances BSF has benefits. This information will be valuable for policymakers and program funders as they consider further investments in the area of healthy marriage and support for two-parent families as well as refinements to the Healthy Marriage and Responsible Fatherhood provisions of the Deficit Reduction Act of 2005 or new federal initiatives. The information about the effects of the BSF intervention will also be of interest and use to program administrators providing services to improve outcomes for children born to unwed parents and for parents themselves. The findings will also be of interest and of use to unwed parents who seek to build strong, stable relationships and marriages.

3. Use of Automated Electronic, Mechanical, and Other Technological Collection Techniques

As with the first follow-up survey, the data collection for the second follow-up survey will use computer assisted telephone interviewing (CATI). The CATI system reduces respondent burden by automating skip logic and question adaptations that allow interviewers to progress from question to question without having to refer back to previous answers to determine whether a follow-up question should be asked or phrasing should be adjusted to properly apply to a respondent’s circumstances. MPR will preload data from the first follow-up survey and embed appropriate skip logic to further reduce respondent burden. CATI minimizes interviewer error through control over the question logic, consistency checks, and probes, and it eliminates the need to call back respondents to obtain missing data since inconsistencies in responses are corrected during the interview process.

The CATI system facilitates survey tracking because of its capability to produce timely reports on screening and interview outcomes, yield rates, item nonresponse rates, and interviewer productivity. CATI improves interviewer supervision through the use of audio and video monitoring. The autodialer, linked to the CATI system, virtually eliminates dialing error and improves interviewer efficiency. The automated call scheduler manages interviewer assignments by scheduling and rescheduling calls to ensure that they are made according to the optimal calling patterns, that all appointments are kept, and that cases requiring special attention or fluency in other languages are routed to the appropriate interviewers.

During the in-home direct assessment, the parent-child interaction tasks will be videotaped for later coding by trained observers. Use of video allows observers to code the same interaction multiple times (for example, with more than one coding scheme) and allows multiple observers to code the same interaction thus establishing inter-rater reliability without requiring that multiple data collectors visit the family’s home.

4. Avoiding Duplication of Effort

Conducting the second follow-up telephone survey will allow us to track impacts of the BSF intervention over time. The in-home direct assessment will allow us to use direct observation and assessment to measure the impacts of BSF on parent-child interactions and children. There is no similar prior or ongoing data collection being conducted that duplicates the efforts of the proposed data collection with low-income unmarried parents. The survey and in-home assessment will not collect any information that can be obtained through reviews of existing records.

5. Sensitivity to Burden of Small Entities

None of the respondents will be small businesses.

6. Consequences to Federal Program Or Policy Activities if the Collection Is Not Conducted Or Is Conducted Less Frequently Than Proposed

The field of federally supported marriage education is young, as is the extension of such services to low-income populations. Little is known about how to develop and implement effective programs especially targeting lower-income couples. Through the Deficit Reduction Act of 2005, Congress provided $150 million per year to support demonstrations of healthy marriage and responsible fatherhood services. Because the BSF project was begun earlier, the collection of information through this evaluation will provide the first findings about the impacts of programs designed to improve couple relationships and marriages. The lessons and impact findings from this study will provide important, useful and timely information to the federal government and state and local agencies operating healthy marriage demonstrations, especially those targeting unwed parents. Early lessons and impact findings may allow for mid-course corrections within other demonstrations, as appropriate. The 36-month findings will provide valuable information about the longer term benefits of this type of intervention as additional or new policy developments are considered at the federal and state levels and within the marriage education field more broadly.

Failure to conduct the second data collection as proposed would limit estimation of the impacts of the program, and limit ACF’s ability to determine whether BSF is meeting its stated goals. It is important to conduct a second data collection because the BSF programs are intended to have long-term influences on relationships, which can affect child well-being. It will be important to understand if program impacts detectable at 15 months (the time of the first telephone follow-up) are sustained and if new impacts emerge over time. For example, movement toward marriage may be a gradual process, with the incidence of marriage increasing over time for program participants. Alternatively, the relationship status of low-income couples, in general, has shown to be unstable (Wood et al. 2003); therefore, couples together at the first follow-up may no longer be together at the second follow-up. If BSF has its intended effect, the relationship status pattern will be more stable in the program group than in the control group.

Measuring effects on children at 36 months is necessary to understand if the BSF programs are meeting the important goal of improving child outcomes. If BSF succeeds in improving the stability and quality of couples’ marriages and relationships, this may lead to improved parenting skills and better child outcomes. Importantly, the second follow-up survey includes child outcome measures that were not collected at the first follow-up because it was too early in the child’s development to assess these outcomes. The outcomes measured in the second follow-up telephone survey include measures of the child’s internalizing and externalizing behaviors, social competence, and health status, each of which may be affected by the BSF intervention through changes in family functioning.

Conducting an in-home direct assessment and observation will provide important information for examining the effect of the BSF programs on children. BSF is expected to have impacts on children’s social and emotional development (including self-regulation and attachment status) through program effects on parenting, father involvement, parent well-being, and family self-sufficiency. BSF may also affect children’s language development, as we expect that a stronger couple relationship will increase the child’s exposure to language in the home. Direct assessment and observation provide the strongest unbiased evidence for these impacts on child development and the parent-child relationship, because they do not rely on parent reports.

7. Special Circumstances

There are no special circumstances.

8. Federal Register Announcement and Consultation

The initial request for comment on the proposed data collection activity and instruments was published in the Federal Register on December 4, 2007 (Vol. 72, No. 232, p. 68166). The second notice was published on February 8, 2008 (Vol 73, No. 27, p 7562).

a. Comments

One person requested a copy of the proposed instrument but provided no further comments after the document was sent to her. Another commenter noted twice (once after the first Federal Register Notice, and again after the second Federal Register Notice) that funds would be better spent helping families than conducting surveys but provided no specific comments on the instrument.

b. Consultation Outside the Agency

During preparation of the data collection instruments, we engaged the professional counsel of a large number of people. These consultants include experts in the study of marriage and relationships, child development and well-being, intervention design, the needs of specific populations, and evaluation design. They also include curriculum developers and program administrators. In June 2005, the BSF Technical Work Group met and provided feedback on our overall study design and data collection plan, and in October 2007, the BSF Technical Work Group met and provided feedback specifically on the second follow-up data collection. The experts consulted for BSF are listed in Table 4.

c. Unresolved Issues

None.

9. Payments and Gifts to Respondents

To secure sufficiently high response rates for the telephone survey, we propose making a $25 incentive payment to all survey respondents. In addition to the incentive offered to each member of the couple who completes an interview, we also propose offering each parent $25 for completing the in-home assessment including the parent-child interaction task, as well as two $5 toys for the child for each parent-child interaction task completed during the in-home assessment. These additional incentives are warranted because of the extra burden being placed on the family to participate in the in-home direct assessment.

TABLE 4

TECHNICAL WORK GROUP, CONSULTANTS, AND OTHER REVIEWERS

Name	Affiliation	Telephone Number
TECHNICAL WORK GROUP
Paul Amato	Department of Sociology Pennsylvania State University	814-865-8868
Thomas Bradbury	Department of Psychology University of California, Los Angeles	310-825-3735
Esther Calzada	Department of Child & Adolescent Psychiatry NYU Medical Center	212-263-8981
Martha J. Cox	Director, Center for Developmental Science University of North Carolina at Chapel Hill	919-966-3509
E. Mark Cummings	Department of Psychology University of Notre Dame	574-631-3404
Lindsay Chase-Lansdale	Institute for Policy Research Northwestern University	847-467-6906
Rolando Díaz-Loving	Facultad de Psicología Universidad Nacional Autónoma de México	525-55622-2326
Ron Haskins	The Brookings Institution	202-797-6057
Edwin Hernandez	Center for the Study of Latino Religion University of Notre Dame	574-631-8558
Linda Malone-Colon	Department of Psychology Hampton University	757-727-5301
Ronald Mincy	Columbia University School of Social Work	212-851-2408
CONSULTANTS
Irv Garfinkel	Columbia University School of Social Work	212-854-8489
John Gottman	Relationship Research Institute University of Washington	206-832-0305
Sara McLanahan	Center for Research on Child Wellbeing Princeton University	609-258-4875
Robert Rector	The Heritage Foundation	202-608-6213
Anne Menard	Domestic Violence Resources Network	717-259-3674
MATHEMATICA POLICY RESEARCH, INC.
Alan Hershey	Project Director (MPR)	609-275-2384
Barbara Devaney	Project Director (MPR)	609-275-2389
Shawn Marsh	Survey Director (MPR)	609-936-2781
Sheena McConnell	Principal Investigator for the Impact Analysis (MPR)	202-484-4518
Robert Wood	Principal Investigator for the Impact Analysis (MPR)	609-936-2776
Robin Dion	Principal Investigator for the Implementation Analysis (MPR)	202-484-5262
Kim Boller	Consultant for Child Outcome Measures (MPR)	609-275-2341
Peter Schochet	Consultant for Evaluation Design (MPR)	609-936-2783
OTHER RESEARCH ORGANIZATIONS
Barbara Goldman	MDRC	212-684-0832
Joanne Hsueh	MDRC	212-532-3200
Matthew Stagner	Chapin Hall Center for Children	773-256-5116
Mary Myrick	Public Strategies, Inc.	405-848-2171

Singer and Kulka (2002), in a review of research on the use of incentives in surveys, found that incentives are cost-effective, lowering the overall cost and burden of most surveys. Studies have also shown that incentives may reduce differential response rates and hence the potential for nonresponse bias (Singer and Kulka 2002). For example, there is evidence that incentives are effective at increasing response rates for people with lower educational levels (Berlin et al. 1992) and low-income and nonwhite populations (James and Bolstein 1990).

Evidence suggests that the incentive cannot be much lower than $25 for adults. An incentive experiment from the 1996 panel of the Survey of Income and Program Participation showed that a $20 incentive significantly increased response rates, while a $10 incentive had no effect relative to those who received no incentive. Burghardt and Homrighausen (2002) found response rates for the third follow-up survey of youth in the National Job Corps Study were low with only a $10 incentive. When OMB approval was received to increase the incentive to $25, the response rate increased and the cost per completed interview was nearly 20 percent lower than those interviews conducted with the $10 incentive.

10. Confidentiality of the Data

Data from the first and second data collection will be maintained by MPR without any information that would allow personal identification of the respondents. Respondents receive information about privacy protections when they consent to participate in the study and

information about privacy will be repeated as part of the survey interviewers’ and in-home data collector’s introductory comments for the second follow-up interview and in-home direct assessment.

Respondents will be informed that the identifying information they provide will be kept private as provided by the Confidentiality Certificate issued by HHS and other provisions of law and that the results of the study will be presented only in aggregate form. A Confidentiality Certificate was received from the National Institute for Child Health and Human Development on October 29, 2007 and expires December 31, 2011 (a copy is provided in Appendix E). The Certificate of Confidentiality issued by HSS (2007) provides that: “persons so authorized to protect the privacy of such individuals may not be compelled in any Federal, State, or local civil, criminal, administrative, legislative, or other proceedings to identify such individuals.”

All interviewers and data-collectors will be knowledgeable about confidentiality and privacy procedures and will be prepared to describe them in detail or to answer any related questions raised by respondents.

The following safeguards will be employed by MPR to carry out privacy assurances:

• All employees at MPR sign a confidentiality pledge that emphasizes the importance of confidentiality and describes their obligations

• Access to identifying information on study respondents is limited to those who have direct responsibility for providing the sample and maintaining sample locating information

• Identifying information is maintained on separate forms and files, which are linked to the interview only by sample identification number

• Access to the file linking sample identification numbers with the respondents’ identification and contact information is limited to a small number of individuals who have a need to know this information

• Computer files are protected with passwords, and access is limited to specific users

11. Additional Justification for Sensitive Questions

Some sensitive questions are necessary in a study of a program designed to affect personal relationships. In the second follow-up survey, all respondents will be informed that their identity will be kept private and that they do not have to answer questions that make them uncomfortable. Table 5 describes the justification for the sensitive questions included in the second follow-up telephone survey. Although these questions are sensitive, they have commonly, and successfully, been asked of respondents similar to those who will be in this study (for example, in the Fragile Families Study and in the Early Head Start Research Evaluation Project). Further, sensitive questions were successfully pretested during the first follow-up. With the exception of the questions concerning the use of child discipline techniques, these questions were also all included in the first follow-up survey. In the first follow-up, the percentage of nonresponse to most of these items has been less than 1 percent. Nonresponse averages to date from the first follow-up survey are noted in Table 5.

TABLE 5

JUSTIFICATION FOR SENSITIVE QUESTIONS

Question Topic	Justification
Whether the BSF partner is the parent of other children born after random assignment (Question FS52)	This question will enable us to examine BSF’s potential impact on multiple partner fertility. Multiple partner fertility has been shown to have negative consequences for child well-being, reducing financial and other support from parents and increasing children’s exposure to unrelated adults, which can increase the risk of child maltreatment (McLanahan and Sandefur 1994; Radhakrishna et al. 2001; Carlson and Furstenburg 2006; Harknett and Knab 2005). This question has been used on follow-up surveys conducted as part of the Fragile Families and Child Wellbeing Study and the first follow-up Building Strong Families survey. In our experience with the first follow-up survey for BSF thus far, nonresponse is less than 1 percent for this item.
Methods of discipline used with the BSF child by BSF parents and current partners (Questions CO5a-CO5j)	These items measure the use of mild to harsh disciplinary practices. These measures will enable us to determine whether BSF’s emphasis on conflict management skills leads to a reduction in the use of harsh discipline techniques among participants. By improving conflict management skills and overall parental well-being, BSF may reduce child maltreatment and the use of harsh discipline. These items are drawn from the Conflict Tactic Scale: Parent Child Version (CTSPC; Straus et al. 2003). The CTSPC is well validated and shown to have good internal consistency, and has been used in large-scale longitudinal surveys including the National Survey of Child and Adolescent Well-Being (NSCAW). (These items were not included on the 15-month survey.)
Whether respondent or his or her BSF partner have cheated; perceived likelihood of cheating in the future (Questions RR8-RR11)	Infidelity has been found to be a major obstacle to marriage for unwed parents (Edin and Kefalas 2005). The BSF curriculum aims to address this issue by discussing the importance of fidelity and trust in building a healthy relationship and marriage. Several large surveys have included similar questions concerning infidelity, such as the Study of Marital Instability Over the Life Course, the Louisiana Fragile Families Study, and the Baseline Survey of Family Experiences and Attitudes in Florida. These questions were used in the first follow-up Building Strong Families survey. In our experience with the first follow-up survey for BSF thus far, nonresponse is less than 1 percent for these items.
Whether respondent has been physically or sexually assaulted by his or her BSF partner or another partner (RR14-RR15.1)	The BSF intervention aims to improve relationship quality and increase the likelihood that couples enter into a healthy marriage. A key characteristic of a healthy romantic relationship is one that is not marred by violence. These questions are drawn from the revised Conflict Tactic Scale (CTS2), the most widely used tool for measuring domestic violence in research studies (Straus and Douglas 2004). The CTS2 has been well validated and shown to have good internal consistency (Straus et al. 1996). Versions of these CTS questions have been used on many surveys, including the National Family Violence Survey, the National Violence Against Women Survey, the first follow-up Building Strong Families survey, and surveys conducted in six states as part of the ASPE-funded TANF Caseload Project. In our experience with the first follow-up survey for BSF thus far, nonresponse is less than 1 percent for these items.
Symptoms of depression (WB1.2-WB3)	Parental depression has been shown to have adverse consequences for child outcomes (Gelfand and Teti 1990, Downey and Coyne 1990). Given BSF’s ultimate goal of improving child well-being, the link between parental depression and child well-being makes this outcome of particular relevance. BSF may reduce depressive symptoms among participants by reducing stress and conflict in relationships. These questions represent the 20-item Centers for Epidemiologic Studies Depression Scale (CES-D), a widely used measure with well-established good psychometric properties (Radloff 1977). The CES-D has been used many large surveys, including those used as part of the Early Head Start Evaluation, the National Longitudinal Survey of Youth, the Project on Devolution and Urban Change, and the first follow-up Building Strong Families survey. In our experience with the first follow-up survey for BSF thus far, nonresponse is less than 1 percent for these items.
Alcohol and drug use (WB4-WB6)	Substance abuse and addiction can have major negative effects on the well-being of individuals and their families. If BSF improves relationship quality and stability, it may reduce substance abuse among participants. The question we include concerning binge drinking was developed by Henry Wechsler and is recommended as a screening tool by the National Institute on Alcohol Abuse and Alcoholism (Wechsler et al. 1995; Wechsler 1998). It has been used in several large national surveys, including the National Survey on Drug Use and Health and the Youth Risk Behavior Surveillance System. The two questions concerning functional impairment resulting from substance use come from Fragile Families surveys and were also used in the first follow-up Building Strong Families survey. In our experience with the first follow-up survey for BSF thus far, nonresponse is less than 1 percent for these items.
Family income (WW1-WW42)	Family income and poverty are important determinants of child well-being (Brooks-Gunn and Duncan 1997; Mayer 1997). BSF aims to enhance child well-being by improving the parental relationship and the likelihood that the parents remain together as a couple. Since two-parent families generally have higher incomes than single-parent ones, increases in family income may be an important avenue by which BSF improves child outcomes. Family income has been collected on many national surveys, including the National Survey of America’s Families and the Fragile Families surveys. The particular questions we use are drawn from the Work First New Jersey study, a large longitudinal study of welfare recipients, and were used in the first follow-up Building Strong Families survey. In our experience with the first follow-up survey for BSF thus far, nonresponse is less than 1 percent for these items, with the exceptions of WW36 (item about handling money) to which 1.3 percent responded that they did not know, and WW4 and WW7 (items about total earnings in the past month) for which nonresponse was 5-10 percent primarily attributable to answers of “do not know.”
Involvement with the criminal justice system (questions WB9-WB33)	Recent research suggests that a history of incarceration and involvement with the criminal justice system may be fairly common among fathers in the BSF target population (Western 2004). Parental incarceration has major negative effects on child and family well-being, reducing the financial support and other types of support the parents can provide to their children and families. BSF may reduce criminal involvement through its potential effects on relationship stability and quality. Similar questions have been included in other large national studies, such as Fragile Families survey, the National Job Corps Study and the first follow-up Building Strong Families survey. In our experience with the first follow-up survey for BSF thus far, nonresponse is less than 1 percent for these items with the exception of WB25.1 (mothers are asked about father’s arrests) to which 3 percent responded that they did not know or refused.
Childhood history of sexual or physical abuse (BP7-BP8)	A history of physical and sexual abuse during childhood has been shown to reduce the likelihood of entering into and sustaining healthy relationships and marriages as an adult (Cherlin et al. 2004). This research also indicates that a history of childhood abuse is fairly common among low-income populations. For these reasons, those with a history of childhood abuse will be an important subgroup to examine as part of the BSF impact analysis. These two questions are from surveys conducted as part of Welfare, Children, and Families: A Three-City Study and were also used in the first follow-up Building Strong Families survey. These questions will only be asked if the respondent did not complete a first follow-up survey (since these questions were included on the first follow-up survey as well). In our experience with the first follow-up survey for BSF thus far, nonresponse is less than 1 percent for these items.
Age of first intercourse (BP9)	The BSF curriculum aims to build commitment and trust among unmarried couples with young children as a means of strengthening and preserving their romantic relationships. Individuals with a large number of sexual partners prior to entering the program may have difficulty establishing the necessary level of commitment and trust to build a healthy and lasting romantic relationship. Therefore, the number of sexual partners prior to random assignment is a variable of potential interest for subgroup analysis. Because of recall difficulties, however, asking about the number of sexual partners prior to random assignment on the first follow-up survey is not practical. Therefore, we will ask instead about the age of first intercourse, which has been shown to be a good proxy for the number of sexual partners (USDHHS 1997]). This question is drawn from the National Survey of Family Growth, and was used in the first follow-up Building Strong Families survey. This question will only be asked if the respondent did not complete a first follow-up survey (since these questions were included on the first follow-up survey as well). In our experience with the first follow-up survey for BSF thus far, nonresponse is 3.7 percent for this item.
Number of sexual partners since random assignment (BP10)	Children who are exposed to the new romantic partners of their parents are placed at increased risk of abuse and other adverse outcomes (Radhakrishna et al. 2001). It is hoped that by increasing the likelihood that participating couples remain together, BSF will reduce the exposure that their children have to the new romantic partners of their parents. Therefore, the number of sexual partners since random assignment is an important variable to examine as part of the impact analysis. This question is drawn from the National Survey of Family Growth surveys and was used in the first follow-up Building Strong Families survey. In our experience with the first follow-up survey for BSF thus far, nonresponse is 2 percent for this item.

12. Estimates of the Hour Burden of the Collection of Information

Table 6 presents the number of respondents, the number of responses per respondent, the average burden hours per response, and the total annual burden hours for the second follow-up data collections for which clearance is being sought.

TABLE 6

ANNUAL BURDEN ESTIMATES FOR SECOND FOLLOW-UP DATA COLLECTION

Activity/Respondent	Number of Respondents	Number of Responses Per Respondent	Average Burden Hours Per Response	Total Annual Burden Hours
Second Follow-up Survey
Mothers	1,443	1	0.88 (53 min avg)	1,275
Fathers	1,358	1	0.88 (53 min avg)	1,199
Total Second Follow-up Survey	2,801			2,474
In-Home Child Assessment
Child direct assessment	850	1	.33 (20 min)	283
Parent-Child Assessment
Mother-child interaction	1,700	1	.33 (20 min)	567
Father-child interaction	1,360	1	.33 (20 min)	453
Total Parent-Child Assessment	3,060			1,020
Grand Total - Burden Second Follow-up Data Collection
Annual Grand Total for All	-	-	-	3,777

The second follow-up telephone survey will involve separate interviews with the mother and father of the BSF focal child. Each survey will be administered once to each respondent. Interviewing will start in 2008 and is expected to be completed in 2011. As with the first follow-up survey, the mother and the father will be asked similar questions although in some cases the mother is asked additional questions that are not included in the father’s interview. For example, questions about the child’s behavior and family routines are asked only of mothers and only if she lives with the child all or most of time. Based on the first follow-up pretests, we expect the mothers’ interview to last an average of 55 minutes and the fathers’ interview to last 50 minutes, for an average of 53 minutes.

Survey interviews will be attempted with a total of 5,095 mothers and 5,095 fathers over the three-year period 2008 to 2011, resulting in interview attempts with about 1,698 mothers and 1,698 fathers annually. Based on our experiences to date with the first follow-up telephone interview, we expect the response rate for the second follow-up survey to be 85 percent for mothers resulting in 1,443 mother respondents (1,698 x .85 = 1,443) annually. We expect the response rate for fathers to be 80 percent resulting in 1,358 father respondents (1,698 x .80 = 1,358) annually.

The in-home child assessments will also be conducted over the three year period 2008 and 2011 and will be attempted with about 1,000 children annually. We expect that we will achieve a response rate of 85 percent. Hence, we expect to complete direct child assessments with 850 children annually (1,000 x .85 = 850).

The observations of parent-child interactions will also be conducted over a three year period and will be attempted with about 1,000 families annually. We expect a response rate of 85 percent for mother-child interactions, resulting in a total of 1,700 respondents—850 mothers and 850 children for the mother-child interaction.

We will only attempt to observe the parent-child interaction when the parent is in regular contact with the child. Based on analysis of Fragile Families data, we expect that 85 percent of fathers will be in regular contact with their child at 36 months and expect to achieve an 80 percent response rate for these fathers, resulting in a total of 1,360 respondents –- 680 fathers and 680 children (1,000/year in sample x .85 in regular contact x .80 response rate = 680).

13. Estimate of Total Annual Cost Burden to Respondents or Record Keepers

None. Respondents will not incur any out-of-pocket costs. Telephone calls will be placed at the expense of the evaluation contractor, and respondents who wish to call the interviewers will be provided with a toll-free number billed to MPR.

14. Estimate of Annualized Costs to the Federal Government

The total estimated cost to the federal government of designing and administering the second follow-up survey and the in-home direct assessment, processing and analyzing the data, and preparing the final report is $4,713,775 (see Table 7).

15. Reasons for Program Changes or Adjustments

Not applicable. This is a new information collection.

TABLE 7

ANNUALIZED COSTS TO THE FEDERAL GOVERNMENT

	FY08	FY09	FY10	FY11	FY12	Total
In-home direct assessment and data coding	$445,781	$445,781	$445,781	$0	$0	$1,337,343
Second survey and impact analysis	$349,705	$1,033,387	$1,096,873	$867,588	$ 28,879	$3,376,432
Total	$795,486	$1,479,168	$1,542,654	$867,588	$28,879	$4,713,775

16. Plans for Tabulations and Publication and Schedule of Project

Our approach to addressing the research questions discussed in Section 1d is to conduct an impact analysis. The goal of the impact analysis is to compare observed outcomes for program participants with outcomes for members of a control group that did not receive the BSF intervention. We will use the experience of the control group as a measure of what would have happened to the program group couples and children in the absence of BSF. Random assignment of couples to a program and a control group ensures that the two groups of couples do not initially differ in any systematic way on any characteristic, observed or unobserved. Any observed differences in outcomes between the program and control group couples can therefore be attributed to BSF with a known degree of precision. Depending on the outcome considered, the unit of observation for the analysis might be the couple (for example, for marital status), an individual parent (for example, for parent well-being), or the child (for example, for child well-being).

Differences of means or proportions in outcomes between the program and control group will provide unbiased estimates of the impacts of BSF. More precise estimates will be obtained using regression models to control for random differences in the baseline characteristics of program and control group members. In their simplest forms, these models can be expressed by the following equation:

(1) Y = X΄ß + δ P + e,

where:

Y is an outcome variable

X is a vector of control variables (including an indicator for each site)

β is the vector of regression coefficient for the control variables

δ is the measure of the impact of BSF

P is an indicator that equals 1 for program group members and 0 for control group members

e is a random error term that is assumed to have a mean of zero conditional on X and P, and is interpreted as the unobserved factors that affect Y.

The statistical techniques used to estimate the regression‑adjusted impacts depend on the form of the dependent variable, Y. If the dependent variable is continuous, then ordinary least squares techniques will produce unbiased estimates of the parameter δ. However, if the dependent variable is binary—for example, whether the couple is married—then consistent parameter estimates can be obtained by using logit or probit maximum likelihood methods. If the dependent variable is censored or truncated—for example, earnings or total income—then tobit maximum likelihood or two-stage procedures will be used.

Control variables in the vector X will include any variables that may affect the outcome that are not affected by the intervention. Hence, X could include the characteristics of the individual or couple for which data are collected on the baseline information form, including the status and quality of the relationship at baseline. X could also include baseline characteristics that can be easily recalled and were measured using the first follow-up survey (such as incarceration prior to random assignment).

We will also estimate impacts for individual sites (as sample size allows) and for groups of sites. BSF may work better in some sites or circumstances than in others. For example, it is possible that BSF works better when it includes home visiting than when the family coordinator’s role is less intensive. Estimates of the impacts by sites or groups of sites will be obtained by introducing interaction terms in the regression model (1) that is the product of the program group indicator (P) and an indicator of membership in the site or group of sites of interest (Site_i):

(2) Y = X΄ß + δ P + γ_i* Site _i * P + e.

The estimated impact of BSF for site _i is given by (γ_i + δ).

Some people may benefit from BSF more than others. For example, it is possible that couples with a strong relationship at baseline gain the most from the BSF services. To address how BSF benefits different subgroups of the population, impacts for key subgroups of the population will be estimated. Subgroups of interest include those defined by the following characteristics at baseline: demographic characteristics (such as age and race/ethnicity, and similarity in these characteristics within the couple); education; whether the baby was wanted or mistimed; relationship status and quality; whether the BSF focal child is the couple’s first child; whether either member of the couple has children with other partners; employment, income, and receipt of public assistance; mental health status; existence of social supports; and expectations of marriage.

Estimates of impacts by subgroup will be obtained by introducing an interaction term in regression model (1) that is the product of the program group indicator (P) and an indicator of membership in the subgroup of interest (Sub_i):

(3) Y = X΄ß + δ P + θ_i* Sub_i * P + e.

The estimated impact of BSF for members of the subgroup of interest is given by (δ + θ_i ).²

c. Publication Plans

We will prepare the following reports:

• Interim Report on Program Impacts. An interim impact analysis report will present the impact estimates using data from the first follow-up survey. Data collection on the first follow-up survey is expected to be completed about August 2009. We expect the interim impact report to be available in early 2010.

• Final Report. The final report will present the findings from an implementation analysis in addition to the impact findings using data from the second follow-up data collections. Data collection for the second follow-up is expected to be completed about June 2011. We expect the final impact and implementation report to be available in late 2011.

• Topical Papers and Research Briefs. These papers and research briefs will describe special topics of interest and will be produced as requested by ACF.

d. Project Schedule

The project began in October 2002. The sites started pilot operations between February and September 2005. Intake for the evaluation began between July 2005 for the earliest site and June 2006 for the last site. Intake in each site lasts between 18 and 29 months, depending on the flow of couples into the program. Data collections are conducted 15 months after random assignment and 36 months after the birth of the BSF focal child. Public use data files will be delivered after all data is collected and prepared. The study is currently scheduled to end in December 2011.

17. Reasons for Not Displaying Expiration Date for OMB Approval

The expiration date will be displayed. The OMB number and expiration date will appear on the second follow-up CATI introductory screens and on the advance letters sent to respondents one week prior data collection (see Appendix F).

18. Exception to the Certification Statement

Exception to the certification statement is not requested.

References

Abidin, R.R., and J.F. Brunner. “Development of a Parenting Alliance Inventory.” Journal of Clinical Child Psychology, vol. 24, no. 1, 1995.

Abidin, Richard R. Parenting Stress Index. Third Edition. Odessa, FL: Psychological Assessment Resources, 1995.

Amato, Paul. “Studying Marital Interaction and Commitment with Survey Data.” Paper presented at Healthy Marriage Interventions and Evaluation Symposium of the Measurement Issues in Family Demographic Conference. Washington, DC, November 13-14, 2003.

Arnett, J. J. and L. A. Jensen. “A Congregation of One: Individualized Religious Beliefs Among Emerging Adults.” Journal of Adolescent Research, vol. 17, 2002, pp. 451-467.

Berlin, M, L. Mohadjer, J. Waksberg, A. Kolstad, I. Kirsch, D. Rock, and K. Yamamoto. “An Experiment in Monetary Incentives.” In Proceedings of the Section on Survey Research Methods. Alexandria, VA: American Statistical Association, 1992.

Brooks-Gunn, Jeanne, and Greg J. Duncan. “The Effects of Poverty on Children.” Children and Poverty, vol. 7, no. 2, summer/fall 1997.

Burghardt, John, and John Homrighausen. “National Job Corps Study: Survey Results.” Princeton, NJ: Mathematica Policy Research, Inc., 2002.

Caldwell, B.M., and Bradley, R.H. Administration Manual: Home Observation for Measurement of the Environment. Little Rock, AR: University of Arkansas at Little Rock, 2003.

Carlson, Marcia. Unpublished tabulations using data from the Fragile Families and Child Well-Being Study, 2002.

Carlson, Marcia J., and Frank F. Furstenberg, Jr. “The Prevalence and Correlates of Multipartnered Fertility Among Urban U.S. Parents.” Working Paper #03-14-FF. Princeton, NJ: Princeton University, Center for Research on Child Wellbeing, January 2006.

Carlson, Marcia, Sara McLanahan, and Paula England. “Union Formation in Fragile Families.” Demography, vol. 41, May 2004.

Carroll, Jason, and William J. Doherty. “Evaluating the Effectiveness of Premarital Prevention Programs: A Meta-Analytic Review of Outcome Research.” Family Relations, vol. 52, no.2, 2003.

Case, Anne, Darren Lubotsky, and Christina Paxson. “Economic Status and Health in Childhood: The Origins of the Gradient.” Working Paper 8344. Cambridge, MA: National Bureau of Economic Research, 2001.

Cherlin A.J., L.M. Burton, T.R. Hurt, and D.M. Purvin. “The Influence of Physical and Sexual Abuse on Marriage and Cohabitation.” American Sociological Review, vol. 69, no. 6, 2004.

Cowan, Carolyn, and Philip Cowan. When Partners Become Parents: The Big Change for Couples. Mahwah, NJ: Lawrence Erlbaum Associates, 2000.

Cummings, E. Mark, M. Ballard, Mona El-Sheikh, and Margaret Lake. “Resolution and Children’s Responses to Interadult Anger.” Developmental Psychology, vol. 27, no.3, 1991.

Cummings, E. M., and P. Davies. Children and Marital Conflict: The Impact of Family Dispute and Resolution. New York: Guilford Press, 1994.

Davies, P. T., E. M. Forman, J. A. Rasi, and K. I. Stevens, “Assessing Children’s Emotional Security in the Interparental Relationship: The Security in the Interparental Subsystem Scales.” Child Development. vol. 73, 2002, pp. 544-562.

Downey G., and J.C. Coyne. “Children of Depressed Parents: An Integrative Review.” Psychological Bulletin, vol. 108, 1990.

Dunn, L. M., and L. M. Dunn. Peabody Picture Vocabulary Test, Fourth Edition. Examiner’s Manual and Norms Booklet. Circle Pines, MN: American Guidance Service, 2006.

Dunn, L. M., E. R. Padilla, D. E., Lugo, and L. M. Dunn. Test de Vocabulario en Imagenes Peabody. Circle Pines, MN: American Guidance Service, 1986.

Edin Kathy, and Maria Kefalas. Promises I Can Keep: Why Poor Women Put Motherhood Before Marriage. Berkeley, CA: University of California Press, 2005.

Ehrle, Jennifer, and Kristin A. Moore. 1997 NSAF Benchmarking Measures of Child and Family Well-Being: Report No. 6. Washington, DC: The Urban Institute, 1999.

Eisenberg, N. “Emotion, Regulation, and Moral Development.” Annual Review of Psychology, vol. 51, 2000, pp. 665-697.

Fuller, Bruce, Gretchen Caspary, Sharon Lynn Kagan, Christiane Gauthier, Danny Shih-Cheng Huang, Judith Carroll, and Jan McCarthy. “Does Maternal Employment Influence Poor Children’s Social Development?” Early Childhood Research Quarterly, vol. 17, 2002, pp. 470-497.

Gelfand, D.M., and D.M. Teti. “The Effects of Maternal Depression on Children.” Clinical Psychology Reviews, vol. 10, 1990.

Goodman, S. H., and Gotlib, I. H. “Risk for Psychopathology in the Children of Depressed Mothers: A Developmental Model for Understanding Mechanisms of Transmission.” Psychological Review, vol. 106, no. 3), 1999, pp. 458-490.

Guerney, Bernard G. Relationship Enhancement. San Francisco, CA: Jossey-Bass, Inc., 1977.

Harknett, Kristin. “Are Children with Married Parents Healthier? The Case of Pediatric Asthma." Working Paper 2005-01-FF. Princeton, NJ: Center for Research on Child Well-Being, 2005.

Harknett, Kristen, and Jean Tansey Knab. “More Kin, Less Support: Multipartnered Fertility and Perceived Support Among Unmarried Parents.” Unpublished working paper. Philadelphia, PA: University of Pennsylvania, 2005.

House, J.S., D. Umberson, K.R. Landis. “Structures and Processes of Social Support.” Annual Review of Sociology, vol. 14, 1988.

Huston, T.L., and A.F. Chorost. “Behavioral Buffers on the Effect of Negativity on Marital Satisfaction: A Longitudinal Study.” Personal Relationships, vol. 1, 1994.

James, J., and R. Bolstein. “The Effect of Monetary Incentives and Follow-Up Mailings on the Response Rate and Response Quality in Mail Surveys.” Public Opinion Quarterly, vol. 54, 1990.

Karney, B.R., and T.N. Bradbury. “The Longitudinal Course of Marital Quality and Stability: A Review of Theory, Method, and Research.” Psychological Bulletin, vol. 118, 1995.

Kirkland, John, David Bimler, Andrew Drawneek, Margaret McKim, and Axel Schölmerich. “An Alternative Approach for the Analysis and Interpretation of Attachment Sort Items.” Early Child Development and Care, vol. 174, 2004, pp. 701–719.

Larzelere, R.E., and T.L. Huston. “The Dyadic Trust Scale: Toward Understanding Interpersonal Trust in Close Relationships.” Journal of Marriage and the Family, vol. 42, 1980.

Leventhal, T., M.B. Selner-O’Hagan, J. Brooks-Gunn, J.B. Bingenheimer, and F.J. Earls. “The Homelife Interview from the Project on Human Development in Chicago Neighborhoods: Assessment of Parenting and Home Environment for 3- to 15-Year-Olds.” Parenting: Science and Practice, vol. 4, 2004, pp. 211-241.

Liu, Shirley, and Frank Heiland. "New Estimates on the Effect of Parental Separation on Child Health." Working Paper 2007-14-FF. Princeton, NJ: Center for Research on Child Well-Being, 2007.

Love, J.M., E.E. Kisker, C.M. Ross, J. Brooks-Gunn, P. Schochet, K. Boller, J. Constantine, C. Vogel, A. Fuligni, and C. Brady-Smith. “Making a Difference in the Lives of Infants and Toddlers and their Families: The Impacts of Early Head Start.” Washington, DC: Administration for Children and Families, June 2002.

Lundberg, Shelly, and Elaina Rose. “Child Gender and the Transition to Marriage.” Demography, vol. 52, no. 2, May 2003.

Markman, Howard, F. Floyd, S. Stanley, and R. Storaasli. “Prevention of Marital Distress: A Longitudinal Investigation.” Journal of Consulting and Clinical Psychology, vol. 56, no. 2, 1988.

Markman, H.J., M.J. Renick, F.J. Floyd, S.M. Stanley, and M. Clements. “Preventing Marital Distress Through Communication and Conflict Management Training: A 4- and 5-Year Follow-Up.” Journal of Consulting and Clinical Psychology, vol. 61, 1993.

Mayer, Susan J. What Money Can’t Buy: Family Income and Children’s Life Chances. Cambridge, MA: Harvard University Press, 1997.

McGroder, S. M. “Parenting Among Low-Income, African American Single Mothers with Preschool-Age Children: Patterns, Predictors, and Developmental Correlates.” Child Development, vol. 71, no. 3, 2000.

McLanahan, Sara, I. Garfinkel, N. Reichman, J. Teitler, M. Carlson, and C. Audigier. “The National Report: The Fragile Families and Child Well-being Study Baseline Report.” Princeton, NJ: Bendheim-Thoman Center for Research on Child Wellbeing, August 2001.

McLanahan, Sarah, and Gary Sandefur. Growing up with a Single Parent: What Hurts, What Helps. Cambridge, MA: Harvard University Press, 1994.

Merrell, Kenneth W. Preschool and Kindergarten Behavior Scales, Second Edition: PKBS-2, Examiner’s Manual. Austin, TX: Pro-Ed, 2002.

Murray, Kathleen T., and Grazyna Kochanska. “Effortful Control: Factor Structure and Relation to Externalizing and Internalizing Behaviors.” Journal of Abnormal Child Psychology, vol. 30, no. 5, 2002.

Office of the Assistant Secretary for Planning and Evaluation. Trends in the Well-Being of America’s Children and Youth 1997. Washington, DC: U.S. Department of Health and Human Services, 1997.

Ouelette, Tammy, Nancy Burstein, David Long, and Erik Beecroft. “Measures of Material Hardship: Final Report.” Washington, DC: U.S. Department of Health and Human Services, April 2004.

Peterson, J., and N. Zill. “Marital Disruption, Parent-Child Relationships, and Behavior Problems in Children.” Journal of Marriage and the Family, vol. 48, 1986, pp. 295-307.

Pettit, G. S., and Bates, J. E. “Family Interaction Patterns and Children’s Behavior Problems from Infancy to Four Years.” Developmental Psychology, vol. 25, 1989, pp. 413-420.

Radhakrishna, Aruna, Ingrid Bou-Saada, Wanda Hunter, Diane Catellier, and Jonathan Kotch. “Are Father Surrogates a Risk Factor for Child Maltreatment?” Child Maltreatment, vol. 6. no. 4, November 2001.

Radloff, L.S. “The CES-D Scale: A Self-Report Depression Scale for Research in the General Population.” Applied Psychological Measurement, vol. 1, 1977.

Rangarajan, Anu, and Philip Gleason. “Young Unwed Fathers of AFDC Children: Do They Provide Support?” Demography, vol. 35, no. 2, May 1998.

Singer, E., and R.A. Kulka. “Paying Respondents for Survey Participation.” Ann Arbor, MI: The University of Michigan, Institute for Social Research, Survey Research Center, 2002.

Stanley, S. M., H. J. Markman, and S. Whitton. “Communication, Conflict, and Commitment: Insights on the Foundations of Relationship Success from a National Survey.” Family Process, vol. 41, no. 4, 2002.

Stanley, Scott. “Assessing Couple and Marital Relationships: Beyond Form and Toward a Deeper Knowledge of Function.” Paper presented at Healthy Marriage Interventions and Evaluation Symposium of the Measurement Issues in Family Demographic Conference. Washington, DC, November 13-14, 2003.

Straus, Murray A., and Emily M. Douglas. “A Short Form of the Revised Conflict Tactics Scales, and Typologies for Severity and Mutuality.” Violence and Victims, vol. 19, no. 5, October 2004.

Straus, Murray A., Sherry L. Hamby, Sue Boney-McCoy, and David B. Sugarman. “The Revised Conflict Tactics Scale.” Journal of Family Issues, vol. 17. no. 3, May 1996.

Straus, Murray A., Sherry L. Hamby, and W. Louise Warren. The Conflict Tactic Scales Handbook. Los Angeles, CA: Western Psychological Services, 2003.

Turner, R.J. and J.B. Turner. “Social Integration and Support.” In Handbook of the Sociology of Mental Health, eds. C.S. Aneshensel and J.C. Phelan. New York: Kluwer Academic/Plenum Publishers, 1999.

U.S. Department of Health and Human Services. “Indicators of Welfare Dependence: Annual Report to Congress.” October 1997.

Vandell, D. L. “Effects of Playgroup Experience on Mother-Son and Father-Son Interaction.” Developmental Psychology, vol. 15, 1979, pp. 379-385.

Wampler, K. S. “An Update on Research on the Couple Communication Program.” Family Science Review, vol. 3, no.1, 1990.

Wechsler, H., G.W. Dowdall, A. Davenport, and E.B. Rimm. “A Gender-Specific Measure of Binge Drinking Among College Students.” American Journal of Public Health, vol. 85, 1995.

Wechsler, H. “Binge Drinking: The Five/Four Measure.” Journal of Studies on Alcohol, vol. 59, 1998.

Western, Bruce. “Incarceration, Marriage, and Family Life.” Working Paper #05-09-FF. Princeton, NJ: Princeton University, Center for Research on Child Wellbeing, 2004.

Wilcox, W. Bradford, “Soulmates? Religion and Marriage Among African Americans.” Paper presented at the Connecting Marriage Research to Practice Conference, University of North Carolina at Chapel Hill, September 2006

Wilson, Chris M., and Andrew J. Oswald. “How Does Marriage Affect Physical and Psychological Health? A Survey of the Longitudinal Evidence.” Discussion Paper No. 1619. Bonn, Germany: Institute for the Study of Labor (IZA), June 2005.

Wood, Robert G., Anu Rangarajan, and John Deke. “WFNJ Clients Under Welfare Reform: How Is an Early Group Faring Over Time?” Princeton, NJ: Mathematica Policy Research, Inc., September 2003.

Wood, Robert G., Anu Rangarajan, and John Deke. “Marriage Patterns of TANF recipients: Evidence from New Jersey.” Princeton, NJ: Mathematica Policy Research, Inc., October 2003.

Zill, N. Behavior Problem Scales Developed from the 1981 Child Health Supplement to the National Health Interview Survey. Unpublished summary, 1985.

1 Subcontractors are MDRC, Chapin Hall Center for Children, Decision Information Resources, and Public Strategies.

2 Information on power to detect effects is included in section 2.a of Part 2.