Supplement A follow-up

OMB No.: 0970-0439

Expiration Date: 09/30/2016

SUPPLEMENT A:

Question-by-Question justification and surveys referenced foR the CSPED 12-month Follow-up survey

Table1. CSPED 12-Month Follow-up Survey: Question-by-Question Justification

Question	Source	Rationale
Introduction and Contact Information
Introduction (i1-i8)	PACT tailored for CSPED	Questions are asked to ascertain whether the respondent is the sample member and to inform the respondent about: the expected length of the interview, the Certificate of Confidentiality, and the possibility that the interview may be recorded.
Contact information (A1a-A2)	BSF 36-month follow-up tailored for CSPED	Obtaining the exact spelling of the respondent’s name is important so that we can send them the respondent payment.
Marital status and educational attainment (A3-A4)	OMB tailored for CSPED; CBRA tailored for CSPED	Obtaining updated information on marital status and educational attainment to assess whether the program had an influence on either of these factors.
Child Roster and Father Involvement
Roster of children born to father since random assignment (B1- B3g)	PACT tailored for CSPED; BSF 15-month follow-up tailored for CSPED	Questions are asked about any children the father has had since random assignment for two reasons. First, some men did not have children at random assignment but were expecting babies; it will be important to know how the program has affected the respondent’s parenting of the baby. Second, the program may reduce the likelihood that the father has had more children since random assignment. The questions about the child are the same as the questions about the father’s other children at baseline.
For all the children the respondent has had since random assignment: age, gender, name of mother of child, establishment of siblingship with respondent’s other children (B3d and e, B3c, B3f, B3g, B3h, B3i, B3g)	PACT
Whether lived with child since random assignment (B4)	FFCWS tailored for CSPED	A key goal of CSPED is to increase parental involvement with children. These questions allow us to assess the program’s impact on the living arrangements of the child and the respondent’s involvement with the child. These questions are asked of all the respondent’s children.
Number of nights stayed with child in last month (B5-B6)	FFCWS tailored for CSPED; WFNJ tailored for CSPED
Any contact with child in last month (B7)	EHS tailored for CSPED
In-person contact with child in last month (B8)	EHS tailored for CSPED
Most recent in-person contact with child (B9-B10)	PACT
Assessment of relationship quality (B11)	EHS tailored for CSPED
Relationship with Mother/Fathers
Status of relationship with mother/father (C1-C2)	BSF tailored for CSPED	These questions ask about marital status, romantic involvement, and contact with the mother. They are asked about all the mothers of the respondent’s children. CSPED might improve noncustodial parents’ relationship skills and co-parenting relationships, increasing the likelihood that parents are involved with the other parent of at least one of their children at follow-up and improving the quality of the co-parent relationship. A father’s romantic relationship with the child’s mother is related to his contact with his children (Tach, Mincy, and Edin 2010).
Any contact with mother/father in the past 30 days (C3)	PACT tailored for CSPED
Lives with mother/father (C4)	BSF tailored for CSPED
Nights in past 30 days stays with mother/father (C5)	PACT tailored for CSPED
Quality of relationship with mother/father (C6)	FFCWS
Quality of the collaborative co-parenting relationship (C7)	PAM
Formal and informal support paid to mother in last month (C8–C17a)	BSF, FFCWS tailored for CSPED	A key goal of CSPED is to promote responsible parenting, including fathers’ material support of their children. Financial support of children through formal and informal monetary payments and in-kind purchases will be important measures of CSPED’s impact. These questions capture whether the father has paid the mother directly or through formal child support orders. These questions are asked about all the mothers of the respondent’s children. Formal payments made through child support orders will also be collected through administrative records; however informal payments will not be included in those records. It is necessary to ask about all child support payments on the survey in order to distinguish between formal and informal payments. In addition to provision of material support to their children, CSPED may affect noncustodial parents’ understanding of their child support responsibilities; this understanding will be reflected in the survey responses and may differ from administrative records.
Barriers to parent involvement (C18-C19)	EHS tailored for CSPED	Because the geographic distance between nonresident fathers and their children is negatively associated with father involvement, CSPED might have smaller impacts on noncustodial parent involvement with the noncustodial parent lives a larger distance from the child.
Parenting time/visitation orders (C20-C22)	New	A key goal of CSPED is to improve the relationship between nonresident parents and their children. Parenting time orders determine the visitation structure between nonresident parents and their children and establish processes for nonresident parents to spend time with their children in person. CSPED programs may offer parenting time assistance as a component of parenting services. These questions will determine whether or not the respondent had a parenting time order prior to or following random assignment, and whether or not that order has been modified since random assignment.
Attitudes toward parental support and involvement (C23-C24)	Fragile Families tailored for CSPED	Parenting programs are a core component of CSPED services. CSPED may improve parenting attitude, quality of parenting and parenting behaviors. Hence, the survey includes attitudinal questions, questions about the types of activities that the father may do with the child, focusing on active engagement, and questions on parenting behaviors and discipline techniques. The question tailors the activities asked about to the age of the child.
Parenting self-assessment (C25)	PSI; IFSS tailored for CSPED
Activities with focal child in the last month (C26-C26_1)	PACT
Warmth, supportiveness and discipline (C27-C27_1)	CTSPC tailored for CSPED; PPQ tailored for CSPED
Presence and structure of parenting time orders (C28-C29_SPECIFY)	New	A key goal of CSPED is to improve the relationship between nonresident parents and their children. Parenting time orders determine the visitation structure between nonresident parents and their children and establish processes for nonresident parents to spend time with their children in person. CSPED programs may offer parenting time assistance as a component of parenting services. These questions will determine whether or not the respondent has a parenting time order for each focal child and how much time the respondent is ordered to spend with the child.
Whether parent has other romantic partner (C30-C31)	PACT, WFNJ tailored for CSPED	Having a spouse or a cohabiting partner is associated with more favorable labor market outcomes for men and economic benefits for women, so noncustodial parents’ relationships with new partners are expected to be predictive of later economic outcomes. Having a new partner and living with the children of a new partner may also be associated with less involvement by fathers in the lives of their children from prior relationships.
Parents’ relationship status with partner (C32)	WFNJ tailored for CSPED
Whether parent lives with partner (C33)	BSF tailored for CSPED
Number of nights parent spent with partner in past 30 days (C34)	CSPED-developed
Whether partner has children under age 18 (C35)	PACT tailored for CSPED
Whether partner’s children stayed with partner and respondent in last 30 days (C36)	PACT tailored for CSPED
Child Support Program: Attitudes and Interactions
Attitudes towards child support system (D1-D5)	Fragile Families; PACT tailored for CSPED	CSPED teaches parents about their legal rights and responsibilities. Further, CSPED programs emphasize that parents can work with child support enforcement to address their needs. These questions ask about respondent attitudes regarding the child support system, interactions with the child support system, and attitudes towards paying child support.
Economic Stability
Information on all jobs respondent has had in the past year (E1-E9c)	WIA; PACT	Encouraging employment is one of the primary goals of CSPED. Therefore it is critical that the evaluation has a complete picture of the employment history of study participants. With that goal, the evaluation will include employment information drawn from both the survey and administrative records. Administrative and survey-based employment information have different strengths. Collecting information from both sources will allow the evaluation team to draw on the distinct strengths of these two data sources. Unlike self-reported survey data, earnings measures based on UI administrative records are not subject to nonresponse or recall error. However, administrative data do not cover all jobs. Workers excluded from UI earnings records include self-employed workers, railroad employees, workers in service for relatives, most agricultural labor, some domestic service workers, part-time employees of nonprofit organizations, and some workers who are casually employed. Workers in these sectors comprise about 10 percent of workers in the U.S. economy (Hotz and Scholz 2002). Informal employment that is not covered in the UI system is likely to be more common for the low wage population that CSPED targets. For that reason, the survey asks specifically about informal employment and earnings from all jobs (survey items E1-E9). These questions ask about all the jobs that the respondent has currently and since random assignment. We ask about when the job began, ended (if applicable), type of job (e.g. full time, part time, self-employed, day laborer), wage rate, hours worked, and fringe benefits. Information on job characteristics will not be available in administrative records.
Income received through odd jobs or other work since random assignment (E10-E11)	PACT	Many people, especially those with low income, do other work for pay that may not be considered a job. For example, they may babysit, or help out with a family business occasionally. These questions ask about whether they have done work that they do not consider “a job” and the amount they made from that work.
Public disability benefits received (E12-E13)	BSF 15-month follow-up tailored for CSPED	CSPED programs may increase employment, thereby reducing the need for public support benefits such as Supplemental Security Income. Because disability benefits are often substantial for those who receive them (and costly to the government), this information will be important for the benefit-cost analysis. Unlike information on receipt of other public support benefits (such as SNAP or TANF), information on disability benefit receipt will not be available through administrative records.
Steps taken to find employment (C14)	PACT	CSPED programs provide services designed to help participants take steps towards finding work. This question asks whether or not the respondent has taken steps towards finding a job since random assignment.
Barriers to employment (E15)	FFCWS, BSF tailored for CSPED	CSPED programs provide services designed to address barriers to employment. These questions ask about the respondent’s perceptions of barriers to finding or keeping a job.
Possession of a bank account (E16)	New	CSPED programs help participants establish bank accounts as a component of promoting economic stability. This question asks whether the respondent currently has a bank account of any kind.
Financial hardship (E17)	IFSS	The program may assist the respondent in improving their financial situation. These questions identify financial hardships experienced by the respondent since random assignment.
Social support networks (E18-E20)	BSF 15-month follow-up tailored for CSPED	CSPED participants may expand the network of people available to help them in case of hardship due to interactions with new individuals through employment or program participation. These questions ask how many people the respondent could turn to given different situational difficulties.
Housing stability (E20-E23a)	WFNJ, HII	Housing instability, including homelessness, eviction, frequent movies, involuntary moves due to being unable to pay rent or mortgage, and living with others without paying rent, is experienced by a considerable share of urban men, especially those who have been incarcerated (Geller and Curtis 2011). Understanding the housing circumstances of the CSPED sample will help capture the extent of the disadvantage. These questions ask about where the respondent lives and who the respondent lives with.
Possession of a valid driver’s license (E24-E25)	New	Lack of transportation can be a barrier to employment for persons seeking to find or keep work. CSPED programs provide license reinstatement services, which may affect the ability of participants to find or keep work. These questions ask whether the respondent has ever had a license, and if so, whether the respondent currently possesses a valid license.
Criminal Justice Involvement
Arrests since random assignment (F1-F3)	BSF; SVORI tailored for CSPED	CSPED programs provide supportive services and employment support to participants, which may help to decrease interaction with the criminal justice system. Recent research suggests that a history of incarceration and involvement with the criminal justice system may be fairly common among fathers in the CSPED target population (Pettit and Western 2004). Because criminal justice involvement is extremely costly to the government and society, this information is crucial for the benefit-cost study. These questions ask about arrests since random assignment. The evaluation will also make use of State administrative records related to criminal justice involvement, however these records will not include criminal justice activity in other States and, in most States, these records will not include criminal justice activity at the county or local level.
Convictions since random assignment (F4)	SVORI tailored for CSPED
Incarceration since random assignment (F5-F6)	SVORI tailored for CSPED
Parole or probation since random assignment (F7-F8)	SVORI tailored for CSPED
Respondent’s Well Being
Depressive symptoms (G1)	PHQ-8	By linking respondents to mental health services, providing opportunities for social interactions, and improving relationships, the CSPED programs may reduce depression. Eight items from the Parental Health Questionnaire are included in this survey. The PHQ-8 has been shown to be a valid measure of depression in population-based studies (Kroenke et al. 2009)
Locus of control and future orientation (G2)	FFCWS tailored for CSPED	Disadvantaged noncustodial parents may feel helpless to change their circumstances and pessimistic about the future. If CSPED programs help noncustodial parents acquire new skills or improve their circumstances, through employment for example, they may develop greater feelings of self-efficacy and the ability to plan for the future.
Service Receipt
Participation in group employment services since random assignment (H1-H3)	BSF 15-month follow-up tailored for CSPED	Asking both members of the program and control group about the receipt of services will provide information about the “counterfactual”—the services that would have been received in the absence of participation in the program. This series of questions asks both program and control group members about participation services that are part of the core CSPED service package, such as individual and group employment services and parenting classes, child support case management services, as well as services that CSPED programs refer participants to, including GED and ESL classes, mental health and substance abuse services, record expungement services, and anger management and domestic violence services. In order to determine the frequency and intensity of services, respondents are asked to indicate participation in each type of service, as well as the number of times they received the service and duration of each service provided.
Participation in individual employment services since random assignment (H4-H6)	BSF 15-month follow-up tailored for CSPED
Participation in job training since random assignment (H7-H9)	BSF 15-month follow-up tailored for CSPED
Receipt of job development services since random assignment (H10)	BSF 15-month follow-up tailored for CSPED
Receipt of transportation services for employment since random assignment (H14)	New
Participation in GED classes since random assignment (H15)	BSF 15-month follow-up tailored for CSPED
Participation in mental health, alcohol, or substance abuse services since random assignment (H16)	BSF 15-month follow-up tailored for CSPED
Participation in domestic violence or anger management services since random assignment (H17)	BSF 15-month follow-up tailored for CSPED
Receipt of record expungement services since random assignment (H18)	New
Participation in group parenting services since random assignment (H19-H21)	BSF 15-month follow-up tailored for CSPED
Receipt of visitation/parenting time assistance services (H22)	New
Receipt of child support case management services (H23-H25)	New
Receipt of subsidized or supported employment since random assignment (H11-H13)	PACT; New	CSPED programs may link participants to subsidized or supported employment programs, which could help participants find and maintain employment. These questions ask both program and control group members about receipt of these services since random assignment.
Contact Information
I1–I5	BSF 15-month follow-up tailored for CSPED	Additional contact information is required to send the $25 appreciation payment to the respondent and in case an additional wave of interviews will be conducted. An email address is collected in case we have difficulty contacting the respondent using the other information provided.

Sources: Parents and Children Together (PACT), Building Strong Families Study (BSF), Fragile Families and Child Well-Being Study (FFCWS), Work First New Jersey (WFNJ), Parenting Alliance Measure (PAM), Patient Health Questionnaire (PHQ), Serious Violent Offender Reentry Initiative Evaluation (SVORI), Early Head Start survey (EHS), Parenting Practices Questionnaire (PPQ), Conflict Tactics Scale-Parent Child (CTSPC), WIA Gold Standard Follow-up Survey (WIA), Housing Instability Index (HII), (IFSS)

SURVEYS REFERENCED

The list below contains brief descriptions of the eleven surveys referenced for the development of the CSPED follow-up survey, as well as locations of the surveys referenced. Descriptions were compiled from websites about the surveys and descriptions of Mathematica studies were gathered from project summaries. When necessary, we tailored questions drawn from these surveys to make them easier to understand or to have the questions align more closely with the baseline survey’s goals.

1. Parents and Children Together (PACT)

Mathematica’s OMB-approved Parents and Children Together (PACT) impact and evaluation study (OMB Control Number 0970-0403) assesses innovative approaches to helping fathers increase involvement in the lives of their children and achieve economic stability. Similarly to the PACT study, the CSPED study uses a random assignment design to examine the effects of parenthood and employment services provided to low-income parents. Thus, the CSPED data collection instruments were designed to draw heavily on the PACT study instruments, which will facilitate comparisons of program outcomes between the two studies. .

Intake processes for the CSPED study are also closely modeled after the PACT study. In addition, the CSPED and PACT studies use an MIS to perform random assignment and to track program participation, and both studies include qualitative interviews with program staff, a web survey of staff and community partners, baseline data collection with parents via telephone survey, and a 12-month follow-up survey on various outcomes related to family and economic well-being.

The PACT baseline data collection instrument served as the starting point for the CSPED baseline data collection instrument. The CSPED evaluation team reviewed each question within the PACT instrument and made modifications. These modifications fall into three general categories:

• Minor wording modifications. The CSPED target population varies slightly from the PACT sample population in that noncustodial mothers are included in the CSPED study. For this reason, gendered pronouns and question wording were modified throughout the instrument to accommodate noncustodial mothers in addition to fathers. Other minor wording changes were also made to reflect programmatic variations, areas of analytical focus, clarify target behaviors, and maximize the reliability and validity of data collected from the target CSPED population.

• Deletion of items excluded from analysis. In order to reduce respondent burden, the CSPED evaluation team removed any baseline items that would not be used for analysis of the CSPED baseline survey. Examples include items about respondent religiosity, country of origin and disability status.

• Addition of items required for analysis. Several items were added in order to better understand program effects on participant outcomes. Examples include questions about employer-provided health insurance coverage, additional detail about the respondent’s living situation, and a self-assessment of the respondent as a parent.

2. Building Strong Families Study (BSF)

The United States Department of Health and Human Services/Administration for Children and Families (ACF) initiated the Building Strong Families (BSF) project to help interested and romantically involved low-income, unwed parents build stronger relationships and thus enhance their child’s well being and their own future. The BSF evaluation conducted by Mathematica was designed to test the effectiveness of these programs for couples and children. BSF data collection included a baseline information form to collect demographic and socioeconomic data along with two follow-up surveys. The follow-up surveys included questions related to mother-father relationships, family structure, fathers’ involvement in child rearing, parent-child relationships and the home environment, family functioning, child well-being and development, and parental well-being.

Surveys are available from Mathematica upon request.

3. Fragile Families and Child Well-Being Study (FFCWS)

The Fragile Families and Child Well-Being Study is a longitudinal study of a cohort of nearly 5,000 children born between 1998 and 2000 from birth through age five. Approximately one-third of the children were born to unmarried parents. Interviews were conducted with both mothers and fathers covering a range of topics including attitudes, relationships, and parenting behavior.

Study protocols and codebooks can be found here: http://www.fragilefamilies.princeton.edu/documentation.asp

4. Work First New Jersey (WFNJ)

Mathematica evaluated the effects of New Jersey’s initiative to help welfare recipients transition from welfare to work. WFNJ interviewed sample members annually for five years documenting changes in household composition, income, employment, and other indicators of well-being.

Surveys are available from Mathematica upon request.

5. Early Head Start (EHS)

The U.S. Department of health and Human Services, Administration for Children and Families funded an evaluation of the Early Head Start program—a program for pregnant women and families with infants and toddlers that is based on the Head Start program model. The Administration for Children and Families sponsored an experimental evaluation of the program based in 17 sites. This study conducted five follow-up surveys after random assignment.

Surveys are available from Mathematica Policy Research

6. Workforce Investment Act Adult and Dislocated Worker Programs Gold Standard Evaluation (WIA)

The U.S. Department of Labor sponsored an experimental evaluation of the Adult and Dislocated Worker programs funded under the Workforce Investment Act of 1998 (WIA). The study is taking place in 28 randomly-selected local workforce investment areas across the USA. Two follow-up surveys are being conducted with over 6,000 study participants at 15 and 30 months after random assignment.

Surveys are available from Mathematica Policy Research.

7. Evaluation of the Serious Violent Offender Reentry Initiative (SVORI)

The Evaluation of the Serious Violent Offender Reentry Initiative (SVORI) was a multi-year, multi-site evaluation funded by National Institute of Justice. The impact evaluation was designed to measure the impact of enhanced reentry programming on post-release outcomes. As part of the evaluation, interviews were conducted at four points in time.

Surveys are available from the National Archive of Criminal Justice Data.

8. Patient Health Questionnaire Screeners (PHQ)

The PHQ is a clinical tool designed to provide clinicians with screening and diagnostic tools for mental health disorders. All PHQ instruments have been tested in clinical settings, and are designed to improve recognition rates of depression and anxiety.

Assessments are available from Pfizer at www.phqscreeners.com.

9. Housing Instability Index (HII)

The HII is a tool created for the Safe Housing Assistance with Rent Evaluation (SHARE) study, a CDC-funded evaluation designed to learn about the connection between domestic violence and housing. The HII provides information about vulnerability, quality of life and health outcomes associated with housing stability.

The index is available through the National Alliance to End Homelessness.

10. The Parenting Stress Index (PSI)

The PSI is designed to identify dysfunctional parenting and identify sources of stress within a family unit. This empirically validated measure has been tested across languages and cultures and can be used as a diagnostic and predictive tool for future parental behavior problems.

Assessments are available for purchase from PAR, Inc.

11. Improving Family Services Study (IFSS)

The IFSS is the baseline survey instrument for the National Evaluation of Partnerships to Demonstrate the Effectiveness of Supportive Housing for Families in the Child Welfare System, a multi-year, multi-site evaluation funded by Robert Wood Johnson Foundation and evaluated by the Urban Institute.

12. Parent-Child Conflicts Tactics Scale (CTSPC)

The CTSPC is the parent-child version of the CTS, an instrument most frequently used to measure psychological and physical attacks on a partner in a marital, cohabiting, or dating relationship and use of negotiation. The CTSPC is used to measure the extent to which a parent has carried out acts of psychological and physical maltreatment and neglect of children, regardless of whether the child was injured. The scales of the CTSPC include: nonviolent discipline, psychological aggression, physical assault (including questions on discipline/corporal punishment), and neglect, as well as a supplemental scale on sexual abuse. 

Assessments are available from Western Psychological Services at http://wpspublish.com/app/.

REFERENCES

Geller, A., and M.A. Curtis. “A Sort of Homecoming: Incarceration and the Housing Security of Urban Men.” Social Science Research, vol. 40, 2011, pp. 1196–1213.

Hotz, V. Joseph, and John Karl Scholz. “Measuring Employment Income for Low-Income Populations with Administrative and Survey Data.” Washington, DC: U.S. Department of Health and Human Services, Assistant Secretary for Planning and Evaluation, 2009.

Kroenke, Kurt, Tara W. Strine, Robert L. Spitzer, Janet B.W. Williams, Joyce T. Berry, and Ali H. Mokdad. “The PHQ-8 as a Measure of Current Depression in the General Population.” Journal of Affective Disorders, vol. 144, no. 1, 2009, pp. 163–173.

Pettit, B. and B. Western. “Mass Imprisonment and the Life Course: Race and Class Inequality in U.S. Incarceration.” American Sociological Review, vol. 69, 2004, pp. 151–169.

Tach, L., R. Mincy, and K. Edin. “Parenting as a ‘Package Deal’: Relationships, Fertility, and Nonresident Father Involvement Among Unmarried Parents.” Demography, vol. 47, 2010, pp. 181–204.