Evaluation of the Community Healthy Marriage Initiative — Impact Evaluation - Wave 2
OMB No.: 0970-0322
Supporting Statement Part A
Office of Planning, Research and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
370 L’Enfant Promenade, SW
Washington, DC 20447
A. Justification
This request is for approval of changes to the OMB approved baseline information collection instrument (OMB No. 0970-0322) for use in the second wave of information collection for the Community Healthy Marriage Initiative (CHMI) impact evaluation. Approval is requested for a twelve month period. As described in the initial submission, the evaluation is designed to measure changes in a range of domains and outcomes on family structure and marriage and other outcomes in three pairs of matched communities. In order to support analyses of these changes, a second wave of information collection is required. To measure change, the instrument to be used for the second wave includes almost all of the questions that were on the baseline instrument. However, the time frames for recalling and reporting events were changed to correspond to the second wave field period. Also, questions were added to the second wave instrument to address changes in relationship status and child well-being since baseline, experience with CHMI programs, and attitudes toward marriage. Some questions, such as those related to the demographic characteristics of baseline respondents, do not need to be asked again and have been deleted from the second wave. Table 1 below shows the questions that were added and deleted from the second wave instrument.
Table 1. Questionnaire Additions and Deletions
Questionnaire Additions |
C2a. Are you married to the same person you were married to when we last interviewed you in (INSERT MONTH AND YEAR OF LAST INTERVIEW)? |
C2b. Our records show that when we last interviewed you in (INSERT MONTH AND YEAR OF LAST INTERVIEW) you were married. Why are you and your spouse no longer married? |
C2c. You mentioned that (PARTNER) is living with you. Was (PARTNER) living with you when we last interviewed you in (INSERT MONTH AND YEAR OF LAST INTERVIEW)? |
C3a. Are you romantically involved with the same person you were involved with when we last interviewed you in (INSERT MONTH AND YEAR OF LAST INTERVIEW)? |
C7a. (IF MARRIED) Have you ever been married to someone else besides (SPOUSE)? (IF NOT MARRIED) Have you ever been married? |
C7b. How many times have you been married? |
C7c. Since you were 18 years old, how many times have you lived with a partner without being married for a period of three months or more? |
Continued
Table 1. Questionnaire Additions and Deletions (Continued)
Question Additions (continued) |
C10i. (AGREE/DISAGREE) People are better off financially if they are married. |
C10j. (AGREE/DISAGREE) Needing to keep government benefits like Welfare and Food Stamps can discourage couples from getting married. |
C11. Sometimes couples decide not to get married so they do not lose public benefits such as Welfare, Medicaid, Food Stamps, WIC, or the Earned Income Tax Credit. Among people you know, does this happen…a great deal of the time, some of the time, hardly ever, or never? |
D7a. Why did you (AND SPOUSE/PARTNER) attend the classes or workshops? |
D12a. Were you encouraged by someone in your personal life, other than your partner, to participate in these classes? |
D12b. Who encouraged you to participate in the classes? |
D19a. Did the classes or workshops help you to improve your relationship with your child(ren)? |
D19b. Why do you think these improvements happened? |
D19c. What knowledge or skills did you learn that helped you improve your relationship with your child(ren)? |
D19d. Overall, would you say the classes or workshop was too long, about right, or too short? |
D19e. How would you rate the teacher or facilitator of the classes? |
D19f. Would you recommend these classes to others? |
G7a. When we last interviewed you in (MONTH AND YEAR), we asked you some questions about your child, (NAME). Does (CHILD) still live with you? |
G7b. Where does (CHILD) live? |
G7c. Why does (CHILD) no longer live with you? |
G7d. In the past month, how often have you spent one or more hours a day with (CHILD)? |
G7e. How much influence do you have in making major decisions about (CHILD), such as (HIS/HER) education, medical care, and religion? |
G7f. Thinking back to when (CHILD) was born, what was the status of your relationship with (CHILD’s) biological (MOTHER/FATHER)? |
G7h. Were you ever married to (CHILD’s) biological (MOTHER/FATHER)? |
G8a. Did (CHILD) graduate from high school or does (CHILD) have a GED? |
G8b. What is (CHILD) doing now? |
G29a. Does (CHILD) have any children of (HIS/HER) own? |
G29b. As far as you know, has (CHILD) ever attended any classes about healthy dating or healthy relationships? |
Continued
Table 1. Questionnaire Additions and Deletions (Continued)
Questionnaire Deletions |
B8. Did you talk to someone, such as a friend, a family member or someone at work about the message of this advertising? |
H1. (IF BASELINE RESPONDENT) Do you consider yourself Hispanic, Latino(a) or Spanish? |
H2. (IF BASELINE RESPONDENT) Do you consider yourself White or Caucasian, Black or African American, American Indian or Alaskan Native, Asian, Native Hawaiian or Other Pacific Islander, or some other race? |
H3. (IF BASELINE RESPONDENT) In what country were you born? |
H4. (IF BASELINE RESPONDENT) In what year did you first come to live in the United States? |
I1. (IF BASELINE RESPONDENT) Is (SPOUSE/PARTNER) Hispanic, Latino(a) or Spanish? |
I2. (IF BASELINE RESPONDENT) Is (SPOUSE/PARTNER) White or Caucasian, Black or African American, American Indian or Alaskan Native, Asian, Native Hawaiian or Other Pacific Islander, or some other race? |
I3. (IF BASELINE RESPONDENT) In what country was (SPOUSE/PARTNER) born? |
I4. (IF BASELINE RESPONDENT) In what year did (SPOUSE/PARTNER) first come to live in the United States? |
I19. (IF BASELINE RESPONDENT) Before (SPOUSE/PARTNER) turned 18, most of the time did (HE/SHE) live with…? |
I20. (IF BASELINE RESPONDENT) Were (HIS/HER) biological or adoptive parents ever married to each other? |
This information collection will provide longitudinal data on the characteristics of persons living in the CHMI treatment communities and comparison communities. Additionally, the treatment sites will have a supplemental sample of persons who participated in a CHMI program. The addition of this participant sample will add precision to the treatment-comparison site analysis and will increase our ability to detect potential impacts of CHMI. Outcomes of interest include marriage quality and stability, attitudes toward marriage, awareness of media messaging, social ties, and household self-sufficiency. The data will provide a means for assessing outcomes and impacts of a wide array of marriage and relationship education and related services offered in the treatment communities.
To reduce the burden on respondents, ensure consistent administration and maximize data quality through the provision of built-in editing, the survey will be administered via computer-assisted personal interviewing (CAPI) using encrypted laptop computers. A web-based control system will be used to track survey completion and refusal rates, case and event information, and receipt of incentive payments.
We are aware of no other studies gathering data from community members to assess the impact of a range of services in support of healthy marriages and relationships. Some organizations awarded healthy marriage grants by ACF have evaluators working with them to document project activities and outcomes. However, these projects are not evaluating the net impact of these services at the community level.
ACF is sponsoring additional rigorous evaluations of healthy marriage and relationship services to specific target populations: low-income unwed parents and married couples. These evaluations will measure the net impact of the services on adults and children in the research sample, not community-level effects.
There is no expected involvement of small entities including small businesses, local governments, or other small entities. Respondents are individuals in selected geographical areas.
In order to assess impacts, data on the characteristics of both the treatment and comparison communities after services are provided in the treatment communities must be collected at least once. In the initial submission, ACF suggested that the CHMI evaluation would include two waves of follow-up data collection. However, the current plan is to conduct a single follow-up 24 months after baseline data were collected. The ACF grants supporting healthy marriage education and related programming in the treatment sites are 5-year grants. Conducting follow-up data collection at 24 months after baseline provides sufficient time for program activities in treatment communities to have reached a “steady state” of operations.
A.7 Special Circumstances
There are no special circumstances associated with this data collection.
In accordance with the Paperwork Reduction Act of 1995 and Office of Management and Budget regulations, ACF published a notice in the Federal Register announcing the agency’s intention to collect this information. The Federal Register notice providing a 60-day comment period was published on October 17, 2008, Vol 73, page 61875. No comments were received. A copy of this notice can be found in Attachment A.
Comments on the baseline data collection were sought from the project’s technical work group and carried over into Wave 2. A list of the work group members is shown below:
Lorraine Blackman, Ph.D., Indiana University School of Social Work
Irwin Garfinkel, Ph.D., Columbia University School of Social Work
Nicholas Ialongo, Ph.D., Bloomberg School of Public Health
Howard Markman, Ph.D., Center for Marital and Family Studies, University of Denver
Rebecca Maynard, Ph.D., University of Pennsylvania
Ronald Mincy, Ph.D., School of Social Work, Columbia University
Robert Moffitt, Ph.D., Johns Hopkins University
Jeffrey Smith, Ph.D., University of Michigan
In addition to the work group members, we solicited feedback on the baseline survey instrument from five survey methodologists and substantive experts. The list is shown below:
Steven Durlauf, Ph.D., University of Wisconsin at Madison
Norval Glenn, Ph.D., University of Texas at Austin
Kristin Moore, Ph.D., Child Trends
Steven Nock, Ph.D., University of Virginia
Theodora Ooms, Ph.D., Center for Law and Policy
There is no change proposed in the amount of the respondent payment. As with the baseline data collection, a $25 payment is planned to offset respondent burden.
There is no change in the assurance of privacy. Each potential participant in the follow-up survey will receive and have read to them a statement of informed consent (see Attachment B). This statement will explain the study and will assure them of their privacy to the fullest extent under the law and rights as respondents. Specifically, the reference to privacy reads:
“We will keep what you tell us in the interview private to the extent allowed by law. After you complete the interview, the interviewer will not be able to look at your answers again. Only the researchers at RTI International will see the information we have collected from study participants. We will combine your information with information from all of the other participants to create group statistics. Also, we will do the interview in a private setting where no one can overhear your answers. We will create an identification (ID) number and use it instead of your name to identify your interview in the computer, which will prevent anyone from finding out what your answers were. Although every effort will be made to keep research records private, there may be times when federal or state law requires the disclosure of such records, including personal information. This is very unlikely, but if disclosure is ever required, RTI will take all steps allowable by law to protect the privacy of personal information.
There are some exceptions to our promise to keep this information private. If you tell us that you are in immediate danger or that you intend to harm yourself or someone else, we may need to inform the appropriate authorities according to state and local law.”
The second wave of information collection asks almost all of the questions from the currently approved instrument. As shown in Table 1, none of the new questions are of a sensitive nature.
As with the baseline data collection, all interviews will be conducted in a private setting so that answers will not be overheard. Respondents will be informed that they may refuse to answer any question they do not wish to answer.
The estimates of respondent hour burdens for the second wave of information collection are shown in Table 2. The interview is expected to average 45 minutes in length, varying between 30 and 60 minutes depending on the respondent’s circumstances. For example, married, employed persons with children will have more questions asked than single, unemployed persons without children. No costs are estimated because respondents are individuals.
Table 2. Respondent Burden
Instrument |
Number of Respondents |
Number of Responses per Respondent |
Average Burden Hours per Response |
Total Burden Hours |
Wave 2 Survey |
4,120 |
1 |
0.75 |
3,090 |
Estimated Total |
|
|
|
3,090 |
The total number of respondents is based on the following assumptions: we will achieve an 80 percent response rate among those continuing from Wave 1/baseline (4,024*.80 = 3,220), and a 75 percent response rate among the participant sample (1,200*.75 = 900), equaling 4,120 respondents (3,220 + 900 = 4,120). We expect the Wave 2 response rates to be higher than the baseline response rate (63%) because we are following respondents in Wave 2 with whom we have built rapport and have an established relationship. In Wave 1, we were soliciting cooperation from individuals with whom we had no prior or programmatic ties.
There are no costs to respondents other than their time to participate in the study.
Total costs associated with the CHMI follow-up survey are estimated to be about $4,000,000, including costs associated with sampling, data collection, processing, and analysis and reporting over about a 24-month period of performance. The annualized cost is about $2,000,000.
The modest reduction in burden is a result of a program change. The reduction in the burden estimate results from not needing to administer a screening interview to establish an individual’s eligibility for the Wave 2 survey. All Wave 2 sample members are known to be eligible. The original burden estimate was 3,332 hours. The current estimate is 3,090 hours.
The operational schedule for the Wave 2 survey for data collection, analysis, and reporting is shown in Table 3.
The analysis plan has not changed from that included in the initial submission. The analyses will include examination of program implementation and neighborhood context and changes from Wave 1/baseline in public perceptions and attitudes toward marriage, parenting and out-of-wedlock birth, marriage and divorce rates, and awareness of media messages. It will also include examination of whether and how baseline differences between treatment and comparison sites have changed over time. Findings from the Wave 2 survey will be summarized in a report that presents descriptive and multivariate results, including overall analyses, site-specific analyses, and cross-site analyses.
Table 3. Time Schedule
Activity |
Time Frame |
Train Interviewers |
September 2009 |
Collect Data |
October 2009–April 2010 |
Prepare Data (cleaning, coding, weighting) |
May 2010–July 2010 |
Analyze Data |
August 2010–February 2011 |
Prepare Report |
March 2011–July 2011 |
The OMB expiration date will be displayed on all data collection materials.
There are no exceptions to the certification statement.
| File Type | application/msword |
| File Title | October 15, 2008 |
| Author | dbond |
| Last Modified By | DHHS |
| File Modified | 2009-06-10 |
| File Created | 2009-06-10 |