CHMI baseline survey OMB Justification doc.3.22.07

February 23, 2007

Evaluation of the Community

Healthy Marriage Initiative

Impact Study

Office of Management and Budget Clearance

Package and Data Collection Instrument

Prepared for

Mark Fucello

Richard Jakopic

Office of Planning, Research and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

370 L’Enfant Promenade, SW

Washington, DC 20447

Prepared by

RTI International

701 13^th Street, NW

Suite 750

Washington, DC 20005-3967

RTI Project Number 08957

Summary

Evaluation of the Community Healthy Marriage Initiative Impact Study

The Administration for Children and Families, U.S. Department of Health and Human Services (DHHS), is conducting a demonstration and evaluation called the Community Healthy Marriage Initiative (CHMI). The Personal Responsibility and Work Opportunity Reconciliation Act of 1996 included the formation of two-parent families and the reduction in the share of children born outside marriage as explicit legislative goals. The Deficit Reduction Act of 2005 authorized ACF to award grants to support programs to provide healthy marriage and responsible fatherhood services. Demonstration programs are funded by the Office of Family Assistance (OFA) to support healthy marriage, improve child well-being, and increase the financial security of children. Among those demonstrations, the CHMI projects involve local coalitions that aim to saturate their communities with marriage education and skills training, media messages, and other related activities. The projects will aim to increase the number of healthy marriages and change the norms in the communities to be more supportive toward healthy marriages.

The objective of the CHMI evaluation is to evaluate the community impacts of these interventions on marital stability and satisfaction, and family and child well-being. The study will help DHHS to identify successful models for the interventions and to determine future policy initiatives.

The impact evaluation will assess the effects of healthy marriage initiatives by comparing family and child well-being outcomes over time. Primary data will come from three rounds of in-person data collection. Data will be collected in three CHMI communities and three comparison communities that are well-matched to the CHMI sites.  This request is for the baseline survey (round 1) of the impact evaluation.

The CHMI baseline survey will be administered via computer-assisted personal interviewing (CAPI) to a sample of 12,960 households in the CHMI communities and comparison sites. The sample frame will consist of postal carrier routes and their associated residential addresses purchased from a commercial vendor. Within each site, demographic information about the postal carrier routes will be used to stratify the residential addresses, and systematic sampling will be used to select the addresses.

The baseline survey will collect data from adults 18-49 years of age in the sampled communities. Within households, respondents will be selected at random. Methods to maximize response rates include (1) sending a lead letter in advance of contact, (2) making multiple attempts at contact, (3) offering a $25 incentive, (4) informing community authorities in advance of the survey, (5) offering the interview in Spanish, and (6) training interviewers in refusal aversion and conversion. An interview completion rate of 75 percent is anticipated, along with a combined screening/interview rate of about 60 percent. The potential for nonresponse bias will be examined using information from external sources, and frame information about respondents and nonrespondents.

Data collection is scheduled for the period July-December 2007. A pretest of the survey questionnaire using 9 respondents was conducted and problems with question wording, flow, and order were identified and corrected.

The average length of the screening interview is 4 minutes, and the average length of the main interview is 40 minutes. This is a new data collection for ACF.

TABLE OF CONTENTS

A. Justification 1

A.1 Circumstances of Information Collection 1

A.2 Purpose and Use of Information 4

A.3 Use on Information Technology 4

A.4 Efforts to Identify Duplication 4

A.5 Involvement of Small Entities 5

A.6 Consequences if Information Collected Less Frequently 5

A.7 Special Circumstances 5

A.8 Consultation Outside the Agency 5

A.9 Payment to Respondents 6

A.10 Assurance of Privacy 8

A.11 Questions of a Sensitive Nature 8

A.12 Estimates of Hour Burden 8

A.13 Estimates of Cost Burden to Respondents 10

A.14 Estimates of Annualized Cost Burden to the Government 10

A.15 Changes in Burden 10

A.16 Time Schedule, Publication and Analysis Plans 10

A.17 Display of Expiration Date 11

A.18 Exceptions to Certification Statement 11

B. Collections of Information Employing Statistical Methods 12

B.1 Respondent Universe and Sampling Methods 12

B.2 Information Collection Procedures 21

B.3 Methods to Maximize Response Rates 24

B.4 Tests of Procedures 25

B.5 Statistical Consultants 26

LIST OF TABLES

1 CHMI Research Questions, Outcome Measures, and Data Sources 2

2 Respondent Burden 9

3 Time Schedule 10

4 Summary of Proposed Sample Design 13

5 Required Sample Size Assumptions 15

ATTACHMENTS

A 60-Day Federal Register Notice 28

B 30-Day Federal Register Notice 31

C Informed Consent Form 33

D Letter of Authorization 36

E Lead Letter 38

F Question & Answer Brochure 40

G Appointment Card 43

H “Sorry I Missed You” Card 45

I Incentive Receipt 48

J Hotline Numbers 50

K Refusal Conversion Letter 52

L Verification Letter 54

M Screener and Survey Questionnaire 56

A1. Circumstances of Information Collection

The Administration for Children and Families, U.S. Department of Health and Human Services, is conducting a demonstration and evaluation called the Community Healthy Marriage Initiative (CHMI). The Personal Responsibility and Work Opportunity Reconciliation Act of 1996 (PRWORA) included the formation of two-parent families and the reduction in the share of children born outside marriage as explicit legislative goals. The Deficit Reduction Act of 2005 authorized ACF to award grants to support programs to provide healthy-marriage and responsible fatherhood services. Demonstration programs are funded through the Office of Family Assistance (OFA) to support healthy marriage, improve child well-being, and increase the financial security of children. One of the priority areas in the OFA healthy marriage demonstration program announcement invited applications for grants to implement Community Healthy Marriage Initiatives. The objective of the CHMI evaluation is to evaluate the community impacts of these interventions on marital stability and satisfaction, and family well-being and child well-being among low-income families.

The CHMI projects generally involve local coalitions that aim to saturate their communities with marriage education, relationship skills training, media messages, and other related activities. In so doing, the projects aim to increase the number of healthy marriages, reduce divorce, and change the norms in the communities to be more supportive toward healthy marriages. Although each site has its specific mix of services, all attempt to engage a coalition of public and private, secular and faith-based organizations to sponsor their own activities and promote the overall goals of the initiative. All are trying to implement community-level strategies to encourage healthy marriages and parenting, and thereby to generate benefits for children as well as couples.

These efforts to support healthy marriages and responsible parenting are based on an evolving literature that links unmarried parenthood to a range of negative outcomes for children and adults. Over half of poor families with children are not headed by a married couple. Chronic poverty is even more concentrated among families headed by unmarried parents. The economic and other disadvantages related to the absence of a parent often hurt children’s academic performance and limit their long-term economic and social well-being (e.g., see McLanahan and Sandefur, 1994). Unfortunately, the share of children living with married, biological parents has declined dramatically over recent decades, especially in low-income and minority communities.

Married-couple families with children have much higher living standards and are less poor than other families. Even among poor and near-poor families with children, married couples are better able to avoid material hardship than families headed by unmarried parents (Lerman, 2002; Lerman, 1996; Thomas and Sawhill, 2001; Lichter, Graefe, and Brown, 2001). Among couples with the same earnings capacity, the economic advantages of marriage over cohabiting remain significant (Lerman, 2001). Among the plausible reasons for economic advantages of married couples relative to similar cohabitating couples are support from family members and a heightened incentive to earn, sometimes called the marriage premium.

The CHMI Evaluation is one component of a research strategy aimed at testing the efficacy of individual service and community efforts to promote healthy marriages. Two other components (Building Strong Families and Supporting Healthy Marriage) involve random assignment experiments to estimate the impact of a specified set of marriage education services on the specific target populations. The focus of CHMI is on the community. The community approach hypothesizes that healthy marriages should be supported not only through direct services used by individuals and couples but also through the broader local environment, including the messages delivered by media, clergy and other opinion leaders, and the experiences of peers and family members. RTI International and its subcontractors, the Urban Institute and Crider Associates, are responsible for the data collection and analysis of the CHMI evaluation.

Primary data for the impact evaluation will come from three rounds of in-person data collection. The impact evaluation will assess the effects of healthy marriage initiatives by comparing family and child well-being outcomes over time in three CHMI communities with similar outcomes in three comparison communities that are well-matched to the CHMI sites.  The impact evaluation will be integrated with the implementation study (approved by OMB in 2005) that involves a qualitative assessment of CHMI activities and the collection of programmatic and administrative data. This request is for the baseline survey (round 1) of the impact evaluation.

Table 1 below lists the primary research questions of the evaluation, the outcome measures, and the data sources that will be used to answer the research questions, estimate impacts, and provide contextual information for understanding the impacts of the CHMI interventions.

Table 1

CHMI Research Questions, Outcome Measures, and Data Sources

Evaluation Research Questions	Outcome Measures	Data Sources
Did the interventions alter attitudes about marriage, non-marital childbearing, father involvement, and divorce?	Proportion of individuals who place high values on marriage, father involvement and low values on non-martial childbearing and divorce.	Survey data on attitudes concerning the value of marriage
Did the interventions increase the share of people using marriage education and other marriage-related activities?	Proportion of individuals reporting the use of marriage education and other marriage-related services (such as counseling from clergy)	Survey data on participation in marriage education and other marriage-related activities
Did the interventions alter the message people hear from media, friends, and family about marriage, divorce, and non-marital childbearing?	Proportion of individuals reporting messages from media, friends, and family members concerning the value of marriage and the importance of raising children in two-parent families	Survey data on reception of media messages and discussions with friends, family, and others in the community about marriage, divorce, and the bearing and raising of children outside marriage
Did the interventions increase the proportion of children in the community living with their own two parents in a healthy marriage?	Proportion of children living with their own 2 parents Proportion of children living with parents in a healthy marriage	Administrative data on marriage and divorce rates Program data on participation in marriage-education and related services Survey data on marital quality and stability, parenting, living arrangements for children, nonresidential parents
Did the interventions increase the number and proportion of healthy married adults?	Proportion of married adults in community Proportion of healthy marriages	Program data on participation in marriage-education and related services Administrative data on marriage and divorce rates Survey data on marital status, quality, stability, conflict, domestic violence, attitudes toward marriage
Did the interventions increase marital quality?	Changes in marriage quality measures	Survey data on marital quality, stability, conflict, domestic violence, satisfaction Program data on reports of marital quality
Did the interventions contribute to marital stability?	Changes in marriage stability Length of marriage	Survey data on marital quality, stability Administrative data on marriage and divorce rates
Did the interventions change the rates of marriage formation and dissolution?	Marriage rates Separation rates Divorce rates Cohabitation rates	Administrative data on marriage and divorce rates Survey data on marital history, status, stability
Did the interventions increase overall child well-being?	Changes in child well-being measures Rates of child abuse and neglect reports	Survey data on child behavior, school attendance, parental involvement, attitudes toward parenting Administrative data on rates of child abuse and neglect
Did the interventions increase the financial well-being of families with children?	Employment status Income TANF participation	Administrative data on employment and earnings, and receipt of benefits Survey data on employment, family income, receipt of benefits, household self-sufficiency
What was the financial effect of the interventions on families?	Economic well-being measures	Survey data on employment, family income, receipt of benefits, household self-sufficiency
Did the interventions increase child support payments?	Proportion of custodial parents receiving a child support payment and amount of support payments	Survey data on child support payments received; satisfaction with payment

A2. Purpose and Use of Information

This information collection will provide baseline information on CHMI impact communities and comparison communities that describe characteristics of the respondents that are not available through secondary data. These characteristics include marriage quality and stability, attitudes toward marriage, awareness of media messaging, social ties, and household self-sufficiency. In conjunction with other secondary data sources, the CHMI data will provide a means for assessing outcomes and impacts of marriage and relationship services. This study will help ACF identify successful models for interventions and to help determine future policy initiatives. The questionnaire asks for the minimum information necessary to meet the needs of Federal policy makers and the marriage/relationship research communities.

A3. Use on Information Technology

The CHMI baseline survey will be administered via computer-assisted personal interviewing (CAPI) to a sample of 12,960 households in the CHMI communities and comparison sites. The use of CAPI will ease interview administration, ensure consistent administration, and maximize data quality through the provision of built-in editing. Anonymity and confidentially will be easier to protect using a computerized instrument compared with using a paper instrument. The data stored on the laptops will be encrypted and password-protected. Moreover, the need for subsequent data entry will be eliminated, which reduces the potential for human error.

This project will also use a web-based control system designed to track data. This system will be created and implemented by RTI and will allow staff to monitor and record information regarding eligibility and participation rates, case and event information, and receipt of incentives. The advantages of using a web-based system include the ability to access information from various locations, increased security as a result of login and identification requirements, and flexibility in terms of processing and reporting on the data.

A4. Efforts to Identify Duplication

In 1996 Congress published findings based on existing research into topics pertaining to marriages and relationships (Section 101 of Pub. L. 104-193). Specifically, it held that “marriage is the foundation of a successful society” and that “marriage is an essential institution of a successful society which promotes the interests of children.”

In order to encourage the strengthening of existing marriages and the creation of future healthy marriages, the Deficit Reduction Act of 2005 created funding in the amount of $150 million each year for five years for programs promoting healthy marriage and responsible fatherhood. Key requirements of the law specify that:

Funds may be used for competitive research and demonstration projects to test promising approaches to encourage healthy marriages and promote involved, committed, and responsible fatherhood by public and private entities and also for providing technical assistance to States and Tribes.

• Applicants for funds must commit to consult with experts in domestic violence; applications must describe how programs will address issues of domestic violence and ensure that participation is voluntary.

• Healthy marriage promotion awards must be used for eight specified activities, including marriage education, marriage skills training, public advertising campaigns, high school education on the value of marriage and marriage mentoring programs.

• Not more than $50 million each year may be used for activities promoting fatherhood, such as counseling, mentoring, marriage education, enhancing relationship skills, parenting, and activities to foster economic stability.

Because CHMI grants were not awarded until September 2006, there has been little opportunity for evaluation research of large-scale, community-wide efforts. It is therefore the case that this evaluation will be an assessment of unexamined information and is unlikely to provide data that has been or could be obtained through other means.

A5. Involvement of Small Entities

For this evaluation, RTI will conduct in-person field interviews with persons living in the sampled communities. There is no expected involvement for small entities including small businesses, local governments, or other small entities. This project will involve only households in selected geographical areas.

A6. Consequences if Information Collected Less Frequently

Three rounds of data collection are planned for the evaluation – at baseline, 18 months, and 36 months. CHMI programs are funded for a five year period. The evaluation is designed to assess impacts at an early and later stage of program operations. If data were collected less frequently, then information about whether the funding set aside by Congress is being used to achieve its goals will be delayed. In addition, having information about program impacts while operations are underway will provide sites the opportunity to make service improvements.

A7. Special Circumstances

This study will be conducted based on the guidelines put forth in CFR Title 5, Section 1320.5. There will be no special circumstances.

A8. Consultation Outside the Agency

The 60-day Federal Register notice published on August 15, 2006 yielded no comments on the data collection. A copy of the 60-day notice can be found in Attachment A.  The 30-day Federal Register notice published on January 8, 2007 yielded two requests for a copy of the survey instrument, which were provided. A third comment questioned the value and utility of the research but did not require a response. A copy of the 30-day notice is provided in Attachment B.

Additional comments on the data collection were sought from the project’s ongoing technical work group. This group met in May 2005 to discuss design and evaluation issues. A list of those members is shown below:

Technical Work Group Members

Lorraine Blackman, Ph.D., Indiana University School of Social Work

Irwin Garfinkel, Ph.D., Columbia University School of Social Work

Nicholas Ialongo, Ph.D., Bloomberg School of Public Health

Howard Markman, Ph.D., Center for Marital and Family Studies, University of Denver

Rebecca Maynard, Ph.D., University of Pennsylvania

Ronald Mincy, Ph.D., School of Social Work, Columbia University

Robert Moffitt, Ph.D., Johns Hopkins University

Jeffrey Smith, Ph.D., University of Michigan

In addition to the work group members, we solicited feedback on the survey instrument from four survey methodologists and substantive experts. The list is shown below:

Consultants on CHMI Questionnaire

Steven Durlauf, Ph.D., University of Wisconsin at Madison

Norval Glenn, Ph.D., University of Texas at Austin

Kristin Moore, Ph.D., Child Trends

Steven Nock, Ph.D., University of Virginia

A9. Payment to Respondents

The incentive plan for the CHMI baseline survey includes a $25 payment to respondents as a token of appreciation for their cooperation. The cash payment will be provided to respondents who make a good faith attempt to complete the 40-minute interview, even if they do not finish the interview or refuse significant numbers of questions.

Offering cash incentives to respondents will provide significant advantages to the study, including:

• Increased responses both from the baseline cohort and the panel cohort that will be followed up in survey rounds 2 and 3

• Reduced potential bias

• Reduced data collection costs

The justifications for the incentive plan may be evident in the makeup of the CHMI sample. In short, the survey sample is drawn randomly among adults 18 to 49 years of age for cross-sectional (one-time) and panel (longitudinal) surveys. The sample will include several non-mutually exclusive subgroups that are of interest to the evaluation: (1) low-income families; (2) unwed or married expectant or new parents; (3) linguistically isolated families; (4) immigrant families; (5) young adults; and (6) families with special needs. These respondent characteristics represent difficult and hard-to-reach populations.

Using incentives for a survey of this kind is supported by the final report at the Symposium on Providing Incentives to Survey Respondents (1992) that was sponsored jointly by OMB and the Council of Professional Associations on Federal Statistics. The report recommended that OMB “seriously consider the use of incentives” for surveys that targeted difficult respondent populations, including surveys…

• that have “small subpopulations of interest”

• where response is affected by relatives who serve as gatekeepers to respondent access

• for which participation may incur out-of-pocket costs to the respondent (e.g. baby sitting costs)

• that are part of longitudinal panels

For the CHMI baseline survey, the provision of a $25 cash incentive will make it convenient for a larger number of low-income sample members to make time for the interview during the defined data collection period, as opposed to using the time to take care of chores or even generate income. This is confirmed by past research that showed that incentives are more effective for low-income households (Singer 2002), although the evidence is mixed.

Additionally, gatekeepers for selected respondents or respondents with limited English language skills may prevent the respondents from knowing about the study. The incentive payment will help us emphasize the importance of their participation.

Furthermore, incentives may counteract resistance among some sample members to participate in future waves of the survey (if they are selected to be on the panel). This retention is important in reducing bias because it will ensure that subgroups with limited representation in the sample participate at a sufficient rate to maintain the overall representativeness of the panel.

In addition to using payment of incentives as a technique to expand response rates, it can also provide cost savings to the government. The National Survey on Drug Use and Health (NSDUH), for example, attributed a net saving of $9 per interview and an increase in the weighted overall response rate from 68% to 71% for the 2002 survey when the use of $30 incentive was approved by OMB. We believe that our remuneration plan is a cost effective way to collect data for the CHMI baseline survey as it will reduce field travel costs and interviewer time required for repeated attempts at contact and refusal conversion. As Warriner et al. (1996) noted, “At issue is not only the expense of the cash outlay for incentives but their effect on other fieldwork costs as well. The costs of follow-ups…means that some of the costs of incentives may be underwritten if an early response negates the need for further reminders.”

In summary, we believe that $25 is a reasonable compensation for respondent burden associated with completing the interview during the data collection period. It will also ensure the collection of useful, cost effective, and policy relevant data for the CHMI evaluation.

A10. Assurance of Privacy

Each potential participant in the baseline survey will receive and have read to them a statement of informed consent (see Attachment C). This statement will explain the study and will assure them of their privacy and rights as respondents. Specifically, the reference to confidentiality reads:

Privacy

We will keep what you tell us in the interview private to the extent allowed by law. After you complete the interview, the interviewer will not be able to look at your answers again. Only the researchers and project staff at RTI International will see the information we have collected from study participants. We will combine your information with information from all of the other participants to create group statistics. Although every effort will be made to keep research records private, there may be times when federal or state law requires the disclosure of such records, including personal information. This is very unlikely, but if disclosure is ever required, RTI will take all steps allowable by law to protect the privacy of personal information.

Exceptions to Privacy

There are some exceptions to our promise to keep this information private. If you tell us that you are in immediate danger or that you intend to harm yourself or someone else, we may need to inform the appropriate authorities according to state and local law.

A11. Questions of a Sensitive Nature

The baseline survey includes some questions that may be considered sensitive. Potential respondents are cautioned prior to their participation that “some questions might make [them] feel uncomfortable or upset.” These questions are about domestic violence and infidelity. To minimize the sensitivity, only a few questions are asked about each topic. The questions about domestic violence and infidelity are indicators of marriage quality and stability, which are key outcome variables for the evaluation.

In addition, we will collect locating information including social security numbers (SSNs) and birth dates from respondents. Because a portion of the baseline respondents will be followed up longitudinally, it is imperative that respondents can be found at a later date. SSNs will be necessary for searching electronic databases to learn the respondents’ location in future years. Birth dates will confirm that the correct respondent has been located.

All interviews will be conducted in a private setting so that answers will not be overheard. Respondents will be informed that they may refuse to answer any question they do not wish to answer. Data will be collected on laptop computers that are fully encrypted and password-protected. Once the interview is complete, the interviewer will not be able to access the data again. Interview data will be transmitted daily to RTI via secured data transmission.

A12. Estimates of Hour Burden

There will be two steps in the baseline survey data collection: the screening process and the interview process. A sample frame of households will be constructed using postal lists; systematic sampling will be used to select households for inclusion in the sample. The household screening instrument includes a brief explanation of the study, and questions to confirm the address of the sample dwelling unit (SDU), to determine missing addresses, to determine eligibility, and to randomly select a respondent. Adults are considered eligible if they are between the ages and 18 and 49. The average length of the screening interview is 4 minutes. The length varies from 3 to 5 minutes depending on whether a household is eligible or not – only eligible households go through the respondent selection process.

If an eligible respondent agrees to participate, a CAPI interview will be administered.

The interview consists of a series of modules, or groups of questions based on similar topics (e.g., marital quality, child well-bring, awareness of media messages, receipt of services, respondent characteristics, and household self-sufficiency.) The interview averages 40 minutes in length, and varies between 30 and 55 minutes depending on the respondent’s circumstances. For example, married, employed persons with children will have a longer interview than single, unemployed persons without children.

The respondent burden for the CHMI baseline survey is shown in Table 2.

Table 2

Respondent Burden

Instrument	Number of Respondents	Number of Responses Per Respondent	Average Burden/ Response	Total Burden (hours)
Screening	9,331	1	.06	560
Interviewing	4,200	1	.66	2772

Estimated Total Annual Burden Hours: 3,332

The number of respondents to the screening interview (9,331) is based on a sample size of 12,960, an occupied rate of 90%, and a completion rate of 80% (12,960*.9*.8 = 9,331) The number of respondents to the main interview (4,200) is based on 9,331 respondents to the screener, an eligibility rate of 60%, and a response rate of 75% (9,331*.6*.75 = 4,200).

The total burden to all respondents for the screening instrument is therefore estimated to be 560 hours. The total burden to all respondents for the interviewing instrument is therefore estimated to be 2772 hours. The estimated total burden hours for both instruments combined is 3,332.

A13. Estimates of Cost Burden to Respondents

There are neither capital or startup costs nor are there any operation and maintenance costs to respondents.

A14. Estimates of Annualized Cost Burden to the Government

Total costs associated with the CHMI baseline survey are estimated to be $3,997,141 for sampling, data collection, processing, and analysis over a 24-month period of performance. The annualized cost is approximately $1,998,570.

A15. Changes in Burden

This is a new data collection for the Administration for Children and Families.

A16. Time Schedule, Publication, and Analysis Plans

The operational schedule for the CHMI baseline survey, including site selection, sampling, data collection, analysis, and reporting is shown in Table 3 below.

Table 3

Time Schedule

Activity	Time Frame
Select Sites for Evaluation	November 2006 to March 2007
Select Sample	March 2007 to April 2007
Program and Test Instrument	January 2007 to May 2007
Hire/Recruit Interviewers	March 2007 to May 2007
Prepare Control Systems	January 2007 to May 2007
Train Interviewers	June 2007
Collect Data	July 2007 to December 2007
Analyze Data	January 2008 to April 2008
Prepare Report	April 2008 to August 2008

Findings from the baseline survey will be summarized in a report that presents descriptive and multivariate results, including overall analyses, site-specific analyses, and cross-site analyses. The memorandum will examine baseline marriage and relationship information, public perceptions and attitudes to marriage, parenting and out- of-wedlock birth, and present information on neighborhood context, access, and awareness of media messaging. The focus of the baseline analysis will be on comparisons with matched comparison sites; further comparisons will be made through later waves of data collection which will allow us to see how baseline differences may change over time.

The evaluation will assess community impacts of the interventions with approaches that increase in complexity using data from subsequent data collections as compared to this baseline collection. We will pilot all our analytic methods using the baseline data, to the extent that this is possible. In later years, approaches that we will use include pre/post comparisons, difference-in-difference models, instrumental variables and ordinary least squares models.

The analysis plan includes several different methods, including difference-in-difference (DD and DDD) methods. These methods involve comparing changes in outcomes in a CHMI community with changes in the associated comparison site or sites. A related approach is to add differences in outcomes between subgroups that were the focus of CHMI activity (say, new parents) and other groups. In this case, we assess how the difference between one group and another changed in the CHMI site as compared to the comparison site. We can implement these DD and DDD models in a regression context to control for other characteristics that may change over time. One can pool the site by time by subgroup outcomes and estimate regressions of the combined impact of CHMI sites after implementation on various outcomes. However, one difficulty with pooling is that CHMI sites are likely to differ markedly in their service mix, coverage, target groups, and community settings. A second issue is that comparison sites may themselves have marriage-related activities. Pooling the observations across sites will depend on the specific analysis and on the nature of the site differences. Differences in the mix of activities across CHMI sites might make it hazardous to pool cases in models using CHMI or CHMI-time interactions as the primary impact variable. Pooling is more appropriate for the models that use access to specific marriage-related services, defined similarly across sites, as their primary impact variables.

One way to take account of the variability in the nature of marriage-related services is to derive CHMI effects in a two-stage instrumental variable (IV) model. The first stage estimates the determinants of exposure to marriage-related services either as a function of CHMI residence in the post-implementation period or as a function of the accessible supply of marriage-related services. This accessibility measure, which is exogenous to marriage outcomes, serves as the instrumental variable. The second stage estimates how exposure to CHMI-type activities affects individuals and couples in terms of such outcomes as healthy marriages and financial well-being. The analytic strategy is a flexible, two-stage, instrumental variable method that not only derives the impact of CHMI on marriage but also the path of the impact in two steps—the CHMI effect on exposure and the effect of exposure on outcomes.

An intermediate strategy between the DD and IV models is to use the accessible supply of marriage-related services to all sample members (both in CHMI and comparison sites) as an independent variable in an OLS model predicting marriage and child well-being. Unless sample members moved to gain access to marriage-related services, this variable should be exogenous. This model has the advantage of simplicity over the comparable IV model and may capture effects of accessible services that are hidden in the IV model. On the other hand, the IV model’s first stage helps us learn how supply translates into exposure to marriage-related services; the second stage potentially provides a causal explanation of how CHMI ultimately affects marriage outcomes.

More detailed information about the outcomes to be measured and methods to be employed will be provided in OMB packages for the follow-up surveys.

A17. Display of Expiration Dates

The OMB expiration date will be displayed on all disseminated data collection materials.

A18. Exceptions to Certification Statement

There are no exceptions to the certification statement.

B1. Respondent Universe and Sampling Methods

The population about which we want to draw inferences is adults between18-49 years of age who live in the CHMI communities (“catchment areas”) and comparison communities. These adults are likely to be in or form couple relationships that may include children -- the population that CHMI projects target.

Because the CHMI and comparison communities will be purposively selected, i.e., not selected at random, the estimates from the survey data will apply only to these specific communities. Estimates from the survey data will not be generalized to any other population.

Survey Design

The CHMI survey design described here is for the baseline survey. The second and third survey rounds will be used to investigate changes over time for the communities in general and for a portion of the respondents to the baseline survey.

Three CHMI communities will be selected for the evaluation of CHMI impacts. These communities will be chosen based primarily on review of Healthy Marriage Demonstration program applications submitted to ACF and on secondary data sources. The selection criteria will derive from aspects of the grant applications and the communities themselves. From the grant applications, we will consider the amount of funding, mix of intended services, the size of the target population, and the probability of the program making an impact in the community. In the communities, we will consider marriage rates, divorce rates, poverty level, and unemployment level. Sites will be selected that demonstrate the strongest likelihood of effecting marital stability, attitudes toward marriage, marital quality, and family and child well-being.

A crucial component of the CHMI evaluation is the selection of well-matched comparison communities. Comparison communities are geographically defined areas that represent, as closely as possible, the characteristics of CHMI communities, except that no CHMI is present. We will use a methodology for choosing comparison sites that involves matching geographic areas using key variables chosen from a review of the literature and expert opinion. The methodology will involve an iterative selection process using Geographic Information Systems (GIS)¹ technology and 2000 Census data. This methodology is closely modeled after an MDRC report by Seith, et al (2003) with several key additions. First, we will measure the degree of spatial autocorrelation present in the key variables in the target community. If variables are spatially clustered in the target community, then comparison communities should be sought that also exhibit spatial autocorrelation or spatial clustering in these variables. This added dimension to the matching will ensure that we do not pair up cultural enclaves in the target group with culturally independent tracts in the comparison group. Second, we will develop a composite score using a linear combination of the key variables in order to better measure the “goodness of fit” of each potential comparison community.

As mentioned, the baseline survey will collect data from adults 18-49 years of age in CHMI and comparison communities. Because there is no complete list of adults in the specified communities, RTI will acquire the postal carrier routes for these communities and the residential addresses that constitute the routes. RTI will purchase both the postal carrier routes and residential addresses information from Marketing Systems Group, Inc. Additional demographic information about postal carrier routes will be purchased from Claritas, Inc.

Within each of the CHMI and comparison communities, RTI will use the demographic information about the postal carrier routes, e.g., counts by age, race, gender, and household composition, to stratify the residential addresses. Next, systematic sampling will be used to select the residential addresses. In order to ensure that there is good coverage of the survey population, RTI will use the procedure called the “half-open interval.” The listings will be defined to include the interval up to, but not including, the next listing for sampling (Kish 1965). This approach allows RTI to improve coverage of the target population by accounting for residential addresses missed by the postal carrier routes. For example, if a new residential housing has been recently constructed or converted from some other type of building, these residential addresses will have an opportunity to be included in the screening process that determines eligibility and, possibly, included in the sample. Administration of the half-open interval procedure is illustrated in the screening instrument (Attachment M).

RTI will screen the residents of the selected addresses to determine eligibility, defined as having at least one adult 18-49 years of age living in the household. In the eligible households, RTI will randomly select one adult 18-49 years of age using the Kish method of respondent selection.

The sample design for the baseline survey is summarized in Table 4 below.

Table 4 Summary of Proposed Sample Design
First Stage: Residential Mailing Addresses
Sampling Frame	All locatable residential mailing addresses in the CHMI and comparison communities
Stratification	Explicit – socioeconomically and racially/ethnically defined neighborhoods, carrier route sequence number, geopolitical entities Implicit – walking sequence
Type of Selection	Systematic sampling (equal probability within community)
Sample Sizes	2,160 addresses per community
SecondStage: Eligible Household Members
Sampling Frame	Roster of eligible household members (aged 18-49)
Stratification	Implicit – sort by gender and age
Type of Selection	Simple random sample
Sample Size	1 eligible resident per household

Precision Requirements

One way to investigate the level of precision for a survey is to examine the power associated with different size differences and different sample sizes. The following graph, produced by PASS software (Hintze 2004), shows the power on the vertical axis, the difference between the two groups on the horizontal axis, and the equal sample sizes for two independent random variables under a two-sided exact test.

The graph shows that sample sizes of 400 or greater can detect a difference of 0.1 between the two independent binomial variables with power 0.8. The sample size of 100 achieves power 0.8 when the difference is 0.2.

Based on the typical fiscal constraint, consideration for modeling procedures, and setting the comparison community proportion to 0.5, i.e., the worst case scenario, RTI concluded that a sample size of 700 for each community would meet the analytic objectives for the aggregated individual level differences expected. When the comparison community estimate is 0.5, RTI will be able to detect differences as small as 0.076 between a CHMI community and a matched comparison community with power 0.8. For example, if there is an estimate of 50% for some characteristic of adults in the matched comparison community, any estimate of 57.6% or greater for the adults in the CHMI community would be detected with power 0.8. Note that these calculations are based on the worst case scenario. Smaller differences will be detected when the matched comparison community estimate moves away from 50%.

Required Sample Size

The required sample size is the desired number of completed interviews divided by the product of the residential address occupancy rate, screening completion rate, eligibility rate, and interview completion rate. The formula is

,

where RSS is the required sample size, C is the desired number of completed interviews, OR is the occupancy rate, SR is the screening completion rate, ER is the eligibility rate, and IR is the interview completion rate. RTI expects a residential address occupancy rate of 90%, a screening completion rate of 80%, an eligibility rate of 60%, and an interview completion rate of about 75% in a CHMI community or a comparison community. Based on these assumptions and the requirement of 700 completed interviews, 2,160 residential addresses will be selected in each CHMI and comparison community. This information is presented in Table 5, Required Sample Size Assumptions.

One other rate of interest is the combined screening/interview rate. This rate is the product of the screening completion rate and interview completion rate. For a community in the CHMI survey, the combined screening/interview rate is about 60% (= 80% * 75%).

Table 5

Required Sample Size Assumptions

CHMI Baseline Survey
Treatment site	Required Completes = 700 Occupied Rate = 90% Screening Completion Rate = 80% Eligibility Rate = 60% Interview Rate = 75% Required Sample Size = 2,160
Comparison site	Required Completes = 700 Occupied Rate = 90% Screening Completion Rate = 80% Eligibility Rate = 60% Interview Rate = 75% Required Sample Size = 2,160
Total Completes Each Pair	1,400
Total Completes - 3 Pairs	4,200
Total Sample Size Each Pair	4,320
Total Sample Size - 3 Pairs	12,960

Sampling Frame

Once the CHMI sites and comparison communities have been selected, RTI will acquire the postal carrier routes and associated residential addresses. This list of residential address will be the sampling frame. RTI expects the list of residential address to be relatively complete and up to date. If there are small numbers of residential address that are not on the frame, the half-open interval procedure will incorporate these residential addresses into the sample.

Sample Selection

Sample selection will be conducted in two stages. The first stage will be the selection of residential address using stratified systematic sampling from the list of residential addresses on the sampling frame. This design will result in an epsem (equal probabilities of selection) sample of households from each stratum. Based on the population and sample sizes (number of addresses or households) we will define a sampling interval, pick a random number within the interval, and select the household corresponding to this random start as the first sample element. Subsequent addition of the sampling interval to this random number, applied repeatedly until we reach the end of the population file, will define the complete sample. Also, the screening will include all residential addresses in the half-open interval identified by the interviewers. The residential addresses that are eligible will be included in the sample. The second stage will be the selection of an adult 18-49 years of age in each sampled residential address using the Kish method.

Residential Address Probability of Selection

The probability of selection for a residential address in a stratum is the number of residential addresses selected into the sample in the stratum divided by the number of residential addresses in the stratum. For residential address i in stratum h, the probability of selection, π_hi, is

,

where N_h is the number of residential addresses in stratum h and n_h is the number of residential addresses selected into the sample in stratum h.

Adult Probability of Selection

The probability of selection for an adult 18-49 years of age in a sampled residential address, the second stage, is one divided by the number of eligible adults at the residential address. For adult j and residential address i, the probability of selection, π_ij, is

,

where A_i is the number adults18-49 at residential address i.

Combined Probability of Selection

The combined probability of selection for an adult is the product of the probability of selection for an adult and the probability of selection for the residential address. For adult j at residential address i in stratum h the combined probability of selection is

,

where π_hi is the probability of selection for residential address i in stratum h and π_ij is the probability of selection for adult j residential address i.

Residential addresses in the half-open interval identified by field staff have the same probability of selection as the other residential addresses in the stratum in which they are located.

Design Weight

The design weight for each sampled adult is the inverse of the combined probability of selection. For adult j at residential address i in stratum h, the design weight, d_hij, is

,

where π_hij is the combined probability of selection for adult j at residential address i in stratum h.

Response Rate Calculations

RTI will calculate response rates in accordance with The American Association for Public Opinion Research (AAPOR) document Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys (AAPOR 2006). Specifically, the response rates will be calculated using response rate calculation number three (RR3). The formula for RR3 is

,

where I denotes completed interviews, P denotes partial interviews, R denotes refusal and break-off, NC denotes non-contact, O denotes other eligible but no interview, e is an estimate of the proportion of cases of unknown eligibility that are eligible, UH unknown if occupied residential address, and UO other unknown. RTI will use the information from the residential addresses with known eligibility to estimate e.

RTI will calculate RR3 for both unweighted counts and, following the recommendation in the section for Some Complex Designs concerning unequal probabilities of selection and multistage designs, weighted counts using the design weight for the sample overall and specified strata at each stage of sample selection. RTI will use a final disposition of case codes consistent with the AAPOR definitions for In-Person Household Surveys. Finally, RTI will consider an interview complete if a core set of 25 questions is answered.

Nonresponse Bias Analysis

Virtually all surveys experience some type of nonresponse. Nonresponse can occur when no information on the sampled unit is collected, called unit nonresponse, or only partial information is collected, i.e., some questions are not answered, called item nonresponse. In either case, estimates of population characteristics from the survey data have the potential for bias. In order to investigate if there is the potential for nonresponse bias in the baseline estimates, RTI will examine various aspects of the sample and external sources of data. These investigations may include some or all of the following four categories: (1) comparisons of survey estimates to external data, (2) evaluation of nonresponse rates, (3) comparisons between respondents and nonrespondents, and (4) comparisons between initial respondents and initial refusers.

Information from External Sources. If appropriate external sources of data can be obtained, RTI will compare the final weighted estimates from the baseline survey to external information to estimate the bias, i.e.,

where is the estimated mean from the baseline survey and is the estimated, or true, mean from the external data source. This estimate of bias incorporates all possible contributions to the bias, e.g., coverage bias. It is not just an estimate of nonresponse bias. One major difficulty with this approach is finding the appropriate external data sources. These external sources could be other surveys, which may have their own biases, or census data.

Information about Response/Nonresponse Rates. If the response rates overall and by specific analytic domain are high, then the potential for biased estimates from the survey data is reduced. Conversely, if the response rates are low, there is a higher potential for biased estimates from the survey data. The calculation of response rates is described in the Response Rate Calculation section. The study team will evaluate response rates for the potential for nonresponse bias and adjust analysis plans accordingly. The nonresponse rate is one minus the response rate.

Information about Respondents and Nonrespondents. If frame information is available for respondents and nonrespondents or other external information, e.g., information about the neighborhood, then either type of information can be compared to estimate the bias using the following approach. Assume we select a simple random sample from a population that consists of two unknown groups: respondents and nonrespondents. For some characteristic of the population, the overall population mean is

,

where is the population mean for the respondents, is the population mean for the nonrespondents, is the number of respondents in the population, is the number of nonrespondents in the population, and is the total population size. Also, let be an approximately unbiased estimator for , i.e., and, if available, let be an approximately unbiased estimator for , i.e., , given this information, the bias of is

.

For the bias of to be small, we must have a small ratio of the number of nonrespondents in the population to the population size, , a small difference between the population mean of the respondents and population mean of the nonrespondents, , or both. Relating this to the actual sample selected, we would have an estimated bias of

,

where is the estimate of nonrespondents in the population, and is the estimate of the total population. For the estimated bias to be small, we must have a small weighted nonresponse rate, a small difference between the weighted sample mean of respondents and the weighted sample mean of nonrespondents, or both.

Information from Respondents Only. In the event that information about nonrespondents is not available, RTI will investigate the differences between initial respondents and initial refusers that eventually respond. The approach is similar to the approach described in the previous section dealing with respondents and nonrespondents. The estimated bias here is the difference between the initial responders and initial refusers, i.e.,

,

where is the mean of the initial respondents and is the mean of the initial refusers that eventually respond.

Post-Survey Weighting

Given that RTI finds no significant nonresponse bias, post-survey weighting will be implemented to produce unbiased estimates by accounting for unknown eligibility and nonresponse. Post-survey weighting will consist of two adjustments: unknown eligibility adjustment and nonresponse adjustment. The unknown eligibility adjustment adjusts the design weights to account for residential addresses for which eligibility cannot be determined. The nonresponse adjustment adjusts the nonresponding unknown eligibility adjusted weights to account for nonresponding adults. The nonresponse adjusted weight is the analysis weight, i.e., the weight used for analysis.

Unknown Eligibility Adjustment. The unknown eligibility adjustment in a stratum is a ratio adjustment. The adjustment factor is the sum of the design weights for eligible, ineligible, and unknown eligibility residential addresses in the stratum divided by the sum of the design weights for eligible and ineligible residential addresses in a stratum. For stratum h, the unknown eligibility adjustment factor, a_h, is

,

where d_hij is the design weight for adult i at residential address j in stratum h, E is the set of eligible residential address, I is the set of ineligible residential addresses, and U is the set of unknown eligibility residential addresses.

The unknown eligibility adjusted weight in a stratum is the unknown eligibility adjustment factor for the stratum multiplied by the design weight for the eligible and ineligible residential addresses in the stratum and 0 for the unknown eligibility residential addresses in the stratum. For adult i at residential address j in stratum h, the unknown eligibility adjusted design weight, e_hij, is

,

where a_h is the unknown eligibility adjustment factor for stratum h and d_hij is the design weight for adult i at residential address j in stratum h.

Nonresponse Adjustment. The nonresponse adjustment in a stratum is a ratio adjustment. The adjustment factor is the sum of the unknown eligibility adjusted weights for eligible residential address, completed interviews and nonrespondents, in the stratum divided by the sum of the unknown eligibility adjusted weights for completed interviews at residential addresses in a stratum. For stratum h, the nonresponse adjustment factor, b_h, is

,

where e_hij is the unknown eligibility adjusted weight for adult i at residential address j in stratum h, E is the set of eligible residential address and R is the set of completed interviews at residential addresses.

The nonresponse adjusted weight in a stratum is the nonresponse adjustment factor for the stratum multiplied by the unknown eligibility adjusted weight for residential addresses with completed interviews in the stratum, 0 for nonresponding residential addresses in the stratum, and 1 for the unknown eligibility residential addresses in the stratum. For adult i at residential address j in stratum h, the nonresponse adjusted weight, f_hij, is

,

where b_h is the nonresponse adjustment factor for stratum h and e_hij is the unknown eligibility weight for adult i at residential address j in stratum h. The nonresponse adjusted weight is the analysis weight.

Estimation

RTI will use SUDAAN^®2 for point estimation and variance estimation. When producing these estimates, SUDAAN^® correctly accounts for the survey design and differential weighting. In contrast, most general software packages expect the data is from a simple random sample and do not correctly calculate point estimates or variance estimates for survey data. The incorrect calculation of variance estimates is particularly troublesome when conducting hypothesis testing.

SUDAAN^® has procedures for producing descriptive statistics for continuous and categorical modeling data. SUDAAN^®’s modeling procedures model dependent variables that are binomial, nominal, ordinal, or continuous using categorical or continuous dependent variables. RTI will use the Taylor series linearization methodology available in SUDAAN to calculate the variances.

B2. Information Collection Procedures

To achieve a high rate of cooperation from eligible households, data will be collected using computer-assisted in-person interviewing (CAPI). In-person interviewing has been shown to yield a higher response rate than other data collection modes because it affords the opportunity to make personal contact with respondents and to make face-to-face appeals for cooperation. The survey instrument will be computer-assisted to maximize data quality, minimize missing data, ease administration, and reduce time for data preparation and processing. Finally, in-person interviewing will allow us to reach respondents who may not be accessible by telephone or who evade calls from research organizations.

Well-qualified field supervisors and field interviewers will be selected from RTI’s National Interviewer File, and supplemented with new hires as necessary to afford full coverage of a site. Field supervisors will attend a 1-day, in-person training session focused on project management responsibilities. Then all field supervisors and field interviewers will attend a 5-day, in-person training session, covering procedures for screening households, selecting a respondent, gaining cooperation, avoiding and converting refusals, administering the interview, and administrative reporting procedures. Training will involve a combination of lecture, demonstration, and hands-on skills practice. All field supervisors and field interviewers will be required to pass a certification exam upon the completion of training, after which they will receive a letter of authorization (see Attachment D). This letter can be shown to individuals who question the survey’s or interviewer’s legitimacy.

Prior to making contact attempts, each sampled address will be mailed a lead letter that explains the study and requests the household’s cooperation. The letter will be sent on RTI letterhead and will bear the electronic signature of the RTI project director. The lead letter will explain how the study data will be used, offer assurances of privacy, and include a telephone number that recipients can call for more information (see Attachment E).

About two weeks after the lead letter is mailed, field interviewers will begin contacting the sampled dwelling units (SDUs). Upon arrival, the interviewer will refer the resident to the lead letter and answer any questions. If the resident has no knowledge of the lead letter, the interviewer will provide another copy. A Question & Answer Brochure (Attachment F) with commonly asked questions about the study may be given to the resident at this time. After these introductions, the interviewer will screen the household for eligibility. Only households with at least one resident between the ages of 18-49 will be eligible for the study. In households in which there is more than one eligible adult, the screener will guide the interviewer to select one respondent at random using the Kish method of respondent selection. To minimize intrusiveness, the screening questions will be administered via a paper and pencil instrument (see Attachment M) so the interviewer does not need to enter the resident’s home to set up the laptop before eligibility is determined. After a respondent has been selected and determined to be at home, the interviewer will ask to be invited into the home to conduct the interview. At the start of the interview, information about household residents gathered during screening will be entered into the survey instrument to verify that the correct respondent was selected.

As necessary and appropriate, the interviewer will make use of an appointment card (Attachment G) to schedule return visits with the selected respondent. If no one is at home at the time of the scheduled appointment, the interviewer will leave a Sorry I Missed You card (Attachment H) informing the resident(s) that the interviewer plans to make another visit at a later date/time. Callbacks will be made as soon as possible. Interviewers will attempt to make at least four callbacks (in addition to the initial call) to each SDU in order to complete the screening process and obtain an interview.

If the selected respondent is currently available, the interviewer will immediately work with the respondent to identify a private setting within the home where informed consent procedures and the interview can take place. Once in a private setting, the interviewer will read the consent form (Attachment C) to the respondent, obtain verbal consent, and give the respondent a blank copy of the consent form to keep.

The interviewer will administer the interview in a prescribed and uniform manner. The questionnaire will be computerized such that the interviewer will read the questions from the screen and enter the respondent’s answers into the laptop. Random portions of each interview will be recorded by the laptop using computer-assisted recorded interview (CARI) technology. Supervisors will review these recordings to ensure interviews are being correctly administered and are not being falsified. Respondents will be informed of the quality control process during the informed consent procedure.

After the interview is completed each respondent will be given a $25 incentive payment and asked to initial a receipt (Attachment I). Respondents who report any domestic violence or who become distressed during the interview will also be provided with a discreet card that lists toll-free hotline numbers (Attachment J).

If a resident refuses to be screened or interviewed, the interviewer will be trained to accept the refusal in a positive manner, thereby avoiding the creation of an adversarial relationship and precluding future contact opportunities. A refusal letter will then be sent by the field supervisor. The refusal letter will be tailored to the specific concerns expressed by the resident and ask him/her to reconsider participation (Attachment K). An in-person conversion will then be attempted either by field supervisors or specially selected interviewers with established conversion records.

At least 10 percent of the completed screeners and interviews will be verified. Supervisors will contact randomly selected respondents to confirm that the screener and/or interview was actually completed and that proper procedures were followed (i.e., screener and interview conducted in person, a consent form was provided to the respondent, the incentive was paid, etc.). Verification letters will be mailed when telephone numbers are unavailable or telephone interviewers are unable to reach the respondent over the phone (Attachment L).

RTI’s project staff will closely monitor interviewer behaviors to ensure compliance with study protocols. Supervisors will receive daily updates on actions taken by interviewers to contact all pending cases. To ensure that cases are worked appropriately, interviewers will transmit case status information to RTI on a daily basis. These data will be processed by a master control system and summarized in web-based reports used to monitor data collection. The control system will allow supervisors to assign and transfer cases to field interviewers to ensure complete coverage of the sample.

During data collection, each supervisor will hold a 1-hour weekly conference with interviewers to discuss progress towards weekly goals, review problem cases, and determine the general priority order for all pending cases. In turn, each supervisor will prepare a written report to the data collection task leader and participate in conference call to address staffing issues and progress and problems at each site. In addition, the data collection task leader will hold a group call with all supervisors to review progress towards production, cost, and schedule goals. The data collection task leader will also provide feedback on the results of in-house data quality procedures such as interview verification calls, quality control checks of preliminary survey data, review of CAPI files, and quality control checks of returned screeners and other data collection forms.

B3. Methods to Maximize Response Rates

We are committed to successfully contacting and screening sample members and achieving the highest possible response rates. As discussed in prior sections, our methods include:

• In-Person Interviewing. We expect to find in general that the sample members are in a lower socioeconomic class than average. Some may not even have regular telephone access. When surveying a hard-to-reach population of this kind, our experience has shown that interviewer-administered mode yields higher response rates than self-administered modes. We will conduct in-person interviews using CAPI because it will help us interview sample members with efficiency.

• Respondent Convenience and Multiple Attempts. We will be flexible in scheduling interviews at the respondent’s convenience. We will also make multiple attempts to reach nonrespondents, including leaving a Sorry I Missed You Card at empty households at the time of the visit. As the study progresses and we learn the best times to reach respondents and make contact attempts, we will adjust our interviewer scheduling accordingly.

• Customized Lead Letters. We will send customized lead letters in advance of fielding to promote respondent cooperation. The lead letter will explain the study objectives, explain that the survey is voluntary, and assure privacy, to the extent provided by law. Moreover, the letter will provide several means for respondents to contact us, including a toll-free telephone number and email address.

• $25 Incentive. We plan to offer a $25 cash incentive to each respondent who makes a good faith effort to complete the survey. We anticipate that this will reduce perceived burden so that respondents will make time for the interview. The incentive payment will also help us emphasize the importance of participating in the study.

• Informing the Community. We will alert community authorities (e.g., the chief of police and authority figures in each community) to the survey. In this way, they will be able to tell respondents that the survey is legitimate.

• Bilingual Approach. We will translate the questionnaire and other respondent materials into Spanish, which is likely to be the most commonly encountered second language in these communities. The bilingual approach, which involves Spanish translation and interviewing, allows respondents with limited English skills to fully understand the nature of their participation. Further, it makes it more likely that those respondents will complete interviews because they will be given the choice of using the language with which they feel more comfortable.

• Comprehensive Interviewer Training. We will provide a multi-day, comprehensive training to the interviewing staff. They will be trained on the study purpose and procedures, interview administration, and the protection of human subjects. Past literature has shown that interviewer effect can be a source for potential survey bias. Therefore, we believe that a thorough understanding of the study and the instrument, and upholding standard protocols and ethical commitment will reduce bias and in turn help interviewers gain respondent trust.

• Refusal Aversion and Conversion. Part of the interviewer training will address in detail specific techniques to avert and convert a refusal from a respondent. Respondents who initially refuse to participate will be assigned to interviewers who have a proven record of turning refusals into completed interviews. Reasons for refusals and barriers to participation will be continually evaluated in light of the experience gained in the data collection process. We will mail letters to respondents and design efficient approaches to target specific reasons for refusing.

• Regular Debriefings with Data Collection Staff. The project management staff will regularly meet with data collection staff to discuss issues related to data collection operations. Methods to enhance response rates will be a standard agenda item at these meetings.

B4. Tests of Procedures

The questionnaire developed for the CHMI baseline survey was based on or taken from recent studies that examined the topics of family and marriage. These studies included the Building Strong Families Project, Supporting Healthy Marriage Project, Fragile Families and Child Well-Being Study, as well as the National Survey of America’s Families, the 2000 Census and others.

Using nine respondents, we conducted a pretest of the survey questionnaire. Respondents were recruited from Raleigh-Durham, North Carolina and resembled our expected survey respondents as much as possible. The field test interviews were administered in-person to replicate the planned data collection mode.

The pretest pointed to the need for five distinct types of changes to the questionnaire. First, questions were deleted to ensure the survey could be completed in an average of 40 minutes. Second, several items were re-worded to address confusion expressed by the respondents. Third, new response options were added to several of the response sets to be more inclusive of the experiences respondents described. Fourth, variable fills were added to some of the questions to acknowledge whether the respondent is male or female, whether the respondent owns or rents their home, and whether a parent had one or multiple children. Finally, show cards were created for respondents to view when questions had lengthy response lists.

In addition, we used RTI’s Question Appraisal System (QAS) to conduct a thorough review of the questionnaire using mock interview scripts. The QAS review focused on question wording, placement, and flow within the questionnaire. The final version of the questionnaire that will be used for fielding the CHMI baseline survey is included in Attachment N.

B5. Statistical Consultants

The following individuals were consulted on the statistical aspects of the survey design:

Dr. Paul Biemer RTI International 919-541-6056

Dr. Karol Krotki RTI International 202-728-2485

Dr. Darryl Creel RTI International 301-770-8229

References

The American Association for Public Opinion Research. 2006. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 4^th ed. Lenexa, KS: AAPOR.

Final Report. 1992. “Providing Incentives to Survey Respondents.” Symposium on Providing Incentives to Survey Respondents.

Hintze, Jerry. 2004. NCSS and PASS. Kaysville, UT: Number Cruncher Statistical Systems.

Kish, Leslie. 1965. Survey Sampling. New York: John Wiley & Sons. Page 56.

Lerman, Robert. 1996. “The Impact of Changing U.S. Family Structure on Child Poverty and Income Inequality.” Economica: Supplement: Economic Policy and Income Distribution. 63 (25OS): S119-39.

__________. 2001. “Marriage as a Protective Force against Economic Hardship.” Paper presented at the 23^rd Annual Research Conference of the Association for Public Policy Analysis and Management, Washington, D.C., November 1-3.

__________. 2002. “Family Structure and Childbearing Before and After Welfare Reform,” in Alan Weil and Kenneth Finegold, eds., Welfare Reform: The Next Act. Washington, DC: Urban Institute Press.

Lichter, Daniel, Deborah Roempki Graefe, and J. Brian Brown. 2001. “Is Marriage a Panacea? Union Formation among Economically Disadvantaged Unwed Mothers.” Paper presented at annual meeting of the Population Association of America, Washington, DC, March 29-31.

McLanahan, Sara and Gary Sandefur. 1994. Growing up with a Single Parent: What Helps, What Hurts. Cambridge, MA: Harvard University Press.

Seith, D., N. Verma, H. Bloom, and G. Galster. 2003. “Exploring the Feasibility and Quality of Matched Neighborhood Research Designs.” MDRC Working Papers on Research Methodology. New York: MDRC.

Singer, E. 2002. “The Use of Incentives to Reduce Nonresponse in Household Surveys.” In Survey Nonresponse, edited by R. Groves, D.A. Dillman, J.L. Eltinge, and R.J.A. Little (163–78). New York: John Wiley.

Thomas, Adam, and Isabel Sawhill. 2001. “For Richer or for Poorer: Marriage as an Antipoverty Strategy.” Brookings Institution Working Paper. Washington, DC.: Brookings Institution.

Warriner, K., Goyder, J., Gjertsen, H., Hohner, P., McSpurren, K. 1996 “Charities, No; Lotteries, No; Cash, Yes: Main Effects and Interactions on a Canadian Incentives Experiment”. Public Opinion Quarterly, 60(4): 542-562.

ATTACHMENT A

60-Day Federal Register Notice

[Federal Register: August 15, 2006 (Volume 71, Number 157)]

[Notices]

[Page 46915-46916]

From the Federal Register Online via GPO Access [wais.access.gpo.gov]

[DOCID:fr15au06-42]

-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Administration for Children and Families

Proposed Information Collection Activity; Comment Request,

Proposed Projects

Title: Evaluation of the Community Healthy Marriage Initiative.

OMB No.: New Collection.

Description: The Administration for Children and Families, United

States Department of Health and Human Services, is conducting a

demonstration and evaluation called the Community Healthy Marriage

Initiative (CHMI). Demonstration programs will be funded to support

healthy marriage directly as well as encourage community changes in

norms that increase support for healthy marriages and improve child and

family well-being. The objective of the impact evaluation is to

evaluate the community effects of these

[[Page 46916]]

interventions on marital stability and satisfaction and child and

family well-being outcomes among low-income families. Primary data for

the impact evaluation will come from three waves of in-person data

collection. This collection is a baseline survey of community members

where CHMI demonstrations are operating, the first of three CHMI

surveys. The impact evaluation will assess the effects of community

healthy marriage initiatives by comparing family and child well-being

outcomes in the CHMI communities with similar outcomes in comparison

communities that are well matched to the demonstration project sites.

Respondents: Community members aged 18-49 in three study sites and

three comparison communities.

Annual Burden Estimates

-----------------------------------------------------------------------

Number of Average burden

Instrument Number of responses per hours per Total burden

respondents respondent response hours

----------------------------------------------------------------------

CHMI Baseline Survey........................ 4,200 1 1 4,200

----------------------------------------------------------------------------------------------------------------

Estimated Total Annual Burden Hours: 4200/

In compliance with the requirements of Section 3506(c)(2)(A) of the

Paperwork Reduction Act of 1995, the Administration for Children and

Families is soliciting public comment on the specific aspects of the

information collection described above. Copies of the proposed

collection of information can be obtained and comments may be forwarded

by writing to the Administration for Children and Families, Office of

Administration, Office of Information Services, 370 L'Enfant Promenade,

SW., Washington, DC 20447, Attn: ACF Reports Clearance Officer. E-mail

address: [email protected]. All requests should be identified

by the title of the information collection.

The Department specifically requests comments on (a) whether the

proposed collection of information is necessary for the proper

performance of the functions of the agency, including whether the

information shall have practical utility; (b) the accuracy of the

agency's estimate of the burden of the proposed collection of

information; (c) the quality, utility, and clarity of the information

to be collected; and (d) ways to minimize the burden of the collection

of information on respondents, including through the use of automated

collection techniques or other forms of information technology.

Consideration will be given to comments and suggestions submitted

within 60 days of this publication.

Dated: August 9, 2006.

Robert Sargis,

Reports Clearance Officer.

[FR Doc. 06-6923 Filed 8-14-06; 8:45am]

BILLING CODE 4184-01-M

ATTACHMENT B

30-Day Federal Register Notice

ATTACHMENT C

Informed Consent

Interview Consent Form

About the Study

You have been randomly selected to participate in an important research study called the Study of Community Family Life. This study is being conducted by RTI International for the Department of Health and Human Services, Administration for Children and Families. The purpose of this study is to provide researchers with an understanding of how people build and keep close relationships with the important people in their lives.

Participation in the Study

If you decide to be in the study, we will ask you to complete an interview. The interviewer will read questions from a computer screen and type your answers into the computer. The interviewer will ask you questions about your relationships with family members and friends, your attitudes about marriage and relationships, advertising or news stories you may have seen about family relationships, your job, and services you may have received. The interview will take about 40 minutes to complete and you will be paid $25 for your participation. If you decide to participate, you will be one of about 4,200 people involved in the study.

Voluntary Participation

The choice of whether to participate in this study is completely up to you. No one will be upset or angry if you decide not to participate. If you decide to participate in the study, you can refuse to answer any of the questions asked in the interview. Just tell the interviewer you want to skip a question and the interviewer will go on to the next question.

Benefits and Risks of Study Participation

There are no physical risks to you from participating in this study. It is possible, however, that some questions might make you uncomfortable or upset. We will ask you a few questions about domestic violence and infidelity. However, you may refuse to answer these questions. There are no benefits to you for participating in the study. However, the results of this study may benefit the public by helping to inform policy makers on issues of family relationships.

Privacy

We will keep what you tell us in the interview private to the extent permitted by law. After you complete the interview, the interviewer will not be able to look at your answers again. Only the researchers at RTI International will see the information we have collected from study participants. We will combine your information with information from all of the other participants to create group statistics. Although every effort will be made to keep research records private, there may be times when federal or state law requires the disclosure of such records, including personal information. This is very unlikely, but if disclosure is ever required, RTI will take all steps allowable by law to protect the privacy of personal information.

Exceptions to Privacy

There are some exceptions to our promise to keep this information private. If you tell us that you are in immediate danger or that you intend to harm yourself or someone else, we may need to inform the appropriate authorities according to state and local law.

Future Contacts

RTI may contact you by phone after you have completed the interview to ask a few brief questions about the work and conduct of the interviewer who visited your home. We may also contact you again through mailings once a year and interview you again in 18 months and 36 months. If you are contacted in the future, you will be able to make a decision about participating at that time.

Questions

If you have any questions about the study or your involvement in it, you may call the study office at RTI toll-free at 1-800-XXX-XXXX. If you have any questions related to your rights as a survey participant, you may call RTI’s Office of Research Protections toll-free at 1-866-214-2043.

Do you have any questions that might help you decide whether or not you want to participate in the study? Would you like to participate?

Random Recordings

We are using a special quality control system on this project. The system runs on the computer and may record what you and I say to each other during random parts of the interview. Neither you nor I will know when the computer is recording what we say. The recording will be reviewed by people at RTI to monitor my work. The recordings will be used only for those purposes, and will be kept confidential. May we use this quality control system during your interview?

ATTACHMENT D

Letter of Authorization

RTI LETTERHEAD

To Whom It May Concern:

This letter certifies that «Fname» «Lname» is a representative for the Study of Community Family Life, sponsored by the Department of Health and Human Services, Administration for Children and Families (ACF). RTI International, a research organization with headquarters in Research Triangle Park, North Carolina, is under contract to the Federal Government to perform all data collection activities associated with this study (DHHS Contract Number: 233-03-0035).

If you need additional assurance that «Fname» «Lname» is a legitimate RTI representative assigned to this government-sponsored study, please contact Ms. Wandy Nieves, National Field Director, at 1-800-334-8571, ext. 28738, or Ms. Kristine Fahrney, Data Collection Leader, at extension 25531, between 9:00 AM and 5:00 PM ET, Monday through Friday.

Thank you for your cooperation.

Sincerely,

Susan Mitchell

RTI Project Director

ATTACHMENT E

Lead Letter

RTI LETTERHEAD

DATE

Screening ID #

Resident

[STREET ADDRESS] [MULTI UNIT]

[CITY], [ST] [ZIP]

Dear Resident:

I am writing to ask for your help with an important study about family life in your community. This study, sponsored by the U.S. Department of Health and Human Services, Administration for Children and Families, will help the agency understand how people build and keep close relationships with the important people in their lives. Study information will be used to design programs to strengthen families and parent-child relationships.

Your household was randomly selected to participate in this study. This letter is addressed to “Resident” because households were selected by address and we do not know your name.

The Administration for Children and Families has contracted with RTI International to conduct this study. RTI is non-profit research company with offices throughout the U.S. and around the world. A staff member from RTI International will be in your neighborhood soon to provide you with more information. When our representative arrives to explain the study, please ask to see his or her personal identification card. (An example is shown below.) He or she will ask a few preliminary questions, and then may ask one adult member of your household to participate in a voluntary interview. It is also possible that no one from your household will be asked to participate. If a member of your household is selected for the interview and chooses to participate, he or she will receive a cash payment of $25.00 at the end of the interview.

Your identity and the information you tell us will be kept private which means we will not release it to anyone not involved with the study, unless required to law. We are required to protect personal information by Federal law, and we have taken steps that will make sure we carefully protect the privacy of the information people tell us. No person will be identified by name in the reports from this study. If you have questions related to your rights as a survey respondent, you may call RTI’s Office of Research Protections toll-free at 1-866-214-2043. If you have further questions about the study, you may contact the Project Staff toll-free at 1-XXX-XXX-XXXX.

Y our help is extremely important to the success of this study, and we thank you in advance for your cooperation.

Sincerely,

Susan Mitchell

RTI Project Director

ATTACHMENT F

Question & Answer Brochure

Questions and Answers

About The Study of Community Family Life

Sponsored by Department of Health and Human Services, Administration for Children and Families and conducted by RTI International

What is the purpose of this study?

RTI International is conducting a study for the Administration for Children and Families. We are hoping that this study will help us understand how we can best help married couples or individuals or couples interested in marriage build and keep healthy marriages.

What is involved and how long will it take?

If you agree to participate, you will take part in a computerized, face-to-face interview conducted by one of our interviewers in your home or another private setting. We may invite you to be interviewed again 18 months and also 36 months from now. You may refuse to answer any question, and you may also stop the interview at any time. The time varies, but each interview generally takes about 40 minutes.

Will I be paid?

Yes. You will receive $25, for completing the first interview. You will also receive $25 for any follow-up interviews.

Where would we do the interview?

We may do the interview in your home, or another private place that is convenient for you.

What types of questions will be asked?

The questions will touch on your attitudes about marriage and reltionships, your relationships with family members and friends, public messages you may have heard, and services you may have received. We will also ask questions you may consider sensitive, such as questions about infidelity and domestic violence.

What about my privacy?

The information you provide will be held and handled in the strictest of confidence. We will create an identification (ID) number and use it instead of your name to identify your interview in the computer, which will prevent anyone from finding out what your answers were. After you complete the interview, the interviewer will not be able to look at your answers again. Only the researchers and project staff at RTI International will see the information we have collected from study participants. We will combine your information with information from all of the other participants to create group statistics.

Do I have to participate?

No. You do not have to participate in this survey or respond to any questions you do not want to answer. Your decision to participate or not will not affect any social service(s) you may be receiving.

Why was I selected?

Your household was chosen through scientific sampling procedures from a list of households in your city. Within each selected household, we are randomly selecting a participant who is also between the ages of 18 and 49.

How can I get more information and/or enroll?

For more information, call 1-XXX-XXXX (toll-free) and leave a message. Someone from the project staff will contact you.

RTI International is an independent, not-for-profit research organization in North Carolina, dedicated to conducting research that improves the human condition. For more information see http://www.rti.org

Additional information about the Department of Health and Human Services, Administration for Children and Families is available at: http://www.acf.dhhs.gov/

ATTACHMENT G

Appointment Card

INTERVIEW APPOINTMENT

JUST A REMINDER: I appreciate your taking time for this important community study. I have you scheduled for the following appointment.

DAY: __________________________

DATE: _____/_____/_____

TIME: __________ AM / PM

I look forward to seeing you then.

INTERVIEWER: __________________________________

RTI INTERNATIONAL

P. O. Box 12194 Research Triangle Park, NC 27709

ATTACHMENT H

“Sorry I Missed You” Card

Sorry I Missed you
		RTI Survey Research Division P. O. Box 12194 Research Triangle Park, NC 27709
Sorry I Missed you
		RTI Survey Research Division P. O. Box 12194 Research Triangle Park, NC 27709

Date: _____/_____/_____ Time: _________ Dear Resident: I stopped by today to talk to you about an important research study being conducted by RTI. I am sorry that I did not find you at home. I will return to talk with you in the next few days. Thank you in advance for your participation. Sincerely, ____________________________

Date: _____/_____/_____ Time: _________ Dear Resident: I stopped by today to talk to you about an important research study being conducted by RTI. I am sorry that I did not find you at home. I will return to talk with you in the next few days. Thank you in advance for your participation. Sincerely, ____________________________

ATTACHMENT I

Incentive Receipt

CASH

INCENTIVE

RECEIPT

Study of Community Family Life

RTI PROJECT # ________--- __________ CASE ID #__________________

________________________________________________________________

To show our appreciation for the time you spent answering our questions for this important study we are authorized to pay you a cash incentive of $25. Since maintaining the privacy of your information is important to us, we ask that you not enter your full name on this form. However, the interviewer must sign and date this form to certify you have received (or declined) the cash payment.

 Accepted $25.00 Cash Incentive

 Declined $25.00 Cash Incentive

Recipient's Initials (PLEASE DO NOT SIGN YOUR NAME) _____________ Date: ___/ ___/ ___

Interviewer's Signature: __________________________________________ FI ID # ___________

Disposition: Send original and yellow to supervisor, retain pink copy; gold copy to respondent.

ATTACHMENT J

Hotline Numbers

ATTACHMENT K

Refusal Conversion Letter

RTI LETTERHEAD

DATE

Screening ID #

Resident

[STREET ADDRESS] [MULTI UNIT]

[CITY], [ST] [ZIP]

We really need your help! My name is Susan Mitchell, the Project Director for the Study of Community Family Life and I would like to make a personal appeal to you.

I know how busy you must be with your own work and family matters. But we're asking for your help because only you can provide the critical information we need to continue our work. We will pay you $25.00 for your time, and we will do everything possible to make your participation convenient and comfortable.

Recently, an interviewer from RTI International contacted you about participation in the study. The study is sponsored by the Administration for Children and Families and includes 4,200 households across the nation. We need information from everyone who is eligible for the study (including you!) so that we can get an accurate picture of family life in your community. Please understand that your experiences and points of view are very important to us. No one else can replace your unique role in this study!

Because your help and participation are essential, we promise to do everything we can to make this a comfortable experience. While you're being interviewed you do not have to answer any questions that seem too personal, and if you change your mind about participating after the interview has begun, you may stop at any time. In addition, our interviewers are willing to conduct the interview in a place that is convenient for you.

The information you provide to us will be held in the strictest confidence according to state and federal laws. It will be used for statistical purposes ONLY. No names or other identifying information will be included in any reports of scientific findings. The information you provide will be combined with the answers of others in the study and reported in summary form. Specific individuals will NOT be identified or singled out in any reports.

I do hope that you will agree to participate in this important study. Your input is absolutely critical. Although you will not directly benefit from continuing to participate in the study, research like this can provide important information about the ways in which we can help improve family life in communities like yours.

Please call [NUMBER] (toll-free) and leave a message letting us know the best way to contact you to arrange for an interview. If we do not hear from you in a few days, an interviewer will contact you to arrange the interview.

Please accept my deepest thanks and appreciation for any help you may provide.

Sincerely,

Susan Mitchell

Project Director, RTI International

ATTACHMENT L

Verification Letter

RTI LETTERHEAD

[DATE]

RESIDENT

[ADDRESS]

In recent weeks, RTI has been conducting a survey called the Study of Community Family Life. Our records indicate that a [AGE]-year-old [GENDER] in your household was interviewed. We would appreciate it if [HE/SHE] would take a minute to complete the following questions.

This information is used only to verify the quality of our interviewer’s performance.

1. Were you interviewed in-person or over the telephone?

In-person ___ Over the telephone___

2. About how long did the interview take?

___ Minutes

3. Were you paid for your participation?

Yes___ No___

If yes, how much were you paid? $__________

4. Was the interviewer professional and courteous?

Yes___ No___

IF NO: Please describe how our interviewer could improve his/her behavior:

________________________________________________________________

________________________________________________________________

A stamped, pre-addressed envelope is enclosed for your convenience in returning this form. Thank you for your cooperation.

Sincerely,

Susan Mitchell

Project Director

ATTACHMENT M

Screener and Survey Questionnaire

1 A GIS is a combination of hardware, software, and data that can be used to help manipulate, analyze and present information that is tied to a spatial location. A GIS can combine many layers of information about a place to give a better understanding of that place.

2 Research Triangle Institute. 2004. SUDAAN Language Manual. Release 9.0. Research Triangle Park, NC: Research Triangle Institute.