Part A_OMB comments and ACF revisions_FINAL

U.S. Department of Health and Human Services

Administration for Children and Families

Administration on Children, Youth, and Families

National Survey on Child and Adolescent Well-Being Second Cohort

(NSCAW II)

Office of Management and Budget

Clearance Package Supporting Statement

and Data Collection Instruments

Supporting Statement

Volume I

Part A

May 10, 2011

Table of Contents

Volume I:

A. Justification

(1) Explanation of Circumstances that Make Information Collection Necessary. . 3

(2) How the Information Will Be Collected and For What Purposes. . . . . . . . . . . 9

(3) The Use of Improved Information Technology to Reduce Burden. . . . . . . . . 19

(4) Efforts to Identify and Avoid Duplication. . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

(5) Efforts to Minimize Burden on Small Businesses and Other Entities. . . . . . . 21

(6) Consequences of Less Frequent Data Collection. . . . . . . . . . . . . . . . . . . . . . .21

(7) Special Circumstances Requiring Collection of Information Inconsistent

with Guidelines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . .. . . . 22

(8) Federal Register Notice and Other Consultation . . . . . . . . . . . . . . . . . . . . . 22

(9) Payment to Respondents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

(10) Assurance of Confidentiality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

(11) Justification for Questions of a Sensitive Nature . . . . . . . . . . . . . . . . . . . . . . 27

(12) Estimates of Annualized Hour Burden . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

(13) Estimates of Annualized Cost Burden to Respondents . . . . . . . . . . . . . . . . . 29

(14) Estimates of Annualized Cost to the Government . . . . . . . . . . . . . . . . . . . . . 29

(15) Reasons for Any Program Changes or Adjustments . . . . . . . . . . . . . . . . . . . 30

(16) Time Schedule, Publication and Analysis Plans . . . . . . . . . . . . . . . . . . . . . . 30

(17) Display of Expiration Date . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

(18) Exceptions to Certification Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

PART A: JUSTIFICATION

The Administration on Children and Families (ACF) requests a revision to a previous Information Clearance Request (OMB control number 0970-0202) in order to conduct a 36-month follow up for the second cohort of the National Survey of Child and Adolescent Well-Being (NSCAW II). ACF conducted the first cohort of NSCAW (NSCAW I) through fives waves of data collection; from 1999 through 2007. NSCAW II is following a similar schedule of follow-up interviews, and has completed the baseline and 18-month follow-up collections. We now seek clearance to conduct the 36-month wave of data collection. We intend to use essentially the same measures that were used in the baseline and 18-month collections, with some minor updates of reference periods and corrections of errors that were found during the previous waves. The NSCAW II measures table is included for reference in Appendix A.

A.1 Explanation of Circumstances That Make the Information Collection Necessary

The Deficit Reduction Act of 2005, Public Law 109-171 reauthorizes Section 429A of the Personal Responsibility and Work Opportunities Reconciliation Act of 1996, which authorizes the Secretary to conduct a longitudinal child welfare study. The NSCAW II fulfills the intent of the reauthorization legislation.

This section examines existing research to demonstrate how the planned study both meets the legislative mandate and also provides vital information for decision-making about policy and practice within the child welfare system. The study is being conducted through a contract with RTI International and its subcontractors, Caliber/ICF and Walter R. McDonald Associates.

The NSCAW II responds to a continuing need for better understanding of the child welfare system, the children and families who come in contact with it, and the services they receive. Since the beginning of NSCAW I in 1999, the context within which the state and local child welfare agencies operate has changed, with increasing demands and fewer resources (e.g., Lindsey, 2003). Federal legislation and policy have imposed greater accountability on agencies and now levy penalties if standards are not met. Client characteristics have changed, as immigration patterns have broadened past large, urban areas. Caseworker turnover continues to be a problem in staffing roles at the frontline. Reform efforts are aiming toward community-based integrated systems of care and other alternatives (Webb and Harden, 2003). All the while agencies are reacting to these demands, state budget cuts have diminished the resources available to meet these and other challenges.

Prior to NSCAW, most research on children and families within the child welfare system used a cross-sectional or cohort design, which limited conclusions about trajectories of care, long-term child and family outcomes, or impact of agency practices over time. Furthermore, much of the data on service patterns within this population had to be drawn exclusively from administrative records. Such records usually included only limited service data or did not capture the full range of the child welfare system (e.g., including only children in foster care). Through its prospective longitudinal design, NSCAW represents a major advance to child welfare research and other fields, and has been a platform from which much has been learned. With its wealth of data and national probability sample, NSCAW is a superior tool for following up on any number of these studies in depth, and providing national population estimates where most studies are only able to provide results for individual communities or states.

Child Safety, Permanence and Well-Being

NSCAW I has illustrated the unique challenges facing children and families involved in the child welfare system.

NSCAW data are being used to examine CWS progress toward the important goal of achieving permanence of children’s living environment. Over three quarters of children and youths remained with their families across all four waves of the study (Testa, Bruhn & Helton, 2010), and over 90% who were in their families of origin at baseline ended with them as well. At baseline a modest 11% of children had been removed from the home (DHHS, 2005). On the other hand, only 51% of children in informal kin care at baseline remained in their home through all four waves. Considering children who were removed from their home, at 36 months, 28.8% of youths aged 0 to 6 and 39.9% of youths aged 7 and older were not in permanent settings (Barth et al., 2009).

NSCAW is a resource to explore new ideas for achieving supportive permanent environments for children caught in impermanent homes. Researchers and policy-makers are beginning to scrutinize traditional adherence to the values of continuity of family and family-like placement arrangements after a child’s safety has been ensured. For example, in part spurred by economic, technological, and societal changes, researchers and policy-makers have begun to consider the overall “social capital” (e.g., education opportunities, health care options) of a placement setting as important, in addition to stability and continuity, when assessing options (Testa et al., 2010).

Service Needs and Use

Despite tremendous need, many children within the child welfare system receive few services if any. For instance, nearly half of the youth in NSCAW with completed child welfare investigations were determined to have clinically significant emotional or behavioral problems; yet only one quarter of these youth had received any specialty mental health care during the previous 12 months (Burns et al., 2004). Stahmer et al. (2005) found high levels of developmental and behavioral needs among toddlers (approximately 42%) and preschoolers (68%) in NSCAW, with only 23% of these young children receiving relevant services. Leslie, Hurlburt, and colleagues (2003) found that less than half of the communities in NSCAW provided comprehensive health and developmental examinations to all children entering out of home care. Meanwhile, a large percentage of the children in foster care had visited an emergency room within a 1‑year period (Jee et al., 2005).

Disparities

Child maltreatment differentially impacts children of different racial and ethnic groups as well as children from different locations (Ortega et al., 2010). Disparities also exist in terms of access to services. For instance, African American children appear less likely to receive specialty mental health services than white children involved in the child welfare system (Horwitz, Hurlburt, & Zhang, 2009). Interestingly, Hurlburt and colleagues (2004) found that linkages between child welfare and mental health at the local level increased mental health access for youth, particularly among ethnic minority youth. This study, in particular, underscores the importance of understanding the impact of contextual factors amenable to change within the child welfare system that might improve the quality of services provided.

Impact of Child Welfare Service Context

We have also learned that provider and agency characteristics influence the quality of services received by families within the child welfare system as well as their outcomes. For example, recent research using NSCAW demonstrates that organizational culture can affect service quality and outcomes independent of the individual characteristics of either the service provider or the children and families receiving services (Glisson, 2010). Having a positive work culture within a child welfare agency increases the chance that a child needing mental health care will actually receive it (Glisson & Green, 2005). These findings stress the continued importance of gathering data at the provider, agency, and policy levels within future NSCAW efforts.

It is important to note that the impact of the NSCAW longitudinal study extends beyond that funded directly by ACF. NSCAW data are licensed through the National Data Archives for Child Abuse and Neglect to over 100 researchers who have published more than 250 journal articles, scientific reports, book chapters, and dissertations. (An extensive bibliography of NSCAW publications can be found at:http://ndacan.cornell.edu/NDACAN/Bibliography.html ).In addition, the NSCAW study has served as a platform for multiple collaborative opportunities used to expand on existing data. For instance, the National Institute of Mental Health supported the Caring for Children in Child Welfare (CCCW) study which supplemented NSCAW county and agency data to better understand contextual influences on service use and outcomes (Leslie et al., 2003). The National Institute of Justice provided funds to examine intimate partner violence among female caretakers within NSCAW (Hazen, Connelly and Kelleher, 2004). The MacArthur foundation funded efforts to combine NSCAW data with data from provider organizations in NSCAW PSUs to assess the impact of agency organizational characteristics on child and family outcomes (Glisson, 2010). NSCAW has been instrumental in supporting these collaborations and intends to actively foster collaborative relationships with scientists interested in expanding upon or capitalizing on the NSCAW data.

Use of a Conceptual Framework

The original NSCAW study was grounded by a conceptual framework that emphasized the child and family’s environmental context as well as the dynamic nature of an ever-changing child welfare system. The conceptual framework acknowledged that many families have repeated contact with child welfare services and that such prior experience with this system likely shapes subsequent decisions about their future. Furthermore, decisions made by the child welfare system and court were considered to be a function of both child and family characteristics as well as governmental and agency practices, which were in turn linked to policies, regulations, resources, norms, and values. This broad ecological framework for NSCAW has not been changed for the current proposal; however, Exhibit A.1-1 illustrates the critical need to portray the child welfare system and the families served by it in their updated and current context.

The contextual forces impacting child welfare systems and the families served by them are dynamic and ever changing. Families entering the child welfare system today face different pressures from those faced by families first enrolled in NSCAW in 1999. For example, federal policy increasingly favors kinship care due to its less restrictive and family-like environment. There is also growing evidence of the value of kinship care for increasing a child’s safety, stability, and permanence (Testa, et al., , 2010. Policies have also changed and are placing different pressures on child welfare agencies and providers. For instance, major changes have occurred in federal legislation, policies, and programs which have posed new challenges to child welfare systems in states. Federal legislation (Adoption and Safe Families Act of 1997) mandated the Children’s Bureau to more closely monitor child safety, permanency, and well-being and hold states accountable for improving outcomes for children and families. Through the Child and Family Services Reviews (CFSRs), the Children’s Bureau is charged with ensuring that state child welfare agency practice is in conformity with federal child welfare requirements and levies penalties against states that do not improve outcomes after implementing Program Improvement Plans (PIPs).

Our goal continues to be to provide data at each level of this framework, to provide for an ecological understanding of children’s development following contact with CWS. Such a goal includes gathering information related to child and family characteristics, characteristics of a family’s natural community, characteristics of child welfare service agencies and their providers, the types and qualities of services received by families served by the child welfare system, and the policies and regulations that impact those services provided. We also aim to inform sites and state administrators not only about the characteristics of their populations but also about agency and worker policies and practices, services provided, and other key findings with a bearing on child and family outcomes.

Exhibit A.1-1 Ecological Framework for NSCAW

We designed this framework based on an ecological conception of how development occurs and use this model to organize the conceptualization of the study issues. As seen above, decisions that are made by the child welfare system and court personnel are a function of both personal/family characteristics and governmental and agency practices, which in turn are linked to policies, regulations, resources, characteristics, norms, and values. Our continuing goal is to provide data at each level so that we can develop a comprehensive picture of the interactions and transactions that lead to child and family outcomes. This permits us to address the effects of factors within and without the child welfare system as well as transactions among the child, family, and larger environment. The framework also recognizes that as issues vary or change in any part of the system, the outcomes will be modified. If children are reported for abuse or neglect, child welfare service providers consider a whole host of variables in determining which path these children will follow. These factors include assessments of individual strengths and vulnerabilities, extended family and community support, the stigma attached to removal from the home, the kind and degree of maltreatment, previous experiences with the child welfare system, and the availability of services. Child welfare services researchers can do no less and, indeed, must bring additional theoretical and methodological sophistication to assessing the ecology of child welfare.

The Continuing Need for Longitudinal Research in Child Welfare

Longitudinal research is needed to understand patterns of care and how those vary by agency, community, family type, service history, and child’s developmental stage, in the current context. Further, such research will help to disentangle the influence that different patterns of care have on the developmental outcomes for children and their families. One of the major foci of NSCAW II is to more completely measure and track service utilization to facilitate that understanding. Additionally, NSCAW II makes use of administrative data advances such as submissions to AFCARS and NCANDS, and the more prevalent sophisticated agency database systems that now exist. These data will be important for accurately tracking additional reports and investigations involving the child and family, changes in out-of-home placements, and, potentially, service utilization.

Additional longitudinal research is needed to understand how to make services (and perhaps policies) more responsive to child and family needs. Although the ages of children receiving child welfare services has changed, there has been no concomitant change in policy or practice that might reflect a developmental perspective; our child welfare law does not speak to the ages of children. Prior to 1996, there was some attention to placing foster care in a developmental framework, but these efforts were modest and had little impact on public policy. As early as 1975, Wald questioned the one-law-fits-all notion that allowed child welfare law to be silent on the issue of the child's age; he asked: How can we build in developmental knowledge to make the laws more sophisticated and more likely to serve the best interests of children? ... (1975, p. 11). Longitudinal research may identify service, family, and child characteristics that interact to show the way toward developmentally sound policies.

To answer important questions for future policymaking, child welfare practice, and effective resource allocation, the conduct of the 36-month follow-up for National Survey of Child and Adolescent Well-Being II is essential. In the next section, we describe the design of the study and the specific research questions to be addressed.

A.2 How the Information Will Be Collected and For What Purposes

A.2.1 Overview of the Design of the Study

 The NSCAW II cohort includes 5,873 children, aged birth to 17 ½ years, who had contact with the child welfare system within a one-year period beginning in March, 2008. These children were selected from 81 Primary Sampling Units (in most cases, counties) in 30 states. As in NSCAW I, the sample of investigated/assessed cases includes both cases that receive on-going services and cases that are not receiving services, either because they were not substantiated or because it was determined that services were not required.

This sample design—with oversampling of infants and children in out of home placement, and undersampling of cases not receiving services to ensure appropriate representation among subgroups of particular instance—allows in-depth analysis of subgroups of special interest (e.g., young children and adolescents in foster care) while providing national estimates for the full population of children and families entering the system. Children in out-of-home placement are especially important to our analyses to examine the process and outcomes of earlier permanency planning. Infants are oversampled to ensure sufficient numbers of cases to assess the outcomes of child maltreatment and services, including out-of-home placements, on early childhood development. Section B.1 details the NSCAW II sample design. Exhibit B.1-1 provides the exact sample allocation and effective sample sizes.

The NSCAW is a longitudinal study with multiple informants associated with each sampled child, in order to get the fullest possible picture of that child. A baseline round of face-to-face interviews or assessments was conducted with children, parents, and non-parent adult caregivers (e.g., foster parents, kin caregivers, group home caregivers), and investigative caseworkers was begun in 2008-2009, with one follow-up administered 18 months after the close of the NSCAW II index investigation/assessment. Baseline data collection was completed between April 2008 and September 2009; the 18-month follow-up data collection began October 1, 2009 and was completed on January 8, 2011. The overall weighted response rate for the NSCAW II baseline (enrollment) wave was 55.8%. Response rates for the first follow-up (Wave 2) were 81% for children, 86% for caregivers, and 94% for services caseworkers.

Both children who remain in the system and those who leave the system will be followed for the full study period.

A.2.2 Purpose of the Study

NSCAW is designed to address crucial program, policy, and practice issues of concern to the federal, state, and local governments, and child welfare agencies. NSCAW I was the first national study of child welfare to collect data from children and families, and the first to relate child and family well-being to family characteristics, experience with the child welfare system, community environment, and other factors. The study examines the interplay among the history and characteristics of children and families, their experiences with the child welfare system, other concurrent life experiences, and outcomes. It brings to bear perspectives from child welfare, child development, and other fields to focus on children’s well-being, including their health and physical well-being, social functioning, academic achievement, mental health, and behavioral adjustment. It relates these to developmental stage, prior experience, caregiver behavior, social services, and community environment. By drawing on these different perspectives, the study provides more complete understanding of how family, child, community, and service factors affect children’s well-being and will provide the foundation for improving policies, programs, and practices. The major research questions the study will address include:

 Who are the children and families that come into contact with the child welfare system in 2008-2009?

 What pathways and services do children and families experience while in the child welfare system?

 What are the shorter- and longer-term outcomes for these children and families, especially as related to permanence, safety, well-being, and service utilization?

Examples of questions within each of these areas are included in Exhibit A.2-1.

Reports planned are similar to those released from NSCAW I, and will focus on the four areas of ACF policy interest – permanence, safety, well-being, and service utilization. Following each wave of data collection, data from the survey will be analyzed by the project team. Additionally, after being stripped of identifying information and analyzed for the possibility of inadvertent disclosure, data sets from NSCAW II will be made available to the larger research and policy community to encourage secondary analyses that will support further research and timely policy decisions. The National Data Archive for Child Abuse and Neglect (NDACAN) at Cornell University is the repository for NSCAW I data, and currently holds licenses with 124 principal investigators; NDACAN will continue to hold the NSCAW data licenses and support users.

Our analyses will focus on the key study issues described above. Examples of the cross-sectional and longitudinal analyses to be performed include:

 Description of characteristics and risk factors for children and families at the point of entry into the child welfare system, overall and for subgroups;

 The investigation/assessment process (e.g., risk factors, decisions, family involvement);

 Children’s and families’ experience of child welfare and other services, and changes in services and placements during their period in the child welfare system;

 The process of permanency planning and implementation for children in out of home care;

 Description of children and families who leave the system quickly and those who stay in for a longer period;

 Analysis of relationship of child, family, caseworker, agency, and other factors to child and family services and outcomes;

 Analysis of how the organization, structure, and resources of agencies relate to the services provided and to whom.

Exhibit A.2-1 Examples of Questions NSCAW Will Address

 Who are the children and families who come into contact with the child welfare system in 2008-2009?

 What are their backgrounds and characteristics?

 What are their prior histories?

 What problems and strengths do they bring?

 How do the characteristics, experiences, and needs of children and families differ by the ways they come into contact with the system?

 What effects do state and agency policies and programs have on the characteristics of those who enter the system?

 What pathways and services do children and families experience while in the child welfare system?

 What placements and services do they experience while they are in the child welfare system?

 What determines the different pathways, placements, and services they experience?

 How do child welfare services interact with other services and supports for children and families involved with the child welfare system?

 What are the shorter- and longer-term outcomes for these children and families?

 How do children and families change during the time they are in contact with the child welfare system?

 How do children and families change after they leave the system?

 How do child, family, system, community, and other factors influence child and family functioning?

 How do these factors affect subsequent child welfare system involvement?

The primary focus of the study is on children and families; however, because data are collected from child welfare agencies, it will also be possible to conduct some limited analyses at the agency level. Agency level data (e.g., staff turnover, use of dual tracking, budget) and caseworker data (e.g., level of experience, specialized training) will be used in analyses of child and family services and outcomes. In addition, data collected during the sampling process will be used to describe such aspects of the child welfare system as outcomes of completed cases (e.g., substantiation rates) and the disposition of substantiated cases (e.g., rates of case opening, placement rate), overall and for subgroups.

A.2.3 Information Elements and Data Sources

Section A.1 presented an ecological-developmental theory of risk and resiliency to examine the antecedents and consequences of child maltreatment and the outcomes of contacts with the child welfare system. With this model in place, we developed the NSCAW I data collection instruments to operationalize this theoretical model of child maltreatment. This involved developing an instrumentation package that validly and reliably measures the constructs of the conceptual model to the highest standards of scientific accuracy, across a broad range of child ages. The selection of instrumentation also took into account the methodological and administrative considerations relevant to collecting data in a cost effective manner that does not burden respondents unduly, the potential consequences of inquiring about sensitive behaviors and topics, and adequately deals with the requirements of a diverse multicultural population of interest. We used the NSCAW I instruments as a starting point for NSCAW II, and made revisions based on feedback from data users and from expert consultants.

Instrument Development Process

The task of instrumentation revision was taken on by a group of experts in child welfare, child maltreatment, child development, social welfare service provision and utilization, psychometrics, survey methodology, and survey research. These teams met regularly through conference calls and discussed the selection of instruments. The decision-making process was interactive and iterative. The draft instrumentation package was presented to the full Consultant Group in a meeting on May 29, 2007, and their feedback was incorporated.

Whenever possible, we endeavored to retain NSCAW I instrumentation. Where additions or revisions were needed, we were guided by the principle that the development of new measures or the substantial adaptation of existing measures (especially partial deletion) should be avoided, to minimize risks to the implementation of the project and maximize utility of the data. For these reasons, wherever possible instruments for collecting family and child well-being outcomes and system/agency characteristics were chosen from the body of existing instruments, and their use without modification was recommended.

Criteria for Choosing NSCAW I Measures

Because of the heavy reliance on NSCAW I measures, it is important then to describe the approach in the original study to the processes by which existing measures with previously established properties were evaluated and selected. The following criteria were used by the NSCAW I Instrument Development Team (IDT).

Psychometric Properties of Standardized Instruments: Each standardized instrument was rigorously evaluated in terms of its psychometric properties: reliability, validity, standardization sample, type of norms. However, because the NSCAW I included a unique target population, even when an instrument was considered reliable and valid, we considered how applicable the norms would be.

Non-Standardized Instruments: Because there were many important constructs for which there were no available standardized measures, we relied on other surveys of similar populations that asked questions about child and family well-being and the social service system. For these instruments we considered how successfully the potential instrument was used in other similar studies.

Age Applicability: Because the project was designed to study children from birth through adolescence, the issue of the sensitivity of measures to changes in age was very important. For instance, the indicators and strategies for assessment of mental health change over the age span. The Instrument Development Team considered instruments that would provide measurement of characteristics are as comparable as possible across the age span.

Administration Factors: There are a variety of administration factors that the Instrument Development Team considered. With fixed resources and a large sample of children to be assessed, issues of training, materials, scoring complexity, analysis, cost, and participant response burden were of major importance. The instruments were selected so that the respondent burden fit within the time parameters budgeted in the data collection plan. Because we conduct the data collection at the homes of the participants, we thought about the materials required to collect the data. Many standardized assessments require elaborate testing materials that are not only expensive but are cumbersome to carry. These factors were weighed in a cost-benefit fashion.

Cultural and Linguistic Considerations: We knew that there would be a considerable degree of ethnic diversity among the NSCAW children and families and that a substantial proportion will be speakers of languages other than English. The Instrument Development Team considered instruments in terms of the cultural assumptions of the respondent groups and not just those of the researchers’ own culture. We also considered the process of how translation would affect the connotations, or implied associations, of the words. The IDT considered the types of questions that asked because indicators of well-being may have a different meaning that depends on the ethnicity of the family. Both the content and the form in which items are presented are important. The Instrument Development Team solicited help from individuals on the NSCAW I Technical Work Group with special expertise in multi-cultural issues in order to develop appropriate criteria.

Other Considerations: While the criteria listed above factored into the review and evaluation of the individual instruments, other issues were apparent only after the assessment instruments were considered collectively. The ordering of instruments was one example. Another consideration was the suitability of the instruments for computerization—the use of open-ended questions where considerable typing of responses would be needed, the use of long lists of response options, screen layout considerations, and the need for “show cards” or other interviewer-presented materials during Audio Computer-Assisted Self-Interview (ACASI) administration, for example.

Obtaining information about child and family functioning and outcomes: Data sources include the child, caregiver(s), caseworkers, and teachers, but the importance of these data sources vary by age. Although older children are able to be interviewed or tested, it is possible to obtain data from young children only by direct assessment. As such, information gathered from caregivers becomes more important. We also know that some children may have changes in placements so that caregivers may change from one assessment to the next. Moreover, the timing of the data collection for a particular child may coincide with a transition to a new home or school even in the absence of frequent moves. Our choice of measurement instruments needed to reflect this by including multiple data sources. Inclusion of multiple data sources also permits different perspectives to be included. For instance, the incidence of behavior problems may be specific to home or school, necessitating collection of data from both caregivers and teachers. The Instrument Development Team also considered the overall burden on the respondent so that in some cases an appropriate instrument was not chosen for a particular respondent because the length of the interview was excessive.

Obtaining information about systems and agencies: In assessing the usefulness of agency measures, we assessed how well they describe a) the characteristics of systems and agencies, and b) the types and characteristics of services provided to children and families. We focused on those system, agency and service measures most directly related to child and family outcomes for different types of families and children. The research questions related to systems, agencies and services stipulated in NSCAW’s conceptual model cover issues related to children's entry to, experience within, and exit from the child welfare system.

For the NSCAW II instruments, every effort was made to minimize the burden of data collection on both participating agencies and on respondents. The agency instruments were selected so that the respondent burden fits within assumed parameters. To achieve those objectives, system and agency measures recently collected within other research, and/or those that are collected on an ongoing basis by other organizations were actively sought out and assessed.

At the state level, NSCAW II will use data collected for other purposes to the extent they are appropriate to our objectives, in order to reduce burden on participating agencies and retain data comparability across sites. This survey’s usefulness to the larger research community is enhanced if system and agency measures are defined in a way that facilitates comparability with other studies. In instances where measures have not been collected at the local levels, or where available data are not sufficiently timely, the appropriateness of adapting measures that have been developed within related research was assessed.

Questionnaire Domains

Exhibit A.2-2 presents the measurement constructs included in the questionnaire for each type of respondent: child, parent, teacher, and caseworker. The ages of the child for which the section is applicable is also included, as is the rationale for inclusion of the constructs.

Exhibit A.2-2 Child, Family, and Caseworker Constructs, Their Rationale and

the Data Source in NSCAW II Data Collection

Construct	Child Age	Data Source	Rationale
Child Areas
1. Social Competence, Relationships			Critical for adaptation in a variety of domains and a frequent area of disruption for children with histories of maltreatment.
Relationships with parents and other significant adults	< 11 > 11	Observation Child	Warm and supportive relationships between children and adults can buffer children against stresses and help children heal from negative effects of maltreatment.
Peer relationships	> 5 > 5	Child Teacher	Success in making and keeping friendships is linked to better school adjustment. Peer friendships provide children a support system and model for future relationships. Peer rejection is related to adolescent conduct disorder.
Global Social Competence	> 5	Caregiver/ Teacher	Children with better developed social skills have greater success in forming social relationships and better long-term academic and occupational achievement.
2. Health, Cognitive status			Health and intellectual functioning are among the most important indicators of well-being and have an influence on development in other domains.
Developmental/cognitive status	All	Child	Cognitive functioning and neurodevelopmental status are important mediators of school success. Cognitive status should be assessed at two visits for children <6 and at one visit for children > 6.
Communication skills	< 6	Child	Language skills are the foundation for literacy skills taught in school.
Health and Disabilities	All ages	Caregiver	Children with chronic health conditions and disabilities are more likely to experience maltreatment. Health status and injuries can be a direct consequence of maltreatment. Health and injuries influence the extent to which children can participate in activities.
3. Adaptive behavior, Functional status			Adaptive behavior reflects competence in achieving personal independence and meeting social demands such as academic adjustment and performance.
Adaptive Skills, Socialization, Communication	All Ages	Caregiver	The ability to function in daily life is an important marker of adjustment. For adolescents in out of home placements, living skills become an important focus of attention as they transition to independent living.

Exhibit A.2-2 Child, Family, and Caseworker Constructs, Their Rationale and

the Data Source in NSCAW II Data Collection (Continued)

Construct	Child Age	Informant	Rationale and other Comments
Academic achievement (including attendance, grade progression)	> 6 > 6	Child Teacher	Academic achievement and completion of high school are critical for future economic viability. Minibattery is a shortened version of the W-J-R
Special ed. status and educational, developmental and support services received	All ages	Teacher/ Caregivers Caseworkers	Children’s developmental and educational needs affect their participation in school and social activities. Documentation of services addressing these needs an indicator of how CW system operates.
School socialization	> 6	Teacher	Maltreated children often experience difficulties in social as well as academic aspects of school adjustment. The ability to function socially in the classroom is highly related to academic performance.
School engagement	> 6	Teacher	Motivations are affected by early experiences and in turn influence children’s dispositions towards learning and school.
Future Expectations	> 10	Child	These expectations are a by-product
4. Behavior regulation, Emotional and Mental Health			Behavioral and emotional processes are developed as part of the caregiver-child relationship and can be disrupted when this relationship is impaired. Their successful development along with mental health are the hallmarks of adjustment and well-being.
Temperament	<3	Caregiver	Temperament can either act as a risk or a resiliency factor by influencing how a child relates to others. It is also one of the antecedents of self-regulation
Behavior Problems	> 2 >5 > 11	Caregiver Teacher Child	Behavior problems are the earliest signs of maltreatment, especially difficulty with impulse control and aggressive behavior. Externalizing, problems interfere with peer acceptance and academic performance, and a precursor to later delinquency. Often setting specific, this measurement requires multiple informants.
Mental health	> 2 > 5 > 7	Caregiver Teacher Child	Mental health is a broad construct that affects all aspects of well-being. In children it is highly associated with behavioral problems. Conduct disorders and other externalizing problems can be assessed with the CBCL. For internalizing problems such as depression and anxiety need to ask child.
Criminality/delinquency	> 11	Child	Elevated risk of delinquency and criminality among maltreated individuals. Avoidance is a marker of successful social adaptation.
Substance Abuse and Risky Behaviors	> 11	Child	Drug abuse and early sexual activities are associated with depression and maltreatment.

Exhibit A.2-2 Child, Family, and Caseworker Constructs, Their Rationale and

the Data Source in NSCAW II Data Collection (Continued)

Construct	Child Age	Informant	Rationale and Other Comments
5. Life experiences
Child Maltreatment	All Ages > 11 All Ages	Caregiver Child Caseworker	It is critical to get more than the maltreatment information that forms the basis of the report. The measure presented is one possibility but no decision has been made.
Family/placement disruptions	All Ages	Caregiver	Frequent placement changes and household disruptions are related to poorer child outcomes
Loss, violence and other stressors in and out of the home	> 5	Child	All violence that is experienced (i.e., viewed as well as directly experienced) has a negative impact on mental health and on how children handle conflicts themselves.
6. Service experiences	All ages > 11	Caregiver/ Caseworker Child	Documentation of services received is critical to understanding the service provision process, the factors that affect the process, and the relationship among individual/family variables, services, and outcomes.
Parent /Caregiver Areas
1. Health Status	All ages		The health of the caregiver affects the functioning of the caregiver and thus how s/he relates to the child.
Mental health and Substance Abuse		Caregiver	Psychiatric disorders, especially parental depression can be especially harmful to the quality of the parent-child relationship. It is one factor predisposing a parent to maltreatment as well as a result of maltreatment. Substance abuse is one of the reasons for reports to CPS. Substance abuse is also associated with parenting difficulties.
Physical health		Caregiver	Affects how well a parent can provide caregiving and function in the larger world.
Services received by biological parent		Investigative and Services Caseworkers	Services received by parents may be critical to their ability to provide appropriate care for their children. If parents do not receive needed services, reunification may be unlikely.
2. Caregiver attributes/behaviors			Parenting attitudes and behaviors are on a continuum, from positive and supportive to negative, with child maltreatment falling at the negative extreme.
Parent/caregiver behaviors Emotional nurturing and Cognitive/verbal responsiveness and stimulation	< 11	Caregiver/ Observation	Supportive parent-child relationships are the foundation upon which all developmental achievements are built. Parental sensitivity to child’s needs and interests, parental ability to structure an interaction, and acceptance of child are highly related to quality of the emotional bond between parent and child. Verbal and behavioral responsiveness are highly linked to children’s communication and intellectual competence.

Exhibit A.2-2 Child, Family, and Caseworker Constructs, Their Rationale and

the Data Source in NSCAW II Data Collection (Continued)

Construct	Child Age	Informant	Rationale and Other Comments
Behavioral Monitoring and Discipline	All ages < 11	Caregiver Caregiver/ Observation	Use of appropriate discipline promotes socialization and behavioral self-regulation in children rather than short-term compliance. It is frequently under the guise of discipline that parents justify physical maltreatment. At the other extreme, lax supervision and the failure to provide any limits can cross the boundary from leniency to neglect.
3. Contextual factors	All		These are influences on caregivers and children that affect perceived stress and well-being.
Neighborhood factors	All	Caregiver	The behavior of Individuals and families has to be understood in terms of the environment of their community.
Family Demographics	All	Caregiver	Will provide background characteristics, roster of who lives in the home, employment, education, and other descriptive information.
Social support and other family resources, including assistance with child-rearing	All	Caregiver	Perceived social support is believed to buffer the child and family against stress, thereby helping them better cope with their problems.
Domestic Violence in the Home	All	Caregiver	Domestic violence is highly associated with child maltreatment, is a source of stress for the child, and may itself be considered a form of psychological maltreatment.
Criminal Involvement of Parents	All	Caregiver	A background factor that is associated with maltreatment.
Risk Assessment	All	Investigative Caseworker	The relationship between caseworker perceived risk and strengths and long term safety and well being needs to be assessed.
Caseworker Areas
Construct	Child Age	Caseworker	Rationale and Other Comments
Job role	All	Services	Related to child outcomes
Work unit	All	Services
Caseload	All	Services	Caseload level will determine amount of time worker is likely to be able to spend on services to sample child
Work environment and Job satisfaction	All	Services	Work environment and job satisfaction are believed to influence worker’s job performance and turnover
Demographics	All	Investigative and Services	Measures of caseworker demographics and professional background; also allows comparison of caseworker race/ethnicity with that of sample child and family.

In summary, the constructs chosen for inclusion in NCAW II are identical to those selected for the NSCAW I survey, and were identified from an understanding of the models used to explain the causes and consequences of maltreatment and how the child welfare system relates to child and family outcomes. Moreover, these constructs were mapped onto the critical research questions that were independently identified. The instrumentation was chosen because the NSCAW I IDT believed that they were the best current measures for operationalizing the constructs as well as adhering to the adopted measurement criteria. Those decisions have been reexamined in light of experience using the NSCAW data in analysis, and the possibility of new or revised measures. Instruments and measures have been working well in NSCAW II, and we are making only minor changes and corrections to the instruments that were used in the first two waves of data collection. Those changes are described in Appendix B.

A.3 The Use of Improved Information Technology to Reduce Burden

With the exception of the Teacher survey, where a mail-in choice will be offered in addition to a web-based questionnaire, all NSCAW II instruments will be programmed for computer-assisted data collection. This computer-assisted interviewing (CAI) technology affords a number of improvements in the collection of survey data. First, this methodology permits more complex routings in the questionnaires, compared to a paper-and-pencil instrument. Given the necessity for a very complex instrument because of the variations in the children’s ages and circumstances, and the detailed information being sought (e.g., the measurement of the child’s cognitive development and emotional well-being through the use of standardized assessments), CAI technology makes possible the administration of these complex questionnaires by interviewers with a level of accuracy that would not otherwise be possible. The questionnaire programs have been developed to implement complex skip patterns based on the child’s age and other variables, and to fill specific wordings based on answers previously provided by the respondent.

A second improvement relates to the consistency of data provided by a respondent. We identified questions that are related and logical errors that respondents might make. If their answers lie outside the logical range, the interviewer is prompted to verify the two seemingly inconsistent pieces of data with the respondent, while their thinking on how the answer was formulated is still fresh. This reduces the need for subsequent data editing, thus saving both time and money. It is likely that respondent-resolved inconsistencies result in data that are more accurate than when inconsistencies are resolved using edit rules. (We do not, however, impose data consistency rules across respondents for the same sampled child, except for a few basic demographic variables like child’s age, race, and gender, nor do we impose consistency across longitudinal waves.)

CAI technology also permits greater expediency with respect to data processing and analysis. A number of back-end processing steps, including editing, coding, and data entry, become a part of the data collection process. Data are transmitted from the field to the RTI central office via modem rather than the mail. These efficiencies save time due to the speed of data transmission.

Sensitive questions asked of adult caregivers and older children are administered via Audio Computer-Assisted Self Interview (ACASI). There is evidence that the ACASI methodology is especially useful for sensitive topics and increases reporting of embarrassing, socially unacceptable, and unlawful behaviors by increasing the respondents’ sense of confidentiality. (Turner et al., 1998; Lessler and O’Reilly, 1997) If one assumes that increased reporting of such behaviors represents increased data accuracy, then the use of ACASI methodology is critical for this particular survey.

Finally, current technologies allow for greater data security. We have implemented a paperless system that obviates the need for consent/assent forms, incentive receipts, authorization for teacher contacting, through the use of an electronic signature pad. These files are transmitted with completed questionnaire data in encrypted files, and reviewed for completeness and accuracy. For any respondent who does not wish to provide an electronic signature, the interviewer completes paper forms, scans them, sends the electronic file, and destroys the paper copy.

A.4 Efforts to Identify and Avoid Duplication

The Adoption and Foster Care Analysis Reporting System (AFCARS) in the Children’s Bureau, NCANDS's Detailed Case Data Component (DCDC), and the Statewide Automated Child Welfare Information Systems (SACWIS) provide aggregate data on child abuse and neglect cases. These data allow researchers to generate population and system level estimates but do not provide child and family well-being measures from multiple sources over time, as the NSCAW does. Thus, this survey is the first national data collection effort of its kind and is the only resource capable of supporting complex analyses (e.g., causal modelling) using longitudinal data.

Regulations establishing AFCARS defined rules and reporting standards for states to use in reporting on children who live in or are adopted from foster care. State reports required annually in order to be in compliance with AFCARS are based on the census of children in care at a specific date; but this does not represent a longitudinal cohort. However, the AFCARS data elements are recorded on all children in care. Foster care data elements in AFCARS include placement beginning and end dates, number of placements during each removal episode, and discharge outcomes for all children in out-of-home care, including family foster care, relative foster care, group homes, emergency shelters, residential facilities and pre-adoptive care.

The NCANDS DCDC is a compilation of case-level information on children who have been investigated by child welfare agencies due to alleged abuse or neglect. It contains 117 data elements. For all cases that are investigated, DCDC includes demographic and geographic data and disposition status (substantiated, indicated, unsubstantiated or closed without finding). For those cases with a finding of substantiated or indicated maltreatment, the record includes data on provision of 24 service types to the child and/or family, including case management, counseling, day care, education, family preservation, mental health services and others. However, DCDC collects only a dichotomous variable stating that the service was or was not provided, and does not record service dates or indicators of utilization level. Currently, most of the states provide DCDC data to NCCANDS, a significant increase from NSCAW I.

All data elements from the DCDC and AFCARS, as well as others, are included in the model data system defined by SACWIS. SACWIS is a comprehensive operational informational system that supports information exchange between multiple service systems. SACWIS data elements for services include service beginning and end dates, and may also include provider names and the reason for service provision. Service categories include all those used by DCDC and multiple additional service types.

Taken together, data elements defined by AFCARS, DCDC and SACWIS represent a common denominator of measures currently in use in many states. There are currently and will continue to be important gaps in the coverage provided by these measures. Most importantly, AFCARS data elements are required for use only for those children in foster care or adopted under state auspices. Comparable information may not be available for children receiving home-based services. Equally important, AFCARS compliance does not necessarily mean that states have the ability to trace children’s placement and service experience longitudinally. The substantial level of adoption for these standardized measures strongly supports their use as appropriate within the NSCAW, but cannot eliminate the need for longitudinal, child- and family-level developmental and well-being data to be collected by this project.

Prior to NSCAW I, we carefully considered the option of adding questions about child abuse and neglect or identifying selected cases for further follow-up from the samples of existing large national surveys such as the National Health Interview Survey sponsored by the National Center for Health Statistics (NCHS) and the Early Childhood Longitudinal Surveys of birth and kindergarten cohorts sponsored by the National Center for Education Statistics (NCES). However, several obstacles to implementing this strategy could not be overcome. The incidence rate of child abuse and neglect is quite small and the sample size of either survey would not have produced adequate numbers of cases for the research objectives of the NSCAW. Additionally, the age ranges for these surveys do not match that needed for this study. Finally, NCHS and NCES staff expressed concerns that the sensitive nature of the questions required to identify instances of child abuse and neglect would be off-putting to many respondents and thereby jeopardize the success of the NHIS and ECLS studies.

A.5 Efforts to Minimize Burden on Small Businesses and Other Entities

This survey does not involve small businesses, but does involve county-level child welfare agencies as the first-stage sampling unit. In developing the study design and data collection procedures, we have made every effort to eliminate or minimize the burden placed on these selected agencies and their staff. For example, the field representative will assemble data necessary for monthly sampling activities and directly contact families. The agency interview will require only about 45 minutes for agency staff to complete with the field representative’s assistance.

A.6 Consequences of Less Frequent Data Collection

Given the purpose of the survey—identifying factors that are associated with child well-being in a very high-risk population—the survey cannot be conducted less frequently during the anticipated study period. Tracking developmental change and emotional and physical well-being status in children undergoing the extreme stresses and traumas encountered in child abuse and neglect situations is especially critical to the research objectives of the NSCAW. In order to measure these in sometimes rapidly changing children and young children developing quickly, the intervals between assessments must necessarily be short. However, resources available limit the number of follow-ups possible.

Researchers generally agree that unaided respondent recall decays significantly after six months. (Biemer et al., 1991) Interviews with adult caregivers would collect more complete and accurate service utilization data with a recall period of six rather than eighteen months; however, for consistency with NSCAW I, we have chosen an eighteen-month follow-up period and will rely on best practices for improving respondent recall. Consideration was given to abstracting service utilization from CPS administrative or case records, but it was concluded that the resulting data would be narrowly confined to only those services funded by the CPS agency, as those would be the only services systematically recorded and available. An option supported by survey methods literature (See for example, Breslin et al., 1995; Kooiker, 1995; Tucker, 1992; McLaws et al., 1990; Sudman, 1971) is diary-keeping, in this application requested of the current adult caregiver at the personal interview. However, we rejected this approach because of our NSCAW I experiences suggesting that these families are in tumult, and engaged in activities of much higher saliency to them than completing NSCAW survey forms. Additionally, we reasoned that a differential in the completeness and quality of service utilization data would exist between types of children in the study if the diary approach were relied upon. For children in out-of-home placements—likely to change between contacts with the study team—the propensity for the new placement family to receive and complete the survey forms without additional information and motivation would be problematic.

A.7 Special Circumstances Requiring Collection of Information Inconsistent with Guidelines

With no exception, this project will produce valid and reliable data that can be generalized to the entire target population, uses no classifications not approved by OMB, will safeguard data, and will prudently share those data only with legitimate researchers through licensing agreements.

A.8 Federal Register Notice and Other Consultation

A.8.1 Federal Register Announcements and Comments

The 60-day federal register announcement was published on January 6, 2011. The federal register citation is: Volume 76, Number 4, Page 778. A copy of the notice, as published, is included in Appendix C. No public comments were received.

A.8.2 Consultation Within the Research Community

To garner ideas for a Request for Proposals for the NSCAW Continuation, ACF held a meeting with key data users on March 16, 2006. The purpose of this meeting was to closely examine issues surrounding analytic use of the NSCAW I data and to set priorities among research needs for additional data. Attendees at this meeting are listed in Exhibit A.8-1.

A consultant group was established once the contract for NSCAW II was awarded. Experts on child welfare agencies and systems, social welfare policy, child and youth development, and other areas serve as members of that Consultant Group. This group meets periodically to provide advice and consultation to the Federal Project Officer and the project team on such areas as policy and research issues; research design, methods, and operations; and priorities and strategies for dissemination of results. In addition, Consultant Group members, and other experts as necessary, participate in survey instrument development, sample design, and other study activities. Exhibit A.8-2 contains a list of the members of the Consultant Group. Exhibit A.8-3 includes members of the NSCAW II Instrumentation teams, including federal, state, and expert advisors.

A.8.3 Federal Consultation

Participation in NSCAW I design and planning from other federal agencies included representatives from the National Institute on Mental Health, the Office of Special Education Programs, the National Center for Education Statistics, the National Center for Health Statistics, the National Institute for Child Health and Human Development, and other agencies. Staff from the National Institute on Mental Health and the Office of Special Education Programs were particularly interested in the NSCAW I and made helpful contributions on mental health and special education services.

Exhibit A.8-1 Participants in the Design and Measures Meeting, March 16^th, 2006

NAME	DEPARTMENT/BRANCH	UNIVERSITY/AGENCY
Rick Barth	School of Social Work	University of Maryland -Baltimore
Cheryl Boyce	Division of Pediatric Translational Research and Treatment Development	National Institute of Mental Health
Barbara J. Burns	Department of Psychiatry/Behavioral Sciences	Duke University
Robert Clyman	Kempe Center	Colorado State University
Byron Egeland	Institute of Child Development	University of Minnesota
E. Michael Foster	School of Public Health	University of North Carolina
William Gardner	Columbus Children’s Research Institute	Ohio State University
Charles Glisson	Children’s Mental Health Services Research Ctr	University of Tennessee
Sally Horowitz	Department of Epidemiology and Biostatistics, School of Medicine	Case Western Reserve University
Brenda Jones-Harden	Department of Human Development	University of Maryland
John Landsverk	Child and Adolescent Services Research Center	Children's Hospital of San Diego
Robert Ortega	School of Social Work	University of Michigan
Matthew Stagner	Center on Labor, Human Services, and Population	The Urban Institute
Mark Testa	Children and Family Research Center Children and Family Research Center	University of Illinois at Urbana-Champaign

Exhibit A.8-2 NSCAW II Consultant Group Members

NAME	DEPARTMENT/BRANCH	UNIVERSITY/AGENCY
Cheryl Boyce	Division of Pediatric Translational Research and Treatment Development	National Institute of Mental Health
Robert Clyman	Kempe Center	Colorado State University
Byron Egeland	Institute for Child Development	University of Minnesota
John Fairbank	Department of Psychiatric and Behavioral Sciences	Duke University
Robert Goerge	The Chapin Hall Center for Children	University of Chicago
Sally Horowitz		Stanford University
Brenda Jones-Hardin	Department of Human Development	University of Maryland
Kelly Kelleher	Columbus Children’s Research Institute	Ohio State University
John Landsverk	Child and Adolescent Services Research Center	Children’s Hospital of San Diego
Robert Ortega	School of Social Work	University of Michigan
Matthew Stagner	The Chapin Hall Center for Children	University of Chicago
Mark Testa	Children and Family Research Center Children and Family Research Center	University of Illinois at Urbana-Champaign

Exhibit A.8-3 NSCAW II Instrumentation Teams

Infancy and Early Childhood	Late Adolescence	Health and Mental Services and Status	CPS/Caseworker and Agency Issues
Cecilia Casanueva, RTI	Greg Aarons, Children’s Hospital-San Diego	Barbara Burns, Duke University	Champana Bernard, Fairfax County Department of Family Services
Rachel Chazan Cohen, ACF	Heather Ringeisen, RTI	Sally Horwitz, Stanford University	Ted Cross, RTI
Mary Dozier, University of Delaware	Cassandra Simmel, Rutgers University	Michael Hurlburt, Children’s Hospital of San Diego	Brecht Donoghue, ACF
Byron Egeland, University of Minnesota	Matt Stagner, University of Chicago	Katina Lambros, Children’s Hospital of San Diego	John Fluke, Walter R. McDonald & Associates
Michael Hurlburt, Children’s Hospital of San Diego	Ellen Wilson, RTI	John Landsverk, San Diego State University, Children's Hospital of San Diego	Charles Glisson, University of Tennessee
Brenda Jones-Harden, University of Maryland	Maria Woolverton, ACF	Laurel Leslie, Children’s Hospital of San Diego	Janet Griffith, ICF Caliber
Laurel Leslie, Children’s Hospital of San Diego		Anne Riley, Johns Hopkins University	Gila Shusterman, Walter R. McDonald & Associates
		Heather Ringeisen, RTI	Sandi Slappey, Fairfax County Department of Family Services
		Aubyn Stahmer, Children’s Hospital of San Diego	Mark Testa, University of Illinois at Urbana-Champaign
		Ruth Stein, Montefiore Medical Center	Rebecca Wells, University of North Carolina at Chapel Hill

A.9 Payment to Respondents

Given the length, complexity, longitudinal design, and sensitive nature of the study, we are requesting cash payments for participants in the NSCAW II, and believe from our experience in NSCAW I that these incentives are necessary to obtain acceptable response rates. We believe that such cash payments are especially relevant due to the inclusion of testing of the children, and the level of suspicion and hostility we anticipate from among the adult caregivers reported to the Child Welfare System. For teachers, we have found that some teachers’ unions require some token payment for survey participation; in addition, completion of the questionnaires often requires teachers to put in additional time beyond their working day.

We requested and had approved for the baseline and 18-month follow-up waves the following incentive amounts, which we request for the 36-month follow-up:

Incentive Amounts by Type of Respondent

	ALL WAVES
Parent/Adult Caregiver/Youth over age 18	$50
Children 10 years of age or younger*	$10
Children 11 years of age or older*	$20
Teacher	$10

* We offer younger children gift cards to toy stores, and for the older children gift cards to music or video stores.

A.10 Assurance of Confidentiality

All data provided by children and families will be kept confidential, with these exceptions: study staff will report in accordance with procedures developed with the participating agency in cases where there is suspicion of ongoing abuse in questionnaire responses. Interviewers may report serious concerns they have, based on their observations in the household. Interviewers will alert an adult caregiver if there is evidence of suicidal intent expressed by a child respondent. These exceptions are the same as those used in NSCAW I and in the first two waves of NSCAW II, and are explained to respondents in the consent and assent forms, completed before the beginning of data collection. There is no evidence to suggest that these procedures and exceptions had any appreciable effects on response rates for either cohort.

Other procedures we use to ensure confidentiality include:

 Responses to sensitive questions are entered directly into the computer by the respondent, without the interviewer being aware of the response. The computer is programmed to flag situations in which there is a risk of ongoing abuse and the need to report. Procedures for study staff to follow in these situations are defined in consultation with the local agency.

 Data are transmitted and stored in such a way that only members of the project team who are authorized and have need to know have access to any identifying information. All project team members have signed confidentiality agreements that provide for termination of employment, civil suit, and financial and other penalties in case of violation.

 All interviewers and other personnel working on the survey must sign affidavits pledging that the data they collect or work with will not be disclosed. Penalties for disclosure include termination of employment and substantial financial fines. These responsibilities are thoroughly explained in the training session for field representatives.

 The contractor has obtained a Certificate of Confidentiality from NIH. This allows researchers to refuse to release identifiable data even under subpoena.

 Reports and data files provided to the research community will not include any individually identifying information.

As noted above, all precautions will be taken against inadvertent disclosure. Identifying information will be maintained in files separate from questionnaire data and analysis files. The contractor protects the project directories and files containing completed questionnaire data and files of identifiers and contacting data through the use of encryption and passwords. The transmission of data from the field staff to the contractor’s central office is protected through file compression and encryption. Field staff computers are protected with strong whole-system encryption.

To protect against systematic disclosure with the release of data into the research community, the project team conducts a Statistical Disclosure Analysis in preparation for construction of General and Restricted Release files. When unacceptable levels of risk of reidentification of participants have been discovered, variables have been suppressed, values recategorized or top coded, or “noise” inserted.

A.11 Justification for Questions of a Sensitive Nature

Since the survey focuses on child abuse and neglect and behaviors thought to be correlated with abuse and neglect, it necessarily deals with a number of topics that are considered confidential and sensitive, such as questions about child maltreatment, alcohol and drug dependence, involvement with the law, and domestic violence. Many of the questions being used have been drawn from surveys of related topics, and our NSCAW I experiences do not suggest that respondents object to these questions. The data are used in analysis to describe the families coming into contact with the child welfare system, to assess the service needs of those families, and to determine factors related to subsequent maltreatment and the effects of maltreatment experiences on child outcomes, as described in Sections A.1 and below in A.16.3.

Respondents are advised of the voluntary nature of participation and their right to refuse to answer any question during the informed consent process. Additionally, at the beginning of the Audio Computer-Assisted Self Interview (A-CASI, in which the respondents hear the questions read by the computer through headphones and enter their responses directly into the computer) portion of the interview containing the most sensitive questions, respondents are reminded of the importance of their honest answers and the strict confidentiality which will be accorded their data, and are also reminded of the exceptions to confidentiality (information indicating suicidal intent or ongoing serious abuse.)

The context in which questions are asked and the auspices of the survey are important factors in overcoming the sensitivity of the subject matter. In NSCAW we take the following steps to create a context which minimizes sensitivity and makes clear to respondents the legitimate need for the information:

 We use laptop computers instead of paper and pencil questionnaires in the NSCAW. The principal privacy concern of respondents in past surveys covering sensitive topics was the possibility that another household member would see their answers on the paper protocol. The computer helps allay those fears. (See Lessler and O’Reilly, 1994.)

 Advance letters, project brochures and interviewer identification badges are used to make clear that the survey is sponsored by ACF and that the information will be put to important uses.

 Field Representatives are carefully and thoroughly trained on project procedures in a ten-day training session.

 The questionnaires are carefully crafted to lead smoothly from one topic to another, with periodic reminders of confidentiality protections.

 As new topics are introduced, the need for the information is explained briefly to the respondent.

 Toward the end of the interviewer-administered interview, we use a self-administered questionnaire with audio capability (ACASI) to collect information on the most sensitive topics. In ACASI, the respondent hears the questions and answer choices read by the computer and enters responses directly into the computer. The use of headphones and a screen-blanking option maximize privacy. For adult caregivers, the ACASI modules include questions on alcohol dependence, drug dependence, involvement with the law, discipline techniques, and domestic violence; the interview for children 11 and older includes questions in ACASI on substance abuse, sexual activity, delinquency, injuries, and maltreatment.

In sum, the quality of the interviewer training, advance information about the survey, the assurances of confidentiality and of the voluntary nature of the interview, the quality of the questionnaire program, and the ACASI administration of the most sensitive questions minimize problems of sensitivity in NSCAW. All interviews are conducted in the most private setting achievable in the household or other setting chosen by the respondent, and no hard copy of the completed questionnaire or other forms are available because all will be administered directly into a laptop computer.

A.12 Estimates of Annualized Hour Burden

Exhibit A.12-1 contains the estimated interview times for each type of respondent in the NSCAW II. Burden was estimated using timing data from interviews.. For the child and caregiver interviews, the questionnaire administration time is averaged over the child age distribution. To compute the total estimated annual cost, the total burden hours were multiplied by the average hourly wage for each adult participant, according to the Bureau of Labor Statistics, Current Employment Statistics Survey, 2011. Estimates are presented below in Exhibit A.12-1.

Exhibit A.12-1 Estimated Response Burden for NSCAW Respondents

RESPONDENT/ INTERVIEW	NUMBER OF RESPONDENTS	NUMBER OF RESPONSES PER RESPONDENT PER YEAR	AVG BURDEN HOURS PER RESPONSE	AVG. ANNUAL BURDEN HOURS	AVG HOURLY WAGE	TOTAL ANNUAL COST
Child Interview	5,873	.5	1.17	3,436
Caregiver Interview	5,873	.5	1.6	4,698	$22.88	$107,499
Caseworker Interview	2,055	2.5	1.0	5,137	$22.88	$117,546
Teacher Questionnaire	2,071	.5	.50	518	$22.88	$11,852
Totals:	15,872			13,789		$236,897

A.13 Estimates of Annualized Cost Burden to Respondents

There are no direct monetary costs to individual participants other than their time to participate in the study.

A.14 Estimates of Annualized Cost to the Government

The estimated annual cost to the government for each year of this contract is $4,962,930.50. This figure is based on a total cost to the Federal government for NSCAW II under the terms of the six-year contract to the RTI International and their subcontractors is $29,777,583. These costs include revision and programming of the questionnaires, design and implementation of the sample, data collection from 5,873 children and associated respondents from 95 Primary Sampling Units (counties) at the baseline, 18-month, and 36-month follow-ups, data processing and analysis, preparation of reports and presentations, and the use of expert consultants. Reimbursement to each agency that participates in the survey, up to $300 per agency, and respondent expenses are included in these costs.

A.15 Reasons for Any Program Changes or Adjustments

Burden estimates have changed as a result of the elimination of agency-director interviews for the 36-month wave. Changes to instruments have been minor, and are not expected to have an appreciable impact on burden. We have deleted the Vineland Social Adjustment Scale, which is part of the caregiver interview; we have added several questions about children’s health that were inadvertently omitted in the previous rounds. Finally, we have added a question about sexual orientation, because of its perceived relevance both to children’s likelihood of being maltreated and to children’s placement and permanency options. Although there is considerable interest in the practice and policy communities around this issue, there is little scientific data on the relationship of sexual orientation to child welfare case dynamics or to children’s outcomes. A summary of these changes is included in Appendix B.

A.16 Time Schedule, Publication and Analysis Plans

A.16.1 Study Schedule

Exhibit A.16-1 provides a schedule of the major activities for the NSCAW II study period.

A.16.2 Publication and Reporting Plans

Reporting plans for NSCAW include submission by the contractor of analysis reports and the production of deliverable data files and documentation. The analysis reports will include basic descriptive analyses, longitudinal analyses of change over time , and additional analyses of policy and program questions identified. In addition to the analysis reports, the project team will produce the following data files:

 CD-ROMs with electronic codebooks (within 8 months after the completion of each data collection wave.) These files will provide analysts with a user-friendly mechanism for examining documentation and creating files for analysis.

 Data files and documentation in accordance with NDACAN guidelines (within 8 months after the completion of each data collection wave.) These data files will provide the research community with access to NSCAW data through licensing agreements with NDACAN. To ensure the preservation of confidentiality of data from children, caregivers, and other respondents, the contractor will conduct thorough statistical disclosure analysis and, as needed, will use such means as excluding, masking, or otherwise altering variables to ensure disclosure is avoided.

Exhibit A.16-1 NSCAW II PROJECT SCHEDULE

Activity	Time frame
	Start date	End date
Revise instrumentation, sample design, and data collection plans	October 2006	July 2007
Recruit states and agencies	July 2007	December 2007
Revise procedures for within-PSU sampling	April 2007	July 2007
Program instruments	April 2007	October 2007
Conduct Pretest	November 2007	November 2007
Obtain OMB clearance	October 2007	January 2008
Obtain human subjects and other reviews	August 2007	January 2008
Recruit and train field staff	November 2007	March 2008
Conduct Wave 1 data collection	April 2008	December 2009
Prepare Wave I analysis report	May 2010	December2010
Release Wave 1 data		May 2010
Conduct Wave 2 data collection	October 2009	January 2011
Release Wave 2 data		June 2011
Prepare Wave 2 analysis report	June 2011	December 2011
Conduct Wave 3 data collection	June 2011	August 2012
Prepare Wave 3 analysis report	October 2012	February 2012
Release Wave 3 data		October 2012

A.16.3 Analysis Plan

The data from NSCAW are archived and available to the research community at the National Data Archive on Child Abuse and Neglect, housed at Cornell University. Most of the analyses of NSCAW data are generated by the research community at large. The Archive provides licensing agreements for two tiers of data release: a restricted release version of the data requires researchers , among other requirements, to obtain IRB approvals, provide a data security plan, and submit to unannounced security inspections. A general release version also is available, but omits many key data elements to provide additional protections against participant reidentification; IRB approvals are also necessary for obtaining the general release data. All government-generated reports from NSCAW are posted on the ACF website and are available to the public.

Our analysis planning will address the most important knowledge gaps about children involved with child welfare, and responsive to ACF’s priorities at the time of concrete planning (the several months just before data become available). This will lead to a plan that will provide timely analyses to address immediate policy and practice questions, and will promote wide use of NSCAW data by investigators in the child maltreatment and child development fields. Large, longitudinal studies like NSCAW require that project staff serve in a lead role not only conducting analyses, but also providing coordination and technical assistance to collaborators across the country involved in analyzing data and disseminating results. An integrated, experienced multidisciplinary team will successfully blend all the diverse elements into an effective approach to lead work on NSCAW once ACF has defined direction.

The planning process will yield a thorough outline that will provide clear direction and focus for analysis. The outline will be organized around the policy priorities and research questions identified by ACF. Each section will include the relevant constructs and the relevant analysis population. Specific independent and dependent variables from the data set will be listed, plus any necessary covariates or interaction variables. Included will be specifications for any variables that need to be derived from interview and/or administrative data. The exact analytic procedures to be used with each research question will be listed as well. Finally, the outline will include table shells that will later be used in NSCAW reports for presenting results. Preliminary analysis with unweighted data will help guide the analysis plan. The proposed outline will be revised based on psychometric assessment of the variables, preliminary analyses, and emerging priorities and research questions. More detail on each step in constructing our analysis plan is below.

Identifying and Articulating the Policy and Practice Concerns that are a Priority to the Field as a Whole. The child welfare field has changed since NSCAW I began. Monitoring and accountability systems have been increasingly emphasized, and the provision of appropriate services to children and families has received particular attention. An example of change in practice is the expansion of the so-called dual track (investigation vs. assessment) system for responding to maltreatment reports. NSCAW II will need to be adapted to current needs of the field. NSCAW has a track record of actively soliciting input from experts across relevant fields, and we will actively engage the Consultant Group on thorough consideration of research priorities. The candidate issues might include the following:

• Linking children and families with services

The first round of Child and Family Service Reviews found deficiencies for most states in terms of ensuring that families and children received services for identified problems. NSCAW data provides an objective assessment of service needs as well as reports from multiple sources on service referral and receipt.

• Overrepresentation of minority populations in child welfare
  Disproportionate representation by race continues to be evident in child welfare agencies across the nation. A new NSCAW cohort will allow examination of change or continuity in these patterns over time.

• The NSCAW II cohort provides the opportunity to examine change in the child welfare workforce characteristics and conditions, and relate these to child and family services and experience with the system.

Translate Policy and Practice Concerns into Testable Research Questions with Clear Constructs. Once research priorities are determined, the team will develop research questions to address these priorities and identify the appropriate constructs to test these research questions. We will provide information on relevant theory and research, through consulting with experts, both within and beyond the Consultant Group.

Research questions and constructs will be at a level of specificity that requires deep knowledge of the child welfare and child development field as well as substantial experience with NSCAW. In choosing research questions and constructs, the research team will take into account the many conceptual subtleties and practical obstacles to matching them well with policy priorities. For example, re-reports in a child maltreatment case open in CWS do not necessarily reflect a failure; increased monitoring may in itself lead to reports that would not otherwise have been made. We will consult with the Consultant Group on the degree to frame this in terms of substantiation—the official CWS decision on maltreatment, but affected by variation in state practice and evidentiary considerations, versus risk assessment—closer to caseworkers’ actual opinion about the child’s safety, but potentially more subjective.

Operationalize Constructs in Terms of NSCAW Variables, Weights, and Populations. Once the above steps are completed, the team will specify NSCAW II variables to operationalize constructs. We will thoroughly assess the data set and select specific variables to match constructs, including programming derived variables. We will (a) take into account multiple variables from the child, caregiver, caseworker, and teacher instruments, (b) be cognizant of what variables were used with which informants in which waves, (c) stay aware of skip rules and coding instructions that are specific to each variable, (d) make sure that code accounts successfully for every case, and e) clearly specify the meaning of every category of the new derived variable.

For each analysis, we will specify in the plan how use of the weights at each wave will ensure the most accurate population estimates. We will also specify which segment of the population will be used in each analysis. Many analyses are appropriate for the whole population, but others must be framed somewhat differently for different segments, and slightly different analyses will result. For example, service delivery can proceed very differently for youths in traditional foster care, kinship care, and in-home, and can depend on whether a family is still engaged with CWS or not.

Target Analyses to Specific Dissemination Products. Part of analysis planning is to envision the final products to be developed and tailor analyses so that results can efficiently be “dropped into place” in products. A key tool is the development of table and figure shells, preliminary graphs in which almost everything is designed and written ahead of time except for the results.

It is premature to specify the exact statistical methods that we will use since these arise from the specific research questions to be addressed. Nevertheless, we can identify several general types of analysis strategies that we will use. First, we will develop accurate population estimates to describe the children involved with child welfare services. Because NSCAW II will be a relatively large national probability sample with sophisticated weighting procedures, it will be possible to use univariate methods to derive estimates (proportions and means in particular) of important variables related to safety, well-being, permanency and service utilization with a high degree of precision. The analysis plan will make calculation and interpretation of these estimates a priority.

Second, we will explore and evaluate a number of predictors of child outcome and service delivery variables, primarily through the use of bivariate methods. These include contingency table (crosstab) analysis with appropriate statistical tests (e.g., Pearson’s ²) and simple regression and correlation procedures. This will enable us, for example, to examine outcomes across age groups and gender, the relationship between maltreatment type and services, and children’s needs relative to service receipt.

We anticipate that multiple predictor analyses will play an important part in the analysis plan. There are circumstances, for example, in which important questions about what variables predict an outcome must use multivariate methods to take into account confounding variables that limit interpretation. Methods with multiple predictors may be needed to address the likelihood that some consumers of NSCAW results will interpret bivariate relationships as causal. Multiple indicator methods like structural equation modeling may be needed to adapt to measurement error that is likely to arise for some NSCAW measures (Biemer et al., 2006). Questions about the relationship between agency variables and child outcomes necessitate multivariate analysis, since they require hierarchical linear models that can take into account sampling both at the agency and child level.

It will also be important to conduct change analyses to assess how children’s safety and permanency are addressed over time, how their development proceeds, and how they recover from maltreatment and other harm. We will use a variety of regression methods that use change scores or use initial levels as covariates to estimate outcomes. For analyses that only involve two waves of data with the same measure, we will employ statistical strategies such as the reliable change index (Jacobson et al., 1999) that provide the ability to control for measurement error and determine rates of clinically significant improvement and deterioration.

Conduct Preliminary Analyses and Plan Final Statistical Methods. Between the end of data collection and the time analysis files are ready, we will prepare assiduously for final analyses. We will begin by constructing programs and conducting analyses with preliminary unweighted data to test use of specific statistical methods with NSCAW II. This will expedite work when the final analysis files become available. The team has substantial experience conducting these analyses with NSCAW, and can make use of dozens of files of existing code. Note that the special requirements of NSCAW preclude using “off the shelf” methods in many circumstances, and require careful programming with sophisticated statistical software such as SUDAAN and MPlus. NSCAW staff are currently working on adapting appropriately sophisticated methods to the special requirements of the data set. For example, a special work group has developed techniques for adapting multilevel modeling to NSCAW I’s specially weighted sample (with children, PSUs or agencies, and possibly states representing different levels).

The final outline will include specific analyses to be conducted, complete with the specific variables to be used, closely tied to research questions and policy priorities. In the spirit of simplicity, most of the analyses will be straightforward and use simple methods, but we will include sophisticated methods in the analysis plan to the extent they are needed for specific research questions.

With all the preparatory work outlined above, we will be well-poised to conduct final analyses. There can be a seamless transition and substantial continuity, with past and current work on NSCAW I feeding immediately into detailed analysis planning in NSCAW II, and setting the stage for final analyses for this contract. Learning curves will not be steep, but experience with previous obstacles will have prepared us for tackling new obstacles that arise, and the productivity of NSCAW work in 2005-2006 can be maintained and then increased.

Although it is premature to include a final analysis plan here, Exhibit A.16-1 presents examples of research questions linked to analytic methods, constructs, and measures in NSCAW II. It includes some questions addressed in NSCAW I that deserve further inquiry, as well as questions that might arise out of emerging priorities related to current policy issues.

Exhibit A.16-1. Elements of Analysis Planning

Examples of Research Questions	Population estimates		Group comparisons/ bivariate correlations	Multivariate models	Change analyses	Constructs	Measures
Well-Being and Child Development
Are children reaching developmental norms for their age? What proportion of children has learning and school difficulties? How does maltreatment affect cognitive development? Does out-of-home placement affect cognitive development? Does special education help ameliorate learning difficulties? How well do children progress in learning over time?	X X	X X		X	X	Cognitive development, learning, school adaptation	Preschool Language Scales, Vineland Adaptive Behavior Scale, Kaufman Brief Intelligence Test, Bayley Infant Neurodevelopmental Screener, Woodcock-Johnson Version III, Battelle Developmental Inventory, Brief Infant Toddler Social Emotional Assessment,
Do children develop adequate social skills during school years? Do children develop adequate peer relationships? Can children in out-of-home placements develop satisfying peer relationships? Do children’s social skills improve over time?	X X	X			X	Social development	Social Skills Scale, Loneliness and Social Dissatisfaction Questionnaire for Young Children, Social Skills Rating System (Teachers), Social Competence – Teacher Report Form
Is children’s physical development adequate? What proportion of children has chronic health problems? Are abuse and neglect related to risk of injuries?	X X X				X	Health	Questionnaire for Identifying Children with Chronic Conditions – Revised
How frequent are mental health problems in this population? How does severity of maltreatment affect mental health? How is placement and stability of placement related to mental health? How does parental substance abuse affect children’s mental health? How do risk and protective factors in combination explain children’s mental health outcomes?	X	X X X		X	X	Behavior problems and mental health	Child Behavior Checklist, Parent Report, Teacher Report Form, and Youth Self Report

Exhibit A.16-1. Elements of Analysis Planning (continued)

Examples of Research Questions	Population estimates	Group comparisons/ bivariate correlations	Multivariate models	Change analyses	Constructs	Measures
Safety
What maltreatment do children experience? To what degree do children suffer multiple victimizations Does CWS response match children’s level of danger? Are children maintained in safe environments over time?	X X	X		X	Maltreatment, violence, and victimization	Project-developed questions on re-reports, Conflict Tactics Scale, Community Environment Scale
Permanence
How often are children placed out of home and in what placements? Is the risk of placement disproportionate for people of color? To what extent is parental substance abuse related to placement? What resources may help prevent placement? Are the qualifications and experience of CWS staff related to placement decisions? How stable are children’s living environments over time?	X	X X X X	X	X	Disruption in living environment	Project-developed questions, UC Berkeley Foster Care Study Questions
How often are children and youths reunified, adopted and emancipated? Are permanency outcomes comparable across racial-ethnic groups? What child, family and resource variables predict reunification? Can permanent living situations be maintained over time?	X	X X	X	X	Permanent living situations	Project-developed questions
Service Delivery
Are children and families receiving social support? Are children and families receiving services? Can caregiver substance abuse services prevent further maltreatment and out-of-home placement? How well does service delivery match needs? How do CWS policies and interventions relate to services delivery? How well and for what duration are services delivered over time?	X X	X X X	X	X	Support and service variables	Duke Functional Social Support Scale Pediatric questions, Early childhood education questions, Child care questions, Child and Adolescent Services Assessment (mental health), Project-developed questions on caregiver services

Considerations in Analysis Planning. Complex, longitudinal data sets like those collected in NSCAW present a substantial number of methodological and analytical challenges that influence analysis planning. Analysis plans will have a number of features that address long-standing ACF research interests related to NSCAW, deal with methodological challenges that have become clear in analyses of the NSCAW I data, and take advantage of unique opportunities to expand research to address important questions more effectively. Several of these challenges are presented below.

Children “Aging Out” of and “Into” Measures. One challenge of longitudinal data is that children change dramatically as they mature. The relevance of constructs and the utility of measurement methods varies depending on developmental level and chronological age. Thus children “grow out” of measures used in early childhood, and “grow into” other measures of constructs relevant to older children. It is insufficient to treat these measures and age groups in isolation because many of the underlying constructs are nevertheless the same or related for different age groups, and because it is essential for advancement of knowledge for developmental trends to be assessed across developmental stages.

Therefore, analysis planning will find methods to bridge these gaps so that certain constructs can be followed across different measures. One choice is to eschew change scores, instead using multiple regression to explain child outcomes in later waves while statistically controlling for children’s baseline levels. Another approach is to identify categories on each measure that represent a clinical or resilient status. We can then examine the extent to which those who initially score in the clinical range on Measure A at a younger age are not on a related Measure B at a later age. In some circumstances it is reasonable to compare standardized scores across different measures, to determine, for example, if children two standard deviations below the mean at Wave 1 are closer to the mean on a related measure at a later wave, although we will need to take care to avoid regression to the mean artifacts (see Campbell and Kenny, 1999). As noted previously, analyses of clinically significant change on standardized measures with differing content and norms at different ages can also be used to determine rates of improvement and deterioration (Walrath, Ybarra, and Holden, 2006).

Capturing and Isolating the Effects of Services. Instrumentation has been augmented to include more detail on the specifics of services children receive. In addition to type of services, other dimensions to be captured include frequency, duration and content of services received. Our analysis plan will include specific analyses of when and how often services are provided, how that varies by child age, maltreatment type, and other case characteristics, and how the CWS intervention relates to other services. Furthermore, we will consider using strategies such as latent class analysis (LCA) (Lanza, Flaherty, and Collins, 2003), to understand underlying classes within the services mix and to explore the relationships among case mix, service mix, and outcomes. This will increase understanding of the synergistic effects of different complex services patterns on child outcomes for varying maltreatment histories.

Hierarchical, Nested Data. NSCAW has substantial data from each state, PSU, and each child. In NSCAW I, data on PSUs were gathered from the Census and from state and local agency surveys (US Department of Health and Human Services, 2001a, b). In NSCAW II, we will also have data from each PSU and each state, from NCANDS, AFCARS, and other sources.

Important research questions about the impact of local and state policy and practices on children need to be addressed by hierarchical linear modeling (HLM) (Byrk and Raudenbush, 1992). HLM develops linear models simultaneously at multiple levels of analysis, here both child and PSU (or child welfare agency). This would treat analysis of children’s data as nested within each agency, and examine variation across agencies as well as across children. HLM can also deal with state effects as well; however, the relatively small number of states in the study would somewhat limit the power of the analysis. Nevertheless, these analyses will allow us to capture effects that are at the level of state and agency as well as at the level of individual children.

HLM is a commonly used analytic approach, but methods for using HLM in weighted data sets have not been well developed. Care will be taken to handle the sophisticated NSCAW weights correctly in HLM. For example, the recent literature suggests that both agency- and child-level weights should be used for two-level models, yet not all software packages have implemented this feature (see, for example, Pfeffermann et al., 1998; Asparouhov, 2004). Project staff have developed these methods for NSCAW I, which will be available for use with NSCAW II.

Longitudinal Comparisons Involving Multiple Sampling Cohorts. Combined with NSCAW I, NSCAW II provides an opportunity to examine recent historical change in child welfare populations, CWS response, child and family services, and child outcomes. The advantage is magnified if NSCAW II is implemented in the same PSUs as NSCAW I so that the change in the PSU sample can be eliminated as an explanation for differences. Such analysis can help assess whether policy and practices changes made over the course of a decade have affected children.

We will use both simple and sophisticated methods as appropriate to compare NSCAW I and NSCAW II results. For some analyses, simple comparison of means and percentages using regression and contingency table methods (e.g., Pearson ²) can be calculated, comparing trends both before and after policy changes. The use of growth curve modeling methods is also possible, although special provisions need to be made to splice together curves associated with two independent samples of children within the PSUs. Growth curve analysis identifies the particular pattern or “curve” each child has over time rather than treating their scores at each time point only in the aggregate. We can summarize trends in growth curves across both samples, and estimate the effect of agency- and PSU-level changes as well as variations in child characteristics, victimization histories, and CWS and other services on child growth. This will allow us to assess not only the net effect of these variables on children’s trajectories, but also the impact of these variables via developmental processes. We will work on methods for making useful comparisons between growth curves from NSCAW II and changes that can be estimated with two data points in NSCAW II.

Bias Arising from Non-Response. An investigation has been conducted in order to provide information on the extent of the bias arising from unit nonresponse in the baseline wave the failure to obtain an interview from a NSCAW II sample member. An estimate of the nonresponse bias is the difference between the sample estimate (based only on respondents) and a version of the sample estimate based upon respondents and nonrespondents. In the NSCAW II, a limited amount of frame information is available for sample children who did not respond to the survey. Thus, it is possible to compare nonrespondents and respondents for some characteristics in order to investigate the potential nonresponse bias in the NSCAW II results. There is also bias in NSCAW II due to frame undercoverage; in particular, unsubstantiated cases were not included on the sampling frame in a few large states, and an adjustment was made to account for this. At future waves, differential attrition of the sample may introduce bias; thus, non-response analysis will be conducted following each wave of data collection.

A.17 Display of Expiration Date

The OMB number and expiration date will be displayed at the beginning of each questionnaire program and in a box on the consent forms. These will be printed on the front cover of the Teacher Questionnaire.

A.18 Exceptions to Certification Statement.

There are no exceptions to the certification statement.

References

Barth, R. P.,  Shenyang, G., Weigensberg, E., Christ, S. L. Bruhn, C. M., & Green, R. L. (2009). Explaining Reunification and Reentry 3 Years After Placement in Out-of-Home Care. . In M. B. Webb, K. Dowd, B. Jones Harden, J. Landsverk & M. F. Testa (Eds.), Child welfare and child well-being: New perspectives from the National Survey of Child and Adolescent Well-Being. New York: Oxford University Press.

Burns, B.J., Phillips, S.D., Wagner, H.R., Barth, R.P., Kolko, D.J., Campbell, Y., and Landsverk, J. (2004). “Mental Health Need and Access to Mental Health Services for Youths Involved with Child Welfare: A National Survey.” Journal of the American Academy of Child and Adolescent Psychiatry, 43(8), 960-970.

Glisson, C. (2010). Organizational climate and service outcomes in child welfare systems. In M. B. Webb, K. Dowd, B. Jones Harden, J. Landsverk & M. F. Testa (Eds.), Child welfare and child well-being: New perspectives from the National Survey of Child and Adolescent Well-Being. New York: Oxford University Press.

Glisson, C., and Green, P. (2005). The Effects of Organizational Culture and Climate on the Access to Mental Health Care in Child Welfare and Juvenile Justice Systems. Administration and Policy in Mental Health and Mental Health Services Research, 33(4), 433-448.

Hazen, A., Connelly, C., and Kelleher, K. (2004). “Intimate Partner Violence Among Female Caregivers of Children Reported for Child Maltreatment.” Child Abuse and Neglect, 28(3), 301-319.

Horwitz, S., Hurlburt, M., Farmer, E., and Zhang, J. (2010). “The Patterns and Predictors of Mental Health Services Use by Children in contact with the Child Welfare System.” In M. B. Webb, K. Dowd, B. Jones Harden, J. Landsverk & M. F. Testa (Eds.), Child welfare and child well-being: New perspectives from the National Survey of Child and Adolescent Well-Being. New York: Oxford University Press.

Hurlburt, M.S. Leslie L.K., Landsverk, J., Barth, R.P., Burns, B.J., Gibbons, R.D., Slymen, D.J., and Zhang, J. (2004). “Contextual Predictors of Mental Health Service Use Among Children Open to Child Welfare.” Archives of General Psychiatry, 61, 1217-1224.

Jee, S.H., Antonucci, T.C., Aida, M, Szilagyi, M.A, .and Szilagyi, P.G. (2005). “Emergency Department Utilization by Children in Foster Care.” Ambulatory Pediatrics, 5, 102-106.

Leslie, L.K., Hurlburt, M.S., Landsverk, J., Rolls, J.A., Wood, P.A., and Kelleher, K.J. (2003). “Comprehensive Assessments for Children Entering Foster Care: A National Perspective.” Pediatrics, 112(1), 134-142.

Lindsey, D. (2003). The Welfare of Children. New York: Oxford University Press.

Ortega, R. M., Grogan-Kaylor, A., Ruffolo, M., Clarke, J., & Karb, R. (2010). Racial and ethnic diversity in the initial child welfare experience: Exploring areas of convergence and divergence. In M. B. Webb, K. Dowd, B. Jones Harden, J. Landsverk & M. F. Testa (Eds.), Child welfare and child well-being: New perspectives from the National Survey of Child and Adolescent Well-Being. New York: Oxford University Press.

Stahmer, A.C., Leslie, L.K., Hurlburt, M., Barth, R.P., Webb, M.B., Landsverk, J., and Zhang, J. (2005). “Developmental and Behavioral Needs and Service Use for Young Children in Child Welfare.” Pediatrics, 116, 891-900.

Testa, M. F., Bruhn, C., & Helton, J. (2010). Comparative safety, stability, and continuity of children's placements in formal and informal substitute care. In M. B. Webb, K. Dowd, B. Jones Harden, J. Landsverk & M. F. Testa (Eds.), Child welfare and child well-being: New perspectives from the National Survey of Child and Adolescent Well-Being. New York: Oxford University Press.

U.S. Department of Health and Human Services, Administration for Children and Families (2005). National Survey of Child and Adolescent Well-Being (NSCAW) CPS Sample Component Wave 1 Data Analysis Report. Washington, DC: DHHS.

Webb, M.B. and Harden, B.J. (2003) “Beyond Child Protection: Promoting Mental Health for Children and Families in the Child Welfare System.” Journal of Emotional and Behavioral Disorders, 11(1), 39-58.