Supporting Statement Part A_FINAL

National Survey of Child and

Adolescent Well-Being Second Cohort

(NSCAW II)

OMB Supporting Statement Part A

Submitted by:

Mary Bruce Webb, Ph.D.

Administration for Children and Families

370 L’Enfant Promenade SW

Washington, DC 20447

PART A: JUSTIFICATION

The Administration for Children and Families (ACF) requests a revision of the current clearance to conduct an 18-month follow-up wave of data collection (Wave 2) for the National Survey of Child and Adolescent Well-Being II, referred to herein as NSCAW II. ACF conducted NSCAW I through five waves of data collection (1999-2007). Baseline data collection on a new cohort (NSCAW II) began in March, 2008 and is ongoing (OMB control number 0970-0202). The current clearance expires on May 31, 2010. Data collection on the baseline wave is scheduled to be completed in August 2009. Wave 2 data collection is scheduled to begin in October 2009. The following documentation provides detailed information about progress on and results of the work conducted under the existing clearance for NSCAW II and the planning process and procedures to be used to conduct the remainder of the study.

A.1 Explanation of Circumstances That Make the Information Collection Necessary

Title 42 of the US Code Section 628b authorizes the Secretary to conduct a longitudinal child welfare study. The NSCAW II fulfills the intent of the reauthorization legislation. See Appendix D for a copy of this legislation.

This section examines existing research to demonstrate how the planned study both meets the legislative mandate and also provides vital information for decision-making about policy and practice within the child welfare system (CWS). The need is described for a new cohort for this national longitudinal study of children and families who come into contact with the child welfare system, with data obtained from interviews with multiple informants over the study period. Additionally, the conceptual model on which the study design and instrument selection activities have been based is presented.

The study is being conducted through a contract with RTI International and its subcontractors, Caliber/ICF and Walter R. McDonald Associates.

Since the beginning of NSCAW I in 1999, the context within which the state and local child welfare agencies operate has changed, with increasing demands and fewer resources (e.g., Lindsey, 2003). Client characteristics have changed, as immigration patterns have broadened past large, urban areas. Caseworker turnover continues to be a problem in staffing roles at the frontline. Reform efforts are aiming toward community-based integrated systems of care and other alternatives (Webb and Harden, 2003). Federal legislation and policy have imposed greater accountability on agencies and now levy penalties if standards are not met. All the while agencies are reacting to these demands, state budget cuts have diminished the resources available to meet these and other challenges.

NSCAW I Results

Prior to NSCAW, most research on children and families within the child welfare system used a cross-sectional or cohort design, which limited conclusions about trajectories of care, long-term child and family outcomes, or impact of agency practices over time. Furthermore, much of the data on service patterns within this population had to be drawn exclusively from administrative records. Such records usually included only limited service data or did not capture the full range of the child welfare system (e.g., including only children in foster care). Through its prospective longitudinal design, NSCAW represents a major advance to child welfare research and other fields, and has been a platform from which much has been learned.

Child Safety and Well-Being

NSCAW I has illustrated the unique challenges facing children and families involved in the child welfare system (CWS). Baseline and 18-month reports of child and family data suggest that many of these children are being abused and neglected; maltreatment also seems to persist for many over time. Over one-fifth of children overall were the subject of at least one re-report to the CWS by the 18 month follow-up (DHHS, forthcoming). Compared to normative data, an especially large number of children had significant threats to their well-being and development. The well-being of children involved with the CWS remained about the same at 18 months as at baseline. There were some areas of improvement (reasonable academic achievement scores for age 6-10, decreased suicide attempts for adolescents, increased contraceptive use among the sexually experienced), and more children improved than worsened (DHHS, forthcoming).

Permanence

NSCAW I data are being used to examine the CWS progress toward the important goal of achieving permanence of children’s living environment. Permanency was common: over three quarters of children and youths remained with their families across all four waves of the study (Testa, Bruhn, and Helton, in press), and over 90% who started with their family ended with them as well. At baseline a modest 11% of children had been removed from the home (DHHS, 2005). On the other hand, only 51% of children in informal kin care at baseline remained in their home through all four waves. Considering children who were removed from their home, at 36 months, 28.8% of youths aged 0 to 6 and 39.9% of youths aged 7 and older were not in permanent settings (Barth et al., in press).

NSCAW is a resource to explore new ideas for achieving supportive permanent environments for children caught in impermanent homes. Researchers and policy-makers are beginning to scrutinize traditional adherence to the values of continuity of family and family-like placement arrangements after a child’s safety has been ensured. For example, in part spurred by economic, technological, and societal changes, researchers and policy-makers have begun to consider the overall “social capital” (e.g., education opportunities, health care options) of a placement setting as important, in addition to stability and continuity, when assessing options (Testa, Bruhn, and Helton, in press).

Service Needs and Use

Despite tremendous need, many children within the child welfare system receive few services if any. For instance, nearly half of the youth in NSCAW I with completed child welfare investigations were determined to have clinically significant emotional or behavioral problems; yet only one quarter of these youth had received any specialty mental health care during the previous 12 months (Burns et al., 2004; Stahmer et al., 2005) found high levels of developmental and behavioral needs among toddlers (approximately 42%) and preschoolers (68%) in NSCAW I, with only 23% of these young children receiving relevant services. Leslie, Hurlburt, and colleagues (2003) found that less than half of the communities in NSCAW I provided comprehensive health and developmental examinations to all children entering out of home care. Meanwhile, a large percentage of the children in foster care had visited an emergency room within a 1‑year period (Jee et al., 2005).

Disparities

Child maltreatment differentially impacts children of different racial and ethnic groups as well as children from different locations (Ortega et al., in press). Disparities also exist in terms of access to services. For instance, African American children appear less likely to receive specialty mental health services than white children involved in the child welfare system (Horwitz, Hurlburt, Zhang, in press). Interestingly, Hurlburt and colleagues (2004) found that linkages between child welfare and mental health at the local level increased mental health access for youth, particularly among ethnic minority youth. This study, in particular, underscores the importance of understanding the impact of contextual factors amenable to change within the child welfare system that might improve the quality of services provided.

Impact of Child Welfare Service Context

We have also learned that provider and agency characteristics influence the quality of services received by families within the child welfare system as well as their outcomes. For example, recent research using NSCAW I demonstrates that organizational culture can affect service quality and outcomes independent of the individual characteristics of either the service provider or the children and families receiving services (Glisson, in presss). Having a positive work culture within a child welfare agency increases the chance that a child needing mental health care will actually receive it (Glisson and Green, 2005). These findings stress the continued importance of gathering data at the provider, agency, and policy levels within future NSCAW efforts.

It is important to note that the impact of the NSCAW longitudinal study extends beyond that funded directly by ACF. NSCAW I data are licensed through the National Data Archives for Child Abuse and Neglect to over 100 researchers who have made multiple conference presentations, and published numerous book chapters and at least 70 articles in refereed journals. In addition, the NSCAW I study has served as a platform for multiple collaborative opportunities used to expand on existing data. For instance, the National Institute of Mental Health supported the Caring for Children in Child Welfare (CCCW) study which supplemented NSCAW I county and agency data to better understand contextual influences on service use and outcomes. The National Institute of Justice provided funds to examine intimate partner violence among female caretakers within NSCAW (Hazen, Connelly and Kelleher, 2004). The MacArthur foundation funded efforts to combine NSCAW I data with data from provider organizations in NSCAW Primary Sampling Units (PSUs) to assess the impact of agency organizational characteristics on child and family outcomes (Glisson, 2007). NSCAW I has been instrumental in supporting these collaborations and intends to actively foster collaborative relationships with scientists interested in expanding upon or capitalizing on the NSCAW I data.

Use of a Conceptual Framework

The original NSCAW I study was grounded by a conceptual framework that emphasized the child and family’s environmental context as well as the dynamic nature of an ever-changing child welfare system. The conceptual framework acknowledged that many families have repeated contact with child welfare services and that such prior experience with this system likely shapes subsequent decisions about their future. Furthermore, decisions made by the child welfare system and court were considered to be a function of both child and family characteristics as well as governmental and agency practices, which were in turn linked to policies, regulations, resources, norms, and values. This broad ecological framework for NSCAW I has not been changed for the current proposal; however, Exhibit A.1-1 illustrates the critical need to portray the child welfare system and the families served by it in their updated and current context.

Our goal continues to be to provide data at each level of this framework, to provide for an ecological understanding of children’s development following contact with the CWS. Such a goal includes gathering information related to child and family characteristics, characteristics of a family’s natural community, characteristics of child welfare service agencies and their providers, the types and qualities of services received by families served by the child welfare system, and the policies and regulations that impact those services provided. We also aim to inform sites and state administrators not only about the characteristics of their populations but also about agency and worker policies and practices, services provided, and other key findings with a bearing on child and family outcomes.

Exhibit A.1-1 Ecological Framework for NSCAW

The Continuing Need for Longitudinal Research in Child Welfare

Longitudinal research is needed to understand patterns of care and how those vary by agency, community, family type, service history, and child’s developmental stage, in the current context. Further, such research will help to disentangle the influence that different patterns of care have on the developmental outcomes for children and their families. One of the major foci of NSCAW II is to more completely measure and track service utilization to facilitate that understanding. Additionally, NSCAW II will make use of administrative data advances such as submissions to AFCARS and NCANDS, and the more prevalent sophisticated agency database systems that now exist. These data will be important for accurately tracking additional reports and investigations involving the child and family, changes in out-of-home placements, and, potentially, service utilization.

Additional longitudinal research is needed to understand how to make services (and perhaps policies) more responsive to child and family needs. Although the ages of children receiving child welfare services has changed, there has been no concomitant change in policy or practice that might reflect a developmental perspective; our child welfare law does not speak to the ages of children. Prior to 1996, there was some attention to placing foster care in a developmental framework, but these efforts were modest and had little impact on public policy. As early as 1975, Wald questioned the one-law-fits-all notion that allowed child welfare law to be silent on the issue of the child's age; he asked: How can we build in developmental knowledge to make the laws more sophisticated and more likely to serve the best interests of children? ... (1975, p. 11). Longitudinal research may identify service, family, and child characteristics that interact to show the way toward developmentally sound policies.

To answer important questions for future policymaking, child welfare practice, and effective resource allocation, the conduct of the National Survey of Child and Adolescent Well-Being (NSCAW) II is essential. In the next section, we describe the design of the study and the specific research questions to be addressed.

A.2 How the Information Will Be Collected and For What Purposes

A.2.1 Overview of the Design of the Study

 The NSCAW II cohort will include 5,341 children, aged birth to 17 ½ years, who have contact with the child welfare system within a one-year period that began in February, 2008. These children are sampled from those entering the system during the reference period.

These 5,341 children are being selected from 81 of the NSCAW I 92 Primary Sampling Units (PSUs) in 83 counties nationwide that agreed to participate in NSCAW II. Retaining most of the NSCAW I PSUs will allow researchers to assess the change in context from the late 1990s, as well as enabling longitudinal analysis of organizational measures such as staff turnover, climate, and work environment The sample of investigated/assessed cases includes both cases that receive on-going services and cases that are not receiving services, either because they were not substantiated or because it was determined that services were not required.

This sample design—with oversampling of infants and children in out of home placement, and undersampling of cases not receiving services to ensure appropriate representation among subgroups of particular instance—allows in-depth analysis of subgroups of special interest (e.g., young children and adolescents in foster care) while providing national estimates for the full population of children and families entering the system. Children in out-of-home placement are especially important to our analyses to examine the process and outcomes of earlier permanency planning. Infants are oversampled to ensure sufficient numbers of cases to assess the outcomes of child maltreatment and services, including out-of-home placements, on early childhood development. However, the sample design has been revised to exclude the calculation of state level estimates for the states with the largest numbers of CPS cases since the NSCAW I data have rarely been used for state estimates. Similarly, the NSCAW I sample design included 727 children selected from among children in out-of-home placement for about 12 months; this sample component has not been repeated in NSCAW II because the data have not been often utilized. Section B.1 details the NSCAW II sample design. Exhibit B.1-1 provides the exact sample allocation and effective sample sizes.

The NSCAW is a longitudinal study with multiple informants associated with each sampled child, in order to get the fullest possible picture of that child. A baseline round of face-to-face interviews or assessments is being conducted with children, parents, and non-parent adult caregivers (e.g., foster parents, kin caregivers, group home caregivers), and investigative caseworkers. Administrative data – those prepared for submission for other purposes – and Census data will be appended to questionnaire data files, for community context, to improve data quality, and to reduce agency burden. Data collection began in March, 2008, and includes one wave of follow-up interviews in 2009-2010 (Wave 2), 18 months after the close of the NSCAW II index investigation/assessment. Additional rounds of follow-up data collection may be conducted, depending on the availability of resources in future years. Both children who remain in the system and those who leave the system will be followed for the full study period.

NSCAW II also includes several supplemental studies. One study by a state CWS agency has increased the size of the sample in the state so estimates can be made at the state region level, and the impacts of the state’s policies examined closely. Additionally, two groups of researchers will be analyzing NSCAW data (from cases where the legal guardian provided signed permission) merged with Medicaid data to estimate the costs of child maltreatment and to study how state and county policies impacts the use of various specific services and treatments (e.g., prescription of psychotropic drugs).

A.2.2 Purpose of the Study

NSCAW is designed to address crucial program, policy, and practice issues of concern to the federal, state, and local governments, and child welfare agencies. NSCAW I was the first national study of child welfare to collect data from children and families, and the first to relate child and family well-being to family characteristics, experience with the child welfare system, community environment, and other factors. The study examines the interplay among the history and characteristics of children and families, their experiences with the child welfare system, other concurrent life experiences, and outcomes. It brings to bear perspectives from child welfare, child development, and other fields to focus on children’s well-being, including their health and physical well-being, social functioning, academic achievement, mental health, and behavioral adjustment. It relates these to developmental stage, prior experience, caregiver behavior, social services, and community environment. By drawing on these different perspectives, the study provides more complete understanding of how family, child, community, and service factors affect children’s well-being and will provide the foundation for improving policies, programs, and practices. The major research questions the study will address include:

 Who are the children and families that come into contact with the child welfare system in 2008-2009?

 What pathways and services do children and families experience while in the child welfare system?

 What are the shorter- and longer-term outcomes for these children and families, especially as related to permanence, safety, well-being, and service utilization?

Examples of questions within each of these areas are included in Exhibit A.2-1.

We have prepared a document which summarizes preliminary plans for analyzing the wealth of data that will flow from NSCAW. Reports planned are similar to those released from NSCAW I, and will focus on the four areas of ACF policy interest – permanence, safety, well-being, and service utilization. Following each wave of data collection, data from the survey will be analyzed by the project team. Additionally, after being stripped of identifying information and analyzed for the possibility of inadvertent disclosure, data sets from NSCAW II will be made available to the larger research and policy community to encourage secondary analyses that will support further research and timely policy decisions. The National Data Archive for Child Abuse and Neglect (NDACAN) at Cornell University is the repository for NSCAW I data, and currently holds licenses with 127 principal investigators; NDACAN will continue to hold the NSCAW data licenses and support users.

Our analyses will focus on the key study issues described above. Examples of the cross-sectional and longitudinal analyses to be performed include:

 Description of characteristics and risk factors for children and families at the point of entry into the child welfare system, overall and for subgroups;

 The investigation/assessment process (e.g., risk factors, decisions, family involvement);

 Children’s and families’ experience of child welfare and other services, and changes in services and placements during their period in the child welfare system;

 The process of permanency planning and implementation for children in out of home care;

 Description of children and families who leave the system quickly and those who stay in for a longer period;

 Analysis of relationship of child, family, caseworker, agency, and other factors to child and family services and outcomes;

 Analysis of how the organization, structure, and resources of agencies relate to the services provided and to whom.

Exhibit A.2-1 Examples of Questions NSCAW Will Address

 Who are the children and families who come into contact with the child welfare system in 2008-2009?

 What are their backgrounds and characteristics?

 What are their prior histories?

 What problems and strengths do they bring?

 How do the characteristics, experiences, and needs of children and families differ by the ways they come into contact with the system?

 What effects do state and agency policies and programs have on the characteristics of those who enter the system?

 What pathways and services do children and families experience while in the child welfare system?

 What placements and services do they experience while they are in the child welfare system?

 What determines the different pathways, placements, and services they experience?

 How do child welfare services interact with other services and supports for children and families involved with the child welfare system?

 What are the shorter- and longer-term outcomes for these children and families?

 How do children and families change during the time they are in contact with the child welfare system?

 How do children and families change after they leave the system?

 How do child, family, system, community, and other factors influence child and family functioning?

 How do these factors affect subsequent child welfare system involvement?

The primary focus of the study is on children and families; however, because data are collected from child welfare agencies, it will also be possible to conduct some limited analyses at the agency level. Agency level data (e.g., staff turnover, use of dual tracking, budget) and caseworker data (e.g., level of experience, specialized training) will be used in analyses of child and family services and outcomes. In addition, data collected during the sampling process will be used to describe such aspects of the child welfare system as outcomes of completed cases (e.g., substantiation rates) and the disposition of substantiated cases (e.g., rates of case opening, placement rate), overall and for subgroups.

A.2.3 Information Elements and Data Sources

Section A.1 presented an ecological-developmental theory of risk and resiliency to examine the antecedents and consequences of child maltreatment and the outcomes of contacts with the child welfare system. With this model in place, we developed the NSCAW I data collection instruments to operationalize this theoretical model of child maltreatment. This involved developing an instrumentation package that validly and reliably measures the constructs of the conceptual model to the highest standards of scientific accuracy, across a broad range of child ages. The selection of instrumentation also took into account the methodological and administrative considerations relevant to collecting data in a cost effective manner that does not burden respondents unduly, provides for the potential consequences of inquiring about sensitive behaviors and topics, and adequately deals with the requirements of a diverse multicultural population of interest. We used the NSCAW I instruments as a starting point for NSCAW II, and made revisions based on feedback from data users and from expert consultants.

Instrument Development Process

The task of instrumentation revision was taken on by a group of experts in child welfare, child maltreatment, child development, social welfare service provision and utilization, psychometrics, survey methodology, and survey research. Members included staff from ACF, RTI, Caliber/ICF, WRMA, and several members of the Consultant Group; several experts were consulted regarding specialized topics. (See Section A.8 for a full description of the Consultant Group and consultants.) Prior to initiating the NSCAW II Instrumentation task, experts were identified to consult on one or more of the following instrumentation work groups: 1) Infancy and early childhood, 2) Late adolescence, 3) Health and mental health status and services, and 4) Child Protective Services/Caseworker and agency issues. These teams met regularly through conference calls and discussed the selection of instruments. The decision-making process was interactive and iterative. The draft instrumentation package was presented to the full Consultant Group in a meeting on May 29, 2007, and their feedback was incorporated.

The overarching goal of instrumentation work group calls was threefold: 1) to identify modules or items that should be updated to reflect changes in policy or ACF priorities since the original NSCAW was launched in 1998, and 2) to identify modules or items that required modification due to issues identified via data analysis of variables and measures in the Wave 1-5 datasets, and 3) to identify measures that have been revised since the original set of instruments were programmed and that should be replaced with new editions. The general decision was made that the development of new measures or the substantial adaptation of existing measures (especially partial deletion) should be avoided, to minimize risks to the implementation of the project and maximize utility of the data. For these reasons, wherever possible instruments for collecting family and child well-being outcomes and system/agency characteristics were chosen from the body of existing instruments, and their use without modification was recommended.

The NSCAW II baseline (Wave 1; currently in progress) instruments and burden have already been reviewed and approved by OMB. The minor changes and additions planned for Wave 2 are described in Section A.15.

Criteria for Choosing NSCAW I Measures

Because of the heavy reliance on NSCAW I measures, it is important then to describe the approach in the original study to the processes by which existing measures with previously established properties were evaluated and selected. The following criteria were used by the NSCAW I Instrument Development Team (IDT).

Psychometric Properties of Standardized Instruments: Each standardized instrument was rigorously evaluated in terms of its psychometric properties: reliability, validity, standardization sample, type of norms. However, because the NSCAW I included a unique target population, even when an instrument was considered reliable and valid, we considered how applicable the norms would be.

Non-Standardized Instruments: Because there were many important constructs for which there were no available standardized measures, we relied on other surveys of similar populations that asked questions about child and family well-being and the social service system. For these instruments we considered how successfully the potential instrument was used in other similar studies.

Age Applicability: Because the project was designed to study children from birth through adolescence, the issue of the sensitivity of measures to changes in age was very important. For instance, the indicators and strategies for assessment of mental health change over the age span. The Instrument Development Team considered instruments that would provide measurement of characteristics are as comparable as possible across the age span.

Administration Factors: There are a variety of administration factors that the Instrument Development Team considered. With fixed resources and a large sample of children to be assessed, issues of training, materials, scoring complexity, analysis, cost, and participant response burden were of major importance. The instruments were selected so that the respondent burden fit within the time parameters budgeted in the data collection plan.

Cultural and Linguistic Considerations: The Instrument Development Team considered instruments in terms of the cultural assumptions of the respondent groups and not just those of the researchers’ own culture. We also considered the process of how translation would affect the connotations, or implied associations, of the words.

Other Considerations: While the criteria listed above factored into the review and evaluation of the individual instruments, other issues were apparent only after the assessment instruments were considered collectively. The ordering of instruments was one example. Another consideration was the suitability of the instruments for computerization—the use of open-ended questions where considerable typing of responses would be needed, the use of long lists of response options, screen layout considerations, and the need for “show cards” or other interviewer-presented materials during Audio Computer-Assisted Self-Interview (ACASI) administration, for example.

Obtaining information about child and family functioning and outcomes: Data sources include the child, caregiver(s), caseworkers, and teachers, but the importance of these data sources vary by age. Moreover, the timing of the data collection for a particular child may coincide with a transition to a new home or school even in the absence of frequent moves. Our choice of measurement instruments needed to reflect this by including multiple data sources. Inclusion of multiple data sources also permits different perspectives to be included

Obtaining information about systems and agencies: In assessing the usefulness of agency measures, we assessed how well they describe a) the characteristics of systems and agencies, and b) the types and characteristics of services provided to children and families. We focused on those system, agency and service measures most directly related to child and family outcomes for different types of families and children.

For the NSCAW II instruments, every effort was made to minimize the burden of data collection on both participating agencies and on respondents. To achieve those objectives, system and agency measures recently collected within other research, and/or those that are collected on an ongoing basis by other organizations were actively sought out and assessed.

At the state level, NSCAW II will use data collected for other purposes to the extent they are appropriate to our objectives, in order to reduce burden on participating agencies and retain data comparability across sites. This survey’s usefulness to the larger research community is enhanced if system and agency measures are defined in a way that facilitates comparability with other studies. In instances where measures have not been collected at the local levels, or where available data are not sufficiently timely, the appropriateness of adapting measures that have been developed within related research was assessed.

Questionnaire Domains

Exhibit A.2-2 presents the measurement constructs included in the questionnaire for each type of respondent: child, parent, teacher, and caseworker. The ages of the child for which the section is applicable is also included, as is the rationale for inclusion of the constructs.

Exhibit A.2-2 Child, Family, and Caseworker Constructs, Their Rationale and the Data Source in NSCAW II Data Collection

Construct	Child Age	Data Source	Rationale
Child Areas
1. Social Competence, Relationships			Critical for adaptation in a variety of domains and a frequent area of disruption for children with histories of maltreatment.
Relationships with parents and other significant adults	< 11 > 11	Observation Child	Warm and supportive relationships between children and adults can buffer children against stresses and help children heal from negative effects of maltreatment.
Peer relationships	> 5 > 5	Child Teacher	Success in making and keeping friendships is linked to better school adjustment. Peer friendships provide children a support system and model for future relationships. Peer rejection is related to adolescent conduct disorder.
Global Social Competence	> 5	Caregiver/ Teacher	Children with better developed social skills have greater success in forming social relationships and better long-term academic and occupational achievement.
2. Health, Cognitive status			Health and intellectual functioning are among the most important indicators of well-being and have an influence on development in other domains.
Developmental/cognitive status	All	Child	Cognitive functioning and neurodevelopmental status are important mediators of school success. Cognitive status will be assessed at two visits for children <6 and at one visit for children > 6.
Communication skills	< 6	Child	Language skills are the foundation for literacy skills taught in school.
Health and Disabilities	All ages	Caregiver	Children with chronic health conditions and disabilities are more likely to experience maltreatment. Health status and injuries can be a direct consequence of maltreatment. Health and injuries influence the extent to which children can participate in activities.
3. Adaptive behavior, Functional status			Adaptive behavior reflects competence in achieving personal independence and meeting social demands such as academic adjustment and performance.
Adaptive Skills, Socialization, Communication	All Ages	Caregiver	The ability to function in daily life is an important marker of adjustment. For adolescents in out of home placements, living skills become an important focus of attention as they transition to independent living.

Exhibit A.2-2 Child, Family, and Caseworker Constructs, Their Rationale and

the Data Source in NSCAW II Data Collection (Continued)

Construct	Child Age	Informant	Rationale and other Comments
Academic achievement (including attendance, grade progression)	> 6 > 6	Child Teacher	Academic achievement and completion of high school are critical for future economic viability.
Special ed. status and educational, developmental and support services received	All ages	Teacher/ Caregivers Caseworkers	Children’s developmental and educational needs affect their participation in school and social activities. Documentation of services addressing these needs an indicator of how CW system operates.
School socialization	> 6	Teacher	Maltreated children often experience difficulties in social as well as academic aspects of school adjustment. The ability to function socially in the classroom is highly related to academic performance.
School engagement	> 6	Teacher	Motivations are affected by early experiences and in turn influence children’s dispositions towards learning and school.
Future Expectations	> 10	Child	These expectations are a by-product of the child’s life experiences.
4. Behavior regulation, Emotional and Mental Health			Behavioral and emotional processes are developed as part of the caregiver-child relationship and can be disrupted when this relationship is impaired. Their successful development, along with mental health, is the hallmark of adjustment and well-being.
Temperament	<3	Caregiver	Temperament can either act as a risk or a resiliency factor by influencing how a child relates to others. It is also one of the antecedents of self-regulation.
Behavior Problems	> 2 >5 > 11	Caregiver Teacher Child	Behavior problems are the earliest signs of maltreatment, especially difficulty with impulse control and aggressive behavior. Externalizing problems interfere with peer acceptance and academic performance, and a precursor to later delinquency. Often setting specific, this measurement requires multiple informants.
Mental health	> 2 > 5 > 7	Caregiver Teacher Child	Mental health is a broad construct that affects all aspects of well-being. In children it is highly associated with behavioral problems. Conduct disorders and other externalizing problems can be assessed with the CBCL. For internalizing problems such as depression and anxiety, we obtain both parent and child reports.
Criminality/delinquency	> 11	Child	Elevated risk of delinquency and criminality among maltreated individuals. Avoidance is a marker of successful social adaptation.
Substance Abuse and Risky Behaviors	> 11	Child	Drug abuse and early sexual activities are associated with depression and maltreatment.

Exhibit A.2-2 Child, Family, and Caseworker Constructs, Their Rationale and the Data Source in NSCAW II Data Collection (Continued)

Construct	Child Age	Informant	Rationale and Other Comments
5. Life experiences
Child Maltreatment	All Ages > 11 All Ages	Caregiver Child Caseworker	It is critical to get more than the maltreatment information that forms the basis of the report.
Family/placement disruptions	All Ages	Caregiver	Frequent placement changes and household disruptions are related to poorer child outcomes
Loss, violence and other stressors in and out of the home	> 5	Child	All violence that is experienced (i.e., viewed as well as directly experienced) has a negative impact on mental health and on how children handle conflicts themselves.
6. Service experiences	All ages > 11	Caregiver/ Caseworker Child	Documentation of services received is critical to understanding the service provision process, the factors that affect the process, and the relationship among individual/family variables, services, and outcomes.
Parent /Caregiver Areas
1. Health Status	All ages		The health of the caregiver affects the functioning of the caregiver and thus how s/he relates to the child.
Mental health and Substance Abuse		Caregiver	Psychiatric disorders, especially parental depression, can be especially harmful to the quality of the parent-child relationship. Mental health disorders may predispose a parent to maltreatment, and they may also be an outcome of maltreatment. Substance abuse is one of the reasons for reports to CPS. Substance abuse is also associated with parenting difficulties.
Physical health		Caregiver	Affects how well a parent can provide caregiving and function in the larger world.
Services received by biological parent		Investigative and Services Caseworkers	Services received by parents may be critical to their ability to provide appropriate care for their children. If parents do not receive needed services, reunification may be unlikely.
2. Caregiver attributes/behaviors			Parenting attitudes and behaviors are on a continuum, from positive and supportive to negative, with child maltreatment falling at the negative extreme.
Parent/caregiver behaviors Emotional nurturing and Cognitive/verbal responsiveness and stimulation	< 11	Caregiver/ Observation	Supportive parent-child relationships are the foundation upon which all developmental achievements are built. Parental sensitivity to child’s needs and interests, parental ability to structure an interaction, and acceptance of child are highly related to quality of the emotional bond between parent and child. Verbal and behavioral responsiveness are highly linked to children’s communication and intellectual competence.

Exhibit A.2-2 Child, Family, and Caseworker Constructs, Their Rationale and the Data Source in NSCAW II Data Collection (Continued)

Construct	Child Age	Informant	Rationale and Other Comments
Behavioral Monitoring and Discipline	All ages < 11	Caregiver Caregiver/ Observation	Use of appropriate discipline promotes socialization and behavioral self-regulation in children rather than short-term compliance. It is frequently under the guise of discipline that parents justify physical maltreatment. At the other extreme, lax supervision and the failure to provide any limits can cross the boundary from leniency to neglect.
3. Contextual factors	All		These are influences on caregivers and children that affect perceived stress and well-being.
Neighborhood factors	All	Caregiver	The behavior of Individuals and families has to be understood in terms of the environment of their community.
Family Demographics	All	Caregiver	Will provide background characteristics, roster of who lives in the home, employment, education, and other descriptive information.
Social support and other family resources, including assistance with child-rearing	All	Caregiver	Perceived social support is believed to buffer the child and family against stress, thereby helping them better cope with their problems.
Domestic Violence in the Home	All	Caregiver	Domestic violence is highly associated with child maltreatment, is a source of stress for the child, and may itself be considered a form of psychological maltreatment.
Criminal Involvement of Parents	All	Caregiver	A background factor that is associated with maltreatment.
Risk Assessment	All	Investigative Caseworker	The relationship between caseworker, perceived risk and strengths and long term safety, and well-being to be assessed.
Caseworker Areas
Construct	Child Age	Caseworker	Rationale and Other Comments
Job role	All	Services	Related to child outcomes
Work unit	All	Services
Caseload	All	Services	Caseload level determines amount of time worker is likely to be able to spend on services to sample child/family.
Work environment and Job satisfaction	All	Services	Work environment and job satisfaction are believed to influence worker’s job performance and turnover
Demographics	All	Investigative and Services	Measures of caseworker demographics and professional background; also allows comparison of caseworker race/ethnicity with that of sample child and family.

In summary, the constructs chosen for inclusion in NCAW II are identical to those selected for the NSCAW I survey, and were identified from an understanding of the models used to explain the causes and consequences of maltreatment and how the child welfare system relates to child and family outcomes. Those decisions have been reexamined in light of experience using the NSCAW data in analysis, and the possibility of new or revised measures.

A.3 The Use of Improved Information Technology to Reduce Burden

With the exception of the Teacher survey, which is conducted by mail and web-based interview with telephone prompting of nonrespondents, all NSCAW I instruments were programmed for computer-assisted data collection. We will continue use of these modes and technologies in NSCAW II Wave 2. This computer-assisted interviewing (CAI) technology affords a number of improvements in the collection of survey data. First, this methodology permits more complex routings in the questionnaires, compared to a paper-and-pencil instrument. Given the necessity for a very complex instrument because of the variations in the children’s ages and circumstances, and the detailed information being sought (e.g., the measurement of the child’s cognitive development and emotional well-being through the use of standardized assessments), CAI technology makes possible the administration of these complex questionnaires by interviewers with a level of accuracy that would not otherwise be possible. The questionnaire programs have been developed to implement complex skip patterns based on the child’s age and other variables, and to fill specific wordings based on answers previously provided by the respondent.

A second improvement relates to the consistency of data provided by a respondent. We identified questions that are related and logical errors that respondents might make. If their answers lie outside the logical range, the interviewer is prompted to verify the two seemingly inconsistent pieces of data with the respondent, while their thinking on how the answer was formulated is still fresh. This reduces the need for subsequent data editing, thus saving both time and money. It is likely that respondent-resolved inconsistencies result in data that are more accurate than when inconsistencies are resolved using edit rules. (We do not, however, impose data consistency rules across respondents for the same sampled child, except for a few basic demographic variables like child’s age, race, and gender, nor do we impose consistency across longitudinal waves.)

CAI technology also permits greater expediency with respect to data processing and analysis. A number of back-end processing steps, including editing, coding, and data entry, become a part of the data collection process. Data are transmitted from the field to the RTI central office via modem rather than the mail. These efficiencies save time due to the speed of data transmission.

Sensitive questions asked of adult caregivers and older children are administered via Audio Computer-Assisted Self Interview (ACASI). There is evidence that the ACASI methodology is especially useful for sensitive topics and increases reporting of embarrassing, socially unacceptable, and unlawful behaviors by increasing the respondents’ sense of confidentiality. (Turner et al., 1998; Lessler and O’Reilly, 1997) If one assumes that increased reporting of such behaviors represents increased data accuracy, then the use of ACASI methodology is critical for this particular survey.

Finally, current technologies allow for greater data security. Through the use of an electronic signature pad, we have implemented a paperless system that obviates the need for paper consent/assent forms, incentive receipts, and authorization for teacher contacting,. These files are transmitted with completed questionnaire data in encrypted files, and reviewed for completeness and accuracy. For any respondent who does not wish to provide an electronic signature, the interviewer completes paper forms, scans them, sends the electronic file, and destroys the paper copy.

A.4 Efforts to Identify and Avoid Duplication

The Adoption and Foster Care Analysis Reporting System (AFCARS) in the Children’s Bureau, NCANDS's Detailed Case Data Component (DCDC), and the Statewide Automated Child Welfare Information Systems (SACWIS) provide aggregate data on child abuse and neglect cases. These data allow researchers to generate population and system level estimates but do not provide child and family well-being measures from multiple sources over time, as the NSCAW does. Thus, this survey is the first national data collection effort of its kind and is the only resource capable of supporting complex analyses (e.g., causal modeling) using longitudinal data.

Taken together, data elements defined by AFCARS, DCDC and SACWIS represent a common denominator of measures currently in use in many states. There are currently and will continue to be important gaps in the coverage provided by these measures. Most importantly, AFCARS data elements are required for use only for those children in foster care or adopted under state auspices. Comparable information may not be available for children receiving home-based services. Equally important, AFCARS compliance does not necessarily mean that states have the ability to trace children’s placement and service experience longitudinally. The substantial level of adoption for these standardized measures strongly supports their use as appropriate within the NSCAW, but cannot eliminate the need for longitudinal, child- and family-level developmental and well-being data to be collected by this project.

A.5 Efforts to Minimize Burden on Small Businesses and Other Entities

This survey does not involve small businesses, but does involve county-level child welfare agencies as the first-stage sampling unit. In developing the study design and data collection procedures, we have made every effort to eliminate or minimize the burden placed on these selected agencies and their staff. For example, the field representative will directly contact families.

A.6 Consequences of Less Frequent Data Collection

Given the purpose of the survey—identifying factors that are associated with child well-being in a very high-risk population—the survey cannot be conducted less frequently during the anticipated study period. Tracking developmental change and emotional and physical well-being status in children undergoing the extreme stresses and traumas encountered in child abuse and neglect situations is especially critical to the research objectives of the NSCAW. In order to measure these in sometimes rapidly changing children and young children developing quickly, the intervals between assessments must necessarily be short. However, resources available limit the number of follow-ups possible. For consistency with NSCAW I, we have chosen an eighteen-month follow-up period and rely on best practices for improving respondent recall

A.7 Special Circumstances Requiring Collection of Information Inconsistent with Guidelines

With no exception, this project will produce valid and reliable data that can be generalized to the entire target population, uses no classifications not approved by OMB, will safeguard data, and will prudently share those data only with legitimate researchers through licensing agreements.

A.8 Federal Register Notice and Other Consultation

A.8.1 Federal Register Announcements and Comments

Specific to this request for a 3-year extension, a Federal Register announcement was posted on April 13, 2009 in the Federal Register, Volume 73, Number 69, Page 16875. No comments were received. The announcement is included as Appendix A.

The second Federal Register notice for was published in the Federal Register, Volume 74, Number 139, Page 36237 on July 22, 2009. A copy of the 30-day notice is included in Appendix B.

A.8.2 Consultation Within the Research Community

To garner ideas for a Request for Proposals for the NSCAW Continuation, ACF held a meeting with key data users on March 16, 2006. The purpose of this meeting was to closely examine issues surrounding analytic use of the NSCAW I data and to set priorities among research needs for additional data. Attendees at this meeting are listed in Exhibit A.8-1.

A consultant group was established once the contract for NSCAW II was awarded. Experts on child welfare agencies and systems, social welfare policy, child and youth development, and other areas serve as members of that Consultant Group. This group meets periodically to provide advice and consultation to the Federal Project Officer and the project team on such areas as policy and research issues; research design, methods, and operations; and priorities and strategies for dissemination of results. In addition, Consultant Group members, and other experts as necessary, participated in survey instrument development, sample design, and other study activities. Exhibit A.8-2 contains a list of the members of the Consultant Group. Exhibit A.8-3 includes members of the NSCAW II Instrumentation teams, including federal, state, and expert advisors.

A.8.3 Federal Consultation

Participation in NSCAW I design and planning from other federal agencies included representatives from the National Institute on Mental Health, the Office of Special Education Programs, the National Center for Education Statistics, the National Center for Health Statistics, the National Institute for Child Health and Human Development, and other agencies. Staff from the National Institute on Mental Health and the Office of Special Education Programs were particularly interested in the NSCAW I and made helpful contributions on mental health and special education services.

Exhibit A.8-1 Participants in the Design and Measures Meeting, March 16^th, 2006

NAME	DEPARTMENT/BRANCH	UNIVERSITY/AGENCY
Rick Barth	School of Social Work	University of Maryland -Baltimore
Cheryl Boyce	Division of Pediatric Translational Research and Treatment Development	National Institute of Mental Health
Barbara J. Burns	Department of Psychiatry/Behavioral Sciences	Duke University
Robert Clyman	Kempe Center	Colorado State University
Byron Egeland	Institute of Child Development	University of Minnesota
E. Michael Foster	School of Public Health	University of North Carolina
William Gardner	Columbus Children’s Research Institute	Ohio State University
Charles Glisson	Children’s Mental Health Services Research Ctr	University of Tennessee
Sally Horowitz	Department of Epidemiology and Biostatistics, School of Medicine	Case Western Reserve University
Brenda Jones-Harden	Department of Human Development	University of Maryland
John Landsverk	Child and Adolescent Services Research Center	Children's Hospital of San Diego
Robert Ortega	School of Social Work	University of Michigan
Matthew Stagner	Center on Labor, Human Services, and Population	The Urban Institute
Mark Testa	Children and Family Research Center Children and Family Research Center	University of Illinois at Urbana-Champaign

Exhibit A.8-2 NSCAW II Consultant Group Members

NAME	DEPARTMENT/BRANCH	UNIVERSITY/AGENCY
Cheryl Boyce	Division of Pediatric Translational Research and Treatment Development	National Institute of Mental Health
Robert Clyman	Kempe Center	Colorado State University
Byron Egeland	Institute for Child Development	University of Minnesota
John Fairbank	Department of Psychiatric and Behavioral Sciences	Duke University
Robert Goerge	The Chapin Hall Center for Children	University of Chicago
Sally Horowitz	Center for Health Policy, Pediatrics	Stanford University
Brenda Jones-Hardin	Department of Human Development	University of Maryland
Kelly Kelleher	Columbus Children’s Research Institute	Ohio State University
John Landsverk	Child and Adolescent Services Research Center	Children’s Hospital of San Diego
Robert Ortega	School of Social Work	University of Michigan
Matthew Stagner	The Chapin Hall Center for Children	University of Chicago
Mark Testa	Children and Family Research Center Children and Family Research Center	University of Illinois at Urbana-Champaign

Exhibit A.8-3 NSCAW II Instrumentation Teams

Infancy and Early Childhood	Late Adolescence	Health and Mental Services and Status	CPS/Caseworker and Agency Issues
Cecilia Casanueva, RTI	Greg Aarons, Children’s Hospital-San Diego	Barbara Burns, Duke University	Champana Bernard, Fairfax County Department of Family Services
Rachel Chazan Cohen, ACF	Heather Ringeisen, RTI	Sally Horwitz, Stanford University	Ted Cross, RTI
Mary Dozier, University of Delaware	Cassandra Simmel, Rutgers University	Michael Hurlburt, Children’s Hospital of San Diego	Brecht Donoghue, ACF
Byron Egeland, University of Minnesota	Matt Stagner, University of Chicago	Katina Lambros, Children’s Hospital of San Diego	John Fluke, Walter R. McDonald & Associates
Michael Hurlburt, Children’s Hospital of San Diego	Ellen Wilson, RTI	John Landsverk, San Diego State University, Children's Hospital of San Diego	Charles Glisson, University of Tennessee
Brenda Jones-Harden, University of Maryland	Maria Woolverton, ACF	Laurel Leslie, Children’s Hospital of San Diego	Janet Griffith, ICF Caliber
Laurel Leslie, Children’s Hospital of San Diego		Anne Riley, Johns Hopkins University	Gila Shusterman, Walter R. McDonald & Associates
		Heather Ringeisen, RTI	Sandi Slappey, Fairfax County Department of Family Services
		Aubyn Stahmer, Children’s Hospital of San Diego	Mark Testa, University of Illinois at Urbana-Champaign
		Ruth Stein, Montefiore Medical Center	Rebecca Wells, University of North Carolina at Chapel Hill

A.9 Payment to Respondents

Given the length, complexity, longitudinal design, and sensitive nature of the study, we request approval to provide cash payments to participants in the NSCAW II, in recognition of their time and their likelihood of incurring incidental expenses such as daycare for other children. We believe that such cash payments are especially relevant due to the inclusion of testing of the children, and the level of suspicion and hostility we anticipate from among the adult caregivers reported to the Child Welfare System. For teachers, we have found that teacher’s unions require some token payment for survey participation.

We began the NSCAW I baseline study with incentives of $25 (cash) to adult caregivers, $10 (gift card) to children over 10 years old, and $10 (cash) to teachers. However, we found that these amounts were not compelling to children and caregivers, given the length and complexity of the interviews. We requested and had approved in July 2000 increased amounts, which are still in effect for NSCAW II – see Exhibit A.9-1 below. An increase in response rates was observed during the time that these increased incentives were offered (approximately 10 months into the study).

As Exhibit A.9-1 indicates, some children will reach the age of 18 or older during the follow-up wave (Wave 2). During the fifth wave of NSCAW I, we were approved to provide young adult respondents with the same incentive amount given to adult caregiver respondents (i.e., $50). We propose to continue this practice in the second wave of NSCAW II.

Exhibit A.9-1 Incentive Amounts by Study Wave and by Type of Respondent

Respondent Type	Baseline (Wave 1)	Wave 2
Parent/Adult Caregiver	$50	$50
Children 10 years of age or younger*	$10	$10
Children 11 years of age or older*	$20	$20
Young Adult (sampled child who has reached the age of 18 years or older)	N/A	$50
Teacher	$10	$10

* We offer younger children gift cards to toy stores, and for the older children gift cards to music or video stores.

A.10 Assurance of Confidentiality

The project has been granted a federal Certificate of Confidentiality (see Appendix C) from the National Institute of Mental Health (NIMH), Department of Health and Human Services (DHHS) that covers any data collected through 2012. This allows researchers to refuse to release identifiable data even under subpoena.

All data provided by children and families will be kept confidential, with these exceptions: study staff will report in accordance with procedures developed with the participating agency in cases where there is suspicion of ongoing serious abuse in questionnaire responses. Interviewers may report serious concerns they have, based on their observations in the household. Interviewers will alert an adult caregiver if there is evidence of suicidal intent expressed by a child respondent. These exceptions are explained to respondents in the consent and assent forms, completed before the beginning of data collection.

Other procedures we use to ensure confidentiality include:

 Responses to sensitive questions are entered directly into the computer by the respondent, without the interviewer being aware of the response. The computer is programmed to flag situations in which there is a risk of ongoing abuse and the need to report. Procedures for study staff to follow in these situations are defined in consultation with the local agency.

 Data are transmitted and stored in such a way that only members of the project team who are authorized and have need to know have access to any identifying information. All project team members have signed confidentiality agreements that provide for termination of employment, civil suit, and financial and other penalties in case of violation.

 All interviewers and other personnel working on the survey must sign affidavits pledging that the data they collect or work with will not be disclosed. Penalties for disclosure include termination of employment and substantial financial fines. These responsibilities are thoroughly explained in the training session for field representatives.

 Reports and data files provided for public use will not include any individually identifying information.

As noted above, all precautions are taken against inadvertent disclosure. Identifying information is maintained in files separate from questionnaire data and analysis files. The contractor protects the project directories and files containing completed questionnaire data and files of identifiers and contacting data through the use of encryption and passwords. The transmission of data from the field staff to the contractor’s central office is protected through file compression and encryption. Field staff computers are protected with strong whole-system encryption.

To protect against systematic disclosure with the release of data into the research community, the project team conducts a Statistical Disclosure Analysis in preparation for construction of General and Restricted Release files. When unacceptable levels of risk of reidentification of participants have been discovered, variables have been suppressed, values recategorized or top coded, or “noise” inserted.

A.11 Justification for Questions of a Sensitive Nature

Since the survey focuses on child abuse and neglect and behaviors thought to be correlated with abuse and neglect, it necessarily deals with a number of topics that are considered confidential and sensitive, such as questions about child maltreatment, alcohol and drug dependence, involvement with the law, and domestic violence. Many of the questions being used have been drawn from surveys of related topics, and our NSCAW I experiences do not suggest that respondents object to these questions. The data are used in analysis to describe the families coming into contact with the child welfare system, to assess the service needs of those families, and to determine factors related to subsequent maltreatment and the effects of maltreatment experiences on child outcomes, as described in Sections A.1 and below in A.16.3.

Respondents are advised of the voluntary nature of participation and their right to refuse to answer any question during the informed consent process. Additionally, at the beginning of the Audio Computer-Assisted Self Interview (A-CASI, in which the respondents hear the questions read by the computer through headphones and enter their responses directly into the computer) portion of the interview containing the most sensitive questions, respondents are reminded of the importance of their honest answers and the strict confidentiality which is accorded their data, and are also reminded of the exceptions to confidentiality (information indicating suicidal intent or ongoing serious abuse.)

The context in which questions are asked and the auspices of the survey are important factors in overcoming the sensitivity of the subject matter. In NSCAW we take the following steps to create a context which minimizes sensitivity and makes clear to respondents the legitimate need for the information:

 We use laptop computers instead of paper and pencil questionnaires in the NSCAW. The principal privacy concern of respondents in past surveys covering sensitive topics was the possibility that another household member would see their answers on the paper protocol. The computer helps allay those fears. (See Lessler and O’Reilly, 1994.)

 Advance letters, project brochures and interviewer identification badges are used to make clear that the survey is sponsored by ACF and that the information will be put to important uses.

 Field Representatives are carefully and thoroughly trained on project procedures in a ten-day training session.

 The questionnaires are carefully crafted to lead smoothly from one topic to another, with periodic reminders of confidentiality protections.

 As new topics are introduced, the need for the information is explained briefly to the respondent.

 Toward the end of the interviewer-administered interview, we use a self-administered questionnaire with audio capability (ACASI) to collect information on the most sensitive topics. In ACASI, the respondent hears the questions and answer choices read by the computer and enters responses directly into the computer. The use of headphones and a screen-blanking option maximize privacy. For adult caregivers, the ACASI modules include questions on alcohol dependence, drug dependence, involvement with the law, discipline techniques, and domestic violence; the interview for children 11 and older includes questions in ACASI on substance abuse, sexual activity, delinquency, injuries, and maltreatment.

In sum, the quality of the interviewer training, advance information about the survey, the assurances of confidentiality and of the voluntary nature of the interview, the quality of the questionnaire program, and the ACASI administration of the most sensitive questions minimize problems of sensitivity in NSCAW. All interviews are conducted in the most private setting achievable in the household or other setting chosen by the respondent, and no hard copy of the completed questionnaire or other forms are available because all are administered directly into a laptop computer.

A.12 Estimates of Annualized Hour Burden

Exhibit A.12-1 contains the estimated annual burden, estimated using actual timing data from NSCAW II interviews conducted to date with caregivers, children, and caseworkers. Burden for the Wave 2 Caseworker interview, as well as burden for the teacher and local agency interview was estimated via simulated interviews conducted with project team members during testing of the questionnaire applications. For the child and caregiver interviews, the questionnaire administration time is averaged over the child age distribution.

Exhibit A.12-1 Estimated Annual Response Burden for NSCAW II Respondents

Instrument	Number of Respondents	Number of Responses per Respondent	Average Burden Hours per Response	Total Burden Hours	Average Hourly Wage	Total Annual Cost
Child Interview	1,424	1.0	1.33	1,894	n/a	n/a
Caregiver Interview	1,424	1.0	1.9	2,706	$15.03	$40,671.18
Caseworker Interview	285	3.0	1.0	855	$25.60	$21,888.00
Teacher Questionnaire	855	1.0	0.50	428	$25.60	$10,956.80
Estimated Total				5,883		$73,515.98

To compute the total estimated annual cost, the total burden hours were multiplied by the average hourly wage for each adult participant, according to the Bureau of Labor Statistics, Current Population Survey, 2008. We also used the CPS table from the first quarter of 2009 for education (detailed occupation earnings are only available annually). For parents, we used the mean salary for full-time employees over the age of 25 who are high school graduates with no college experience ($15.03 per hour). For caseworkers and teachers, we used the mean salary ($25.60) for full-time employees over the age of 25 with a bachelor’s degree.

A.13 Estimates of Annualized Cost Burden to Respondents

There are no direct monetary costs to individual participants other than their time to participate in the study.

A.14 Estimates of Annualized Cost to the Government

The estimated annual cost to the Federal government is $4,773,054.40. The total cost to the Federal government for NSCAW II under the terms of the contracts to the RTI International and their subcontractors is $23,865,272. These costs include revision and programming of the questionnaires, design and implementation of the sample, data collection from children and associated respondents from 81 Primary Sampling Units (83 counties) at the baseline and 18-month follow-up, data processing and analysis, preparation of reports and presentations, and the use of expert consultants. Agency and respondent expenses are included in those costs. Payment is made to the agencies to cover out-of-pocket costs related to participation in the study; these range from $100.00 up to $1000.00 per year.

A.15 Reasons for Any Program Changes or Adjustments

The one new measure not administered at baseline, but planned for Wave 2 is the Organizational Social Climate Scale (The University of Tennessee Children’s Mental Health Services Research Center, Copyright 2006, 2000, 1998, 1988, 1978). This measure is intended to collect detailed information about the caseworker’s experiences with and perceptions of their agency.

The following additional Wave 2 changes are noted:

• Various logic errors identified during the baseline Wave in the child, caregiver, and caseworker questionnaires have been corrected for Wave 2.

• Updates to the consent and assent forms were made so the language is consistent with a follow-up wave (Wave 2). See Appendix E.

• Consistent with the original NSCAW protocol, caseworker interviews will be obtained only for those families who have received services paid for or provided by the child welfare agency since the baseline interview.

• Because the NSCAW II cohort sampled children up to the age of 17 ½ years at baseline, Wave 2 will involve data collection from children who will then be considered young adults (18 years and older). Consistent with the original NSCAW protocol, these young adults will sign a consent form and complete the child interview, but no caregiver or caseworker interviews will be pursued. See Appendix E.

A.16 Time Schedule, Publication and Analysis Plans

A.16.1 Study Schedule

Exhibit A.16-1 provides a schedule of the major activities for the NSCAW II study period covered in this request for extension (i.e., completion of baseline and Wave 2 data collection, data analysis, and data delivery efforts).

Exhibit A.16-1 NSCAW II Project Schedule

Activity	Time frame
Obtain OMB clearance to conduct Wave 2	Fall 2009
Prepare Wave I analysis report	February 2010 - November 2010
Release Wave 1 data	March 2010
Conduct Wave 2 data collection	October 2009 - November 2011
Release Wave 2 data	June 2012
Prepare Wave 2 analysis report	November 2012 - January 2013

A.16.2 Publication and Reporting Plans

Reporting plans for NSCAW include submission by the contractor of analysis reports and the production of deliverable data files and documentation. The analysis reports will include basic descriptive analyses, longitudinal analyses of change over time (Wave 2 only), and additional analyses of policy and program questions identified. In addition to the analysis reports, the project team will produce the following data files:

 CD-ROMs with electronic codebooks (within 8 months after the completion of each data collection wave.) These files will provide analysts with a user-friendly mechanism for examining documentation and creating files for analysis.

 Data files and documentation in accordance with NDACAN guidelines (within 8 months after the completion of each data collection wave.). These data files will provide the research community with access to NSCAW data through licensing agreements with NDACAN. To ensure the preservation of confidentiality of data from children, caregivers, and other respondents, the contractor will conduct thorough statistical disclosure analysis and, as needed, will use such means as excluding, masking, or otherwise altering variables to ensure disclosure is avoided.

A.16.3 Analysis Plan

Our analysis planning will address the most important knowledge gaps about children involved with child welfare, and responsive to ACF’s priorities at the time of concrete planning (the several months just before data become available). This will lead to a plan that will provide timely analyses to address immediate policy and practice questions, and will promote wide use of NSCAW data by investigators in the child maltreatment and child development fields. Large, longitudinal studies like NSCAW require that project staff serve in a lead role not only conducting analyses, but also providing coordination and technical assistance to collaborators across the country involved in analyzing data and disseminating results. An integrated, experienced multidisciplinary team will successfully blend all the diverse elements into an effective approach to lead work on NSCAW once ACF has defined direction.

The planning process will yield a thorough outline that will provide clear direction and focus for analysis. The outline will be organized around the policy priorities and research questions identified by ACF. Each section will include the relevant constructs and the relevant analysis population. Specific independent and dependent variables from the data set will be listed, plus any necessary covariates or interaction variables. Included will be specifications for any variables that need to be derived from interview and/or administrative data. The exact analytic procedures to be used with each research question will be listed as well. Finally, the outline will include table shells that will later be used in NSCAW reports for presenting results. Preliminary analysis with unweighted data will help guide the analysis plan. The proposed outline will be revised based on psychometric assessment of the variables, preliminary analyses, and emerging priorities and research questions. More detail on each step in constructing our analysis plan is below.

Translate Policy and Practice Concerns into Testable Research Questions with Clear Constructs. Once research priorities are determined, the team will develop research questions to address these priorities and identify the appropriate constructs to test these research questions. We will provide information on relevant theory and research, through consulting with experts, both within and beyond the Consultant Group.

Research questions and constructs will be at a level of specificity that requires deep knowledge of the child welfare and child development field as well as substantial experience with NSCAW data. In choosing research questions and constructs, the research team will take into account the many conceptual subtleties and practical obstacles to matching them well with policy priorities. For example, re-reports in a child maltreatment case open in CWS do not necessarily reflect a failure; increased monitoring may in itself lead to reports that would not otherwise have been made.

Operationalize Constructs in Terms of NSCAW Variables, Weights, and Populations. Once the above steps are completed, the team will specify NSCAW II variables to operationalize constructs. We will thoroughly assess the data set and select specific variables to match constructs, including programming derived variables. For each analysis, we will specify in the plan how use of the weights at each wave will ensure the most accurate population estimates. We will also specify which segment of the population will be used in each analysis. Many analyses are appropriate for the whole population, but others must be framed somewhat differently for different segments, and slightly different analyses will result. For example, service delivery can proceed very differently for youths in traditional foster care, kinship care, and in-home, and can depend on whether a family is still engaged with CWS or not.

Target Analyses to Specific Dissemination Products. Part of analysis planning is to envision the final products to be developed and tailor analyses so that results can efficiently be “dropped into place” in products. A key tool is the development of table and figure shells, preliminary graphs in which almost everything is designed and written ahead of time except for the results. It is premature to specify the exact statistical methods that we will use since these arise from the specific research questions to be addressed. Nevertheless, we can identify several general types of analysis strategies that we will use. First, we will develop accurate population estimates to describe the children involved with child welfare services. Because NSCAW II will be a relatively large national probability sample with sophisticated weighting procedures, it will be possible to use univariate methods to derive estimates (proportions and means in particular) of important variables related to safety, well-being, permanency and service utilization with a high degree of precision. The analysis plan will make calculation and interpretation of these estimates a priority.

Second, the analysis team will explore and evaluate a number of predictors of child outcome and service delivery variables, primarily through the use of bivariate methods. These include contingency table (crosstab) analysis with appropriate statistical tests (e.g., Pearson’s ²) and simple regression and correlation procedures. This will enable us, for example, to examine outcomes across age groups and gender, the relationship between maltreatment type and services, and children’s needs relative to service receipt.

We anticipate that multiple predictor analyses will play an important part in the analysis plan. There are circumstances, for example, in which important questions about what variables predict an outcome must use multivariate methods to take into account confounding variables that limit interpretation. Methods with multiple predictors may be needed to address the likelihood that some consumers of NSCAW results will interpret bivariate relationships as causal. Multiple indicator methods like structural equation modeling may be needed to adapt to measurement error that is likely to arise for some NSCAW measures (Biemer et al., 2006). Questions about the relationship between agency variables and child outcomes necessitate multivariate analysis, since they require hierarchical linear models that can take into account sampling both at the agency and child level.

It will also be important to conduct change analyses to assess how children’s safety and permanency are addressed over time, how their development proceeds, and how they recover from maltreatment and other harm. We will use a variety of regression methods that use change scores or use initial levels as covariates to estimate outcomes. For analyses that only involve two waves of data with the same measure, we will employ statistical strategies such as the reliable change index (Jacobson et al., 1999) that provide the ability to control for measurement error and determine rates of clinically significant improvement and deterioration.

Conduct Preliminary Analyses and Plan Final Statistical Methods. Between the end of data collection and the time analysis files are ready, the analysis team will prepare assiduously for final analyses. We will begin by constructing programs and conducting analyses with preliminary unweighted data to test use of specific statistical methods with NSCAW II. This will expedite work when the final analysis files become available. Note that the special requirements of NSCAW preclude using “off the shelf” methods in many circumstances, and require careful programming with sophisticated statistical software such as SUDAAN and MPlus. NSCAW staff are currently working on adapting appropriately sophisticated methods to the special requirements of the data set. For example, a special work group has developed techniques for adapting multilevel modeling to NSCAW I’s specially weighted sample (with children, PSUs or agencies, and possibly states representing different levels).

The final outline will include specific analyses to be conducted, complete with the specific variables to be used, closely tied to research questions and policy priorities. In the spirit of simplicity, most of the analyses will be straightforward and use simple methods, but we will include sophisticated methods in the analysis plan to the extent they are needed for specific research questions. With all the preparatory work outlined above, we will be well-poised to conduct final analyses.

Although it is premature to include a final analysis plan here, Exhibit A.16-1 presents examples of research questions linked to analytic methods, constructs, and measures in NSCAW II. It includes some questions addressed in NSCAW I that deserve further inquiry, as well as questions that might arise out of emerging priorities related to current policy issues.

We have included a list of NSCAW I publications as of December 2008 (see Appendix G), to indicate the types of peer-reviewed research also being conducted by members of the research and practice communities.

Exhibit A.16-1 Elements of Analysis Planning

Examples of Research Questions	Population estimates	Group comparisons/ bivariate correlations	Multivariate models	Change analyses	Constructs	Measures
Well-Being and Child Development
Are children reaching developmental norms for their age? What proportion of children has learning and school difficulties? How does maltreatment affect cognitive development? Does out-of-home placement affect cognitive development? Does special education help ameliorate learning difficulties? How well do children progress in learning over time?	X X	X X	X	X	Cognitive development, learning, school adaptation	Preschool Language Scales, Vineland Adaptive Behavior Scale, Kaufman Brief Intelligence Test, Bayley Infant Neurodevelopmental Screener, Woodcock-Johnson Version III, Battelle Developmental Inventory, Brief Infant Toddler Social Emotional Assessment,
Do children develop adequate social skills during school years? Do children develop adequate peer relationships? Can children in out-of-home placements develop satisfying peer relationships? Do children’s social skills improve over time?	X X	X		X	Social development	Social Skills Scale, Loneliness and Social Dissatisfaction Questionnaire for Young Children, Social Skills Rating System (Teachers), Social Competence – Teacher Report Form
Is children’s physical development adequate? What proportion of children has chronic health problems? Are abuse and neglect related to risk of injuries?	X X X			X	Health	Questionnaire for Identifying Children with Chronic Conditions – Revised
How frequent are mental health problems in this population? How does severity of maltreatment affect mental health? How is placement and stability of placement related to mental health? How does parental substance abuse affect children’s mental health? How do risk and protective factors in combination explain children’s mental health outcomes?	X	X X X	X	X	Behavior problems and mental health	Child Behavior Checklist, Parent Report, Teacher Report Form, and Youth Self Report

Examples of Research Questions		Population estimates	Group comparisons/ bivariate correlations	Multivariate models		Change analyses		Constructs	Measures
Safety
What maltreatment do children experience? To what degree do children suffer multiple victimizations Does CWS response match children’s level of danger? Are children maintained in safe environments over time?	X X		X		X		Maltreatment, violence, and victimization			Project-developed questions on re-reports, Conflict Tactics Scale, Community Environment Scale
Permanence
How often are children placed out of home and in what placements? Is the risk of placement disproportionate for people of color? To what extent is parental substance abuse related to placement? What resources may help prevent placement? Are the qualifications and experience of CWS staff related to placement decisions? How stable are children’s living environments over time?	X		X X X X	X	X		Disruption in living environment			Project-developed questions, UC Berkeley Foster Care Study Questions
How often are children and youths reunified, adopted and emancipated? Are permanency outcomes comparable across racial-ethnic groups? What child, family and resource variables predict reunification? Can permanent living situations be maintained over time?	X		X X	X	X		Permanent living situations			Project-developed questions
Service Delivery
Are children and families receiving social support? Are children and families receiving services? Can caregiver substance abuse services prevent further maltreatment and out-of-home placement? How well does service delivery match needs? How do CWS policies and interventions relate to services delivery? How well and for what duration are services delivered over time?	X X		X X X	X	X		Support and service variables			Duke Functional Social Support Scale Pediatric questions, Early childhood education questions, Child care questions, Child and Adolescent Services Assessment (mental health), Project-developed questions on caregiver services

A.17 Display of Expiration Date

The OMB number and expiration date will be displayed at the beginning of each questionnaire program and in a box on the consent forms. These will be printed on the front cover of the Teacher Questionnaire.

A.18 Exceptions to Certification Statement.

There are no exceptions to the certification statement.

References

Barth, R. P.,  Shenyang, G., Weigensberg, E., Christ, S. L. Bruhn, C. M., & Green, R. L. (in press). Explaining Reunification and Reentry 3 Years After Placement in Out-of-Home Care. . In M. B. Webb, K. Dowd, B. Jones Harden, J. Landsverk & M. F. Testa (Eds.), Child welfare and child well-being: New perspectives from the National Survey of Child and Adolescent Well-Being. New York: Oxford University Press.

Burns, B.J., Phillips, S.D., Wagner, H.R., Barth, R.P., Kolko, D.J., Campbell, Y., and Landsverk, J. (2004). Mental Health Need and Access to Mental Health Services for Youths Involved with Child Welfare: A National Survey. Journal of the American Academy of Child and Adolescent Psychiatry, 43(8), 960-970.

Glisson, C. (in press). Organizational climate and service outcomes in child welfare systems. In M. B. Webb, K. Dowd, B. Jones Harden, J. Landsverk & M. F. Testa (Eds.), Child welfare and child well-being: New perspectives from the National Survey of Child and Adolescent Well-Being. New York: Oxford University Press.

Glisson, C., and Green, P. (2005). The Effects of Organizational Culture and Climate on the Access to Mental Health Care in Child Welfare and Juvenile Justice Systems. Administration and Policy in Mental Health and Mental Health Services Research, 33(4), 433-448.

Glisson, C. (2007). Organizational climate, job satisfaction, and service outcomes in child welfare agencies. Manuscript submitted for publication.

Horwitz, S.M., Hurlburt, M.S., and Zhang, J. (in press). Patterns and Predictors of Mental Health Services Use by Children in Contact With the Child Welfare System. In M. B. Webb, K. Dowd, B. Jones Harden, J. Landsverk & M. F. Testa (Eds.), Child welfare and child well-being: New perspectives from the National Survey of Child and Adolescent Well-Being. New York: Oxford University Press.

Hurlburt, M.S. Leslie L.K., Landsverk, J., Barth, R.P., Burns, B.J., Gibbons, R.D., Slymen, D.J., and Zhang, J. (2004). Contextual Predictors of Mental Health Service Use Among Children Open to Child Welfare. Archives of General Psychiatry, 61, 1217-1224.

Jee, S.H., Antonucci, T.C., Aida, M, Szilagyi, M.A, .and Szilagyi, P.G. (2005). Emergency Department Utilization by Children in Foster Care. Ambulatory Pediatrics, 5, 102-106.

Leslie, L.K., Hurlburt, M.S., Landsverk, J., Rolls, J.A., Wood, P.A., and Kelleher, K.J. (2003). Comprehensive Assessments for Children Entering Foster Care: A National Perspective. Pediatrics, 112(1), 134-142.

Lindsey, D. (2003). The Welfare of Children. New York: Oxford University Press.

Ortega, R. M., Grogan-Kaylor, A., Ruffolo, M., Clarke, J., & Karb, R. (in press). Racial and ethnic diversity in the initial child welfare experience: Exploring areas of convergence and divergence. In M. B. Webb, K. Dowd, B. Jones Harden, J. Landsverk & M. F. Testa (Eds.), Child welfare and child well-being: New perspectives from the National Survey of Child and Adolescent Well-Being. New York: Oxford University Press.

Stahmer, A.C., Leslie, L.K., Hurlburt, M., Barth, R.P., Webb, M.B., Landsverk, J., and Zhang, J. (2005). “Developmental and Behavioral Needs and Service Use for Young Children in Child Welfare.” Pediatrics, 116, 891-900.

Testa, M. F., Bruhn, C., & Helton, J. (in press). Comparative safety, stability, and continuity of children's placements in formal and informal substitute care. In M. B. Webb, K. Dowd, B. Jones Harden, J. Landsverk & M. F. Testa (Eds.), Child welfare and child well-being: New perspectives from the National Survey of Child and Adolescent Well-Being. New York: Oxford University Press.

U.S. Department of Health and Human Services, Administration for Children and Families (2005). National Survey of Child and Adolescent Well-Being (NSCAW) CPS Sample Component Wave 1 Data Analysis Report. Washington, DC: DHHS.

U.S. Department of Health and Human Services, Administration for Children and Families (forthcoming b). “The Well-Being of Children 18 Months Following a Child Maltreatment Investigation: A Report from the National Survey of Child and Adolescent Well-Being.” Washington, DC: DHHS.

Webb, M.B. and Harden, B.J. (2003) “Beyond Child Protection: Promoting Mental Health for Children and Families in the Child Welfare System.” Journal of Emotional and Behavioral Disorders, 11(1), 39-58.

13