Responses to April 24, 2008 Questions

Responses to April 24, 2008 Questions

National Survey of Child and Adolescent Well-Being II

1. On page 29 of the Supporting Statement, the following text describes a data security precaution “All interviewers and other personnel working on the survey must sign affidavits pledging that the data they collect or work with will not be disclosed. Penalties for disclosure include termination of employment and substantial financial fines.” Please provide additional information about the financial fines.

The financial penalty referred to in the text above describes a clause in the Headway Data Collection Agreement. Headway Staffing is the company that RTI uses for payroll and other management functions for field staff. This form is a standard RTI form that data collection staff sign during training for any field project. The financial penalty is part of the Headway employment agreement.

2. Please provide additional information about release of the NSCAW data.

Data obtained in the conduct of the National Survey of Child and Adolescent Well‑Being are available through licensing agreements with the National Data Archive on Child Abuse and Neglect (NDACAN) at Cornell University (www.ndacan.cornell.edu). Two levels of data access are specified in the licensing agreements: a General Release data file and a Restricted Release data file. Both release versions contain the Child Protective Services (CPS) and Long Term Foster Care (LTFC) sample components.

This tiered approach to data release was developed because of the considerable risk to participants if their data were re‑identified with them as individuals. The children are especially vulnerable to social stigmatization if details of their experiences and perceptions were to become known. Therefore, release of these data to the research community is more restrictive than in most federally funded studies involving human subjects.

The General Release data are more accessible by researchers, requiring only the completion of the General Release order form; provision of the licensing agreement signed by the researcher, the institution, and authorized research staff; and certification of approval or review exemption by the researcher’s Institutional Review Board (IRB). The General Release does not, however, contain geographic or sampling strata identifiers and can therefore only be used for limited analytic purposes. The data have been analyzed for disclosure risks, and some variables have been dropped or recoded to mitigate risks of participant re‑identification. The General Release provides researchers with a detailed understanding of the data files (e.g., their structure and contents) and can be used for many descriptive analyses. Researchers are encouraged to obtain access to the General Release before making application for the Restricted Release license.

While the extremely high value that is placed on protecting participantsnot only by federal regulation, but also by ACF and contractor standardsjustifies these alterations of the data, we recognize that some of these protections against re‑identification may at times reduce the analysis potential of certain variables in the data set. For example, when only ranges of percentages are given for a variable, threshold points that may be important for some analyses may be obscured, or non‑linearities in relationships hidden. No matter how thoughtfully continuous variables are transformed into categorical form, different cut points for the categories may be desirable, depending on the particular analytic purpose. For these reasons, the NSCAW data are also available in the Restricted Release, providing more flexibility in analysis.

Because microdata (that is, individual‑level data from multiple sources) carries with it some risk of statistical disclosure of institutional or individual identities, the NSCAW data have been extensively analyzed to determine which items of information, used alone or in conjunction with other variables, have significant disclosure potential. Variables that were found to pose significant risk of re‑identification were suppressed or altered to remove or reduce such risks. For example, in some cases continuous variables have been recast as categorical variables, or fine‑grained categorical variables have been more grossly re‑categorized. In a few instances, data elements have been suppressed or changed. Because of this, a particular individual child might be characterized in terms of a certain variable on the Restricted Release version of the NSCAW data, but be coded to missing or to a different adjacent value in the General Release data.

The Restricted Release data are more complete and have been only minimally altered through suppression and recoding, but have significantly greater controls on access. To obtain the Restricted Release data, a researcher must complete an application and provide certification of approval for the project by the researcher’s IRB; a licensing agreement signed by the researcher, the institution, and authorized research staff; a data protection plan; and payment of a $500 fee to cover the cost of unannounced visits to the researcher’s site to monitor compliance with the data protection plan.

To further protect study participants, individuals who could know of the participation status of sampled children, families, and the caseworkers who investigated the case or provided services are prohibited from having access to the data at either release level. This is because the data cannot be sanitized sufficiently to eliminate the possibility of re‑identification of participants by individuals who have information about the children and families. State and county government and service provider staff interested in the NSCAW data are encouraged to seek a collaborative relationship with academic researchers who can directly access the data, conduct the statistical analyses of interest, and provide the non‑identifying results to their collaborators. Individuals who have interests other than legitimate researchare expressly prohibited from accessing the data in any form.

Detailed information such as the terms of the data licensing agreements can be found on the NDACAN website provided above.

3. On page 33 of the Supporting Statement, the second bullet indicates that data files and documentation are prepared according to “NDACAN guidelines”. Please provide these guidelines.

The “NDACAN guidelines” was a document used in 2000 to determine record layout, file structure, and the like. Because we have, for the most part, retained these over the length of the NSCAW project, we are unaware if such a document currently exists at the Archive.

4. The citation is missing in the References section for a Biemer et al. publication on the use of Structural Equation Modeling in assessing measurement error. Please provide the citation.

Biemer, P., Christ, S., and Wiesen, C., “A General Approach for Estimating Scale-Score Reliability for Panel Data”, Psychological Methods, in review.

The paper is attached for your convenience.

4. We have many questions on the B.1 section “Statistical Power”, and believe that there may be sentences missing or typographical error. Please review and submit a corrected section.

A corrected section (including the paragraph above the section) is attached.

6. Please explain the reason for the significant increase in the Teacher Survey response rate for Wave 5 as specified in Exhibit B.3-1.

When the table was prepared in July 2007, the Teacher Survey had been completed only through the Wave 5 Young Child Follow-Up and therefore the 83.2% weighted response rate applies only to the Wave 5 Infant Follow-Up and the Wave 5 Young Child Follow-Up. We speculate that the higher response rate is partially a function of differences in participation propensities among teachers; it has been our experience that elementary school teachers are more likely to participate in the NSCAW Teacher Survey.

Since preparation of the Supporting Statement we have completed the Wave 5 Adolescent Follow-Up, and its Teacher Survey. Preliminary unweighted response rates are lower than the 83.1% achieved for the younger sample members but higher than the response rates obtained in previous waves. We attribute these increases to improvements in the project materials, and in implementation of additional prompting mechanisms. For example, as email addresses for teachers have become more widely available from both caregivers and on the internet, we now routinely use both telephone and email to prompt nonresponders.

7. Please provide additional information to help us interpret the Nonresponse Bias analysis section.

Additional information from the NSCAW Data File User’s Manual is attached.

8. On page 22 of the Supporting Statement in the discussion of the benefits of using CAI technology, the term “lay interviewers” is used to describe NSCAW field interviewers. Given the extensive training required, we believe this term trivializes the skill levels required to conduct the study.

We agree, and have deleted the word “lay” from this section of the Supporting Statement.