Supporting Statement B_Oct27_clean

For the sake of comparability across cohorts, the National Survey for Child and Adolescent Well-being (NSCAW III) sample design will mirror the original design used in NSCAW I and replicated in NSCAW II.¹ However, unlike NSCAW II which reused the NSCAW I primary sampling units (PSUs), NSCAW III will select a new sample of PSUs using a procedure that maximizes the overlap of the PSU sample. The sample design chosen for NSCAW III is based on the lessons learned from NSCAW I and NSCAW II and incorporates enhancements to improve the sampling precision.

Key features of the NSCAW III sampling design are as follows:

Rather than carry over all former PSUs from prior cohorts of NSCAW, a new sample of PSUs will be selected in order to update the probability proportional to size (PPS) selection probabilities for the current distribution of the child welfare population.
A “maximal PSU sampling coordination” approach will be used that maximizes the probability of sampling PSUs (or agencies) in the NSCAW II sample.
AFC states—i.e., states having legal statutes requiring the agencies to contact families and obtain written permission to allow their information to be released—will be removed from the frame.

This OMB submission is for Phase I, which includes the recruitment of the child welfare agencies and the collection of sample frame files that will be used to select the sample of children involved with the child welfare system (CWS). A subsequent OMB package will cover Phase II, which will include all tasks related to collecting data from the CWS-involved children, their caregivers and their caseworkers.

B1.1 Target Population

For the baseline and 18-month follow-up data collection (Phase II), the target population for NSCAW III includes all children ages 0-17½ who come to the attention of the CWS during the 12-month sampling period. Specifically, the target population includes children who were (1) were investigated or assessed for child abuse or neglect and (2) entered state legal custody through other pathways (e.g., juvenile justice). These children, who are placed into legal guardianship, may comprise as much as 20 percent of children in out-of-home placement. This target population for Phase II of NSCAW III is shown in Exhibit B1.1.

Exhibit B1.1. Phase II of NSCAW III Target Population

According to 2014 data from the National Child Abuse and Neglect Data System (NCANDS), an estimated 3.6 million referrals of abuse or neglect, concerning approximately 6.6 million children, were received by child protective services (CPS).Almost 61 percent of those referrals were accepted for investigation or assessment.

B1.2 Sampling Frame Used and Its Coverage of the Target Population

NSCAW III will construct a sampling frame consisting of all counties in the U.S. except for (1) very small ones, namely those with less than 55 investigations or assessments annually and (2) counties whose state law (AFC states) prohibits the release of information. The first exclusion was used for previous NSCAW samples and for cost efficiency. Only about 1 percent of the child welfare population resides in these small counties so their exclusion has a negligible effect on population coverage and estimation bias. The second exclusion (AFC states) is necessary because child welfare agencies in these states are prevented by state law from participating in the NSCAW.

The NSCAW I target population represented approximately 94.6 percent of the U.S. population of children investigated or assessed for child abuse or neglect during the sampling period. In NSCAW II, it was approximately 88 percent as a result of the additional AFC states that were dropped from the study. In NSCAW III, population coverage is likely to be approximately at the NSCAW II levels or perhaps lower if more states have passed AFC legislation necessitating their removal from the target population. Neither NSCAW I or II included children who enter and are served by the CWS without a maltreatment investigation. These children will be included in the NSCAW III target population (see Figure B1.1).

B1.3 Design of the Sample (Including Any Stratification or Clustering)

NSCAW III proposes to use a stratified, cluster sample design, similar to prior cohorts of NSCAW.

Phase I: Sampling of Child Welfare Agencies. The first stage of sampling will be the selection of primary sampling units (PSU), which for this study are child welfare agencies. The frame PSUs will be ordered by Census region, by state within Census region and then by urban/rural status to ensure that regions and states in both urban and rural areas will be sampled in proportion to their child welfare populations. A frame of all children in the target population will be developed for each sample PSU using lists obtained from the agencies during each month of the sampling period. Then children will be selected disproportionately in each PSU according to their sampling domain to achieve the desired sample size in each domain.

Biemer (2007) determined that a sample size of 55 to 60 completed cases per PSU, per year, is ideal for the general NSCAW design in terms of cost versus error optimization. Thus, for an overall sample size of 4,565 cases, 83 PSUs/cooperating child welfare agencies is optimal. Using a maximum sampling coordination approach, a sample of 83 PSUs/cooperating child welfare agencies via PPS sampling using composite size measures that incorporate the population sizes of the selected domains in each PSU. Data from the most recent NCANDS file will supply these population counts. The composite size measure method (Folsom, Potter and Williams, 1987) provides a means to control domain sample sizes that maximizes the efficiency of the design by minimizing weight variation for units within sampling domains. PSUs will be defined essentially as they were in NSCAW II (i.e., geographic areas that encompass the population served by a single child welfare agency). In most cases, these areas correspond to counties or contiguous areas of two or more counties. In larger metropolitan areas with branch offices, the county will be subdivided into areas served by a single agency/office.

PSUs for NSCAW III will be reselected so that PPS selection probabilities will be based upon the current child welfare population distribution. Updating the probabilities of selection of the PSUs will reduce data collection costs as well as improve sample precision and data collection efficiency.

An independently selected sample of PSUs will contain some child welfare agencies that participated in NSCAW II because they are located in certainty PSUs. Noncertainty PSUs could be re-selected by chance. Additionally, a dependent sampling approach will be used to maximize the overlap between the NSCAW II and NSCAW III PSUs. This procedure is known in the sampling literature as maximum sample coordination (see, for example, Keyfitz, 1951; Kish and Scott, 1971; and Ernst, 1998).

As in NSCAW I, the selection of primary sampling units (PSUs) will involve the following four steps:

1. Partition the target population into PSUs

2. Compute a size measure for each PSU

3. Stratify the PSU sampling frame

4. Select the sample of PSUs

The activities for carrying out each of these four steps are outlined below:

Step 1: Partitioning the Target Population into PSUs. The administrative structure of the child welfare system varies considerably across the states and even within states. Therefore, a single definition of a PSU is not feasible since it depended on the administrative structure of the state system, as well as the jurisdictions of child welfare agencies within the state. For most areas of the country, the best definition of a PSU was the county since it corresponds to a clearly defined political entity and geographic area of manageable size. In other areas, the definition of a PSU was not as straightforward, as in a single child welfare agency that had jurisdiction over several counties, the PSU was defined as a part of or the entire area over which the child welfare agency had jurisdiction, depending how large the area and population were. Extremely large counties or MSAs have child welfare agencies with many branch offices, each with its own data system. Such PSUs will be divided into smaller units, such as areas delineated by branch office jurisdictions, to create manageable PSUs. For the purpose of the first-stage sampling discussion, counties are referred to as PSUs, for simplicity’s sake. Step 2: Compute a Size Measure for Each PSU. The second-stage sampling units will be stratified into nine domains of interest to control the second-stage sample allocation so that domains of interest have sufficient sample sizes. The second-stage NSCAW III domains and the allocation of achieved sample sizes are shown in Exhibit B1.2. Note that there are a total of nine sampling domains defined by columns 1 and 2 of the table. Column 3 is the expected sample size under proportionate sampling. Because some of these sample sizes are inadequate for research purposes, they will be increased to the levels shown in Column 4 (Target sample size). Also shown in this table is the unequal weighting effect (UWE), clustering effect (based upon the expected intercluster correlation), the effective sample size (which is the target sample size divided by the UWE and the clustering effect) and the minimum detectable effect size (MDES) which is the smallest effect size that can be detected with 80 percent power and a type I error of 5 percent.

Exhibit B1.2. Second-Stage Sampling Domains

Age group	Services	Proportionate sample size	Target sample size	UWE	Clustering effect	Effective sample size	MDES
Infant (under 1 year)	Services in home	109	533	1.000	1.358	392	0.200
	Services out of home	73	533	1.000	1.358	392	0.200
	No services	182	533	1.000	1.358	392	0.200
Ages 1 to 11	Services in home	914	397	1.000	1.250	318	0.222
	Services out of home	291	189	1.000	1.084	174	0.300
	No services	1,799	782	1.000	1.556	503	0.177
Ages 12 to 17	Services in home	366	533	1.000	1.358	392	0.200
	Services out of home	123	533	1.000	1.358	392	0.200
	No services	708	533	1.000	1.358	392	0.200
Total		4,565	4,565	1.754	4.564	570	0.166

The simplest size measure for PSUs in NSCAW would be simply the Census counts of children 14 and younger. This simple size measure gives counties with a higher number of children in the eligible age group a higher probability of being selected. However, such a design would be inefficient because we want to oversample children with certain characteristics in the PSU. One of the consequences of using simple size measures when the design calls for second-stage over-sampling of certain domains is that the variability of the sampling weights will be increased and, thus, so will the sampling variances. The composite size measure method, described in Folsom, Potter, and Williams (1987), will provide a method for controlling domain sample sizes while maximizing the efficiency of the design. The composite size measure reflects the size of the sample that would fall into the PSU if a national random sample of children were selected with the desired sampling rates for all domains but without PSU clustering.

After the composite size measures are computed, each of the 3,141 counties on the initial PSU frame will be checked to determine whether it was large enough to support the planned selection of 55 CWS-involved children per PSU during the twelve-month data collection period. Approximately 700 counties with fewer than 55 CWS-involved children will be deleted from the frame will account for approximately 1 percent of the target population. The estimated total size of the NSCAW III target population is approximately 2,600,000 eligible CWS-involved children in 2,441 counties nationwide.

Step 3: Stratifying the First-Stage Frame. The PSU frame will be implicitly stratified by nine census regions and urbanicity within region prior to sampling. The urbanicity of a PSU will be defined by whether the county was part of an MSA (extremely large county). Stratifying PSUs by region and urbanicity allows for controlled allocation of sample PSUs in these implicit strata.

Step 4: Selecting the PSUs. Given the first-stage stratification and the size measure , the selection frequency of the kth PSU in the hth first-stage stratum is calculated as

(2)

where is the number of PSUs selected from the hth first-stage stratum and, is the total size measure of all PSUs in the hth first-stage stratum.

PSUs will be selected using an algorithm that maximizes the expected number of PSUs that will overlap NSCAW I and NSCAW III. Given the sample of NSCAW I PSUs in stratum h, denoted , the algorithm produces a set of conditional probabilities while preserving the unconditional probabilities specified in (2). PSUs will then be sampled from each stratum using the conditional probabilities produced by the algorithm. Algorithms for computing the conditional probabilities of selection are described in Pollock (1984), Causey, Cox, and Ernst (1985), and Ernst (1995).

Design of Second Stage Unit Sample and Overview of the Process

After selecting the PSUs for the study, the process of recruiting the child welfare agencies associated with the PSUs will begin. As these agencies are recruited, we will work with them individually to refine our projections of the expected sizes of the domains of analysis for sampling. The nine domains for the study are shown in Exhibit B.1.2. As shown in this exhibit, the number of children that will be selected in each domain will be sufficient to achieve a minimum detectable effect size (MDES) of 0.2 in all nine strata. When calculating the necessary sample sizes, we assumed an intercluster correlation of 0.066 based upon an analysis of the NSCAW II key characteristics.

As previously noted, a sample size of about 55 completed cases per PSU is ideal for NSCAW for cost and error optimization. We will use the data available from both NSCAW I and II to establish initial sample allocations for each domain within PSU. Then we will adjust those sample allocations throughout the data collection process by following steps:

1. Each month, the contractor (RTI International) will receive files from each child welfare agency containing all children with completed investigations/assessments as well as children entering legal custody through alternative pathways such as the juvenile justice system.

2. These files will be processed and any duplications will be removed.

3. The contractor will compute the number of cases to select in each domain, in each PSU, in any given month using an algorithm we developed in NSCAW II. The algorithm determines the number of cases to select so that target sample sizes are achieved by the end of data collection, then the algorithm optimizes the allocation of sample across PSUs so that UWEs are minimized while equalizing interviewer assignments.

4. The sample for each domain in each PSU is selected, reviewed for accuracy, and transmitted to the field. These steps are depicted in Exhibit B.1.3.

In prior NSCAW studies, some child welfare agencies can be slow to enter cases and their outcomes into their agency data systems. In fact, there could be a lag of up to three months before an investigated case is finally entered into the system. In these agency, a sampling process that only obtained cases completed in month t, say, would missing cases that were not entered into the system until month t+1, say. For that reason, we will obtain four files from the agency for each month of sampling: the month t file as of month t as well as the month t file after it has been updated in months t+1, t+2, and t+3. This will ensure there will be no loss of coverage as a result of delayed data entry.

Exhibit B1.3. Flow Diagram of the Sampling Process

B1.4 Size of the Sample and Precision Needed for Key Estimates

The sampling of agencies (or PSUs) during the agency recruitment phase (Phase I) will result in the selection of 83 agencies. From the 83 participating agencies, approximately 4,565 CWS-involved children, their caregivers, and their caseworkers will be interviewed in Phase II. It is quite difficult to specify the statistical power of NSCAW child-level analysis overall because it will include so many different research questions, variables and subpopulations. However, in Exhibit B1.2 we show the MDES’ for estimates within the primary domains of analysis. The target sample size for NSCAW III is 4,565 completed cases where a completed case is defined as a completed interview for the key respondent (i.e., either the child or his/her caregiver if the child is unable to be interviewed). Based on experience with the NSCAW II analysis, this sample was adequate for many types of analysis that were conducted, both for cross-sectional and longitudinal analysis.

To determine the number of cases to draw from each PSU, the initial sample needs was estimated as 8,695 sampled children to reach a completed sample size of 4,565. The rationale is presented in Exhibit B1.4.

Exhibit B1.4. Calculations for Child-Level Sample for Phase II of NSCAW III

Steps	Number
1. initial sample	8,695
2. assume 25% of the initial cases will be ineligible due to factors including the following: the sampled child is the sibling of another child already in the study the sampled child was not the target of the investigation into abuse or neglect the investigation date for the sampled child occurred outside the sampling period the sampled child/family moved out of the sampled agency’s service area the sampled child was determined to be deceased	.75 x 8,695 = 6,521
3. assume 60% of the cases will cooperate with the initial data collection efforts	.60 x 6,521 = 3,913
4. subject 50% of the remaining nonresponders (n = 3,143) to intensive data collection efforts	.50 x 2,609= 1,304
5. assume 50% of the nonresponders ultimately participate	.50 x 1,304 = 652
Total number of completed cases	3,913 + 652 = 4,565

With the number of completed cases, the average number of completes per PSU, and the oversampled domains, the extensive data available from NSCAW I and II will be used to update response rates by domain and by PSU in order to establish initial sample allocations for each domain.

B1.5 Expected Response Rate

NSCAW III will target 83 PSUs/cooperating child welfare agencies for the study. The agencies that refuse to participate will be replaced following the process described in Section B2. The response rate is not affected if a replacement PSU is found for a child welfare agency that refuses to participate. In addition, “agency first contact” (AFC) states that have laws preventing their participation will get excluded from the frame, and while their exclusion affects the coverage of the frame, it does not affect response rates.

Below are equations showing how the NSCAW I and NSCAW II response rates of 100 percent and 98 percent were calculated:

NSCAW I: ((100 PSUs selected − 8 AFC PSUs − 6 refused PSUs) + 6 replaced PSUs) / (100 PSUs selected − 8 AFC PSUs) = 100%

NSCAW II: ((100 PSUs selected − 17 AFC PSUs − 14 refused PSUs) + (12 replaced PSUs + 5 IL supplement PSUs)) / (100 PSUs selected − 17 AFC PSUs) + 5 IL supplement PSUs = 98%

Based on the agency response rates that were achieved in the previous NSCAW cohorts, an agency response rate between 95 and 98 percent for NSCAW III is expected.

Overall, 114 agencies (102 initially selected plus 12 replacements) will be contacted to participate in NSCAW III. This assumes that 102 PSUs will be initially selected, 14 will refuse to participate, 17 will be from AFC states, 12 will be selected as replacements, and 83 will agree to participate.

Below is an equation that shows the agency sampling assumptions that will result in achieving a 98% response rate and 83 cooperating agencies for NSCAW III.

((102 PSUs selected − 17 AFC PSUs − 14 refused PSUs) + (12 replacement PSUs)) / (102 PSUs selected − 17 AFC PSUs) = 83/85 = 98%

B1.6 Expected Item Nonresponse Rate for Critical Questions

The most critical data collection for this Phase I of the project will be the collection of the monthly sample frame files from child welfare agencies, which is the focus of this section. Prior experience collecting sampling frame files from the participating agencies suggests that there are varying levels of technical capabilities that affect an agency’s ability to provide data files containing the recent closed investigations along with the desired variables for sample selection. Examination of the type of data provided by each agency on the 2013 NCANDS child files and the amount of missing data for various data fields indicates that some child welfare agencies may still having difficulties providing child level electronic files on investigations and service usage.

Out of more than 900 sample files processed in NSCAW II, approximately 2 percent of them presented missing values in the variables used to classify children into the analysis domains. In NSCAW III, if child welfare agencies have more than 5 percent of the records removed from the files due to missing variable information, distributions of those missing records will be compared to the distributions of those not missing by gender, age, ethnicity and race. If the results are significantly different, the child welfare agency will be contacted to find out the reason of the missing values and whether it is possible to obtain this information.

In NSCAW III, the project team will work closely with each agency to assess their technical capabilities in providing data on service receipt and customized solutions for obtaining the necessary child sampling frame data that minimized burden on agency staff. For agencies not able to easily provide a usable data file for sampling, a computer-aided data entry (CADE) application will be offered to agencies which will allow agency staff to enter the requested sample data. Even with the implementation of Statewide Automated Child Welfare Information System (SACWIS) and Comprehensive Child Welfare Information System (CCWIS) data systems and the wider distribution of technological advances, some agencies in NSCAW II were still not capable of generating electronic data files on recent closed investigations and assessments. If this is the case in NSCAW III, the project team will work with the agencies to determine the best mode of data acquisition to obtain the required data for all children eligible for the sample each month.

For Phase II of the project, information regarding the expected item nonresponse rate for critical questions will be included in a subsequent package.

B2. Procedures for Collection of Information

Information package for child welfare agency administrators. After the sample of child welfare agencies is drawn, child welfare agency administrators will be sent a package of materials asking them to participate in NSCAW III. The purpose of this mailout package is to introduce the study’s design and goals, and explain the agency’s role in the sampling and data collection process. Materials to be sent to agencies are included in the Appendices. Agency administrators will first receive a prenotification letter (Appendix A) signed by the ACF Project Officer that introduces NSCAW, details the importance of the survey, and encourages the agency to participate when contacted by the contracting organization. Approximately 3-5 days later, agencies will receive a cover letter (Appendix B), a brochure answering “frequently asked questions” (Appendix C), and a set of 1-page Child Well-Being Spotlight reports that present key study findings from NSCAW I and II (Appendix D).

Initial visit or call with agency staff. Within a week of project information being received, agency recruiters will contact child welfare agency administrators to describe the study, discuss their involvement, answer questions, address concerns, determine whether additional approvals are required, and obtain the agency’s cooperation. The plan for protecting client data will also be discussed with agency staff and supplementary materials (e.g., copies of the Sensitive Information Plan or IRB approval) may be sent to the agency upon request. An agency recruiter checklist and discussion guide is provided in Appendix E. As needed and/or requested, in-person visits with agency personnel will be held to describe the study, address concerns, and make presentations to stakeholders. The telephone calls and visits with the agency staff are necessary steps in the agency recruitment process that will allow the project to answer any questions the agency may have and directly address any concerns the agency has about participating. As part of the agency recruitment process, the project team will engage individuals to champion and legitimize the study, such as state and county stakeholders and the National Association of Public Child Welfare Administrators (NAPCWA). If an agency cannot be persuaded to participate, the project team will attempt to identify and recruit a suitable replacement PSU. Details about the selection of replacement PSUs can be found in Section B3.2.

The project team will secure data sharing agreements and county or state-level IRB approvals as needed, to allow for site participation. Once a child welfare agency has agreed to participate, the project team will collect a signed Memorandum of Agreement (MOA; Appendix F) from each site. In addition to submitting monthly files to be used for sampling, each agency will be asked to:

Identify an “agency liaison” to the study who will serve as the main NSCAW contact.
Identify a contact for the study who will submit NCANDS and AFCARS files during the study period for the purposes of linking child survey data to administrative data.
Assist field staff during data collection by providing contact information for sampled families.
Support caseworkers to meet with field representatives and complete in-person interviews focused on sampled children and families.
Provide or arrange for consent for children in legal custody to be interviewed.

Visit or call with agency staff explaining the sample file process. Upon receipt of a signed MOA, a member of the project team will work with designated data systems staff from the participating child welfare agency to prepare the site for submission of monthly data files. Designated agency staff are typically responsible for maintaining all agency administrative data. An initial data request call, and subsequent calls as needed, will be held. During these calls, a project team member will review the specifications for the monthly sampling files (Appendix G) to ensure that the agency staff fully understands all of the data elements that are being requested. Topics and questions to be addressed during this meeting are provided in Appendix H.

In NSCAW I and II, agencies varied considerably in their data systems, including terminology, children classification conventions, policies for handling reports, and record keeping procedures. The visit or call with each agency will allow the sampling team to address the issues unique to the agency’s data systems and cross-walk between agency policy and practice and data fields needed for child-level sample selection. Data definitions will be based on the SACWIS, NCANDS and AFCARS data definitions as closely as possible, since agency staff should be familiar with those data systems.

Child welfare agency staff monthly sample file generation and transmission. Child welfare agencies have varying levels of technical capabilities affecting their ability to provide appropriate data files. As previously noted, a member of the project team will work with the child welfare agency to determine the best mode of data acquisition. A test file from the agency will be requested so the project team can process and assess data quality before the first month’s sample data is submitted. The data elements to be included in the sample file are specified in Appendix G. The test file will be evaluated and the project team will work with agency technical staff to resolve problems before the files are used to sample actual cases. A secure transmission website will allow agencies to easily and securely transmit data files to RTI International for processing and sample selection. For the 12-month sampling period scheduled to begin in June 2017, monthly sample files will be transmitted by the agencies for a period of 15 months. The three additional months allow for the submission of up to four corrected files for each sampling month in the 12-month period. Reminder emails will be sent to agencies ahead of each monthly submission date. A template of the reminder email is included in Appendix I.

For Phase II of the project, information regarding the procedures for future data collection will be included in a subsequent OMB package.

B3. Methods to Maximize Response Rates and Deal with Nonresponse

This section discusses methods to maximize response rates and deal with nonresponse for Phase I of the NSCAW III, involving child welfare agency recruitment and collection of files for sampling children. Methods to maximize response rates and deal with nonresponse for Phase II of the project will be included in a subsequent package.

B3.1 Expected Response Rates

The expected agency response rate of between 95 and 98 percent is discussed in Section B1.5. This estimate is based on the agency response rates achieved in previous NSCAW cohorts.

B3.2 Dealing with Nonresponse

The most important source of nonresponse at the agency-level is the legal requirement referred to as “agency-first contact” or AFC that affected four states in NSCAW I and four more states in NSCAW II. Consequently, these states were deemed “outside the target population” for NSCAW II thus eliminating a total of 17 PSUs from the sample. In NSCAW II, RTI International constructed special weights referred to as “calibration weights” that adjusted the NSCAW II sample for the nine AFC PSUs that participated in NSCAW I but had adopted AFC laws and thus did not participate in NSCAW II. These calibration weights compensated for the bias due to agency-level nonresponse for the NSCAW II and provided valid inferences to the NSCAW I target population for the NSCAW II sample. These weights also provided valid comparisons between NSCAW I and NSCAW II despite the differences in their target populations.

If an agency refuses to participate in NSCAW III, a replacement agency as similar as possible to the refusing agency from within the same state will be substituted. This substitute PSU will be given the same sample weight and sample allocation as the refusing agency. This approach, which was used quite successfully in NSCAW I and II, is common practice in institutional surveys (National Center for Education Statistics, 2003). If the refusal comes at the state level and no agency within the state is allowed to participate in the study, it may be possible to find a closely matching agency from a neighboring state that can serve as a substitute. In either situation, if a suitable substitute cannot be found, the refusing agency will be removed from the sample and all children that were allocated to that agency will be treated as nonrespondents. In some cases, it may be more statistically acceptable to exclude the state from the target population as was done in both NSCAW I and II for the AFC states.

The consequences of excluding a state from the scope of NSCAW III, either for AFC or other reasons, is a loss in coverage of the CW population. RTI International evaluated the coverage bias resulting from the elimination of the eight AFC states in NSCAW II and found that the AFC states represented approximately 11.3 percent of the CW population and that, at least for the characteristics on the NCANDS data file for these states, the coverage bias was small using criteria similar to Cohen effect size (Cohen, 1992). Still there was concern that, for other characteristics, the coverage bias could be appreciable thus limiting inference to only 88.7 percent of the CW population. This prompted the development of calibration weights (see Biemer, 2013). Calibration weighting uses all the available information on the AFC states combining NCANDS and AFCARS data with data from NSCAW I and II to achieve the greatest bias reduction possible through post-survey weighting adjustments. We will apply this approach to the NSCAW III data to compensate for noncoverage of the AFC states in NSCAW III.

B3.3 Maximizing Response Rates

Strategies to maximize child welfare agency response and cooperation rates for this information collection include:

Advance Informative Materials. Sending child welfare agencies advance materials, such as a prenotification letter signed by the ACF Project Officer and, before the initial call and visit that clearly describes the agency’s critical role in the study, key study findings, and the importance of continuing to monitor the well-being of children and families served by the CWS. These advance materials can be found in Appendices A – D.
Trained and Experienced Recruitment Staff. Staff with prior organizational contacting and child welfare research experience will lead agency recruitment activities. These staff understand the heavy workloads and other administrative many agencies face and will work around their schedules and availability. Senior staff, including a personnel with child welfare administration backgrounds will assist with telephone calls, site visits, and presentations to agencies that have unique circumstances or concerns about participation.
Engagement of Child Welfare Agency Communities. Prior to contacting agencies, the project team will engage state and county stakeholders and the child welfare agency community (e.g., NAPCWA) to request their support and advocacy of the study.
Reminder Emails to Transmit Monthly Sample Files. To facilitate the transmission of the monthly sample files, reminder emails will be sent to participating agencies (see Appendix I).

These agency recruitment steps were used successfully in the previous NSCAW studies.

B4. Tests of Procedures or Methods to be Undertaken

No pretests will be conducted for the sampling and agency recruitment phases of the study. As described in Supporting Statement Part A, the procedures and methods are based on NSCAW I and II.

B5. Individual(s) Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

RTI International and its subcontractors (University of North Carolina at Chapel Hill, Native American Management Services, Inc., and HR Directions, LLC) are conducting this project under contract number HHSP233201500039. RTI International developed the plans for statistical analyses for this study. The team is led by the following individuals:

Mary Bruce Webb, ACF, Contracting Officer’s Representative
Christine Fortunato, ACF, Co-Contracting Officer’s Representative
Melissa Dolan, RTI International, Project Director
Heather Ringeisen, RTI International, Co-Investigator
Paul Biemer, RTI International, Co-Investigator
Mark Testa, University of North Carolina at Chapel Hill, Co-Investigator

In addition, statistical consulting for the sampling plan was provided by M. A. Hidiroglou, PhD, from Statistics Canada in Ottawa, Ontario.

References

Biemer, P. P. (2007). “NSCAW II Design Methodology and Recommendations,” internal RTI International design report, May 4, 2007.

Biemer, P. and Wheeless, S. (2013). Comparing NSCAW I and NSCAW II estimates using children in the calibrated weights, internal RTI International memorandum.

Causey, B.D., Cox, L.H., and Ernst, L.R. (1985). Applications of transportation theory to statistical problems. Journal of the American Statistical Association. 80(392): 903-909.

Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159.

Ernst, L.R. (1995). Maximizing and minimizing overlap of ultimate sampling units. In JSM Proceedings, Survey Research Methods Section. Alexandria, VA: American Statistical Association. 706-711.

Ernst, L. R. (1998). Maximizing and Minimizing Overlap When Selecting a Large Number of Units per Stratum Simultaneously for Two Designs. Journal of Official Statistics, 14, 297-314.

Folsom, R.E., Potter, F.J., and Williams, S.R. (1987). Notes on a composite size measure for self-weighting samples in multiple domains. In Proceedings of the American Statistical Association, Section on Survey Research Methods, 792-796.

Keyfitz, Nathan (1951), "Sampling with Probabilities Proportionate to Size: Adjustment for Changes in Probabilities," Journal of the American Statistical Association, 46, 105-109.

Kish, Leslie, and Scott, A. (1971), "Retaining Units After Changing Strata and Probabilities," Journal of the American Statistical Association, 66, 461-470.

National Center for Education Statistics (2003). NCES Statistical Standards, downloaded from https://nces.ed.gov/pubs2003/2003601.pdf on March 29, 2106.

Office of Management and Budget (1999). National Survey of Child and Adolescent Well-Being (NSCAW). OMB Information Collection Request. OMB Control No: 0970-0202, ICR Reference No: 199906-0970-002, Conclusion Date: 8/18/1999.

Office of Management and Budget (2008). National Survey of Child and Adolescent Well-Being. OMB Information Collection Request. OMB Control No: 0970-0202, ICR Reference No: 200803-0970-002, Conclusion Date: 3/4/2008.

Pollock, J. (1984). PPES sampling of two subdomains with independent probabilities. In JSM Proceedings, Survey Research Methods Section. Alexandria, VA: American Statistical Association. 223-227.

1 The NSCAW II OMB package contains additional detail about the previous sample designs and can be found here: http://www.reginfo.gov/public/do/PRAViewDocument?ref_nbr=200803-0970-002

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Sroka, Christopher
File Modified	0000-00-00
File Created	2021-01-23

Supporting Statement B_Oct27_clean