Supp Stmt Part B

State and Local Area Integrated Telephone Survey

OMB # 0920-0406

Supporting Statement B

Three-year generic clearance granted 04/25/11

Expires 04/30/14

GenIC request to add additional topics:

2013 National Survey of Children in Nonparental Care (NSCNC)

Prepared by:

Matthew D. Bramlett, Ph.D.

Statistician - Health

CDC/NCHS

3311 Toledo Road, Room 2111

Hyattsville, MD 20782

301-458-4070 (voice)

301-458-4035 (fax)

[email protected]

January 16, 2013

The State and Local Area Integrated Telephone Survey (SLAITS)

OMB clearance number 0920-0406

Expiration 04/30/14

GenIC: 2013 National Survey of Children in Nonparental Care (NSCNC)

B: Collection of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

For the National Survey of Children in Nonparental Care (NSCNC), the sample frame consists of approximately 3,000 English- and Spanish-speaking households which were already screened as part of the 2011 – 2012 National Survey of Children’s Health (NSCH) and were identified as having a resident child living apart from his or her biological or adoptive parents (The NSCH webpage is located at http://www.cdc.gov/nchs/slaits/nsch.htm). These households will be recontacted to participate in the follow-up NSCNC. The respondent will be the same caregiver or guardian who completed the 2011 – 2012 NSCH, or the child’s current caregiver or guardian for children no longer living with the original NSCH respondent (this may include the child’s parent if the child has moved back with the parent(s)). Absolutely no children will be interviewed during the course of data collection. Children who have aged to age 18 between the original interview and re-contact will be classified as out-of-scope to avoid asking an adult to provide information on another adult who has not provided informed consent.

The NSCNC sampling frame of all children in nonparental care in the 2011-2012 NSCH is nationally representative of all noninstitutionalized children ages 0-17 years in nonparental care in the United States in 2011-2012 because the NSCH itself is nationally representative of all noninstitutionalized children ages 0-17 years in the United States in 2011-2012. The NSCH sampling frame included both landline and cell phone numbers and the sampling weights were adjusted to account for nonresponse to the NSCH and to match to demographic control totals from external (Census) population counts. We expect approximately 1,600 completed NSCNC telephone interviews. At least a 55% interview completion rate, calculated as the number of completed interviews divided by the number of eligible children sampled for recontact, would be consistent with the 65% interview completion rate achieved by the National Survey of Adoptive Parents of Children with Special Health Care Needs, which was a follow-up survey of the 2005-2006 National Survey of Children with Special Health Care Needs conducted by the SLAITS program that also followed up with children 1-2 years after original interview, and allowing for the NSCNC population of children to be more transient than the general population. It is entirely possible that our estimated interview completion rate is a conservative estimate. Nonresponse to the NSCNC is correctable given that we already have NSCH data for the eventual NSCNC respondents and nonrespondents alike. (See additional detail in Section 3 (non-response bias) below).

Beginning with the implementation of the 2009-2010 National Survey of Children with Special Health Care Needs, SLAITS modules have employed a dual-frame landline and cell phone sample. Approximately one third of NSCH interviews in 2011 – 2012 were cell phone interviews. Thus, approximately one third of NSCNC interviews are expected to be completed by cell phone, and the rest by landline telephone.

Because the sample of eligible households for NSCNC consists of households that completed the 2011-2012 NSCH (which is now out of the field), the pool of eligible households is pre-screened for the presence of children in nonparental care and ready for calling. Because the NSCH was nationally representative, so will be the NSCNC, after weighting adjustments to account for re-interview nonresponse.

NSCNC

The dress rehearsal to test the Computer-Assisted Telephone Interview (CATI) programming will consist of the first 50 NSCNC interviews. NSCNC dress rehearsal questionnaires will be administered in English only. Children who will have reached age 18 by re-interview will be classified as out-of-scope.

Because the NSCNC sample is pre-screened for the presence of children in nonparental care by the 2011-2012 NSCH, and the population is difficult and costly to locate in a general population survey, there will not be a full pretest using an independent sample of households with no parents present. Instead, we will conduct a CATI dress rehearsal in which a small portion of the pre-screened nationally-representative sample of households identified during the administration of the 2011-2012 NSCH will be used to test the instrument and procedures. After 50 dress rehearsal interviews are completed, interviewing will pause as results are evaluated. We anticipate that the dress rehearsal will identify programming errors and will suggest few (if any) changes to the instrument, and expect to simply include the dress rehearsal cases with the final sample of completed interviews. This methodology has been successfully implemented in a prior SLAITS followback survey with similar constraints (the Survey of Pathways to Diagnosis and Services, which was a followback to selected respondents from the 2009-2010 National Survey of Children with Special Health Care Needs).

If the dress rehearsal indicates a need for changes to question text or a need to add or delete questions, we will submit a second GenIC OMB package including the revised (final) questionnaire before implementing the revised questionnaire in interviews. Otherwise, following any necessary programming revisions, interviewing will resume with the remaining eligible cases. We anticipate that the most likely need for changing the questionnaire may be deleting some questions if the interview length is too long, but wish to preserve the possibility of resuming interviewing immediately if this turns out not to be necessary. If questions are deleted from the questionnaire as a result of the dress rehearsal, the data records for the dress rehearsal cases will have the fields for the deleted questions blanked out when the dress rehearsal cases are included in the final sample.

After interviewing resumes, a total of approximately 1,550 additional prescreened households are expected to complete the NSCNC survey. The remainder of the pool of eligible cases will be included in calling. The NSCNC questionnaire will be administered in English and Spanish following resumption of data collection after the dress rehearsal. Children who will have reached age 18 by re-interview will be classified as out-of-scope.

2. Procedures for the Collection of Information

At the end of the NSCH interview, respondents were told that they may be contacted for future surveys. In order to locate the original respondent at a future date, the interviewer asked for additional contact information. This information will be used to contact and interview eligible respondents for the NSCNC.

Computer-assisted telephone interviews (CATI) will be conducted using the procedures described in the original SLAITS information collection request. The advance letter and introductory telephone script are found in Attachments B & E. All telephone interviews (Attachment A) will be conducted from the data collection contractor’s telephone center in Chicago, Illinois. Interviews will be regularly monitored by the contractor. The study interviews will only be conducted in English and Spanish (NSCH interviews were similarly conducted only in English and Spanish).

All NSCNC-eligible cases from the NSCH will be released for calling, and those with a known mailing address (approximately 90%, because most NSCH respondents received incentives) will be mailed an advance letter (attachment B). Respondents will be offered $10 at first contact (to be mailed after completion of the interview) and an additional $5 after they have refused participation twice (although dress rehearsal cases will not be offered the additional incentive as we expect to complete the dress rehearsal before implementing the refusal conversion procedures). The interview will be approximately 30 minutes in length, although we recognize that it may be slightly longer in the dress rehearsal and may need to be reduced by deleting some questions. The dress rehearsal interview will be conducted in English. After 50 cases have been completed, interviewing will stop while the results are evaluated. The dress rehearsal is designed to test the CATI programming to ensure the questionnaire has been accurately programmed and to examine frequencies and interview time in order to delete low-prevalence or low-variability questions and reduce interview time, if necessary.

After the dress rehearsal is evaluated and changes to the questionnaire programming are made, interviewing will resume (although if question text is changed, or questions are added or deleted, the revised questionnaire will be submitted for OMB approval before implementation of the revised questionnaire in interviews). All telephone interviews will be conducted from the contractor telephone center in Chicago, Illinois. The NSCNC interviews (after the dress rehearsal) will be conducted in English and Spanish. The advance letter, introductory telephone scripts, and the answering machine and voice mail scripts are found in Attachments B and E.

We anticipate completing all telephone interviews by the summer of 2013. Once all necessary steps are taken to ensure that the identity of survey subjects is protected, a publicly available data file should follow shortly thereafter. To reduce the risk of inadvertent disclosure, it is likely that the data file will be coarsened by suppressing survey variables, collapsing multiple variables into one, collapsing response categories for other variables, and/or introducing noise in the data. All data files are reviewed and approved by the NCHS disclosure review board prior to public release. Analysts interested in working with data that were suppressed to protect confidentiality may apply to access unmodified data files through the NCHS Research Data Centers in a protected environment where output is monitored to ensure confidentiality is not impacted in any way.

Advance Letter

Addresses are available for approximately 90% of eligible households. Most NSCH respondents provided their addresses so that they could receive their incentive payment for participating in NSCH. Because the NSCH shares the sample frame and follows the National Immunization Survey (NIS) and all households screened for NSCH eligibility are first screened for NIS eligibility, the NIS advance letter was used for the 2011-2012 NSCH during the original data collection. For the NSCNC follow-back, a new NSCNC-specific advance letter will be used (Attachment B).

The NSCNC letter invites recipients to participate in a study on the health and well-being of children and provides survey information in a concise format. The letter also

• advises recipients of their eligibility for the survey based on responses to a previous survey, and indicates they will be called in the next few weeks;

• briefly explains the purpose of the survey and the intended uses of the data;

• cites and explains the authorizing legislation;

• tells potential respondents that they may refuse to participate completely or refuse individual items, that their responses will be held in strictest confidence, the survey is voluntary, and that their responses will be used for statistical purposes only;

• includes a list of ‘frequently asked questions’ or FAQs on the back of the letter; and

• invites hard-of-hearing or deaf respondents to contact the contractor with a TTY machine at a toll free number to be interviewed.

Estimate Precision

It is important to note that the survey is subject to the usual variability associated with sample surveys. Small differences between survey estimates may be due to random survey error and not to true differences among subgroups of children. The precision of the survey estimates is based on the sample size and the measure of interest. Estimates at the national level are based on larger sample sizes than those at the state level. While the NSCH 2011-2012 is representative at both the national and state levels, the NSCNC will be representative at the national level only.

Power Calculation

A power calculation was not conducted to determine the target NSCNC sample size, because the 2011-2012 NSCH was planned and implemented before the NSCNC followback was conceived. All households with eligible children identified in the NSCH who were living with no parents present will be invited to participate in the NSCNC. Based on past experience with similar followback surveys of adopted children from the 2007 NSCH and the 2005-2006 National Survey of Children with Special Health Care Needs, we do expect to be able to examine and detect differences in NSCNC outcomes for children in various subgroups (for example, caregiver-type subgroups could include children living with grandparents only, children living with grandparents and other relatives, and children living with relatives but not grandparents).

3. Methods to maximize response rates and deal with non-response

Response rates provide one measure of the potential for nonresponse bias but are not the sole indicator of survey and data quality. Although telephone survey response rates have declined, the telephone as a mode of data collection is still one of the most useful and economical means to obtain population-based data. Successful conduct of a SLAITS telephone module depends on a combination of techniques to maximize response rates and understand the impact of nonresponse on data quality. Standard proven survey procedures have been refined through deliberate testing and experience over time. Among those techniques routinely implemented in SLAITS modules are the:

• use of a carefully constructed advance letter for those households where a name and address are available (Attachment B),

• effective interviewer recruitment and training,

• thorough review of confidentiality, privacy, and security requirements,

• maintenance of a toll-free number and website to facilitate participation,

• flexible interview schedules to maximize convenience to the respondent,

• judicious use of incentives,

• quality control and interviewer monitoring,

• refusal aversion/conversion training with experienced interviewers, and

• A carefully constructed refusal conversion letter (Attachment B).

Answering machine messages are left on every third call where no contact with the household is made so long as no more than one message is left in any given week. An exception to this rule is made if the respondent missed a scheduled appointment. In cases of missed appointments an answering machine message is always left to let the respondent know an attempt was made to keep the appointment, regardless of the frequency of previous answering machine messages.

These measures do not assure high response rates. For each SLAITS module, analysis is conducted to evaluate the extent to which nonsampling error impacts data quality. Comparison to other surveys and related data, expected demographic characteristics, interview breakoffs, and other qualitative and quantitative measures will be constantly reviewed and assessed. Should changes to the survey design be warranted due to low response rates, OMB will be consulted.

Generally, nonresponse bias can be thought of as the degree to which nonrespondents differ from respondents in key survey variables. This quantity is generally unknown, but in the case of NSCNC, a follow-back survey to a previous module, we will have demographic and other information collected in the NSCH for all NSCNC-eligible households, those that respond and those that do not. This enables a very powerful nonresponse bias analysis, and also allows us to adjust the sampling weights very precisely to ensure that the final sample of completed interviews represents the same population of children as that represented by the full pool of eligible cases identified in the NSCH. Please see Appendix 9 of the NSAP methodology report¹ (available online) for a detailed example of how this nonresponse bias analysis and weighting correction works. (NSAP was a followback to the 2007 NSCH.) After this process was completed for NSAP, we had very strong evidence that nonresponse bias in the NSAP was no greater than nonresponse bias in the 2007 NSCH. Comparisons of demographic and socioeconomic outcomes and of variables from the NSCH that were topically similar to the NSAP data elements, using the unadjusted and adjusted weights, demonstrated that nonresponse bias accruing from NSAP nonresponse on variables that were used to adjust the weights was reduced to zero after adjustment. For other variables that were not used in the weighting adjustment, nonresponse bias accruing from NSAP nonresponse was greatly reduced (to near-zero) and any remaining bias was small and inconsistent in direction after adjustment. We expect similar results for the NSNCN. Because the pool of eligible NSCNC cases is nationally representative of all noninstitutionalized children ages 0-17 years in nonparental care in the United States in 2011-2012, after the adjustment to the weights to account for NSCNC nonresponse, the final sample of NSCNC respondents will also represent this population accurately.

Note that the nonresponse bias analysis described above using data from the NSCH for all NSCNC-eligible respondents and nonrespondents does not address bias resulting from nonresponse to the underlying NSCH – the pool of eligible NSCNC sample cases consists only of households that completed the NSCH. Based on our experience with NSAP (as shown in Appendix IX of the cited report and described above) and our expected adjustments to the final sampling weights to account for NSCNC nonresponse, we are confident that nonresponse bias in the NSCNC should be no greater than nonresponse bias in the NSCH. The NSCH 2011-2012 nonresponse bias analysis has now been completed, but has not yet been published or posted to the web (it will be included as an appendix in the NSCH 2011-2012 Design and Operation report, which should be published in late 2013). Our experience with previous SLAITS surveys has consistently shown that estimated biases resulting from nonresponse tend to be small and inconsistent in direction, and this has generally held true for the 2011-2012 NSCH as well. Nonresponse bias was assessed using multiple methods, and the typical findings were that either biases were small and insignificant or the direction of bias found depended on the method used (that is, where bias was found, different methods of assessing bias suggested either a positive or negative bias, and this was not consistent across different variables). Overall, there was evidence that the responding sample was more likely than the nonrespondents to live in rural areas and areas with lower household density, lower home values, higher rates of home ownership and a higher percentage of non-Hispanic white persons, but after weighting adjustments, only minor differences by home ownership, home values and race remained.

The final overall weighted response rate for the follow-up survey is dependent on NSCNC response and the underlying response rates for the 2011 – 2012 NSCH. After NSCNC data collection is completed, we will calculate and report this overall weighted response rate in the required methodology report. The calculation will be consistent with the Standard Definitions provided by the American Association for Public Opinion Research (AAPOR), and the methodology report will include the specific calculations and summaries of the disposition of the sample cases. We expect to achieve at least a 55% interview completion rate, calculated as the number of completed interviews divided by the number of eligible children sampled for recontact. This would be consistent with the 65% interview completion rate achieved by the National Survey of Adoptive Parents of Children with Special Health Care Needs, which was a follow-up survey of the 2005-2006 National Survey of Children with Special Health Care Needs conducted by the SLAITS program that also followed up with children 1-2 years after original interview, and allowing for this population of children to be more transient than the general population. It is entirely possible that our estimated interview completion rate is a conservative estimate.

4. Test of Procedures or Methods to be Undertaken

The new NSCNC questions were cognitively tested using standardized qualitative techniques (probes, ‘think aloud’ responses, etc.). The entire draft instrument that is included in this package was further refined in six in-person ‘mock interviews’ that were administered to a convenience sample of adult (18 years of age and over) nonparental caregivers.

A CATI dress rehearsal of the first 50 interviews will be conducted with the same materials, interviewers, and questionnaires. The dress rehearsal is intended to:

• test the questionnaires, procedures, software, and instrument integration;

• ensure that the questions and questionnaires flow properly (i.e., transitions are sufficient and skip patterns are working as intended); and

• assess section administration times to help identify the extent to which reductions may be needed;

The NSCNC dress rehearsal will also examine the procedures used to recontact NSCH respondents using information collected in the NSCH.

If no significant problems are found, the cases from the dress rehearsal will be kept in the data set and data collection will continue. If significant problems are found, corrections will be made and data collection will pause until the problems are fixed. If question text is changed or questions are added or deleted, a revised questionnaire will be submitted in a revised package to OMB for approval before the revised questionnaire is implemented in interviews.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The following persons were consulted on the statistical aspects of design, data collection, and analysis:

Matthew Bramlett, Ph.D.

Survey Statistician

National Center for Health Statistics

Division of Health Interview Statistics

[email protected]

301-458-4070

Stephen Blumberg, Ph.D.

Senior Health Scientist

National Center for Health Statistics

Division of Health Interview Statistics

[email protected]

301-458-4107

List of Attachments

1. 2013 NSCNC Questionnaire

2. 2013 NSCNC Advance letter, refusal conversion letter, thank you letter

3. SLAITS/ASPE Collaborative Effort to Improve Data on Vulnerable Subpopulations of Children 2005-2012:

Selected analytic and methodological published references

4. Selected persons involved in the NSCNC planning process

5. 2013 NSCNC Introductory Scripts and Answering Machine/Voice Mail Messages

6. NCHS Research Ethics Review Board letter

1 Bramlett MD, Foster EB, Frasier AM, et al (2010). Design and operation of the National Survey of Adoptive Parents, 2007. National Center for Health Statistics. Vital and Health Statistics 1(50). Available online at:

http://www.cdc.gov/nchs/data/series/sr_01/sr01_050.pdf

Page 10 of 10