Non Substantive Change to the NSCH ICR

Sample, Incentives, and Experiments Non-Substantive Change Request

National Survey of Children's Health

OMB Control No. 0607-0990

The U.S. Census Bureau requests a non-substantive change to the currently approved National Survey of Children’s Health (NSCH) Information Collection Request. The Census Bureau received a three year clearance by OMB for the NSCH data collection on May 19, 2017 (OMB#0607-0990, expires May 31, 2019). Subsequently, we requested a non-substantive change to include an additional incentive in two planned mailings of topical questionnaires in order to help improve paper topical response rates and reduce non-response bias for the 2017 survey for which we received OMB approval on November 8, 2017. On December 29, 2017, we also received approval of a separate non substantive change request to address a sponsor request for a slight net increase in the number of survey questions, modification to one of the mailing treatment experiments, and the selection of a new, unduplicated sample.

For calendar year 2018 data collection the following survey lifecycle operations, which already have OMB clearance, will remain unchanged from the original PRA package request: sample frame development, sampling selection and stratum definition, Internet likelihood flag, use of unconditional incentives, weighting procedures, and within household sampling (also known as subsampling). In addition, the majority of screener and topical questionnaire content remains consistent with the 2017 NSCH questionnaires. The only additional changes that will be made for the 2018 NSCH data collection are described below.

This non-substantive change request for the 2018 NSCH data collection reflects 1) the use of a higher denomination bill for the monetary incentive treatment groups, 2) the addition of a certified test, which entails applying an official USPS certified mail sticker on the outside of the mail package envelope , within the topical mail groups, and 3) the selection of an additional 20,000 unduplicated sample addresses than was selected in 2017 from the Master Address File (MAF). All three requested revisions listed above are in response to a combination of items. Along with streamlining the NSCH processes and creating a mailing strategy that has proven effective in increasing early response therefore reducing non-response follow-up and bias, there is now a realized cost savings. This additional funding can be directed towards other survey operations. After consulting with the sponsor of the survey, the Health Resources and Services Administration’s Maternal and Child Health Bureau (HRSA MCHB), it was determined that the preferred option would be to seek out ways to obtain additional topical responses in order to increase the power for measuring key indicators and rare conditions. Another key piece of information is that preliminary results from the 2017 NSCH production survey show that the same response expectations for the 2018 NSCH cycle are no longer realistic. While the survey experienced an increase in early screener response, obtaining response overall was more difficult due to the inclusion of more households with children in the sample. Households without children have very low response effort. Making a more efficient sample, by including more households with children, lowered response because a greater proportion of the sample needed to expend more response effort. However, the overall total burden hours as well as annualized burden costs from 2017 remain less than the estimates provided in the original OMB clearance package.

The request for the addition of a certified test within the topical mail groups would allow for responsive decision making to proceed with additional certified mailings in the initial topical mail groups if the results of the certified screener mailings prove effective. As holds true from the previous request, the primary goal for this test would be to evaluate whether we can increase the likelihood that households would respond to the survey, and correspondingly reduce bias. The mechanism for increasing response would be through increasing the likelihood that a household respondent would open the survey mailing with a certified mail identification. This premise is supported by research advocating for the use of FedEx mailers (Hagedorn et al. 2006)⁰, but at a substantially lower cost. Higher initial response can reduce follow-up costs and nonresponse bias. Additional details for this change can be found in the tables below.

Estimate of Hour Burden

For the 2017 NSCH, 58,345 respondents were expected to complete the screener and 23,460 respondents were expected to complete one of the three age-based topical questionnaires. This resulted in an estimated 81,805 responses annually. The total number of annual burden hours for that estimate is 16,573. This included an average burden per response of 0.083 hours for the screener and 0.5 hours for each of the topical instruments. The estimated total annual respondent cost was $430,898.00. Since the data collection period has ended for 2017, we were able to use the preliminary results to obtain better burden estimates. The actual number of respondents was 75,431 and total number of burden hours for that estimate is 14,819. The different between the two is 1,754 less actual burden hours and $45,595 less annual respondent cost. See Table 1 and Table 2.

Table 1: Expected vs. Actual Annualized Burden Hours

Type of Respondent	Questionnaire Name	Expected Number of Respondents0	Number of Responses per Respondent	Expected Average Burden per Response (in hours)	Expected Total Burden Hours	Actual Number of Respondents0	Average Burden per Response (in hours)	Actual Total Burden Hours
NSCH Production
Adult Parent or Caregiver	Screener	58,345	1	.083	4,843	54,907	.083	4,557
Adult Parent or Caregiver	0-5 Topical Instrument	7,820	1	.5	3,910	6,841	.5	3,421
Adult Parent or Caregiver	6-11 Topical Instrument	7,820	1	.5	3,910	6,841	.5	3,421
Adult Parent or Caregiver	12-17 Topical Instrument	7,820	1	.5	3,910	6,841	.5	3,421
Total		81,805			16,573	75,431		14,819

Table 2: Expected vs. Actual Annualized Burden Costs

Type of Respondent	Expected Total Burden Hours	Expected Hourly Wage Rate	Expected Total Respondent Costs	Actual Total Burden Hours	Actual Hourly Wage Rate	Actual Total Respondent Costs
NSCH Production
Adult Parent or Caregiver (Screener)	4,843	$26.00	$125,918.00	4,843	$26.00	$118,489.00
Adult Parent or Caregiver (0-5 Topical Instrument)	3,910	$26.00	$101,660.00	3,910	$26.00	$88,938.00
Adult Parent or Caregiver (6-11 Topical Instrument)	3,910	$26.00	$101,660.00	3,910	$26.00	$88,938.00
Adult Parent or Caregiver (12-17 Topical Instrument)	3,910	$26.00	$101,660.00	3,910	$26.00	$88,938.00
Total	16,573		$430,898.00	14,819		$385,303.00

With the sample, incentive, and experiment modifications we are requesting, the expected screener and topical burden per response is unchanged from the previous non-substantive change (which increased topical burden hours per response from .5 to .55) approved on December 29, 2017. However, the expected number of respondents for both the screener and topical instruments will increase. This results in a revised burden estimate for the screener and topical interviews totaling 2,094 hours as well as a revised annualized burden cost increase of $54,433. The proposed revised burden and cost estimates for the 2018 cycle of the NSCH are detailed in Table 3 and Table 4.

Table 3: Burden Hours Estimate for Increased Sample (+20,000), Increased Incentive ($5), and Addition of a Certified Test to the Topical Mailings

Type of Respondent	Questionnaire Name	Expected Number of Respondents0	Number of Responses per Respondent	OMB Approved Average Burden per Response (in hours)	OMB Approved Total Burden Hours	Estimated Number of Respondents0	Estimated Burden per Response (in hours)	Estimated Total Burden Hours
NSCH Production
Adult Parent or Caregiver	Screener	58,345	1	.083	4,843	63,306	.083	5,254
Adult Parent or Caregiver	0-5 Topical Instrument	7,820	1	.5	3,910	8,129	.55	4,471
Adult Parent or Caregiver	6-11 Topical Instrument	7,820	1	.5	3,910	8,129	.55	4,471
Adult Parent or Caregiver	12-17 Topical Instrument	7,820	1	.5	3,910	8,129	.55	4,471
Total		81,805			16,573	87,692		18,667

Table 4: Annualized Burden Cost Estimate for Increased Sample (+20,000), Increased Incentive ($5), and Addition of a Certified Test to the Topical Mailings

Type of Respondent	OMB Approved Total Burden Hours	Hourly Wage Rate	OMB Approved Total Respondent Costs	Estimated Total Burden Hours	Hourly Wage Rate	Estimated Total Respondent Costs
NSCH Production
Adult Parent or Caregiver (Screener)	4,843	$26.00	$125,918.00	5,254	$26.00	$136,614.00
Adult Parent or Caregiver (0-5 Topical Instrument)	3,910	$26.00	$101,660.00	4,471	$26.00	$116,239.00
Adult Parent or Caregiver (6-11 Topical Instrument)	3,910	$26.00	$101,660.00	4,471	$26.00	$116,239.00
Adult Parent or Caregiver (12-17 Topical Instrument)	3,910	$26.00	$101,660.00	4,471	$26.00	$116,239.00
Total	16,573		$430,898.00	18,667		$485,331.00

Modification to Incentives and Mailing Experiments

In the OMB non-substantive request approved on December 29, 2017, we obtained approval to revise our mailing experiment to include a certified-mail test in the initial screener mailing. We would like to incorporate this mailing strategy into the paper topical mailings as well if proven effective. According to research done on increasing response rates to postal questionnaires (Edwards et al. (2002))⁰, the use of recorded delivery more than doubled the odds of response (2.21; 1.51 to 3.25). The certified test would assess the effectiveness of an official USPS certified mail sticker placed on the outside of the mail package envelope. While effective, unconditional incentives are only salient once the respondent opens the survey package. The certified mail test evaluates the effectiveness of getting respondents to open the survey package, as observed by differential response. Since the certified sticker experiment will be crossed with the incentive test, effects within incentive and mailing strategy will also be able to be evaluated.

Survey research indicates that incentives are a necessary and cost‐effective expense for achieving a response rate that minimizes nonresponse bias⁰. Along with the certified test, we are also proposing to incorporate the use of a $5 cash incentive in the screener and topical initial mailings. Under the current OMB clearance, we have approval to mail a $2 cash incentive to 90% of the initial screener mailings as well as a $2 cash incentive to 90% of two topical mail groups containing addresses receiving their initial, first, second, and third paper topical invites. Both the screener and topical incentive groups keep a 10% control group that do not receive any incentive. In order to adequately plan for the cost savings we are anticipating for this survey cycle, we would like to propose testing differing incentive amounts. For the screener, we are requesting an even split of incentives ($2 and $5 bills) to 90% of the sample, while keeping the control group consistent with 2017 at 10%. For the topical mailings, we are proposing replacing the $2 cash incentive with a $5 cash incentive and instead of mailing to 90% of two full topical mailings, mailing to 80% of each initial topical mailing group. This way the incentive will be crossed with the proposed certified mail test for each initial mailing, similar to what will be done for the screener.

Table 5 shows the maximum sample sizes for replacement of the certified test as the mailing experiment within each incentive group. At both the 90% and 95% confidence levels, there is enough sample in all treatment groups to detect a significant difference (power=1). Table 6 shows the estimated maximum sample sizes for the initial topical mail groups. Due to the smaller sample size in the topical mailings, there would be enough sample to detect a significant different (power ≥ 0.8) at the 90% confidence level for both the incentive comparison ($5 vs. control) and certified test (certified vs. non-certified).

Table 5: Revised Screener Mailing Treatment Group Comparisons – Maximum Sample Sizes

Incentive	Initial Cases	Certified Test Treatment Status	Maximum Sample for Certified Test Comparison	Treatment Groups (TG)
$5	79,224 (39%)	Certified Mail Package	39,612 (40.1%)	1
		Non-certified Mail Package	39,612 (37.9%)	2
$2	79,224 (37%)	Certified Mail Package	39,612 (38.1%)	3
		Non-certified Mail Package	39,612 (35.9%)	4
Control $0	17,606 (34%)	Certified Mail Package	8,803 (35.1%)	5
		Non-certified Mail Package	8,803 (32.9%)	6

Table 6: Revised Topical Mailing Treatment Group Comparisons – Maximum Sample Sizes

Incentive	Initial Cases	Certified Test Treatment Status	Maximum Sample for Certified Test Comparison	Treatment Groups (TG)
$5	12,120 (50%)	Certified Mail Package	6,060 (51.1%)	1
		Non-certified Mail Package	6,060 (48.9%)	2
Control $0	3,030 (45%)	Certified Mail Package	1,515 (46.1%)	3
		Non-certified Mail Package	1,515 (43.9%)	4

0 Hagedorn Mary Collins, Montaquila Jill, Carver Priscilla, O’Donnell Kevin, Chapman Christopher, National Household Education Surveys Program of 2005: Public-Use Data File User’s Manual, Volume I, Study Overview and Methodology, 2006 (NCES 2006-078). U.S. Department of Education. Washington, D.C.: National Center for Education Statistics. http://nces.ed.gov/nhes/dataproducts.asp#2005dp

0 The expected number of respondents is an estimate of the expected number of completed screener and topical questionnaires, discussed in section B.1.3 of the previously approved OMB package. This is different from the number of respondents that were mailed a screener or topical questionnaire.

0 The actual number of respondents is a more realistic estimate of completed screener and topical questionnaires based on preliminary 2017 NSCH results.

0 The expected number of respondents is an estimate of the expected number of completed screener and topical questionnaires, discussed in section B.1.3 of the previously approved OMB package. This is different from the number of respondents that were mailed a screener or topical questionnaire.

0 The estimated number of respondents considering all proposed revisions (additional 20,000 sample addresses, use of a $5 incentive, and a certified mail test in the topical mailings).

0 Edwards, P., Roberts, I., Clarke, M., DiGuiseppi, C., Pratap, S., Wentz, R., and Kwan, I. (2002). Increasing response rates to postal questionnaires: systematic review. British Medical Journal 2002: 324-1183. Stable URL: http://www.bmj.com/content/324/7347/1183?tab=full

0 Brick JM, Williams D, Montaquila JM. 2011. “Address‐Based Sampling for Subpopulation Surveys”. Public Opinion

Quarterly, 75(3): 409‐28; Foster EB, Frasier AM, Morrison HM, O’Connor KS, Blumberg SJ. 2010. “All Things

Incentive: Exploring the Best Combination of Incentive Conditions”. Paper presented at the American Association

for Public Opinion Research annual conference, Chicago, IL.

5