Non-Substantive Change to the NSCH ICR

Non-Substantive Change to the National Survey of Children’s Health Information Collection Request

The Census Bureau requests approval of a non-substantive change to the currently approved National Survey of Children’s Health (NSCH) Information Collection Request. In order to help improve paper topical response rates for the 2017 survey, the NSCH sponsor has requested the Census Bureau expand the current incentive plans to include an additional incentive in two planned mailings of topical questionnaires. Census recommended to the sponsor that we also implement the additional incentive with an experimental design, as was done for the prior year survey cycle.

The initial sample size and completion estimates are included in Table 1. Approximately 70% of the sample was sent only a web invitation for the first and second invitation, while 30% of the sample was sent paper instruments in addition to the web invitation with their initial mailing. Overall, we expect between a 40% and 43% completion rate for the screener section of the interview for the combined modes based on incentive group.

Table 1: Expected Sample Sizes of 2017 NSCH Incentive Treatment Groups

Initial Sample	Stratum		Incentive Group		Valid Addresses	Completed Screeners		Households		Completed
	Stratum 1: 60.4%		$0: 10%		87.60%	$0: 40%		With Kids		Topicals
	Stratum 2a: 39.6%		$2: 90%			$2: 43%		Stratum 1: 78%		69%
								Stratum 2a: 17%
156,690	1	94,623	$0	9,451	8,723	3,489		2,753		1,902
			$2	85,172	78,614	33,804		26,671		18,430
	2a	62,067	$0	6,225	5,005	2,002		356		246
			$2	55,842	44,897	19,306		3,436		2,375
Totals					137,239		58,601		33,216	22,953
per State					2,691		1,149		651	450

The incentives used in August 2017 were included with the first survey invite letter mailed to sampled households and were split into 10% of households receiving $0 and 90% of households receiving a $2 cash monetary incentive.  There have been no further incentives offered.

The initial invitation letter began mailout on August 7, 2017 to all mailable addresses in the sample (See Table 2 for mail-out schedule).

Table 2. 2017 NSCH Mail-out Schedule

Low-Paper* Mailing Dates		High-Paper** Mailing Dates		Topical Mailing Dates
Initial mailout 8/7/2017		Initial mailout 8/7/2017		1st – 9/15	6th – 11/24
Pressure-sealed postcard reminder 8/14		Pressure-sealed postcard reminder 8/14		2nd – 9/29	7th (1)– 12/6
1st Follow-up web invite only 9/6		1st Follow-up 9/13		3rd – 10/13	8th (1)– 12/20
2nd Follow-up w/ 1st Paper 10/5		2nd Post-card 9/20		4th – 10/27	9th – 1/3/18
2nd Post-card 10/11		2nd Follow-up 10/18		5th – 11/9	Optional 10th – 1/19/18
3rd Follow-up w/ 2nd Paper 11/8		3rd Follow-up 11/22
Optional 4th Follow-up w/ 3rd paper 12/13					(1) Proposed topical incentive mailout period
Survey Closeout (no optional mailing) 1/26/18
(with optional mailing) 2/16/18

*The Low-Paper group is considered Web-push + mail because they were mailed a Web invite letter only with their initial and 1^st follow-up mailings and received their 1^st paper questionnaire with the 2^nd follow-up mailing.

**The High-Paper group is considered Mixed-mode because they received both a Web invite letter as well as a paper questionnaire with each mailing.

Table 3 below details the number of households by incentive treatment and contact group.

Addresses that received an incentive were more likely to respond. Addresses in both the 70% “Low Paper” group and the 30% “High Paper” group had achieved approximately five percentage points more returns than the non-incentive control group by day 44. The addresses which were sent paper questionnaires with the initial mailing, “High Paper,” have returned survey responses at nearly ten percentage points higher rates than households sent only Web response invitations. While the “Low Paper” or web-first addresses will be sent paper questionnaires at the beginning of October, special attention is warranted for ensuring topical responses are received from screeners received later in the collection period. To facilitate the reduction of this potential bias, we are proposing a topical incentive be tested experimentally to 90% of the eligible households in the 7^th and 8^th topical mail-out cycles.

Table 3. NSCH Return Rates by High Paper and Incentive (as of 44 days since first mail-out)

		Incentive
	Total	$0	$2
Sample	156,690	15,676	141,014
Total Screener Returns	32,397	2,561	29,836
% of sample	20.7%	16.3%	21.2%
% of Low Paper	17.6%	13.7%	18.0%
% of High Paper	27.8%	22.4%	28.4%

We believe focusing on procedures that could help reduce the effect of high non-response in the topical questionnaire returns, and provide critical information for planning topical incentives that align with the mode efficiencies afforded by identifying “High Paper” and “Low Paper” addresses. Generating unbiased estimates while reducing costs requires this additional evaluation to plan for the next cycle of the NSCH.

In the 2016 NSCH, a test was done to identify the response benefit associated with including a $0, $2, $5, or $10 cash monetary incentive and a Thank You note in the material that was sent to respondents in the third mailing. We propose following that procedure, and testing incentives at the topical stage to see if a cash incentive could be effective in increasing the response to the topical interview, especially among the initially slower responding “Low Paper” addresses. Due to the limited funding this survey cycle, we are proposing to test a $2 cash incentive against a $0 cash incentive in topical mailings 7 and 8. These mailing groups contain addresses that are receiving their first, second, third, and final topical questionnaire mailing. This test would require the use of $16,894 in remaining available incentive funds, but is expected to increase response (estimates in Table 5), and provide valuable information for us to use in planning the 2018 NSCH to develop and refine a cost-effective data collection strategy. There is no increase in the respondent burden, as this was a planned mailing. The only change to this mailing is adding the monetary incentive. Below is a table of the expected mail-out distribution.  This distribution produces power estimates (see Table 4 below) above .8 for all comparisons when p=.1.

Table 4. Expected NSCH Mail-Out – Incentivized Topical Questionnaire

	Total	Third and Fourth topical recipients	First and Second topical recipients
Total estimated mailing	9,386	2,736	6,650
Control Group (0$)	939	273	665
$2 Group	8,447	2,463	5,985

Initial estimates below in Table 5 are based on the mail-out in Table 4, with 9,386 topical questionnaires.  The estimated response rates below in Table 5 are estimated primarily on the effectiveness of the incentive in the screener mailing discussed above.

Table 5. Estimated NSCH Response Rates – Incentivized Topical Questionnaire

 

	Expected Response Rate to third topical mailing, no incentive (%)	Expected Response Rate Difference due to Incentive (%)	Expected Response Rate (%)	Estimated Number of Mailings	Estimated Additional Topicals Received Above a 14.7% Response
Total			--	9,386
Control Group (0$)	14.7	0	14.7	939	0
$2 Group	14.7	5.0	19.7	8,447	427 (+34.4%)

Table 5. Power Calculations

	p=0.05	p=0.10
Control ($0) vs $2	0.967*	0.985
Control ($0) vs $2 (1st or 2nd Topical Mailing)	0.918	0.961**
Control ($0) vs $2 (3rd or 4th Topical Mailing)	0.732	0.841

 

 

 

* With 939 households receiving no incentive and 8,447 households receiving a $2 incentive, there will be a 97% chance of detecting a significant difference. Stated differently, a power of 0.967 means that 97% of the time a significant difference will be detected, with a 5% probability of a false positive result.

** With 665 households receiving no incentive and 5,985 households receiving $2 incentive in their first or second topical mailing, there will be a 96% chance of detecting a significant difference. Stated differently, a power of 0.961 means that 96% of the time a significant difference will be detected, with a 10% probability of a false positive result.