Incentive Timing Study Proposal

APPENDIX G

Incentive Timing Study Proposal

Incentive Timing Study Proposal
Description: In the 2010 NSCG new cohort data collection effort, a sample of cases identified
as hard to enumerate cases had not responded to the 2010 NSCG after numerous contact
attempts. These cases were of high analytical interest to the NSCG because they tended to have
large sampling weights and tended to include a large proportion of minority cases.
To encourage response among these cases, we conducted an incentive study to examine the
impact of a late-stage $20 and $30 incentive against a control group that receive additional
contacts but no incentive. As expected, the use of the incentive led to an increase in response for
these hard to enumerate cases. Final responses rates for the cases included in the late-stage
incentive were 6.4% for the control group, 24.1% for the $20 incentive treatment group, and
29.5% for the $30 incentive group.
In addition to the incentive resulting in a response rate increase, the use of the incentive also had
a profound effect on the overall representation of the responding sample. The attached graphic
shows the R-indicator value throughout the data collection period. The R-indicator is a
representativity indicator that provides insight on how similar the demographic composition of
the responding sample is to the overall set of sample cases. The R-indicator ranges from 0 to 1,
with 1 indicating complete representation. The incentive was offered in June 2011 and the
immediate impact on the increased representation of the responding sample is evident in the
graphic. This increased R-indicator means that the incentive was successful in bringing in
respondents that were demographically different from the set of respondents prior to the
incentive stage.

R-Indicator by Date for 2010 NSCG
1
0.9
0.8

R

0.7
0.6
0.5
0.4
0.3

Response Date

1

As we consider the plans for interviewing the hard-to-enumerate cases in the 2013 survey cycle,
questions exist about whether the late-stage timing is the optimal way to use the incentive in
future survey cycles. By offering an incentive at an earlier stage of data collection, we could
increase the response rate for these hard to enumerate cases and decrease the need for
nonresponse follow-up efforts. However, the question remains of whether the cost of offering
the incentive to a larger number of cases exceeds the nonresponse follow-up costs had the
incentive not been offered at an earlier stage. In order to explore the issue of optimal incentive
timing, we propose an incentive timing study as part of the 2013 data collection effort. The 2013
NSCG incentive timing study will include cases identified as hard to enumerate using the same
criteria used to identify hard to enumerate cases in the 2010 NSCG. These cases will be
randomly allocated to five treatment groups as part of the 2013 NSCG data collection effort:
No incentive
Incentive offered at week 1 of data collection
Incentive offered at week 7 (coincides with the introduction of a mail response option)
Incentive offered at week 12 (coincides with the introduction of a telephone response
option)
Incentive offered at week 25 (coincides with the final contact phase)
Research Questions: This study is designed to provide insight on the following questions:
What incentive timing approach provides the optimal balance of response and cost?
Are there differences in the demographic composition of the respondents across the
treatment group when examining those that responded before the incentive offer and
examining those that responded after the incentive offer?
Eligibility, Treatment Groups, and Sample Size: In the 2013 NSCG new cohort sample of
83,000 cases, we expect to have approximately 18,000 cases that meet our criteria of hard to
enumerate. Within the set of hard to enumerate cases, the cases with a mailable address will be
eligible for the incentive timing study. The eligible cases will be randomly allocated across the
five treatment groups with each incentive group being allocated the approximately same sample
size that meets our statistical significance testing detection level requirements. The balance of
the eligible cases will then be allocated to the no incentive treatment group.
Additional Methodology/Implementation Information: Both $20 and $30 incentives were
offered in the 2010 NSCG incentive study with the $30 incentive leading to a slightly higher
response as noted above. For the incentive timing study, we will only use the $30 incentive
amount for all cases. The incentive for this study will be a $30 prepaid debit card incentive that
is similar to the debit card incentive used in the 2010 NSCG survey cycle. These debit cards will
have a six month usage period at which time the cards will expire and the unused funds will be
returned to Census and NSF (minus the predetermined per card fee).

2

The cases included in this study will follow the default data collection pathway proposed for the
2013 NSCG new cohort cases which uses a web first data collection methodology.
Assuming a 3,000 case sample for each incentive treatment group and the use of a $30 incentive,
the maximum additional data collection cost associated with this study is in the range of
$420,000 ($30 incentive + $5 per card processing fee for cases that would receive an incentive).
However, please note that the groups that are offered an incentive at a later stage will not require
an incentive be sent to all 3,000 cases since some cases will response in an earlier data collection
phase prior to the incentive being offered. Furthermore, please note that any unused funds
associated with the incentive debit cards will be returned to Census and NSF.
The attached table shows the proposed data collection methodology for the incentive timing
study.

3

Data Collection
Phase

Web Invitation
Phase
(Weeks 1-3)

Web Reminder
Phase
(Weeks 4-6)

Mail Invitation
Phase
(Weeks 7-11)

Production CATI
Phase
(Weeks 12-25)

Final
Contact Phase
(Weeks 25-28)

2013 NSCG New Cohort Data Collection Methodology
Web First
Hard to Enumerate Cases
(DEFAULT)
Week 0 (01/25/13)
Prenotice letter

Week 0 (01/25/13)
Prenotice letter

Week 1 (02/01/13)
Web invite letter

Week 1 (02/01/13)
Web invite letter
*$30 Incentive for Treatment Group #2 (TG2)

Week 2 (02/08/13)
Reminder postcard

Week 2 (02/08/13)
Reminder letter

Week 4 (02/22/13)
PRIORITY Web invite letter

Week 4 (02/22/13)
PRIORITY Web invite letter

Week 5 (03/01/13)
Automated phone call

Week 5 (03/01/13)
Automated phone call

Week 7 (03/15/13)
Web/Mail invite letter and questionnaire

Week 7 (03/15/13)
Web/Mail invite letter and questionnaire
*$30 Incentive for TG3

Week 8 (03/22/13)
Reminder postcard

Week 8 (03/22/13)
Reminder letter

Week 10 (04/05/13)
Automated phone call

Week 10 (04/05/13)
Automated phone call

Week 12 (04/19/13)
Web/Telephone Invite Letter

Week 12 (04/19/13)
Web/Telephone Invite Letter
*$30 Incentive for TG4

Week 12 (04/19/13) –Week 25
(07/05/13)
Telephone calls

Week 12 (04/19/13) –Week 25 (07/05/13)
Telephone calls
Week 13 (04/26/13)
Reminder letter

Week 18
PRIORITY Web/Mail invite letter and
questionnaire

Week 18
PRIORITY Web/Mail invite letter and
questionnaire

Week 25 (07/05/13)
Web invite letter

Week 25 (07/05/13)
Web invite letter
*$30 Incentive for TG5
Week 26 (07/12/13)
Reminder letter

Notes: (1) In the Hard to Enumerate methodology, Treatment Group #1 will not receive an incentive
(2) The bolded text in the “hard to enumerate” column identifies differences from the default pathway

4