19 PATH Study Field Test Experiment 2012-11-08_V3a

Population Assessment of Tobacco and Health (PATH) Study (NIDA)

Attachment 19
PATH Study Field Test Experiment
November 16, 2012

Population Assessment of Tobacco and Health (PATH) Study (NIDA)

PATH Study Field Test Experiment
The field-test experiment is designed to empirically determine the most cost-effective combination
of incentive amount and household screener version that maximizes both the accuracy of the
screener data and the response rate. To inform decisions on the household screener incentive and
length, the field test will vary two factors in an experiment. The first factor will be the value of the
incentive offered for completing the household screener. Three levels of incentives are proposed for
testing ($0 versus $5 versus $10); respondents in both the $5 and $10 groups will get a check when
they complete the screener, on the spot. The second experimental variable will be the format of the
screening questions. Two versions are proposed for testing. Version A asks a brief set of questions
about adults in the household (after a roster of the household has been compiled); Version B asks a
more detailed set of questions about the different forms of tobacco use by adults (after a roster has
been compiled). (Version B is the household screener included in this information collection
request.)
The main outcome variables will be the screener response rates, the percentage of households
classified as containing a tobacco user, and the response rate to the baseline interview. Detailed
questions will also be asked of a certain number of cases classified as non-users (n=508 in total) to
try to estimate a misclassification rate. These detailed questions about tobacco use will be
administered directly to the target person, not to a proxy. This is the same method that has been
used to estimate error rates in several previous studies of tobacco reporting (Gilpin, Pierce, Cavin,
Berry, Evan, Johnson, and Bal, 1994; Hyland, Cummings, Lynn, Corle, and Giffen, 1997; Navarro,
1999).
Three-thousand one-hundred and sixty-six (3,166) addresses are proposed to be fielded. Assuming
that about 87 percent of these are occupied dwelling units, there will be approximately 2,800
households to screen. Two-thirds of these households (1,870 households in total) will be assigned
to get the promised incentive for completing the screener, with half (n=935) being promised $10
and the other half promised $5; the remaining third will be assigned not to receive an incentive. The
screener version variable will be crossed with the incentive factor; 1,400 households will be assigned
to get each version of the screener. Both of these experimental variables will be varied withinsegment. Power calculations indicate power well above 0.80 for detecting differences between the
two screener groups of five percentage points in response rates. For example, assuming a design
effect of 1, there will be power of 0.99 to detect a difference between a 92 percent screener response
rate in the short screener group versus an 87 percent response rate in the long screener group.
Similarly, if the response rates are 72 percent in the short screener group but 67 percent in the long
screener group, the power is still above .80. The power is also adequate for detecting incentive
effects. There will be power of 0.60 or above to detect 5 percent differences in response rates
between pairs of incentive groups (for example, $10 versus $5 or $5 versus no incentive). Power will
be even higher if analytical models that incorporate covariates are used.
Finally, baseline individual screener interviews (which include detailed tobacco use questions) will be
conducted with 840 adults (including 332 tobacco users and 508 cases classified as non-users based
on the household screener). Extended interviews also will be conducted with 590 of these adults,
including all 332 of the tobacco users and approximately 258 of the non-users; individual screeners
only will be conducted with the remaining 250 non-users (to allow detection of false negative
classifications from the household screener). There will be adequate numbers of persons who have
been screened to support these sample sizes.
1

Population Assessment of Tobacco and Health (PATH) Study (NIDA)

Final decisions about which version of the screener and which incentive level to use will be based on
three criteria: 1) the screener response rate; 2) the percentage of persons classified as tobacco users;
and 3) the agreement between screener classifications and classifications based on the more detailed
questions administered directly to the sample person. The longer screener is predicted to produce
more accurate identification of tobacco uses (i.e., more persons classified as tobacco users and
greater agreement between the screener and the more detailed questions) but to have lower screener
response rates. The combination of a long screener and the $10 incentive may represent the best
approach for minimizing response errors without lowering response rates. Analysis of field test data
will also examine the impact of the incentive on callbacks and main interview response rates. In
summary, the field test experiment is expected to provide evidence on the most cost-effective
combination of incentive amount and screener version that maximizes the accuracy of the screener
data and the response rates for the screener and the main interview.
References
Gilpin, E.A., Pierce, J.P., Cavin, S.W., Berry, C.C., Evan, N.J., Johnson, M., & Bal, D.G. (1994).
Estimates of population smoking prevalence: Self- vs proxy reports of smoking status.
American Journal of Public Health, 84, 1576-1579.
Hyland, A., Cummings, K.M., Lynn, W.R., Corle, D., & Giffen, C.A. (1999). Effects of proxy
reported smoking status on population estimates of smoking prevalence. American Journal
of Epidemiology, 145, 746-751.
Navarro, A. (1999). Smoking status by proxy and self report: Rate of agreement in different ethnic
groups. Tobacco Control, 8, 182-185.

2