Memo Outlining Non Material Changes

MEMORANDUM
TO:

Office of Management and Budget

FROM:

Brad Hesse, HINTS Project Officer

DATE:

November 26, 2007

RE:

HINTS 2007 Nonsubstantive Change Request
OMB #0925-0538

This memo summarizes the Health Information National Trends Survey (HINTS) 2007 pilot study and
resulting decisions about the study design plan, in accordance with the stipulations in the OMB approval
letter for HINTS 2007 received on August 2, 2007. The changes described below can be classified as
either reductions in OMB-approved instruments or the elimination of OMB-approved embedded
experiments. None of the changes that are being requested are substantive or contain new elements. NCI
is therefore requesting that these changes be considered under the Nonsubstantive Change Request
procedure.
As outlined in the Supporting Statement of the OMB package submitted for HINTS 2007, the HINTS
target population is all adults aged 18 or older in the civilian noninstitutionalized population of the United
States. The sample design for HINTS 2007 consists of two samples with each sample being selected from
a separate sampling frame. One sample will be a list-assisted random digit dial (RDD) sample selected
from all telephone exchanges in the United States, following the design of HINTS 2003 and HINTS 2005.
This will result in a nationally representative sample of households with a landline telephone. The second
sample is comprised of addresses selected from a list based on U.S. Postal Service (USPS) administrative
records. The purpose of using the dual-frame approach is to directly address the increasing migration of
landline telephone to mobile-only telephone households in a cost-effective manner. Pilot testing was
conducted on both the telephone and mail samples.

1. Mail Survey Response Rates
The mail pilot survey (USPS sample) was conducted from August 23 through October 15, 2007. The
pilot study included:
- mailing advance letters to households;
- mailing a 1st package of instruments to households;
- mailing a postcard reminder to households that had not responded;
- mailing a 2nd package of instruments to households that had not responded; and
- conducting telephone follow-up interviews with a randomly selected adult in households that had
not responded to the mail survey.
Results
The overall response rate for the mail survey was 27.6%. The rate differed significantly by sampling
stratum. Those in the low minority stratum had a rate of 31%, while the rate in the high minority
stratum had a rate of 18.7%. Most of the responses came from the mail survey, with only a small
proportion being picked up during the telephone follow-up (see table below).

1

Overall Response Rate
Mail Response Rate
Telephone Follow-up Response Rate

27.59%
23.74%
3.85%

Low
Minority
30.97%
26.03%
4.93%

Household-level Mail Response Rate
Within-household Mail Response Rate

31.32%
77.24%

33.86%
78.91%

23.84%
75.03%

Proportion of delivered mail with listed phones
Household-level Mail Response Rates for Households with Listed
Phones
Telephone Response Rate

53.43%

56.95%

43.05%

37.62%

37.01%

40.00%

9.27%

10.93%

2.56%

Overall

High
Minority
18.73%
17.75%
0.99%

Decisions
 Because overall response rates for the pilot mail survey are in the range expected, NCI will
continue the dual-frame study design as originally planned.
 Because response rates for the telephone follow-up portion of the mail pilot were much lower
than expected, NCI will not continue this follow-up in the main study and will instead focus
resources elsewhere (see number 2 below).

2. Incentive and Mail Mode Experiments
Two embedded experiments were conducted during the mail pilot. The incentive treatment consisted
of including a $2 incentive in the initial mailing of the questionnaires for half of the sample. The
other half did not receive any money. The express mail treatment consisted of mailing the 2 nd mailing
of the questionnaires using an express mail carrier (Federal Express) to half the sample. The other
half received the 2nd mailing via USPS.
Results
Both the incentive and mail treatments significantly increased the return of the mail survey. Each of
these treatments increased the household-level response rate by approximately 10 percentage points.
The two treatments seem to complement each other. When each is applied separately, the householdlevel response rate increases from 22% to 31%. When both are used together, the response rate goes
up an additional 10 percentage points to 41%.
Incentive
$2

$0

Express

Yes

41.1

30.9

36.1

Mail

No

31.0

21.8

26.3

35.8

25.9

Decisions
 Because these 2 treatments appear to have a significant impact on household response rates, NCI
will eliminate the experimental status of these 2 treatments and instead will send all mail
respondents the $2 incentive in the first mailing and will send all second mailings via Federal
Express.
2

3. Short vs. Long Form
In the mail pilot, an experiment was conducted to test whether sending a shorter questionnaire would
increase the response rate. For this purpose, two questionnaires were designed. The long instrument
included all items that were on the CATI interview. The short instrument cut the instrument 25% 30%.
Results
There was no difference in the response rates for the two different questionnaires (30.8% vs. 30.8%).
Decisions
 The long instrument was shortened to keep it compatible with the CATI instrument which was
also shortened. Please see Attachment A for the mail instrument with changes from the pilot
highlighted in yellow.

4. CATI Extended Timing
The CATI pilot test was conducted from September 17 through October 15, 2007. One of the main
purposes of conducting the pilot of the RDD study was to get an accurate estimate of the amount of
time the extended CATI interview takes to complete.
Results
The CATI extended instrument took an average of 40.12 minutes to complete.
Decisions
 A number of items were cut from the CATI extended instrument in order to bring the time down
to 30 minutes. Please see Attachment B for the revised CATI extended instrument, with cuts
indicated in yellow.

5. Type of Letter and Telephone Introduction
Two different experiments were conducted with the telephone sample. One experiment tested which
of two different letters would increase the response rate. One letter had a text format, which
discussed different aspects of the study. The other letter included a set of bullets highlighting results
from the study.
Two different introductions for the screener were also tested. One introduction characterized the
study as a “national study on people's needs for health information”. The other introduction
characterized it as a “national health study”.
Results
Neither of these experiments yielded statistically significant results. For the type of letter, the rates
were 29.0% vs. 25.4%. For the introduction, the two response rates were 27.9% vs. 26.5%. There
were approximately 400 cases in each of the experimental groups.
Decisions
 Based on results from the cognitive testing which was conducted prior to the pilot test, NCI will
use the bulleted letter (see Attachment C—with no changes since the pilot).
 Both introductions will be provided to interviewers on the screen of the RDD and interviewers
will be allowed to select whichever introduction they feel is most appropriate
3