Overview of SCV Design

Attachment 8. Overview of SCV Design by
Condition and Wave
Additional information about the SCV study design, along with illustrations of the data
collection flow, are provided below by condition and wave.
Condition 1
Wave 1. In Condition 1, CAPI interviews will be attempted with all eligible households.
Household interviews will be completed in-person, via CAPI. However, individual interviews
will be completed via CAPI or CATI, with initial contact attempts for individual respondents by
CAPI and nonresponse follow-up by CATI. Condition 1 ($0 incentive) is considered a control1
group because the protocol closely resembles the current NCVS collection procedures Cases will
be transferred to CATI if a phone number is available and the address has been visited at least 10
times during the first 12 weeks of data collection. The control condition is needed to ensure
comparability between the national panel survey and the experimental conditions.2
After evaluating performance measures, such as the number of completed surveys by day and the
average time between letters and the completion of the household and individual interviews,
RTI, in collaboration with BJS, will determine an optimal time to switch all individual
respondent cases to a centralized telephone nonresponse follow-up. Switching cases to a
telephone follow-up too early in data collection may have a cost impact and change the response
propensities, affecting overall response rates. Studies on mode preference suggest that
respondents have different propensities to respond to different modes (Groves and Kahn, 1979)
and providing alternative modes (as in mixed-mode designs) may be an effective way of
improving response rates (Shettle and Mooney, 1999). Conversely, switching cases to telephone
follow-up too late may result in insufficient time to work the cases, and thus, prevent the study
from achieving the desired effect on response rate and nonresponse error.
Wave 2. The second wave of data collection will begin 6 months after the completion date for
each case in Wave 1. Bounded interviews require data from Wave 1 to be collected, thus we can
assume that households who responded in Wave 1 will be interviewed in Wave 2, or the number
of completed household interviews in Wave 1 will be the starting sample size for Wave 2. The
goal of Wave 2 is to evaluate whether less costly, self-administered follow-up yields acceptable
response rates given the different modes used for the initial survey request in Wave 1.

1

For purposes of this research, the term “control” refers to the comparison group in the SCV experimental
design that most closely resembles the national panel study.
2
Using the most current NCVS data instead of having Condition 1 would not provide comparable data as
multiple survey factors impact the data collection process (e.g., response rates can be affected by the
geographic area of the experiment, the interviewer pool, the recruitment procedures, coding of call
outcomes, and other differences between survey organizations and sample design).

All Wave 1 respondents will be mailed a package that includes a request to go to the study
Website to complete the survey or call a toll-free number to complete the interview via
telephone. We will also use e-mail addresses collected at Wave 1 to alert respondents to the
Wave 2 survey request, providing them a link to the survey website and a unique access code to
log in and complete the survey via the Web. A week later, following the Tailored Design Method
(Dillman, 2000), a thank you postcard will be sent to all households to express appreciation for
taking part in the survey, and remind those who have not done so already to complete the Web
survey or call the toll-free number. Four weeks after the original mailing, a reminder letter will
be sent to nonrespondents. This sequencing of the distribution of materials is a common feature
in self-administered modes (when used as stand-alone modes or in mixed-mode designs) and has
been proven to maximize response rates in mail surveys (see Dillman, 1978; Dillman, 2000). In
parallel with the reminder letter mailing, we will also prompt nonrespondents via email,
providing the survey website link and unique survey access code. Two weeks after the
replacement package has been mailed, nonrespondents will be contacted by telephone (collected
in Wave 1 or available from address matching efforts) in an effort to secure the interview.
Because the address will be the sampling unit in the SCV field test, we will not follow and
interview Wave 1 respondents who move away from the sampled address and are no longer
living there at Wave 2. Moreover, because an important objective of the study is to examine how
a respondent’s survey mode at Wave 1 impacts their propensity to respond at Wave 2, it will not
be necessary to include Wave 1 nonrespondents or new residents of a sampled address at Wave 2
since they did not respond in the first wave. Procedures will be implemented at Wave 2 to
identify movers and new residents through U.S. Postal Service forwarding orders and address
confirmation questions embedded in the Screener and to exclude them from the sample.
Condition 2
Wave 1. Condition 2 will test the use of only telephone mode, subjecting to telephone as many
cases as possible. The attractiveness of this condition is that telephone is a less-costly alternative
to face-to-face interviewing and may be even more desirable (due to the increased social distance
between the interviewer and the respondent), given the sensitive nature of the questions. For the
initial contact with Condition 2 households, letters will be sent to all sample addresses asking
them to call a toll-free number to complete the roster and household interview. The use of an
incentive in this design is expected to be a critical component in motivating sample members to
initiate the call and complete the interview via inbound CATI (see Attachment 1 for a full
discussion of incentives).
Utilizing incentives in this condition should still lead to substantial savings from not having to
make numerous call attempts to cases, whether by telephone or in-person. After three weeks,
calls (outbound CATI) will be made to all telephone numbers (land-line and cell) that can be
matched to nonrespondent sample addresses, a rate expected to approach 50 percent. A greater
proportion of the sample is expected to be attempted by telephone among the remaining
household members, as household respondents would have provided telephone numbers. Finally,
the remaining nonrespondents will be approached via CAPI, which is a more expensive mode.
As much of the CAPI cost is associated with travel and particularly with travel that results in
noncontact, additional individual household members who are at home at the time of a CAPI

interview will be asked to participate in the survey at that time. This condition will be
particularly useful in the event that the inbound CATI mode performs poorly as a form of initial
contact. Additional efficiency can be achieved through the use of centralized CATI, particularly
when implemented on a large scale. Although this efficiency may not hold for all surveys and
survey organizations it is expected to occur under the current protocol.
Wave 2. The data collection protocol for Condition 2, Wave 2, will mirror that of the Condition 1
respondents, with the primary survey modes being inboiund CATI and Web, and nonresponse
follow-up via outbound CATI. Moreover, as in Condition 1, we will not follow and interview
Wave 1 respondents who move away from the sampled address and are no longer living there at
Wave 2, or include Wave 1 nonrespondents or new residents of a sampled address at Wave 2.

Exhibits 1 and 2 illustrate the flow of the field test data collection by condition at Waves 1 and 2,
respectively.

Exhibit 1. Wave 1 Data Collection Flow for Conditions 1 and 2

Exhibit 2. Wave 2 Data Collection Flow for Conditions 1 and 2

*BSQ = Basic Screening Questionnaire (or Screener)