Download:
pdf |
pdf2003 NATIONAL SURVEY ON
DRUG USE AND HEALTH
NONRESPONSE AMONG SAMPLE
MEMBERS 50 OR OLDER IN NSDUH:
METHODOLOGICAL IMPROVEMENT
PROTOCOL (MIP) FINAL REPORT
Contract No. 283-98-9008
RTI Project No. 07190
Authors:
Joe Murphy
Joe Eyerman
Project Director:
Tom Virag
Prepared for:
Substance Abuse and Mental Health Services Administration
Rockville, Maryland 20857
Prepared by:
RTI International
Research Triangle Park, North Carolina 27709
August 31, 2005
2003 NATIONAL SURVEY ON
DRUG USE AND HEALTH
NONRESPONSE AMONG SAMPLE MEMBERS
50 OR OLDER IN NSDUH:
METHODOLOGICAL IMPROVEMENT
PROTOCOL (MIP) FINAL REPORT
Contract No. 283-98-9008
RTI Project No. 07190
Authors:
Project Director:
Joe Murphy
Joe Eyerman
Tom Virag
Prepared for:
Substance Abuse and Mental Health Services Administration
Rockville, Maryland 20857
Prepared by:
RTI International
Research Triangle Park, North Carolina 27709
August 31, 2005
Acknowledgments
This report would not be possible without the guidance and input of staff from the Center for
Behavioral Health Statistics and Quality. In particular, Joel Kennet provided useful comments.
At RTI International (a trade name of Research Triangle Institute), Valerie Garner and Richard
Straw provided report production assistance.
ii
Table of Contents
Chapter
Page
1.
Executive Summary ............................................................................................................ 1
1.1
Introduction ............................................................................................................. 2
1.2
Organization of This Report ................................................................................... 3
2.
Previous Research on Nonresponse and Age...................................................................... 5
2.1
Strategies Employed on Other Studies ................................................................... 6
3.
Nonresponse and Age in NSDUH ...................................................................................... 9
3.1
Components of Nonresponse ................................................................................ 10
4.
Potential Explanations for Refusals among the 50+ ......................................................... 19
4.1
Respondents' Reasons for Refusal ........................................................................ 19
4.2
Age-Specific Reactions to Incentives ................................................................... 21
4.3
Additional Characteristics Influencing Refusal among the 50+ ........................... 21
4.4
Need for Qualitative Information.......................................................................... 27
5.
Focus Groups .................................................................................................................... 29
5.1
FI Focus Groups .................................................................................................... 29
5.2
Potential Respondent Focus Groups ..................................................................... 30
6.
Tailored Field Strategies for the 50+ Age Group ............................................................. 35
6.1
Assessment of the Impact on Prevalence Rates .................................................... 36
References ..................................................................................................................................... 39
Appendix
Appendix A: Tables Relating to Figures in the Report............................................................... A-1
iii
iv
List of Tables
Table
Page
4.1
Incentive Experiment Weighted IRR, by Age ...................................................................22
4.2
Respondent, Household, Environment, and FI Characteristics .........................................22
4.3
Weighted IRRs for the 50+ Age Group, by Characteristics: 2002 ....................................24
4.4
Logistic Regression Model Odds Ratios and Significance: 2002 ......................................26
A.1.
Weighted Interview Response Rate (IRR), by Age: 1999 to 2004 ................................. A-1
A.2.
Weighted IRR by 5-Year Age Groups: 1999 to 2004..................................................... A-1
A.3.
Weighted Nonresponse Rates, by 5-Year Age Groups: 2001......................................... A-2
A.4.
Impact of Age-Specific Nonresponse on Overall IRR: 2001 ......................................... A-2
A.5.
Weighted Nonresponse Rates, by 5-Year Age Groups: 2002......................................... A-3
A.6.
Impact of Age-Specific Nonresponse on Overall IRR: 2002 ......................................... A-3
A.7.
Contribution of Nonresponse Components to Percentage Change in IRR: 2001 to
2002................................................................................................................................. A-4
A.8.
Reasons for Refusal, by Age: 2001................................................................................. A-5
A.9.
Reasons for Refusal, by Age: 2002................................................................................. A-6
A.10. Contribution of Refusal Reasons to Reduction in Refusal Rate: 2001 to 2002 .............. A-7
v
vi
List of Figures
Figure
Page
3.1
Weighted Interview Response Rate (IRR), by Age: 1999 to 2004 ....................................10
3.2
Weighted IRR, by 5-Year Age Groups: 2001 and 2002 ....................................................12
3.3
Weighted Nonresponse Rates, by 5-Year Age Groups: 2001............................................13
3.4
Impact of Age-Specific Nonresponse on Overall IRR: 2001 ............................................14
3.5
Weighted Nonresponse Rates, by 5-Year Age Groups: 2002............................................15
3.6
Impact of Age-Specific Nonresponse on Overall IRR: 2002 ............................................15
3.7
Contribution of Nonresponse Components to Percentage Change in IRR: 2001 to
2002....................................................................................................................................16
4.1
Reasons for Refusal by Age: 2001 and 2002 .....................................................................20
4.2
Contribution of Refusal Reasons to Reduction in Refusal Rate: 2001 to 2002 .................20
vii
viii
1. Executive Summary
This report examines the relationship between age and the propensity to respond in the
National Survey on Drug Use and Health (NSDUH). Previous analyses have demonstrated that
sample members aged 50 or older (50+) are consistently less likely to complete the interview
than younger sample members (Office of Applied Studies [OAS], 2003). A more detailed
examination of this age group is conducted in this report to identify the characteristics of the
nonrespondents, diagnose the causes of the nonresponse, consider the impact of the nonresponse
on prevalence rates, and suggest possible remedies. This analysis is being conducted according
to the Methodological Improvement Protocol (MIP), which specifies identification of the
problem, diagnosis of the causes, consideration of potential solutions to the problem, and
assessment of the impact of potential changes on prevalence rates. The results are summarized
below.
•
Lower response rates for the 50+ age group are due to higher refusal rates across the age
group and increasingly higher rates of physical or mental incapability beginning at age
60. Noncontact rates were lower for the 50+ group than for the younger age groups
(Chromy, Bowman, Crump, Packer, & Penne, 1999; Gfroerer, Lessler, & Parsley, 1997).
•
The methodological changes introduced in 2002 were generally effective in improving
response rates (OAS, 2003). However, the changes had little effect on the response rates
for the 50+ age group (see Figure 3.1 in Section 3).
•
The 50+ age group generally was not responsive to the provision of an incentive,
although respondents aged 50 to 59 did respond to the incentive in the incentive
experiment in a manner similar to the younger age groups (RTI International, 2002; see
also Table 4.1 in Section 4.2).
•
More refined codings of the 50+ age group demonstrate that the least cooperative ages
were 55 to 69 and the most cooperative were 70 to 79 (RTI International, 2002; see also
Figure 3.5 in Section 3.1).
•
Respondents aged 55 to 69 were less responsive to the provision of the incentive and
other enhancements (OAS, 2003). This suggests that a method tailored to the older
groups may be required, such as an appeal to civic duty (see Figure 3.2 in Section 3.1).
•
The 50+ age group represented about one fourth of all refusals in 2001 and about one
third in 2002. The percentage of the full sample that was 50+ remained the same in these
2 years.
•
Even when controlling for interviewer, environment, and other respondent characteristics,
the older sample members were more likely to refuse than younger sample members in
2002 (see Table 4.4 in Section 4.3).
•
Focus groups with field interviewers (FIs) revealed that the older sample members were
less trusting in the legitimacy of the survey and less comfortable with the survey topic
(Murphy & Schwerin, 2003; see also Section 5).
•
Focus groups with FIs also revealed that older sample members were less likely to be
persuaded by the incentive than younger groups. In fact, some may be more likely to
1
refuse because they believe that the provision of an incentive resembles a confidence
scam or other fraudulent practice (Murphy & Schwerin, 2003; see also Section 5).
•
The FIs also reported in the focus groups that sample members with children in the
household were more likely to participate because the survey topic was more salient.
Older sample members may place less salience on the survey topic because they are less
likely to have children living in the household (Murphy & Schwerin, 2003; see also
Section 5).
•
Focus groups with potential 50+ respondents confirmed much of the information
gathered in the FI focus groups. These groups made clear the importance of topic
understanding and interest, courtesy and flexibility in the survey approach, the selection
process, descriptive materials, and trust in gaining cooperation from members of this age
group (Murphy, Schwerin, Hewitt, & Safir, 2005; see also Section 5).
The report contains suggestions for a series of methodological enhancements that may
improve response rates for the older age groups, including additional interviewer training,
adjustments to the lead letter and refusal letter, alternative data entry modes, and improved
public outreach. The possible impact on trend data of any considered change should be fully
evaluated with an experiment before implementation.
1.1
Introduction
Response rates in NSDUH are lower for sample members aged 50+ than for any other
age group. Low response rates for this age group represent a potential threat to the accuracy of
both age-specific and overall prevalence rates. With the introduction of a $30 incentive given to
respondents in 2002, response rates improved among the younger age groups, but there was only
a small increase among sample members aged 50+.
This report analyzes the relationship between age and response as part of NSDUH's MIP.
The MIP is a systematic process that allows methodological enhancements to survey
instruments, forms, and data collection protocol while minimizing the impact on trend data. The
MIP specifies that survey improvements are to be implemented through a minimum of five steps:
1. a detailed evaluation of existing information to carefully identify the need for the
change;
2. an assessment of the impact that the current need for change is having on substance
use prevalence reports;
3. a detailed description of the protocols that will be followed if the change is applied to
the main study;
4. an evaluation of the expected impact of the changes on prevalence rates and trend
data that may result from the application of the change to the main study; and
5. application of the experiment results to develop statistical adjustments for prevalence
rates and trends; use of the field results to introduce change to the main study in a
way that minimizes the impact on trend data and allows tracking of changes in
prevalence rates associated with the enhancement.
2
This report addresses the first three steps of the MIP by providing a detailed evaluation of
the problem of nonresponse among sample members aged 50+, presenting potential tailored field
strategies that could ameliorate the problem, and considering the potential impact of these
changes on prevalence rates and trend data. The goal of this investigation is to inform targeted
methodological improvements to increase age-specific and overall response rates, thereby
reducing the potential for bias in the survey estimates.
1.2
Organization of This Report
This report is organized into six sections. Section 1 summarizes the response rate
problem and provides a summary of the MIP process. Section 2 reviews the relationship between
age and response rates seen in other surveys and methods used to address nonresponse among
older sample members. Section 3 describes this relationship in NSDUH over the past 5 years and
examines changes between 2001 and 2002 in more detail. The roles of noncontacts, refusals, and
other incompletes are discussed, and the 50+ group is disaggregated into smaller age categories
to identify the segments of this group that contribute most to the lower rate of response and
higher rate of refusals. Section 4 focuses on the relationship between age and other potential
correlates accounting for higher nonresponse among older sample members. Section 5 describes
the results of focus groups conducted with NSDUH FIs and potential 50+ respondents on the
topic of nonresponse among those aged 50+ and ways of addressing it. Section 6 presents
potential solutions to the problem and considers the possible effects of these solutions on
prevalence rates.
3
4
2. Previous Research on Nonresponse and
Age
Herzog and Rodgers (1988) analyzed data from several face-to-face surveys, including
the Americans View Their Mental Health (AVMH) study and the American National Election
Studies, and found a linear decline in response rates with increasing age. They reported that
refusal as a proportion of all nonrespondents increased for the middle years (35 to 74), then
declined, reaching particularly low proportions among the oldest old (75+). The reason for
nonresponse among the oldest age groups was less often outright refusal than among the middle
age groups. The authors argued that the surveys examined did not take advantage of special
characteristics of the older population to optimize the approach to this age group. Interviewers
were not trained in specific problems with older respondents. In addition, these surveys held no
particular relevance or interest for the older population, a factor that might be critical for
obtaining high response rates among older adults (Hoinville, 1983). They stressed that future
research should make a vigorous attempt to learn more about older persons who do not get
interviewed and incorporate new methods designed to lower nonresponse among older sample
members.
The negative correlation between response rates and age has been noted elsewhere in the
nonresponse literature. Cohen, Machlin, and Branscome (2000) found that nonrespondents in the
second round of the 1996 Medical Expenditure Panel Survey (MEPS) were more likely to be
elderly. McQuillan, Khare, Karon, Schable, and Vlahov (1997) analyzed nonresponse in the
National Health and Nutrition Examination Survey (NHANES III) and found that white and
black men 40 to 59 years of age were least likely to participate in the survey. DeMaio (1980)
reported that middle-aged and older people cooperate at lower rates than those under 30.
Zimowski, Tourangeau, Ghadialy, and Pedlow (1997) alluded to lower completion rates in
transportation surveys in a review of nonresponse in such surveys. They suggested that elderly
respondents may fail to fill out or return the travel diaries used in such surveys because "they
either have no trips to report or feel that the survey is not relevant to them." Response rates were
negatively correlated with age in the 1992 National Adult Literacy Survey as well (Kirsch et al.,
2001). This in-person survey of adults aged 16 or older consisted of a screening survey followed
by interviews with one or two selected adults in each household. The interview consisted of a
series of background questions (about 20 minutes) followed by completion of a literacy task
booklet. Respondents were provided with a $20 incentive. Response rates in this survey showed
consistent declines in response rate by age group, from 85 percent for those aged 16 to 24 to 77.4
percent for those 65 or older. Others have also noted that increased age of household members
has a negative effective on survey cooperation (Comstock & Helsing, 1973; Paul & Lawes,
1982; Redpath & Elliot, 1988; Smith, 1995).
Groves and Couper (1998) found evidence of a curvilinear relationship between age and
cooperation. They found that middle-aged households were less likely to cooperate than young
and old households, and although elderly persons are more frequently at home due to their low
employment rate and reduced mobility, their poor health may prevent them from survey
participation.
5
Chiu, Riddick, and Hardy (2001) analyzed data from the National Health Interview
Survey (NHIS) and found a different relationship between response and age. They reported that
households containing senior adults and members with activity limitations were negatively
correlated with difficult interviews when controlling for all other predicting variables. They
believed that this is because these people are more likely to be home during the day and because
the topic of health is viewed favorably among elderly persons. Similarly, Kautter, Khatutsky,
Pope, and Chromy (2003) found no significant relationship between age and nonresponse in their
analysis of the Medicare Beneficiary Survey (MCBS). Like the NHIS, the MCBS is a large-scale
household health survey, although the majority of respondents are aged 65 or older. It may be
that the topic of health is salient to older respondents and that they are more likely to respond to
surveys that deal directly with health topics. The issue of topic salience among older respondents
in NSDUH is examined in Section 5 of this report.
Tremblay and Moore (1995) reported higher response rates among elderly persons in the
1993 National Survey of College Graduates, a mail survey with telephone and in-person followup of U.S. residents under age 76 with a bachelor's degree or higher. The sample was drawn
from a frame of respondents to the long form questionnaire of the 1990 decennial census. The
response for those aged 60 or older was 85.9 percent, while it was 69.3 percent for those aged 16
to 29 and 80.1 percent for persons between 30 and 59 years of age. The fact that the sample
consisted entirely of college graduates would seem to limit the relevance of these response rates.
But in their discussion of the effects of age on response rate, Groves and Couper (1998) noted
that older cohorts are less educated and have lower socioeconomic status, factors that are also
associated with lower willingness to participate in surveys. As such, these may be factors that
confound or intervene in the relationship between age and survey response on NSDUH.
Additional evidence on the relationship between age and survey participation can be
found in NSDUH itself prior to 1999.1 Chromy et al. (1999) noted that for the period from 1993
to 1998, interview response rates for those aged 50+, overall, were lower than for all other age
groups. However, the median response rate for those 50+ (76.2 percent) over this time period
was not all that different from the median response rates for those in the other age groups, except
for response rates for those aged 12 to 17.
2.1
Strategies Employed on Other Studies
To combat the effects of lower response rates among older sample members, a variety of
methods has been implemented on other surveys. In this section, we present a summary of these
methods. This information will be helpful in determining methods to be tested and implemented
for NSDUH:
•
Tailoring the questionnaire for older respondents. Sensory deficits are frequently
experienced as people age, including hearing and/or vision loss (Herzog & Rodgers,
1988). Both can inhibit communication between the respondent and the interviewer
(Jobe, Keller, & Smith, 1996). Although large-print surveys or written communications
cannot always compensate for poor eyesight and loss of hearing can make telephone
interviews impossible and in-person interviews extremely difficult (Rodgers & Herzog,
1
Prior to 2002, NSDUH was called the National Household Survey on Drug Abuse (NHSDA).
6
1987), data collection instruments can be tailored to address these concerns. For the frail
elderly population or as a refusal conversion technique, a shorter version of the
questionnaire containing only the core measures can be developed and administered.
In the Second Longitudinal Study on Aging (LSOA II), a shortened instrument was
administered over the telephone as a last resort to respondents who, because of the length
of the computer-assisted telephone interviewing (CATI) instrument, would have
otherwise refused to participate (National Center for Health Statistics [NCHS], 2002).
•
Allowing proxies to respond for older sample members. A substantial number of
interviews on the Asset and Health Dynamics Among the Oldest Old (AHEAD) study
were completed by a proxy respondent rather than by the designated respondent. This
typically occurred because the designated respondent was ill, cognitively impaired, or
unable to participate in a relatively lengthy interview. The incidence of proxy responses
varied with the age of the designated respondent, with almost one third of the interviews
in the oldest age group being conducted with proxy respondents rather than with the
designated respondent. For the proxy interviews, the survey material was generally the
same as for designated respondents except that cognitive test questions were not used on
proxy interviews and expectation/subjective perception questions were not asked (Myers,
Juster, & Suzman, 1997).
•
Employing interviewers with strong interpersonal skills. Increased success may be
obtained by interviewers with knowledge of the constraints under which older
respondents may labor in responding to surveys (New England States Consortium
[NESC], 2002).
•
Developing a training module. A special section can be devoted to contacting and
interviewing older respondents (NESC, 2002).
•
Emphasizing interviewer rapport. Familiarity may be an important factor in
establishing rapport with older respondents (Blake, Korovessis, & Pickering, 2002).
•
Pacing the survey. A slower pace may make older respondents more comfortable
(NESC, 2002).
•
Making the primary purpose of the survey clear. Some older respondents seek
assurance that the survey is designed to gather the data that policymakers are seeking
(NESC, 2002).
•
Soliciting advice from the geriatric research community. (NESC, 2002).
•
Emphasizing anonymity and confidentiality. For elderly home care clients, fear of
losing services is a particular concern. This creates difficulties for researchers to fully
capture their home-care experience (Forbes & Neufeld, 1997; Nehring & Geach, 1973).
Emphasizing anonymity and confidentiality during the survey process may help to
alleviate this bias (Sudman & Bradburn, 1982).
•
Alleviating respondent fears. Respondent fears may be alleviated by interviewers'
offering to meet at another location (e.g., local hospital or health department), talking
with a family member to confirm the legitimacy of the study, letting the participant know
that it is acceptable to have another person present for the interview, or reassuring them,
when appropriate, that existing medical conditions are not barriers to participation
(Moorman, Newman, Millikan, Tse, & Sandler, 1999).
7
•
Reaching out to the public. The U.S. Bureau of the Census and the Department of
Health and Human Services Administration on Aging (AoA) coordinated in a
government-wide outreach effort to encourage Americans, in particular older persons and
their caregivers, to complete and return their census forms. They mounted education and
information campaigns to get the word out on the importance of Census 2000 (FirstGov
for Seniors, 2000).
•
Providing mode options. On the AHEAD study, persons 80 years of age or older are
assigned to a personal interview, although they can be converted to a telephone interview
at their request (Myers et al., 1997).
•
Converting refusers with a financial incentive.2 The Health and Retirement Survey
uses an "end game" strategy of offering reluctant respondents a large financial bonus for
participation ($100) and asking for an immediate "yes" or "no" decision (NESC, 2002).
2
There are questions about the ethics and fairness of the use of targeted incentives for certain subgroups of
interest or for refusal conversion (Groves & Couper, 1998).
8
3. Nonresponse and Age in NSDUH
In NSDUH, response rates have generally decreased with an increase in respondent age.
Gfroerer, Lessler, and Parsley (1997) noted that age in NSDUH was related to interview
nonresponse, with refusal rates increasing as age increased. Ideally, response rates would be
consistently high across all age groups. This would reduce the potential for bias in both agespecific and overall prevalence rates.
This section presents a detailed analysis of the relationship between nonresponse and age
in NSDUH, with particular emphasis on changes in this relationship accompanying the
introduction of several methodological changes in 2002. The goals of this section are to identify
the particular ages and components of nonresponse contributing most to lower response rates
among those aged 50+ and determine the impact that these nonrespondents have on the overall
interview response rate (IRR).
We first consider response rates3 by age for the survey years from 1999 to 2004. The year
1999 is selected as the starting point to ensure comparability of results.4 As Figure 3.1 shows,
IRRs were successively lower for each sampled age group (12 to 17, 18 to 25, 26 to 34, 35 to 49,
and 50+) in each year.5 Table A.1 in Appendix A provides the values corresponding to this
figure. Across all years, response rates were lowest for the 50+ age group and highest for the 12
to 17 age group. The difference in response rates between these two groups remained around 13
percent in 1999, 2000, and 2001 and was around 18 percent in 2002, 2003, and 2004. Response
rates for each age group increased from 2001 to 2002, but less for the 50+ age group than for the
other age groups. These increases were statistically significant for all age groups except for those
aged 50+.
The increase in response rates between 2001 and 2002 correlates with several
methodological changes introduced during this period (OAS, 2003):
•
The name of the survey was changed in 2002 from the National Household Survey on
Drug Abuse (NHSDA).
•
A $30 incentive was given to interview respondents beginning in 2002.
•
Improved data collection quality control procedures were introduced in the survey during
2001 and 2002.
3
Unless otherwise specified, all rates presented in this report are calculated using weighted data that do not
include nonresponse or population adjustments.
4
The redesign of NSDUH in 1999 resulted in major changes in data collection procedures. In particular,
the sample size was increased, there was a transition from a paper to a computerized instrument, and there was a
transition from paper-and-pencil interviewing (PAPI) to computer-assisted interviewing (CAI). An analysis of the
split PAPI/CAI sample in 1999 showed that the transition from PAPI to CAI had a positive effect on response rates,
but the increase did not occur evenly across age groups, with the smallest increase occurring for respondents 35
years old or older (Eyerman, Odom, Wu, & Butler, 2002).
5
Because age for all sample members is not collected until the screening interview, screening response
rates by age are not available.
9
•
Population data used in NSDUH sample weighting procedures were based on the 2000
decennial census for the first time in the 2002 NSDUH.
•
The pair selection algorithm was changed in 2002 to increase the pairs selected in the
50+ age group.
Given the pattern of change in response rates by age between 2001 and 2002, it is likely
that these methodological changes had a larger positive effect on the response propensity of
younger respondents than older respondents, thereby creating even larger differences in response
rates by age.
Figure 3.1 Weighted Interview Response Rate (IRR), by Age: 1999 to 2004
95%
90%
Weighted IRR
85%
80%
75%
70%
65%
60%
1999*
2000
2001
2002
2003
2004
Year
12-17
18-25
26-34
35-49
50+
* Includes 66,706 CAI interviews and excludes 13,809 PAPI interviews.
3.1
Components of Nonresponse
In this section, we examine response rates by age in more detail by disaggregating age
into 5-year groups; disaggregating nonresponse into the components of noncontacts, refusals,
and other incompletes; and narrowing the scope of the analysis to 2001 and 2002 (the year before
and the year after significant methodological changes and an increase in the response rate
differences between the 50+ and all other age groups). Patterns for 1999 and 2000 closely
resemble those for 2001, and patterns for 2003 and 2004 closely resemble those for 2002. For
ease of interpretation, we limit the analyses to 2001 and 2002.
10
Because the five traditional age categories used for sampling and analysis purposes on
NSDUH are very wide, analysis at this level may mask some heterogeneity that exists within age
groups. For example, it may be that sample members aged 50 to 59 respond at the same rate as
those under 50, but sample members aged 60 or older respond at a much lower rate, thereby
decreasing the overall 50+ response rate. It is important to understand whether this phenomenon
is found across the 50+ age group or only in small subgroups to determine where the MIP should
focus on improving response rates.
Figure 3.2 presents the weighted IRR for 2001 and 2002 by 5-year age groups.6 Table
A.2 in Appendix A provides the values corresponding to this figure for 1999 to 2004. For both
years, response rates decreased almost linearly with an increase in age. In 2001, response rates
stayed about the same or decreased slightly between the ages of 20 and 69. Response rates for
those 70 or older decreased sharply with age, with the lowest age-specific response rate of 59.1
percent occurring among the 80+. In 2002, response rates were about 6 to 10 percent higher for
all age groups between 12 and 49, with the exception of the 30 to 34 age group, whose response
rate increased only 3.6 percent from 2001. The response rate for all age groups 50 or older
increased less than 3 percent, with the exception of the 70 to 74 age group, whose response rate
increased by 6.5 percent. In general, response rates between 2001 and 2002 improved more
among younger sample members than among older sample members.
To gain a better understanding of response rate differences by age, we next examine the
components of nonresponse. To assess these possible differences, we present the weighted
noncontact, refusal, and other incomplete rates7 by 5-year age groups for the 2001 and 2002
surveys. It is important to understand how these components factor into nonresponse if we are to
effectively design and implement strategies to address them. For example, a strategy that is
designed to reduce noncontacts may have no effect on refusals. Response rates may differ by age
because certain sampled persons are more difficult to contact, are more likely to refuse
participation, or do not complete the survey for other reasons. In other words, are response rates
for the older age groups lower than for the younger age groups because older sample members
are more likely to be unavailable or unlocatable at the time of the interview, do they simply
refuse to participate in the interview more often, or are there other reasons?
6
Five-year age groups were chosen because they provide sufficient detail to detect differences by age and
sufficient cases to present reliable measures. The lower bound for each age group ends in 0 or 5 for ease of
interpretation, with the exception of the 12 to 14 group, which represents only 3 years but contains a great number of
cases due to the oversampling of 12 to 17 year olds. Because sample sizes for those aged 80 or older are small, all
persons 80 or older were included in the 80+ category. Exact age was not obtained for a small percentage of cases.
In 2001, the weighted IRR for the 419 cases missing an exact age was 36.6 percent. In 2002, the weighted IRR for
the 188 cases missing an exact age was 25.0 percent.
7
Noncontacts include cases in which no one was at the housing unit after repeated visits, the respondent
was unavailable after repeated visits, and access to the building was denied. Refusals include final respondent
refusals and final parental refusals (for respondents aged 12 to 17). Other incompletes include all other cases not
finalized as completes, noncontacts, or refusals: physical/mental incapability, language barriers, and other
noninterviews.
11
Figure 3.2 Weighted IRR, by 5-Year Age Groups: 2001 and 2002
95%
90%
85%
Weighted IRR
80%
75%
70%
65%
60%
55%
50%
1214
1519
2024
2529
3034
3539
4044
4549
5054
5559
6064
6569
7074
7579
80+
Age
2001
2002
Figure 3.3 presents the 2001 nonresponse component rates for each 5-year age group.
Table A.3 in Appendix A provides the values corresponding to this figure. Noncontacts increased
from 2.7 percent for the 12 to 14 age group to 8.36 percent for the 20 to 24 age group.
Noncontacts then decreased steadily with age, beginning with the 20 to 24 age group and ending
in a noncontact rate of 1.7 percent for the 80+. Refusals were the most common reason for
nonresponse for each age group except the 80+. Refusals accounted for 12.4 percent of the 12 to
14 age group and increased steadily with age. The highest refusal rate occurred among those
aged 50 to 54 (19.3 percent). Refusals then generally declined to a rate of 17.3 percent among
those 75 to 79, then dropped off sharply to 10.6 percent among the 80+. Nonresponse due to
other incompletes remained below 4 percent for all age groups until those 60 to 64. The rate of
other incompletes increased steadily from 3.3 percent among those aged 55 to 59 to 12.7 percent
among those aged 75 to 79. The other incomplete rate rose dramatically to 28.8 percent for the
80+.8
8
Nearly one fourth of the 80+ sample members did not complete an interview because they were physically
or mentally incapable.
12
Figure 3.3 Weighted Nonresponse Rates, by 5-Year Age Groups: 2001
45%
40%
Nonresponse Rate
35%
30%
25%
20%
15%
10%
5%
0%
1214
1519
2024
2529
3034
3539
4044
4549
5054
5559
6064
6569
7074
7579
80+
Age
Noncontacts
Refusals
Other Incompletes
All Nonresponse
Because we are examining components of nonresponse as they relate not only to agespecific IRRs, but also to the overall IRR, it is important to examine the impact each of these age
groups has on the overall IRR. When examining Figure 3.3, keep in mind that the number of
sample members in each group is not equivalent and that larger groups have a larger impact on
overall rates. In general, the younger age groups have greater sample sizes than the older age
groups. To account for this, Figure 3.4 takes the weighted sample size into account and presents
the impact each of these age- and component-specific nonresponse rates had on the 2001 overall
IRR. Table A.4 in Appendix A provides the values corresponding to this figure.
Nonresponse was highest for those aged 50+ in 2001, but Figure 3.4 shows that the
overall IRR was affected more by nonresponse in the middle age categories. Each group between
15 and 59 decreased the overall IRR by an increment of more than 1 percent. The majority of
this decrease was due to refusals. Noncontacts among the younger age groups had a larger
overall impact on the IRR than noncontacts among the older groups. Other incompletes had a
larger overall impact for the older group—especially for those aged 80+, for which they were
responsible for a 0.7 percent decrease in the overall IRR.
13
Figure 3.4 Impact of Age-Specific Nonresponse on Overall IRR: 2001
0.0%
-0.2%
Effect on Overall IRR
-0.4%
-0.6%
-0.8%
-1.0%
-1.2%
-1.4%
-1.6%
-1.8%
-2.0%
1214
1519
2024
2529
3034
3539
4044
4549
5054
5559
6064
6569
7074
7579
80+
Age
Noncontacts
Refusals
Other Incompletes
All Nonresponse
In 2002, the pattern of nonresponse by age changed. As shown in Figure 3.5,
nonresponse was below 25 percent for all age groups under 50 and above 25 percent for all age
groups 50+. Table A.5 in Appendix A provides the values corresponding to this figure. The main
reason for this difference was a higher rate of refusals among those aged 50+. In 2001, the 50+
age group represented about one fourth of all refusals, and in 2002 they represented about one
third. Refusals for all ages below 50 in 2002 were below 15 percent, and refusals for all ages 50+
were above 15 percent. The relatively higher rate of other incompletes among the oldest sample
members also contributed to higher nonresponse for those age groups.
Figure 3.6 shows the impact each of these age-specific nonresponse components had on
the overall IRR in 2002 (also see Table A.6 in Appendix A). Whereas in 2001, refusals among
those aged 12 to 54 accounted for a 8.9 percent drop in the overall response rate, the impact of
these refusals in 2002 was only 6.8 percent. The effect of refusals among those aged 55+ actually
increased between 2001 and 2002 from 3.0 to 3.5 percent.
14
Figure 3.5 Weighted Nonresponse Rates, by 5-Year Age Groups: 2002
45%
40%
Nonresponse Rate
35%
30%
25%
20%
15%
10%
5%
0%
1214
1519
2024
2529
3034
3539
4044
4549
5054
5559
6064
6569
7074
7579
80+
Age
Noncontacts
Refusals
Other Incompletes
All Nonresponse
Figure 3.6 Impact of Age-Specific Nonresponse on Overall IRR: 2002
0.0%
-0.2%
Impact on Overall IRR
-0.4%
-0.6%
-0.8%
-1.0%
-1.2%
-1.4%
-1.6%
-1.8%
-2.0%
1214
1519
2024
2529
3034
3539
4044
4549
5054
5559
6064
6569
7074
7579
Age
Noncontacts
Refusals
15
Other Incompletes
All Nonresponse
80+
We summarize these age- and component-specific rates in Figure 3.7, which presents the
increase in IRR and improvement in nonresponse reduction for each 5-year age group. Table A.7
in Appendix A provides the values corresponding to this figure. The line connecting the bars
indicates the change in IRR between 2001 and 2002, and the bars indicate the improvement in
each nonresponse component. Bars falling below 0 percent in the x-axis indicate an increase in
nonresponse and a negative impact on the response rate. We find that the reduction in refusals
and, to a lesser extent, noncontacts among sample members younger than 55 in 2001 led to
increases in age-specific and overall response rates. IRRs among those aged 55 to 59 decreased
by almost 3 percent due to increases in refusals and noncontacts. Generally, response rates
among those aged 60 to 69 and 80+ increased, but not as much as for the younger sample
members. Although there were improvements in the rate of other incompletes among these
groups, the percentage of refusals increased, preventing great improvements in response rates for
these groups. Response rates among those aged 70 to 79 increased due to a reduction of other
incompletes and, to a lesser extent, refusals.
Figure 3.7 Contribution of Nonresponse Components to Percentage Change in IRR: 2001 to 2002
It appears that lower response rates among older respondents were due to relatively
higher refusal rates and, in the oldest age categories, other incompletes, as compared with
younger sample members. With the introduction of methodological changes in 2002,
improvements in refusal rates among younger respondents led to improvements in age-specific
and overall response rates. Refusals among older respondents actually increased for sample
members aged 55 to 69 and 80+.
In order to gain an understanding of why older respondents may be more likely to refuse
participation and why the methodological improvements made in 2002 were unsuccessful in
16
reducing their likelihood to refuse, we next examine other correlates of refusal among older
respondents. If these other correlates help explain the differences in response propensity between
younger and older respondents, we can address them by designing targeted methodological
enhancements.
17
18
4. Potential Explanations for Refusals
among the 50+
Other than age, we know that several important factors relate to whether a sample
member will respond or refuse participation (Groves & Couper, 1998). In this section, we
consider several potential explanations for reduced response rates among older sample members
that are measurable using NSDUH data. Specifically, we investigate the roles of the following:
4.1
•
reasons for refusal given by sample members,
•
age-specific reactions to incentives,
•
respondent and household characteristics,
•
environmental characteristics, and
•
FI characteristics.
Respondents' Reasons for Refusal
The most obvious source from which to begin examining potential reasons for refusal in
NSDUH are the reasons for refusal given by nonrespondents. These reasons are selected from a
set of common refusal reasons by FIs on the screening device. The distribution of refusal reasons
by age group for 2001 and 2002 are presented in Figure 4.1.9 Tables A.8 and A.9 in Appendix A
provide the values corresponding to this figure. For both years, the most common reason for
refusal among sample members of all age groups was "Nothing in it for me." The second and
third most common reasons were "No time" and "Government/surveys too invasive,"
respectively. "House too messy/too ill" was given as a reason for refusal among less than 1
percent of cases for each age group in each year except for those aged 70+, for which it was
mentioned in 1 to 2 percent of cases.
Comparing 2001 with 2002, the rates for "Nothing in it for me" decreased for sample
members under 55 and those aged 70 to 79, but increased for sample members aged 55 to 69 and
80+. This suggests that the introduction of the $30 incentive in 2002 did not result in those aged
55 to 69 and 80+ seeing "something in it for them." The trend can be seen in Figure 4.2, which
presents the reduction (positive number = reduction) in the weighted IRR by age and reason for
refusal. The line connecting the bars represents the overall reduction in refusal rate by age group
between the 2 years. The percentage of sample members refusing with the reason "No time"
improved among the younger age groups, but showed an opposite trend for all sample members
55+. All reasons for refusal became more prevalent between 2001 and 2002 among sample
members aged 80+.
9
Because the focus here is on respondents aged 50+, that age group has been split into 5-year categories.
Age categories for those under 50 follow the standard NSDUH groupings (12 to 17, 18 to 25, 26 to 34, and 35 to
49).
19
Figure 4.1 Reasons for Refusal by Age: 2001 and 2002
20%
18%
16%
Refusal Rate
14%
12%
10%
8%
6%
4%
2%
2001
Age
2002
M is s i n g
O th e r
H o u s e t o o m e s s y / T o o i ll
C o n fid e n t i a l it y o r s u r ve y l e g i t im a c y c o n c e r n s
G a t e k e e p e r/ H o u s e h o l d m e m b e r w o n 't a ll o w p a r t i c i p a t i o n
G o ve rn m e n t / S u rve y s t o o in va s ive
N o tim e
N o t h in g in it fo r m e
Figure 4.2 Contribution of Refusal Reasons to Reduction in Refusal Rate: 2001 to 2002
20
80+
75-79
70-74
65-69
60-64
55-59
50-54
35-49
26-34
18-25
12-17
80+
75-79
70-74
65-69
60-64
55-59
50-54
35-49
26-34
18-25
12-17
0%
The fact that the rate of refusal for almost all reasons for all age groups under 50
decreased between 2001 and 2002 is likely due in part to the introduction of the $30 incentive.
Regardless of the reasons given for refusal, the incentive may have decreased the propensity to
refuse for sample members under 50. The same cannot be said for the 50+ group. These facts
suggest that the incentive may not be a convincing factor for the 50+ group overall. To further
test this hypothesis, we next review age-specific results from the 2001 incentive experiment.
4.2
Age-Specific Reactions to Incentives
In 2001, an experiment was conducted to evaluate the effectiveness of respondent
incentives in improving response rates in NSDUH (RTI International, 2002). A randomized,
split-sample, experimental design was included with the main study data collection to compare
the impact of $20 and $40 incentive treatments with a $0 control on measures of respondent
cooperation, data quality, survey costs, and population substance use estimates. The experiment
found that, overall, refusal rates were significantly lower for the incentive groups than for the
control group (19.6 percent for $0, 12.8 percent for $20, and 9.4 percent for $40). Refusal rates
were not examined by age, but response rates were. Overall, the $40 incentive resulted in a
significantly higher response rate than the $20 incentive, and the $20 incentive resulted in a
significantly higher response rate than the $0 treatment. For respondents aged 18 to 25 and 26+,
the increase in response rates due to the $40 incentive versus $20 was not significant, but the
difference between $20 and $0 was. The results of the incentive experiment led NSDUH to begin
offering all respondents a $30 cash incentive in 2002.
Table 4.1 reexamines the incentive experiment data by 5-year age groups and shows that
for all age groups between 15 and 29, response rates increased significantly from the $0 to $20
treatments and also from $20 to $40. For those aged 50+, this pattern was only replicated among
those aged 55 to 59. For the other sample members aged 50+, there was an increase in response
from $0 to $20, although the increase was not significant for each age group. The increase from
$20 to $40 did not correspond with a significant increase in response and actually correlated with
a significant decrease in response for those aged 70 to 74 and 80+. These results suggest that an
increase in the current $30 incentive for older respondents may not result in an increase in
response rates.
4.3
Additional Characteristics Influencing Refusal among the 50+
To assess the impact of important respondent, household, environmental, and interviewer
characteristics on the likelihood of refusal among those aged 50+, we analyzed the available
NSDUH data on the possible correlates of gender, race, ethnicity, number of respondents
selected per household, household composition, population density, socioeconomic status (SES),
region, and interviewer experience. This analysis was conducted on 2002 data because they
represent the first complete year of results after the introduction of incentives and other
methodological changes. Noncontacts and other incompletes are excluded from this analysis
because their inclusion in the refusal rate denominator could result in misleading conclusions.
Therefore, refusal rates for this analysis include only final refusals and final completes. Table 4.2
presents the definitions used for each of these measures in this analysis.
21
Table 4.1 Incentive Experiment Weighted IRR, by Age
Age
12-14
15-19
20-24
25-29
30-34
35-39
40-44
45-49
50-54
55-59
60-64
65-69
70-74
75-79
80+
Total
a
b
Incentive Amount
$20
92.9a
89.0a
85.8a
80.0a
70.7
81.3a
74.9
75.1
77.1a
77.7a
67.5
81.6
79.4a
85.7a
71.7
78.8a
$0
81.1
76.6
71.2
72.3
66.1
70.1
71.2
73.8
61.3
69.5
68.0
74.7
62.1
40.3
60.9
69.2
$40
94.9a
93.2a,b
91.8a,b
92.6a,b
87.4a,b
84.7a
85.1a,b
76.0
76.9a
84.5a,b
69.5
86.0a
65.5b
82.5a
51.8b
83.3a,b
Significantly different from $0 at the 0.05 level.
Significantly different from $20 at the 0.05 level.
Table 4.2 Respondent, Household, Environment, and FI Characteristics
Characteristic
Number of selected respondents
Number of persons in household
Presence of a minor (<18) in the
household
Single parent household
Population density
SES1
Region
FI and respondent gender
FI and respondent race/ethnicity
FI age
FI experience
Source of Data
Screener
Screener
Screener
Screener
Possible Values
1, 2
1, 2, 3, 4, 5+
Yes, no
Yes (only one 18+ and at least one 0-17 in household)
No (all other cases)
MSA ≥ 1 million persons, MSA < 1 million persons
High SES, low SES
Northeast / Midwest / South / West
All combinations of male / female
Census
Census
Census
Screener / FI
characteristics
Screener / FI
characteristics
FI characteristics
FI characteristics
All combinations of Hispanic / black / white or other
Under 50 / 50+ / missing2
Inexperienced (no prior NSDUH experience) /
experienced (40-99 interviews in prior years) /
highly experienced (100 + interviews in prior years)
MSA = metropolitan statistical area.
1
In order to define SES, block-group-level median rents and property values were given a rank (1...5) based on State
and MSA quintiles. The rent and value ranks were then averaged, weighted by the percentage of renter- and owneroccupied dwelling units, respectively. If the resulting score fell in the lower 25th percentile by State and MSA, the
area was considered "low SES"; otherwise, it was considered "high SES."
2
Because FI age was not available for a significant percentage of cases, "missing" was included as a distinct
category. No other variable had a significant percentage of missing values.
22
Table 4.3 presents the IRRs by age and the aforementioned additional characteristics for
all sample members. Unless noted, all of the data in this table refer to the 50+ portion of the
sample. The percentage of cases with these characteristics among refusals and completes is also
presented. Among all sample members, refusals were most common for sample members with
these characteristics:
•
in households in which one sample member was selected for the interview,
•
in two-person households,
•
in households with no members under age 18,
•
in non-single-parent households,
•
in MSAs with 1 million or more residents,
•
in high-SES segments,
•
in the Northeast region,
•
male being interviewed by a female FI,
•
white/other being interviewed by an Hispanic FI,
•
interviewed by FI under age 50,10 and
•
interviewed by inexperienced FI.
The correlates of refusal were similar for sample members aged 50+ compared with all
sample members. There were few differences in terms of characteristics of sample members
most likely to refuse. Sample members aged 50+ in households in which two respondents were
selected refused more often than those in households in which one was selected. The
FI/respondent race combination that had the highest rate of refusals was that in which the FI was
black and the respondent was white/other. Refusal rates were significantly higher for the 50+ age
group for most types of sample members; exceptions were households of more than five persons
and some FI/respondent race combinations, as noted in Table 4.3.
To simultaneously test the effects of these measures, we ran logistic regression models in
which the dependent measure is refusal (0 = complete, 1 = refusal) and the independent variables
are the characteristics listed in Tables 4.2 and 4.3. We ran one model that included sample
members of all ages and a second that included only those 50+. These models can measure the
significance of age overall and for the 50+ while controlling for the other characteristics. By
comparing the results of the two models, we can identify relationships that may be unique among
sample members aged 50+.
10
The highest refusal rate was actually found most often where the FI's age was unavailable, but this rate
was very close to that for FIs under 50.
23
Table 4.3 Weighted IRRs for the 50+ Age Group, by Characteristics: 2002
Characteristic
1 R selected
2 Rs selected
1-person household
2-person household
3-person household
4-person household
5+ person household
Any household members
< 18
No household member
< 18
Single-parent household
Not single-parent
household
MSA ≥ 1 million
MSA < 1 million
High SES segment
Low SES segment
Northeast
Midwest
South
West
FI male, R male
FI male, R female
FI female, R male
FI female, R female
FI Hispanic, R Hispanic
FI Hispanic, R black
FI Hispanic, R
white/other
FI black, R Hispanic
FI black, R black
FI black, R white/other
FI white/other, R
Hispanic
FI white/other, R black
FI white/other, R
white/other
FI aged < 50
FI aged 50+
FI age missing
FI inexperienced
FI experienced
FI highly experienced
a
b
Unweighted
N
3,870
2,815
1,518
3,519
936
415
297
838
Refusal to
Complete
Rate (1249)
12.4
11.3
11.4
13.7
12.3
11.5
9.8
Refusal to
Complete
Rate (50+)
19.6b
21.2b
19.8b
22.1b
19.3b
16.7a
13.2
Percent of
Refusals
44.8
55.3
12.7
38.1
19.2
16.7
13.4
Percent of
Completes
41.4
58.6
10.8
28.7
19.8
20.6
20.2
Refusal to
Complete
Rate (All Rs)
15.5
13.7
16.5
18.4
14.1
12.1
10.1
35.6
49.5
10.8
10.4
15.0b
64.4
2.1
50.5
5.3
17.7
6.1
13.9
6.0
21.1b
7.9
98.0
48.4
51.7
76.9
23.1
22.4
22.3
31.8
23.5
11.7
12.8
40.4
35.0
2.9
0.8
94.7
43.7
56.3
72.2
27.8
18.2
23.0
36.6
22.2
11.2
12.0
36.5
40.2
4.4
0.6
14.9
15.8
13.3
15.3
12.3
17.2
14.1
12.8
15.2
15.0
15.3
15.8
12.8
10.1
17.4
12.1
12.7
10.8
12.6
9.4
14.1
11.6
10.0
12.3
13.0
12.1
13.1
9.7
9.7
13.1
20.4b
22.8b
18.5b
20.7b
19.1b
23.3b
19.2b
18.5b
22.0b
19.8b
21.5b
21.5b
19.0b
11.6
30.8
4.5
0.6
2.9
5.8
3.2
1.4
3.6
5.1
19.6
6.2
11.8
16.2
16.3
5.9
8.5
11.1
25.3a
8.8
20.5b
25.8b
4.7
4.3
6.9
7.5
10.3
8.8
9.0
8.0
16.6b
11.1
73.6
39.8
54.5
5.7
24.8
26.6
48.7
67.4
37.7
57.1
5.2
21.7
27.0
51.3
15.6
15.2
13.9
15.6
16.1
14.3
13.8
12.7
11.8
11.5
12.4
13.1
11.4
11.2
20.8b
22.4b
18.7b
22.9b
22.7b
20.5b
19.2b
5,847
69
6,616
2,222
4,463
4,952
1,733
1,329
1,873
2,095
1,388
630
775
2,387
2,892
147
19
199
35
182
337
194
321
5,223
2,504
3,381
350
1,381
1,666
3,637
Significantly different from the 12-49 rate at the 0.05 level.
Significantly different from the 12-49 rate at the 0.01 level.
24
Table 4.4 presents the results of these models. In Model 1, which contains all ages, we
find that compared with the 50 to 54 age group, refusal was significantly less likely among all
age groups under 50 except those aged 40 to 44. No 55+ age groups were significantly different
from those 50 to 54 on refusal. Sample members in households of two and three individuals were
significantly more likely to refuse than were sample members in households of five or more
persons. Sample members in households containing a minor (under age 18) were significantly
less likely to refuse than those in households without a minor. Those in single-parent households
were also significantly less likely to refuse than those in non-single-parent households.
As for regional characteristics, sample members in densely populated and high-SES areas
were significantly more likely to refuse than those not living in such areas. Sample members in
the Northeast were significantly more likely and those living in the South were significantly less
likely to refuse compared with those in the West.
The combination of FI and respondent gender was a significant correlate of refusal, with
all combinations significantly more likely to refuse compared with the female FI/female
respondent scenario. Compared to the scenario in which the FI and respondent were both
white/other, refusal was significantly less likely when the respondent was Hispanic, regardless of
FI race/ethnicity. When the FI was white/other and the respondent was black, refusal was also
significantly less likely compared with the scenario in which both were white/other. Cases
finalized by inexperienced FI were significantly more likely to result in a refusal than those
worked by highly experienced FIs.
Although many of these relationships were also evidenced in Model 2 (50+ only), the
results were not exactly the same. When one respondent in the household was selected for the
interview, the 50+ sample members were significantly less likely to refuse than when two
respondents were selected. This may be due to older respondents' not having time or not being
willing to devote their collective available time to the survey. Households in which one member
of the pair was aged 50+ may not view a household-level incentive of $60 ($30 for each
respondent) as adequate compensation for their participation. This means that it is possible that
the increase in the number of selected pairs containing an older person in 2002 may have had a
detrimental effect on response rates among older sample members.
Compared with those under 50, the relationship between household size and refusal
propensity was different among those aged 50+. Respondents in households with one or two
members were significantly more likely to refuse than those in households with five or more
members. However, the presence of a minor and single-parent status were not significant
predictors among those aged 50+. This suggests that older sample members living in small
households (one or two members) are much more likely to refuse, regardless of the age of the
other household members. Although earlier analyses conducted under the MIP found that sample
members aged 50+ were significantly less likely to refuse when a minor was in the household or
when the household was headed by a single parent, the logistic models have the advantage of
simultaneously controlling for other correlates.
Another difference between Models 1 and 2 is the absence of a significant relationship
between SES and refusal propensity among those aged 50+. It may be that younger low-SES
sample members are more attracted to the incentive compared with younger high-SES sample
25
Table 4.4 Logistic Regression Model Odds Ratios and Significance: 2002
Characteristic
Intercept
R 12-14
R 15-19
R 20-24
R 25-29
R 30-34
R 35-39
R 40-44
R 45-49
R 50-55
R 56-59
R 60-64
R 65-69
R 70-74
R 75-79
R 80+
1 R selected
2 Rs selected
1-person household
2-person household
3-person household
4-person household
5+ person household
Any household members <18
No household member < 18
Single-parent household
Not single-parent household
MSA ≥ 1 million
MSA < 1 million
High SES segment
Low SES segment
Northeast
Midwest
South
West
FI male, R male
FI male, R female
FI female, R male
FI female, R female
FI Hispanic, R Hispanic
FI Hispanic, R black
FI Hispanic, R white/other
FI black, R Hispanic
FI black, R black
FI black, R white/other
FI white/other, R Hispanic
FI white/other, R black
FI white/other, R white/other
FI aged < 50
FI aged 50+
FI age missing
FI inexperienced
FI experienced
FI highly experienced
a
b
Logistic Model 1 (All Rs)
Odds Ratio
P Value
0.16
<0.0001
0.46
<0.0001
0.36
<0.0001
0.43
<0.0001
0.60
<0.0001
0.79
0.0180
0.71
0.0014
0.85
0.1065
0.78
0.0092
--1.06
0.6344
0.97
0.8200
1.06
0.6809
0.87
0.3582
0.95
0.7488
1.33
0.0661
0.94
0.2135
--1.02
0.8810
1.25
0.0183
1.19
0.0139
1.12
0.1251
--0.84
0.0153
--0.57
<0.0001
--1.21
0.0001
--1.12
0.0492
--1.16
0.0453
0.94
0.3285
0.87
0.0362
--1.19
0.0092
1.21
0.0063
1.29
<0.0001
--0.66
0.0046
1.20
0.5018
1.12
0.3668
0.40
<0.0001
0.76
0.0901
0.91
0.3748
0.74
0.0036
0.63
<0.0001
--1.09
0.0881
--1.10
0.3533
1.25
0.0004
1.09
0.0967
---
Estimate significant at the 0.05 level.
Estimate significant at the 0.01 level.
26
b
b
b
b
b
a
b
b
a
a
a
b
b
a
a
a
b
b
b
b
b
b
b
b
Logistic Model 2 (50+)
Odds Ratio
P Value
0.14
<0.0001
------------------1.05
0.6917
0.94
0.6278
1.02
0.9082
0.85
0.2687
0.91
0.5625
1.25
0.1512
0.76
0.0052
--1.76
0.0483
1.88
0.0210
1.51
0.0956
1.28
0.3832
--0.89
0.5452
--0.38
0.1102
--1.26
0.0080
--0.97
0.7329
--1.16
0.2319
0.89
0.3470
0.91
0.4343
--1.06
0.6350
1.16
0.2091
1.18
0.0252
--0.45
0.0235
1.57
0.4009
1.09
0.6943
0.32
0.0703
0.87
0.6499
1.07
0.7027
0.79
0.3216
0.48
0.0018
--1.17
0.0796
--1.25
0.2226
1.25
0.0368
1.14
0.1519
---
b
b
a
a
b
a
a
b
a
members, but older persons do not differ in their attitudes toward the incentive. This idea is
investigated further in Section 5.
In general, FI characteristics were not as often significant predictors of refusal among
those aged 50+ as they were among the full sample. The scenario in which the FI was female and
the respondent was male was significantly more likely to yield a refusal than when both were
female. Situations in which the FI and respondent were both Hispanic or the FI was white/other
and the respondent was black were significantly less likely to result in a refusal than when both
were white/other.
4.4
Need for Qualitative Information
The relationships revealed by the characteristic-specific refusal rates and logistic
regression models present opportunities to better understand why some sample members
participate while others refuse and how these reasons differ by age under the current NSDUH
methodology. For instance, one can examine the relationship found for respondents aged 50+ in
households in which two respondents were selected and postulate that these respondents do not
feel that the positives associated with participation (e.g., service to the country, accurate
prevalence estimates, $60 incentive for the household) outweigh perceived negatives (e.g., 2hour household time commitment, invasion of privacy, risk of becoming the victim of a scam).
Similarly, one could interpret the respondent-provided and interviewer-recorded reasons for
refusal as the underlying causes for nonresponse. But these refusal reasons may, in part,
represent convenient explanations for refusal that do not fully explain why older respondents are
less likely to participate in the NSDUH. The data are limited in that they cannot directly answer
these questions. However, such questions can be explored qualitatively, as described in
Section 5.
27
28
5. Focus Groups
5.1
FI Focus Groups
Although statistical analyses show the correlation of age, household composition, and
response, they do not offer explanations or evidence of causal links. The analyses indicate a need
for qualitative information on the underlying causes for lower response propensity among sample
members aged 50+. Because NSDUH FIs have the most direct contact and experience with
respondents, their input was sought to address this issue. To draw from the experience of the FIs,
three focus groups were conducted to explore the issue of nonresponse among those aged 50+
and ways to address it. This section summarizes the ideas and themes resulting from these focus
groups. A detailed account of the focus group design and results has been provided by Murphy
and Schwerin (2003).
Fears and Misperceptions. The most common reasons that respondents aged 50+ gave
for refusing to participate in the NSDUH were "There's nothing in it for me," "I have no time,"
and "Government/surveys are too invasive." In most cases, these categories capture the true
reasons for refusal. However, there may be additional information that is not captured in these
general categories or additional reasons for nonparticipation that respondents aged 50+ do not
overtly state. The FIs in these focus group interviews reported that many respondents aged 50+
refused due to certain fears and misperceptions concerning trust. A fear of scams among this
group may lead to an aversion to inviting unknown persons into their households. Fear of and
disdain for the government's motives also appear to be motivating factors for refusal among a
subset of respondents. Sensitivity to the survey topic and fear of divulging private information to
unknown individuals appear to be factors. Also, apprehension toward the handheld screening
device (similar in size and appearance to a smart phone) or the idea of using the laptop audio
computer-assisted self-interview (ACASI) may affect participation among those aged 50+. This
is consistent with studies that have found that older adults have significantly higher computer
anxiety than younger adults (Laguna & Babcock, 1997). Another commonly reported
misperception among older respondents is that they have nothing to offer the study. FIs report
that many respondents said, "I do not use drugs, so you don't need to interview me," or "My
experiences are irrelevant to this study."
Children May Have a Positive Impact. As a result of preliminary analyses of data from
selected pairs, it was found that sample members aged 50+ were less likely to refuse when the
second selected sample member was aged 12 to 17. Looking more closely at the data, it was
found that sample members aged 50+ were less likely to refuse when there was someone in the
household under age 18, regardless of whether that person was sampled. Similar results were
reported by Groves and Couper (1998), who found that respondents in households with no
children were less likely to cooperate in a survey. They also found that sample members in
households with more than one adult were more likely to cooperate than those with only one
adult. As the logistic models in Section 4.3 show, this relationship is not statistically significant
when controlling for other factors, but the focus groups gave us the opportunity to research this
possible explanation in a different way.
29
More than half of the FIs said that they noticed that respondents aged 50+ were more
likely to participate if a child aged 12 to17 from the household was also selected for the survey.
These FIs believed that the survey provides an opportunity for parents or grandparents and
children to communicate on the subject of drugs and provides a positive shared experience. FIs
also reported that these respondents aged 50+ may be motivated by their concern for children
and society in general. Some FIs mentioned that respondents aged 50+ sometimes prevent a
teenager from responding because they are not comfortable having their children exposed to the
topic of the survey. Nearly half of the FIs mentioned that they noticed increased participation
among those aged 50+ when a child under 18 was present in the household but not selected as a
respondent.
Taking It Slowly. All FIs agreed that in order to gain the cooperation of sample members
aged 50+, a great deal of patience and friendly professionalism was needed. Gaining the trust of
the respondent was an important first step that needed to be taken before attempting to complete
a screener or interview. Often, this meant that the interviewer needed to approach the respondent
in a casual, friendly manner and start a conversation of interest to the respondent (e.g.,
gardening, pets) before mentioning NSDUH. Public awareness of the study could be another
important factor in encouraging participation. FIs reported that some potential respondents called
the State and local public health departments as well as local police to verify the credentials of
the NSDUH project and the NSDUH FIs. Increasing the public's awareness of the study through
contact with local police and public health departments, as well as press releases to local
newspapers, could help raise awareness among residents in the community and enhance the
perceived legitimacy of the study.
More Money May Not Be the Answer. Although FIs reported that the $30 incentive
was helpful in gaining the cooperation of most respondents, certain subsets of the population
aged 50+ may not be as receptive. They reported that high-income respondents aged 50+ were
less likely to be persuaded by the prospect of receiving $30 after completing the survey. They
also reported that respondents aged 50+ on a fixed income and retirees were not as responsive to
the incentive. FIs reported that money was not the prime motivator for this group and that $30
actually raised suspicions of fraud or scams. They reported that the prime motivator for this
group was more likely to be community service. These findings are consistent with those from
the Survey of Program Dynamics that showed that respondents who accepted incentives were
less likely to have someone in the household aged 65+ (Kay, Boggess, Selavel, & McMahon,
2001).
5.2
Potential Respondent Focus Groups
Input was also sought from potential respondents in the 50+ age group to provide a basis
for viable methods to test and implement. Twelve focus groups were conducted to explore the
issue of nonresponse among those aged 50+ and ways of addressing it. Four groups were
conducted in Raleigh, North Carolina, four in Washington, DC, and four in Oakbrook, Illinois, a
suburb of Chicago. A full report summarizing the focus group design and results was prepared
for the Substance Abuse and Mental Health Services Administration (SAMHSA) by Murphy et
al. (2005). Highlights from that report follow.
30
Topic Understanding and Interest. After hearing only the brief FI introduction and
name of the survey, most participants did not have a clear understanding of the survey topics.
After reading the lead letter, most participants still did not understand the topic of the survey and
believed the study objective described in the letter was vague. Many focus group participants
described their initial impression of NSDUH as a study of prescription drug benefits, prescription
drugs, and health insurance. There were no discernible differences by geographic location,
household size, or participant age in perceptions or misperceptions of the survey topic. Once the
topic of the survey was fully explained to the focus group participants, almost all expressed the
opinion that drug use and health is an important topic to research. All participants believed that
interest in the topic would make them more likely to participate, but it was obvious that this
would not be the sole deciding factor. Nearly all groups recommended that the Q&A brochure,
or similar summary of the survey, be included in an advance mailing with the lead letter so that
potential survey respondents would have advance knowledge of what the study is all about and
what will be expected of them.
Courtesy and Flexibility. Across age groups, household size groups, and focus group
sites, a number of participants felt that the interviewing process showed a lack of courtesy to the
respondent. Many also felt that the survey recruitment process might benefit from a more
flexible and accommodating approach. Focus group participants said almost uniformly that it
was inappropriate for a stranger to make a personal visit to solicit time from a respondent
without scheduling the visit in advance, whether by calling on the telephone, or through some
other form of prior notification. A number of participants stated that they would be more likely
to participate if an appointment was scheduled before the first in-person visit. Participants felt
that it was impolite for an FI to appear at a respondent's doorstep unannounced and inconsiderate
to expect the respondent to make time for the interview. In addition, many participants
considered the repetitiveness of the screening interview questions to be a misuse of the
respondent's time, particularly because the possibility existed that the respondent may not even
be selected to participate in the survey. To address this, we could experiment with lead letter
verbiage to provide respondents with a more precise range of time that an FI will be in their area.
FI trainings could address the specific concerns of participants regarding the presumption of
availability and the repetitiveness of the screener questions.
Selection Process. Participants from each focus group expressed confusion over the
language used to describe the selection process or "qualification process," as some called it.
There were many questions about the meaning of "random" selection of households.
Respondents and participants wanted more information about this process to feel more
comfortable with the study objectives and protocol. Most participants believed that RTI or the
Federal Government also had access to their names and phone numbers. All participants in both
age groups wanted the screening script and questions to get directly to the point. The repetition
of the questions was a major issue, specifically for those in households of two or three where
roster questions are asked for all household members. For some group members, the possibility
of having another person in their household selected for the interview would make a difference.
For others, it would not have made a difference to their participation. One recommendation
would be to include a better explanation of the selection process in advance materials and the FI
introduction and possibly add some text that explains that eligibility is based on who lives at that
residence during a specific time period. Also, it should be made clear, as needed, that RTI and
31
SAMHSA/U.S. Public Health Service/Federal Government do not have participants' names or
phone numbers on file.
Descriptive Materials. Remarks about the project materials provided to the focus group
participants mirrored the overall focus group feedback of desiring additional detailed information
about the purpose and benefits of the research and information that would facilitate trust and
legitimacy to the research organization and FI. The lead letter was seen as a good tool, and the
information in the Q&A brochure addressed the issues being raised by the group. The newspaper
articles received a mixed review. The refusal letter appeared to address many participants'
concerns, but some said it would not have changed their minds. Several experiments may prove
beneficial from the focus group feedback on materials. Researching a better way to address the
recipient of the lead letter to be used in place of "Resident," and mailing the letters using a first
class postage stamp in a higher-quality envelope with a pre-printed Research Triangle Institute
return address with a logo may help get more people to open and read the letter.
Safety, Trust, and Confidentiality. Concerns were raised about the survey approach and
physical safety, security of the household, and fear of "scams" or other uses of information for
reasons other than what was specified by the FI or in the survey materials. Concerns about
physical safety were most prevalent in the groups of participants living alone, especially among
those aged 65+, and in the Oakbrook, Illinois, location. Suspicion was raised concerning some of
the screening questions. Some participants were confused as to why the FI asked about separate
residences on the property, convinced she was asking about separate entrances to the residence.
More than physical or household safety, concern was raised over safety from scams and mistrust
of the FI's intentions.
The importance of trusting the FI, the research organization, and the study purpose were
expressed throughout all of the focus groups. The importance of the FI establishing rapport and
creating a level of trust with the respondent was communicated by many participants. Trust in
the research organization and legitimacy of the survey were special concerns of the 65+ age
group.
Overall, confidentiality was not a major concern voiced by the participants. However,
participants expressed major concerns about the questions being intrusive, invasive, and too
personal. Experiments in streamlining the screening questions and adding purpose statements
with specific questions, such as the "Missed DU" question on dwelling units (DUs), might lead
to solutions that would alleviate some respondent concerns. Experiments with advance materials
focused on maximizing the extent to which household members read and retain the information
may shed light on improving methods for gaining trust, eliminating fear, and gaining the
participation of respondents in this age group. Similarly, additional FI training focused on
increasing respondent trust and familiarity with the purpose of the survey and questions,
especially among respondents in this age group, could prove beneficial. Finally, research into or
experimentation with alternative FI identification (ID cards rather than just badges or larger
badges) could lead to increased trust and participation among this age group.
FI Issues and Training. On the whole, focus group participants said they would be more
likely to respond to an FI who was prepared and polished, without being "slick." They expect FIs
to perform their task in a professional manner, which includes being polite and positive, while
32
displaying knowledge of the survey questions. Participants also expressed they would not
respond well to an FI who was timid or who presented a weak approach. An approach issue that
should be included in training is FIs need to be aware that they are "guests" to the respondent's
property and understand how the respondent feels about someone unknown coming to their door.
Training interviewers to be sensitive to these matters may improve their ability to build rapport
with the respondent.
Incentives. In general, the offer of a $30 incentive was not seen as persuasive by the
focus group participants. Very few mentioned they would be convinced to do the interview for
that amount. In some cases, participants felt that being offered money by the government to
complete the survey was inappropriate. Still others were suspicious of the $30 offer, thinking it
was a trick, part of a sales pitch, or that something other than completing the survey would be
expected in return. Most participants agreed that money, while potentially a persuasive tool,
would not be a sufficient enough incentive on its own to gain their participation. Although no
solid suggestions for noncash incentives were offered, these participants felt that the most
important factors in deciding whether to participate was trust in the motives of the FI and survey
and an understanding and appreciation for the topic and value of the data.
33
34
6. Tailored Field Strategies for the 50+ Age
Group
The evidence presented earlier in this report suggests that response patterns for the 50+
age group were systematically different from those of the younger age groups. In addition, the
results of the focus groups suggest that the difference in response behavior with the population
aged 50+ is probably a function, in part, of their perception of the interview process. This
suggests that response rates for the 50+ age group could be improved with field protocols
tailored to meet the specific concerns of this age group.
Several protocol changes and methodological enhancements have been considered to
improve the response rates for the 50+ age group. These possible changes are summarized
below.
•
Adjust existing training modules to better cover the concerns of the 50+ age group.
A module for this group would draw on the information included in the focus group
report, as well as additional information collected in any future focus groups on the topic.
Such a module may provide field staff with the additional skills necessary to overcome
barriers to cooperation that are unique to the 50+ age group.
•
Alter the lead letter and refusal conversion letter to emphasize concepts that are salient to
the older population, such as civic duty or the problems of drug-related crime. These
modifications could be made to all lead and refusal letters if we assume that the impact of
the change would be positive or neutral for all age groups. Alternatively, these changes
could be made to a special set of letters for distribution to only those segments with high
concentrations of older people, if we assume the letter would encourage cooperation
without introducing measurement error in those segments. Lab and field experimentation
on the lead letter's contents and appearance is being considered, and a work plan is being
prepared for SAMHSA review.
•
Develop alternative modes for interfacing with the ACASI interview, such as a larger
keyboard, a keypad tailored to the instrument, or a touch screen. This may reduce the
anxiety that older respondents have about using the laptop.
•
Conduct a public health communications campaign at the local level prior to data
collection. This may lend legitimacy to the data collection effort. A national campaign
might also be effective. An endorsement from the national American Association of
Retired Persons (AARP), or at least a letter of acknowledgement regarding the NSDUH
project, could help communicate the importance of the study to sample members aged
50+. FIs could be supplied with these letters to use when encountering these sample
members.
•
Assess higher incentive amounts for older respondents. The FI focus groups provided
little evidence that the incentive amount was a factor in the response propensity of older
persons. We recommend continuing the qualitative research into the response behaviors
of the 50+ age groups through additional focus groups or other methods, such as video
vignettes with potential respondents in the 50+ age group and other age groups. These
35
focus groups could be used to guide any changes in field protocols and to test the changes
before they are implemented.
•
Stress helping the younger generation (i.e., grandchildren), so those aged 50+ leave a
legacy.
•
Tailor a few brief video clips using individuals recognized by the general public and
well-respected by the population aged 50+. These video clips would be loaded on the
handheld screening device and could be shown by FIs when encountering reluctant
respondents.
In general, we should be extremely careful when implementing any change that is unique
to the 50+ age group in order to avoid a differential measurement error. Although we want to
reduce nonresponse error differentially, we do not want to introduce additional measurement
error to the 50+ age group through changes in the survey materials or in the interaction between
the interviewer and the respondent that cause respondents to self-report differently. For this
reason, we suggest that all changes be tested with an experiment prior to implementation on the
full study.
6.1
Assessment of the Impact on Prevalence Rates
The MIP specifies that once a problem is diagnosed, the possible influence of the
problem on prevalence rates should be assessed. The question to consider here is, does
differential nonresponse among the 50+ age group lead to positively or negatively biased
prevalence rates for this group, or are responders and refusers similar in regard to substance use?
Because prevalence data are not available for the sample members aged 50+ who refused the
interview, this assessment cannot be directly supported by existing NSDUH data. Without direct
measures of prevalence rates among older nonrespondents, we could not conclusively state
whether nonresponse among this group may be resulting in biased estimates. However, an
examination of indirect measures may be possible and will be explored as part of a forthcoming
NSDUH study on nonresponse bias. The relationship between the number of call attempts and
prevalence rates could provide indirect measures of nonresponse bias (Wang, Murphy, Baxter, &
Aldworth, 2005).
A number of studies have used reluctant respondents as predictors of nonrespondents,
with mixed success (Smith, 1984; Stinchcombe, Jones, & Sheatsley,1981). Preliminary analysis
of NSDUH data shows that drug prevalence rates are slightly lower among respondents who ever
refused to participate compared with those who never refused to participate. The opposite
relationship is seen for certain substances in the younger age groups. But we cannot assume that
respondents who have refused and then participated resemble those who refused and never
participated regarding drug use. Lin and Schaeffer (1995) examined two different methods of
using reluctant respondents and decided that both were flawed and more arbitrary than
anticipated. Cohen et al. (2000) found that "reluctant respondents as a whole appear to be a
distinctly separate group, sharing one set of characteristics with the cooperative respondent
group, another set with those who refused during the second round of the survey, and a yet a
third set of characteristics that are uniquely their own" (p. 131).
36
As demonstrated in the 2002 NSDUH's national findings report (OAS, 2003), the survey
changes introduced in 2002 did not significantly increase the response rate among those aged
50+, but this group's prevalence rates did increase significantly. The upper bound analysis
showed that the increase in prevalence rates could not be fully explained by the slightly higher
response rate. Given this finding, there is reason to assume that changes to survey protocol can
affect prevalence rates for this group, but there is no evidence that nonresponse bias is occurring.
A forthcoming NSDUH methods study on nonresponse bias should provide some evidence of the
presence or absence of nonresponse bias among the 50+ age group and other age groups. Any
changes to survey protocol resulting from this report will be considered in this regard prior to
experimentation, prior to implementation, and again after implementation by examining marginal
prevalence rates in an upper bound analysis, as was done in the 2002 national findings report.
37
38
References
Blake, M., Korovessis, C., & Pickering, K. (2002, January). National Adult Learning Survey
2001 technical report (Report P2034, prepared for the Department of Education and Skills,
United Kingdom). Retrieved from http://www.esds.ac.uk/doc/4455/mrdoc/pdf/a4455uab.pdf
Chiu, P., Riddick, H., & Hardy, A. (2001). A comparison of characteristics between late/difficult
and non-late/difficult interviews in the National Health Interview Survey. Presented at the annual
meeting of the American Statistical Association, Atlanta, GA.
Chromy, J. R., Bowman, K. R., Crump, C. J., Packer, L. E., & Penne, M. A. (1999). Population
coverage in the National Household Survey on Drug Abuse. In 1999 Joint Statistical Meetings,
American Statistical Association, Survey Research Methods Section, Baltimore, MD (pp. 576580). Alexandria, VA: American Statistical Association.
Cohen, S., Machlin, J., & Branscome, J. (2000). Patterns of survey attrition and reluctant
response in the 1996 Medical Expenditure Panel Survey. Health Services & Outcomes Research
Methodology, 1(2), 131-148.
Comstock, G. W., & Helsing, K. J. (1973). Characteristics of respondents and nonrespondents to
a questionnaire for estimating community mood. American Journal of Epidemiology, 97, 233239.
DeMaio, T. (1980). Refusals: Who, where and why. Public Opinion Quarterly, 44, 223-233.
Eyerman, J., Odom, D., Wu, S., & Butler, D. (2002). Nonresponse in the 1999 NHSDA. In J.
Gfroerer, J. Eyerman, & J. Chromy (Eds.), Redesigning an ongoing national household survey:
Methodological issues (HHS Publication No. SMA 03-3768, pp. 23-51). Rockville, MD:
Substance Abuse and Mental Health Services Administration, Office of Applied Studies.
FirstGov for Seniors. (2000). Census 2000 – Older Americans count.
http://www.seniors.gov/articles/0300/older_americans_census.html
Forbes, D. A., & Neufeld, A. (1997). Strategies to address the methodological challenges of
client-satisfaction research in home care. Canadian Journal of Nursing Research, 29(2), 69-77.
Gfroerer, J., Lessler, J., & Parsley, T. (1997). Studies of nonresponse and measurement error in
the National Household Survey on Drug Abuse. In L. Harrison & A. Hughes (Eds.), The validity
of self-reported drug use: Improving the accuracy of survey estimates (NIH Publication No. 974147, NIDA Research Monograph 167, pp. 273-295). Rockville, MD: National Institute on Drug
Abuse.
Groves, R. M., & Couper, M. (1998). Nonresponse in household interview surveys. New York,
NY: John Wiley & Sons, Inc..
Herzog, A. R., & Rodgers, W. (1988). Age and response rates to interview sample surveys.
Journal of Gerontology, 43(6), S200-S205.
39
Hoinville, G. (1983). Carrying out surveys among the elderly. Journal of the Market Research
Society, 25, 223-237.
Jobe, J., Keller, D., & Smith A. (1996). Cognitive techniques in interviewing older people. In
Schwartz, N., & Sudman, S. (Eds.), Answering questions: Methodology for determining
cognitive and communicative process in survey research. San Francisco, CA: Jossey-Bass.
Kautter, J., Khatutsky, G., Pope, G., & Chromy, J. (2003). Impact of nonresponse on MCBS
estimates final report (prepared by RTI for the Centers for Medicare & Medicaid Services).
Research Triangle Park, NC: RTI International.
Kay, W., Boggess, W.S., Selavel, K., & McMahon, M. (2001). The use of targeted incentives to
reluctant respondents on response rate and data quality. Presented at the annual meeting of the
American Statistical Association, Atlanta, GA.
Kirsch, I., Yamamoto, K., Norris, N., Rock, D., Jungeblut, A., O'Reilly, P., Campbell, A.,
Jenkins, L., Kolstad, A., Berlin, M., Mohadjer, L., Waksberg, J., Goksel, H., Burke, J., Rieger,
S., Green, J., Klein, M., Mosenthal, P., & Baldi, S. (2001). Technical report and data file user's
manual for the 1992 National Adult Literacy Survey. Retrieved from
http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2001457
Laguna, K., & Babcock, R. L. (1997). Computer anxiety in young and older adults: Implications
for human-computer interactions in older populations. Computers in Human Behavior, 13(3),
317-326.
Lin, I.-F., & Schaeffer, N. C. (1995). Using survey participants to estimate the impact of
nonparticipation. Public Opinion Quarterly, 59, 236-258.
McQuillan, G. M., Khare, M., Karon, J. M., Schable, C., & Vlahov, D. (1997). Update on the
seroepidemiology of human immunodeficiency virus in the United States household population:
NHANES III, 1988-1994. Journal of Acquired Immune Deficiency Syndromes, 14, 355-360.
Moorman, P. G., Newman, B., Millikan, R. C., Tse, C. K., & Sandler, D. P. (1999). Participation
rates in a case-control study: The impact of age, race, and race of interviewer. Annals of
Epidemiology, 9(3), 188-195.
Murphy, J., & Schwerin, M. (2003). Nonresponse among respondents aged 50 and older:
NSDUH field interviewer focus group report (prepared by RTI for the Substance Abuse and
Mental Health Services Administration). Research Triangle Park, NC: RTI International.
Murphy, J., Schwerin, M., Hewitt, D., & Safir, A. (2005). Nonresponse among respondents aged
50 and older potential respondents focus group report (prepared by RTI for the Substance Abuse
and Mental Health Services Administration). Research Triangle Park, NC: RTI International.
Myers, G. C., Juster, F. T., & Suzman, R. M. (1997). Asset and Health Dynamics Among the
Oldest Old (AHEAD): Introduction. Journals of Gerontology, 52B, v-viii.
40
National Center for Health Statistics. (2002). 1994-2000 Second Longitudinal Study of Aging:
Wave 3 survivor file, codebook version SF 2.1. Retrieved from
ftp://ftp.cdc.gov/pub/Health_statistics/NCHs/lsoa2/survivor_interview/w3sf/codebooks/W3_SF_
CB_SF21.pdf
Nehring, V., & Geach, B. (1973). Patients' evaluation of their care: Why they don't complain.
Nursing Outlook, 21(5), 317-321.
New England States Consortium. (2002). Getting feedback from frail elders and people with
disabilities: Factors to consider when selecting a method, an instrument, a vendor. Retrieved
August 31, 2005, from http://www.hhp.umd.edu/AGING/MMIP/TApapers/TApaper13.pdf
Office of Applied Studies. (2003). Results from the 2002 National Survey on Drug Use and
Health: National findings (HHS Publication No. SMA 03-3836, NHSDA Series H-22).
Rockville, MD: Substance Abuse and Mental Health Services Administration.
Paul, E., & Lawes, M. (1982). Characteristics of respondent and non-respondent households in
the Canadian Labor Force Survey. Survey Methodology, 8, 48-85.
Redpath, B., & Elliot, D. (1988). National Food Survey: A second study of differential response,
comparing census characteristics of NFS respondents and non-respondents; also a comparison of
NFS and FSs response bias. Statistical News, 80, 6-10.
Rodgers, W., & Herzog, R. (1987). Interviewing older adults: The accuracy of factual
information. Journal of Gerontology, 42, 387-394.
RTI International (2002). 2001 National Household Survey on Drug Abuse incentive experiment
(prepared for the Substance Abuse and Mental Health Services Administration). Research
Triangle Park, NC: Author.
Smith, T. W. (1984). Estimating nonresponse bias with temporary refusals. Sociological
Perspectives, 27, 473-489.
Smith, T. W. (1995). Trends in non-response rates. International Journal of Public Opinion
Research, 7, 157-171.
Stinchcombe, A. L., Jones, C., & Sheatsley, P. (1981). Nonresponse bias for attitude questions.
Public Opinion Quarterly, 45, 359-375.
Sudman, S., & Bradburn, N. (1982). Asking questions: A practical guide to questionnaire design.
San Francisco, CA: Jossey-Bass.
Trembaly, A., & Moore III, T. F. (1995). Nonresponse issues of the National Survey of College
Graduates. In Proceedings of the Section on Survey Research Methods, American Statistical
Association (pp. 459-464). Alexandria, VA: American Statistical Association.
41
Wang, K., Murphy, J., Baxter, R., & Aldworth, J. (2005). Are two feet in the door better than
one? Using process data to examine interviewer effort and nonresponse bias. To be presented at
the 2005 Conference of the Federal Committee on Statistical Methodology. Washington, DC.
Zimowski, M., Tourangeau, R., Ghadialy, R., & Pedlow, S. (1997) Nonresponse in household
travel surveys (FHWA/PD-98/029, prepared for Federal Highway Administration). Chicago, IL:
NORC.
42
Appendix A: Tables Relating to Figures in the Report
Table A.1. Weighted Interview Response Rate (IRR), by Age: 1999 to 2004
Year
Age
12-17
18-25
26-34
35-49
50+
Total
1999*
78.07%
71.21%
69.45%
67.75%
64.63%
68.60%
2000
82.58%
77.34%
74.92%
73.89%
69.53%
73.93%
2001
82.18%
75.51%
74.82%
72.38%
69.92%
73.31%
2002
89.99%
85.16%
79.41%
78.95%
71.54%
78.56%
2003
89.83%
83.73%
78.72%
77.40%
71.22%
77.67%
2004
88.86%
83.96%
79.05%
76.54%
71.66%
77.51%
2003
89.90%
88.75%
82.85%
79.49%
78.75%
78.91%
77.25%
76.52%
73.43%
71.83%
75.25%
75.61%
70.29%
66.53%
58.40%
77.67%
2004
89.09%
88.05%
83.66%
80.88%
77.51%
78.10%
76.27%
75.37%
75.16%
73.94%
72.74%
72.47%
72.57%
69.23%
58.56%
77.51%
* Includes 66,706 CAI interviews and excludes 13,809 PAPI interviews.
Table A.2. Weighted IRR by 5-Year Age Groups: 1999 to 2004
Year
Age
12-14
15-19
20-24
25-29
30-34
35-39
40-44
45-49
50-54
55-59
60-64
65-69
70-74
75-79
80+
Total
1999*
78.96%
75.85%
70.74%
70.58%
69.27%
69.56%
68.56%
67.34%
65.69%
67.80%
65.57%
67.87%
68.46%
64.55%
57.21%
68.60%
2000
83.82%
81.21%
77.33%
75.78%
75.25%
74.86%
74.30%
73.10%
71.76%
72.54%
72.18%
73.98%
73.46%
71.09%
59.86%
73.93%
2001
83.24%
80.20%
74.42%
75.15%
74.81%
74.29%
72.20%
71.63%
71.09%
75.02%
71.94%
72.80%
68.56%
68.24%
59.14%
73.31%
2002
89.97%
89.37%
84.57%
81.24%
78.38%
80.03%
77.93%
79.48%
74.06%
73.16%
74.55%
73.87%
75.10%
72.94%
56.02%
78.56%
* Includes 66,706 CAI interviews and excludes 13,809 PAPI interviews.
A-1
Table A.3. Weighted Nonresponse Rates, by 5-Year Age Groups: 2001
Age
12-14
15-19
20-24
25-29
30-34
35-39
40-44
45-49
50-54
55-59
60-64
65-69
70-74
75-79
80+
Noncontacts
2.73%
4.42%
8.36%
7.95%
6.98%
5.80%
6.37%
5.60%
5.95%
4.00%
4.32%
2.80%
1.84%
1.71%
1.42%
Refusals
12.41%
13.42%
14.26%
14.15%
15.41%
17.16%
18.16%
19.23%
19.26%
17.73%
16.60%
16.29%
17.52%
17.33%
10.62%
Other Incompletes
1.62%
1.96%
2.96%
2.75%
2.81%
2.76%
3.28%
3.54%
3.70%
3.25%
7.14%
8.11%
12.08%
12.73%
28.82%
All Nonresponse
16.76%
19.80%
25.58%
24.85%
25.19%
25.71%
27.80%
28.37%
28.91%
24.98%
28.06%
27.20%
31.44%
31.76%
40.86%
Table A.4. Impact of Age-Specific Nonresponse on Overall IRR: 2001
Age
12-14
15-19
20-24
25-29
30-34
35-39
40-44
45-49
50-54
55-59
60-64
65-69
70-74
75-79
80+
Total
Noncontacts
-0.10%
-0.28%
-0.48%
-0.43%
-0.44%
-0.40%
-0.45%
-0.37%
-0.37%
-0.18%
-0.14%
-0.09%
-0.05%
-0.03%
-0.03%
-3.84%
Refusals
-0.46%
-0.85%
-0.83%
-0.78%
-0.97%
-1.20%
-1.30%
-1.29%
-1.22%
-0.81%
-0.54%
-0.54%
-0.48%
-0.38%
-0.25%
-11.90%
A-2
Other Incompletes
-0.06%
-0.12%
-0.17%
-0.15%
-0.17%
-0.19%
-0.23%
-0.23%
-0.23%
-0.14%
-0.23%
-0.27%
-0.33%
-0.28%
-0.68%
-3.48%
All Nonresponse
-0.62%
-1.25%
-1.48%
-1.36%
-1.58%
-1.79%
-1.98%
-1.89%
-1.82%
-1.13%
-0.91%
-0.90%
-0.86%
-0.69%
-0.96%
-19.22%
Table A.5. Weighted Nonresponse Rates, by 5-Year Age Groups: 2002
Age
12-14
15-19
20-24
25-29
30-34
35-39
40-44
45-49
50-54
55-59
60-64
65-69
70-74
75-79
80+
Noncontacts
1.57%
3.03%
5.31%
6.20%
6.40%
5.75%
5.39%
4.70%
5.21%
5.59%
3.50%
1.87%
1.57%
1.73%
1.58%
Refusals
7.27%
6.23%
7.99%
10.42%
12.66%
11.58%
13.94%
13.80%
17.31%
18.78%
18.09%
19.12%
15.85%
16.71%
16.92%
Other Incompletes
1.19%
1.37%
2.13%
2.13%
2.57%
2.64%
2.73%
2.02%
3.42%
2.47%
3.87%
5.13%
7.48%
8.62%
25.49%
All Nonresponse
10.03%
10.63%
15.43%
18.75%
21.62%
19.97%
22.07%
20.52%
25.94%
26.84%
25.45%
26.13%
24.90%
27.06%
43.98%
Table A.6. Impact of Age-Specific Nonresponse on Overall IRR: 2002
Age
12-14
15-19
20-24
25-29
30-34
35-39
40-44
45-49
50-54
55-59
60-64
65-69
70-74
75-79
80+
Total
Noncontacts
-0.07%
-0.21%
-0.34%
-0.38%
-0.43%
-0.42%
-0.41%
-0.33%
-0.33%
-0.28%
-0.13%
-0.06%
-0.04%
-0.04%
-0.04%
-3.51%
Refusals
-0.31%
-0.43%
-0.52%
-0.65%
-0.85%
-0.85%
-1.08%
-0.99%
-1.11%
-0.95%
-0.69%
-0.57%
-0.45%
-0.39%
-0.45%
-10.29%
A-3
Other Incompletes
-0.05%
-0.09%
-0.14%
-0.13%
-0.17%
-0.19%
-0.21%
-0.14%
-0.22%
-0.12%
-0.15%
-0.15%
-0.21%
-0.20%
-0.68%
-2.85%
All Nonresponse
-0.43%
-0.73%
-1.00%
-1.16%
-1.45%
-1.46%
-1.70%
-1.46%
-1.66%
-1.35%
-0.97%
-0.78%
-0.70%
-0.63%
-1.17%
-16.65%
Table A.7. Contribution of Nonresponse Components to Percentage Change in IRR: 2001 to 2002
Age
12-14
15-19
20-24
25-29
30-34
35-39
40-44
45-49
50-54
55-59
60-64
65-69
70-74
75-79
80+
Noncontacts
1.16%
1.39%
3.06%
1.75%
0.58%
0.05%
0.97%
0.91%
0.74%
-1.59%
0.82%
0.92%
0.27%
-0.02%
-0.16%
Refusals
5.14%
7.20%
6.27%
3.73%
2.75%
5.58%
4.21%
5.43%
1.95%
-1.05%
-1.49%
-2.83%
1.67%
0.62%
-6.30%
A-4
Other Incompletes
0.43%
0.59%
0.83%
0.62%
0.24%
0.12%
0.55%
1.51%
0.28%
0.78%
3.28%
2.98%
4.61%
4.10%
3.33%
All Nonresponse
6.73%
9.17%
10.16%
6.10%
3.57%
5.74%
5.73%
7.85%
2.97%
-1.86%
2.60%
1.07%
6.54%
4.70%
-3.13%
Table A.8. Reasons for Refusal, by Age: 2001
A-5
Age
12-17
18-25
26-34
35-49
50-54
55-59
60-64
65-69
70-74
75-79
80+
Nothing in
it for me
1.80%
6.34%
6.71%
7.82%
9.69%
7.80%
8.98%
7.08%
8.00%
7.27%
5.63%
No time
1.02%
4.28%
5.29%
6.24%
5.60%
5.04%
2.63%
3.12%
3.46%
2.55%
1.58%
Government/
Surveys too
invasive
0.46%
1.11%
1.24%
2.26%
2.06%
3.03%
2.57%
3.21%
3.04%
2.42%
1.02%
Gatekeeper/
Household member
won't allow
participation
0.48%
0.89%
0.28%
0.35%
0.36%
0.30%
0.26%
0.23%
0.17%
0.42%
0.42%
Confidentiality
or survey
legitimacy
concerns
0.02%
0.23%
0.36%
0.34%
0.44%
0.62%
0.41%
0.50%
0.70%
1.45%
0.27%
House too
messy/ Too ill
0.02%
0.07%
0.10%
0.16%
0.09%
0.30%
0.60%
0.41%
1.45%
1.64%
0.85%
Other
0.21%
0.56%
0.66%
0.84%
0.71%
0.37%
0.78%
1.10%
0.46%
1.57%
0.86%
Missing
0.13%
0.50%
0.42%
0.42%
0.31%
0.26%
0.39%
0.63%
0.24%
0.00%
0.00%
Table A.9. Reasons for Refusal, by Age: 2002
A-6
Age
12-17
18-25
26-34
35-49
50-54
55-59
60-64
65-69
70-74
75-79
80+
Nothing in it
for me
0.93%
3.30%
4.89%
5.57%
6.72%
8.49%
10.06%
9.83%
6.79%
7.14%
6.07%
No time
0.33%
2.00%
4.74%
4.46%
4.94%
6.31%
4.78%
4.72%
3.54%
3.10%
2.57%
Government/
Surveys too
invasive
0.18%
0.63%
0.87%
1.50%
2.65%
1.93%
1.43%
2.16%
2.33%
2.51%
2.67%
Gatekeeper/
Household member
won't allow
participation
0.26%
0.94%
0.55%
0.53%
0.59%
0.44%
0.47%
0.40%
0.46%
0.66%
1.40%
Confidentiality
or survey
legitimacy
concerns
0.01%
0.15%
0.26%
0.45%
0.81%
0.53%
0.51%
0.45%
0.81%
1.20%
0.56%
House too
messy/ Too
ill
0.00%
0.03%
0.05%
0.08%
0.45%
0.40%
0.50%
0.57%
1.12%
0.83%
1.90%
Other
0.04%
0.33%
0.43%
0.46%
0.99%
0.58%
0.34%
0.48%
0.47%
1.16%
1.34%
Missing
0.05%
0.18%
0.16%
0.21%
0.15%
0.12%
0.00%
0.50%
0.33%
0.12%
0.42%
Table A.10. Contribution of Refusal Reasons to Reduction in Refusal Rate: 2001 to 2002
A-7
Age
12-17
18-25
26-34
35-49
50-54
55-59
60-64
65-69
70-74
75-79
80+
Nothing in
it for me
0.86%
3.04%
1.82%
2.26%
2.96%
-0.69%
-1.08%
-2.75%
1.21%
0.13%
-0.44%
No time
0.69%
2.28%
0.54%
1.78%
0.65%
-1.26%
-2.16%
-1.60%
-0.08%
-0.54%
-0.99%
Government/
Surveys too
invasive
0.28%
0.48%
0.37%
0.76%
-0.59%
1.10%
1.13%
1.05%
0.71%
-0.08%
-1.64%
Gatekeeper/
Household member
won't allow
participation
0.22%
-0.05%
-0.27%
-0.18%
-0.23%
-0.14%
-0.21%
-0.16%
-0.29%
-0.23%
-0.98%
Confidentiality
or survey
legitimacy
concerns
0.01%
0.08%
0.10%
-0.11%
-0.37%
0.09%
-0.10%
0.04%
-0.12%
0.25%
-0.30%
House too
messy/ Too
ill
0.02%
0.04%
0.05%
0.08%
-0.36%
-0.09%
0.10%
-0.16%
0.33%
0.81%
-1.05%
Other
0.17%
0.23%
0.23%
0.38%
-0.28%
-0.21%
0.44%
0.62%
-0.01%
0.41%
-0.48%
Missing
0.08%
0.32%
0.27%
0.21%
0.16%
0.15%
0.39%
0.13%
-0.09%
-0.12%
-0.42%
A-8
| File Type | application/pdf |
| File Title | Microsoft Word - Attachment X_NonresponseAmongSampleMembers50andOlder |
| Author | cjewett |
| File Modified | 2023-11-15 |
| File Created | 2023-11-15 |