Attachment V_NonresponseAmongSampleMembers50andOlder

Attachment V_NonresponseAmongSampleMembers50andOlder.docx

2022 National Survey on Drug Use and Health (NSDUH)

Attachment V_NonresponseAmongSampleMembers50andOlder

OMB: 0930-0110

Document [docx]
Download: docx | pdf

2003 NATIONAL SURVEY ON
DRUG USE AND HEALTH

Nonresponse among Sample Members 50 or Older in NSDUH:
Methodological Improvement Protocol (MIP) Final Report

Contract No. 283-98-9008
RTI Project No. 07190

Authors: Project Director:

Joe Murphy Tom Virag
Joe Eyerman



Prepared for:

Substance Abuse and Mental Health Services Administration
Rockville, Maryland 20857

Prepared by:

RTI International
Research Triangle Park, North Carolina 27709

August 31, 2005



2003 NATIONAL SURVEY ON DRUG USE AND HEALTH

Nonresponse among Sample Members
50 or Older in NSDUH:
Methodological Improvement Protocol (MIP) Final Report

Contract No. 283-98-9008
RTI Project No. 07190


Authors: Project Director:

Joe Murphy Tom Virag
Joe Eyerman


Prepared for:

Substance Abuse and Mental Health Services Administration
Rockville, Maryland 20857

Prepared by:

RTI International
Research Triangle Park, North Carolina 27709

August 31, 2005

Acknowledgments

This report would not be possible without the guidance and input of staff from the Center for Behavioral Health Statistics and Quality. In particular, Joel Kennet provided useful comments. At RTI International (a trade name of Research Triangle Institute), Valerie Garner and Richard Straw provided report production assistance.


Table of Contents

Chapter Page


Appendix





List of Tables

Table Page




List of Figures

Figure Page





  1. Executive Summary

This report examines the relationship between age and the propensity to respond in the National Survey on Drug Use and Health (NSDUH). Previous analyses have demonstrated that sample members aged 50 or older (50+) are consistently less likely to complete the interview than younger sample members (Office of Applied Studies [OAS], 2003). A more detailed examination of this age group is conducted in this report to identify the characteristics of the nonrespondents, diagnose the causes of the nonresponse, consider the impact of the nonresponse on prevalence rates, and suggest possible remedies. This analysis is being conducted according to the Methodological Improvement Protocol (MIP), which specifies identification of the problem, diagnosis of the causes, consideration of potential solutions to the problem, and assessment of the impact of potential changes on prevalence rates. The results are summarized below.

  • Lower response rates for the 50+ age group are due to higher refusal rates across the age group and increasingly higher rates of physical or mental incapability beginning at age 60. Noncontact rates were lower for the 50+ group than for the younger age groups (Chromy, Bowman, Crump, Packer, & Penne, 1999; Gfroerer, Lessler, & Parsley, 1997).

  • The methodological changes introduced in 2002 were generally effective in improving response rates (OAS, 2003). However, the changes had little effect on the response rates for the 50+ age group (see Figure 3.1 in Section 3).

  • The 50+ age group generally was not responsive to the provision of an incentive, although respondents aged 50 to 59 did respond to the incentive in the incentive experiment in a manner similar to the younger age groups (RTI International, 2002; see also Table 4.1 in Section 4.2).

  • More refined codings of the 50+ age group demonstrate that the least cooperative ages were 55 to 69 and the most cooperative were 70 to 79 (RTI International, 2002; see also Figure 3.5 in Section 3.1).

  • Respondents aged 55 to 69 were less responsive to the provision of the incentive and other enhancements (OAS, 2003). This suggests that a method tailored to the older groups may be required, such as an appeal to civic duty (see Figure 3.2 in Section 3.1).

  • The 50+ age group represented about one fourth of all refusals in 2001 and about one third in 2002. The percentage of the full sample that was 50+ remained the same in these 2 years.

  • Even when controlling for interviewer, environment, and other respondent characteristics, the older sample members were more likely to refuse than younger sample members in 2002 (see Table 4.4 in Section 4.3).

  • Focus groups with field interviewers (FIs) revealed that the older sample members were less trusting in the legitimacy of the survey and less comfortable with the survey topic (Murphy & Schwerin, 2003; see also Section 5).

  • Focus groups with FIs also revealed that older sample members were less likely to be persuaded by the incentive than younger groups. In fact, some may be more likely to refuse because they believe that the provision of an incentive resembles a confidence scam or other fraudulent practice (Murphy & Schwerin, 2003; see also Section 5).

  • The FIs also reported in the focus groups that sample members with children in the household were more likely to participate because the survey topic was more salient. Older sample members may place less salience on the survey topic because they are less likely to have children living in the household (Murphy & Schwerin, 2003; see also Section 5).

  • Focus groups with potential 50+ respondents confirmed much of the information gathered in the FI focus groups. These groups made clear the importance of topic understanding and interest, courtesy and flexibility in the survey approach, the selection process, descriptive materials, and trust in gaining cooperation from members of this age group (Murphy, Schwerin, Hewitt, & Safir, 2005; see also Section 5).

The report contains suggestions for a series of methodological enhancements that may improve response rates for the older age groups, including additional interviewer training, adjustments to the lead letter and refusal letter, alternative data entry modes, and improved public outreach. The possible impact on trend data of any considered change should be fully evaluated with an experiment before implementation.

    1. Introduction

Response rates in NSDUH are lower for sample members aged 50+ than for any other age group. Low response rates for this age group represent a potential threat to the accuracy of both age-specific and overall prevalence rates. With the introduction of a $30 incentive given to respondents in 2002, response rates improved among the younger age groups, but there was only a small increase among sample members aged 50+.

This report analyzes the relationship between age and response as part of NSDUH's MIP. The MIP is a systematic process that allows methodological enhancements to survey instruments, forms, and data collection protocol while minimizing the impact on trend data. The MIP specifies that survey improvements are to be implemented through a minimum of five steps:

1. a detailed evaluation of existing information to carefully identify the need for the change;

2. an assessment of the impact that the current need for change is having on substance use prevalence reports;

3. a detailed description of the protocols that will be followed if the change is applied to the main study;

4. an evaluation of the expected impact of the changes on prevalence rates and trend data that may result from the application of the change to the main study; and

5. application of the experiment results to develop statistical adjustments for prevalence rates and trends; use of the field results to introduce change to the main study in a way that minimizes the impact on trend data and allows tracking of changes in prevalence rates associated with the enhancement.

This report addresses the first three steps of the MIP by providing a detailed evaluation of the problem of nonresponse among sample members aged 50+, presenting potential tailored field strategies that could ameliorate the problem, and considering the potential impact of these changes on prevalence rates and trend data. The goal of this investigation is to inform targeted methodological improvements to increase age-specific and overall response rates, thereby reducing the potential for bias in the survey estimates.

    1. Organization of This Report

This report is organized into six sections. Section 1 summarizes the response rate problem and provides a summary of the MIP process. Section 2 reviews the relationship between age and response rates seen in other surveys and methods used to address nonresponse among older sample members. Section 3 describes this relationship in NSDUH over the past 5 years and examines changes between 2001 and 2002 in more detail. The roles of noncontacts, refusals, and other incompletes are discussed, and the 50+ group is disaggregated into smaller age categories to identify the segments of this group that contribute most to the lower rate of response and higher rate of refusals. Section 4 focuses on the relationship between age and other potential correlates accounting for higher nonresponse among older sample members. Section 5 describes the results of focus groups conducted with NSDUH FIs and potential 50+ respondents on the topic of nonresponse among those aged 50+ and ways of addressing it. Section 6 presents potential solutions to the problem and considers the possible effects of these solutions on prevalence rates.


  1. Previous Research on Nonresponse and Age

Herzog and Rodgers (1988) analyzed data from several face-to-face surveys, including the Americans View Their Mental Health (AVMH) study and the American National Election Studies, and found a linear decline in response rates with increasing age. They reported that refusal as a proportion of all nonrespondents increased for the middle years (35 to 74), then declined, reaching particularly low proportions among the oldest old (75+). The reason for nonresponse among the oldest age groups was less often outright refusal than among the middle age groups. The authors argued that the surveys examined did not take advantage of special characteristics of the older population to optimize the approach to this age group. Interviewers were not trained in specific problems with older respondents. In addition, these surveys held no particular relevance or interest for the older population, a factor that might be critical for obtaining high response rates among older adults (Hoinville, 1983). They stressed that future research should make a vigorous attempt to learn more about older persons who do not get interviewed and incorporate new methods designed to lower nonresponse among older sample members.

The negative correlation between response rates and age has been noted elsewhere in the nonresponse literature. Cohen, Machlin, and Branscome (2000) found that nonrespondents in the second round of the 1996 Medical Expenditure Panel Survey (MEPS) were more likely to be elderly. McQuillan, Khare, Karon, Schable, and Vlahov (1997) analyzed nonresponse in the National Health and Nutrition Examination Survey (NHANES III) and found that white and black men 40 to 59 years of age were least likely to participate in the survey. DeMaio (1980) reported that middle-aged and older people cooperate at lower rates than those under 30. Zimowski, Tourangeau, Ghadialy, and Pedlow (1997) alluded to lower completion rates in transportation surveys in a review of nonresponse in such surveys. They suggested that elderly respondents may fail to fill out or return the travel diaries used in such surveys because "they either have no trips to report or feel that the survey is not relevant to them." Response rates were negatively correlated with age in the 1992 National Adult Literacy Survey as well (Kirsch et al., 2001). This in-person survey of adults aged 16 or older consisted of a screening survey followed by interviews with one or two selected adults in each household. The interview consisted of a series of background questions (about 20 minutes) followed by completion of a literacy task booklet. Respondents were provided with a $20 incentive. Response rates in this survey showed consistent declines in response rate by age group, from 85 percent for those aged 16 to 24 to 77.4 percent for those 65 or older. Others have also noted that increased age of household members has a negative effective on survey cooperation (Comstock & Helsing, 1973; Paul & Lawes, 1982; Redpath & Elliot, 1988; Smith, 1995).

Groves and Couper (1998) found evidence of a curvilinear relationship between age and cooperation. They found that middle-aged households were less likely to cooperate than young and old households, and although elderly persons are more frequently at home due to their low employment rate and reduced mobility, their poor health may prevent them from survey participation.

Chiu, Riddick, and Hardy (2001) analyzed data from the National Health Interview Survey (NHIS) and found a different relationship between response and age. They reported that households containing senior adults and members with activity limitations were negatively correlated with difficult interviews when controlling for all other predicting variables. They believed that this is because these people are more likely to be home during the day and because the topic of health is viewed favorably among elderly persons. Similarly, Kautter, Khatutsky, Pope, and Chromy (2003) found no significant relationship between age and nonresponse in their analysis of the Medicare Beneficiary Survey (MCBS). Like the NHIS, the MCBS is a large-scale household health survey, although the majority of respondents are aged 65 or older. It may be that the topic of health is salient to older respondents and that they are more likely to respond to surveys that deal directly with health topics. The issue of topic salience among older respondents in NSDUH is examined in Section 5 of this report.

Tremblay and Moore (1995) reported higher response rates among elderly persons in the 1993 National Survey of College Graduates, a mail survey with telephone and in-person follow-up of U.S. residents under age 76 with a bachelor's degree or higher. The sample was drawn from a frame of respondents to the long form questionnaire of the 1990 decennial census. The response for those aged 60 or older was 85.9 percent, while it was 69.3 percent for those aged 16 to 29 and 80.1 percent for persons between 30 and 59 years of age. The fact that the sample consisted entirely of college graduates would seem to limit the relevance of these response rates. But in their discussion of the effects of age on response rate, Groves and Couper (1998) noted that older cohorts are less educated and have lower socioeconomic status, factors that are also associated with lower willingness to participate in surveys. As such, these may be factors that confound or intervene in the relationship between age and survey response on NSDUH.

Additional evidence on the relationship between age and survey participation can be found in NSDUH itself prior to 1999.1 Chromy et al. (1999) noted that for the period from 1993 to 1998, interview response rates for those aged 50+, overall, were lower than for all other age groups. However, the median response rate for those 50+ (76.2 percent) over this time period was not all that different from the median response rates for those in the other age groups, except for response rates for those aged 12 to 17.

    1. Strategies Employed on Other Studies

To combat the effects of lower response rates among older sample members, a variety of methods has been implemented on other surveys. In this section, we present a summary of these methods. This information will be helpful in determining methods to be tested and implemented for NSDUH:

  • Tailoring the questionnaire for older respondents. Sensory deficits are frequently experienced as people age, including hearing and/or vision loss (Herzog & Rodgers, 1988). Both can inhibit communication between the respondent and the interviewer (Jobe, Keller, & Smith, 1996). Although large-print surveys or written communications cannot always compensate for poor eyesight and loss of hearing can make telephone interviews impossible and in-person interviews extremely difficult (Rodgers & Herzog, 1987), data collection instruments can be tailored to address these concerns. For the frail elderly population or as a refusal conversion technique, a shorter version of the questionnaire containing only the core measures can be developed and administered. In the Second Longitudinal Study on Aging (LSOA II), a shortened instrument was administered over the telephone as a last resort to respondents who, because of the length of the computer-assisted telephone interviewing (CATI) instrument, would have otherwise refused to participate (National Center for Health Statistics [NCHS], 2002).

  • Allowing proxies to respond for older sample members. A substantial number of interviews on the Asset and Health Dynamics Among the Oldest Old (AHEAD) study were completed by a proxy respondent rather than by the designated respondent. This typically occurred because the designated respondent was ill, cognitively impaired, or unable to participate in a relatively lengthy interview. The incidence of proxy responses varied with the age of the designated respondent, with almost one third of the interviews in the oldest age group being conducted with proxy respondents rather than with the designated respondent. For the proxy interviews, the survey material was generally the same as for designated respondents except that cognitive test questions were not used on proxy interviews and expectation/subjective perception questions were not asked (Myers, Juster, & Suzman, 1997).

  • Employing interviewers with strong interpersonal skills. Increased success may be obtained by interviewers with knowledge of the constraints under which older respondents may labor in responding to surveys (New England States Consortium [NESC], 2002).

  • Developing a training module. A special section can be devoted to contacting and interviewing older respondents (NESC, 2002).

  • Emphasizing interviewer rapport. Familiarity may be an important factor in establishing rapport with older respondents (Blake, Korovessis, & Pickering, 2002).

  • Pacing the survey. A slower pace may make older respondents more comfortable (NESC, 2002).

  • Making the primary purpose of the survey clear. Some older respondents seek assurance that the survey is designed to gather the data that policymakers are seeking (NESC, 2002).

  • Soliciting advice from the geriatric research community. (NESC, 2002).

  • Emphasizing anonymity and confidentiality. For elderly home care clients, fear of losing services is a particular concern. This creates difficulties for researchers to fully capture their home-care experience (Forbes & Neufeld, 1997; Nehring & Geach, 1973). Emphasizing anonymity and confidentiality during the survey process may help to alleviate this bias (Sudman & Bradburn, 1982).

  • Alleviating respondent fears. Respondent fears may be alleviated by interviewers' offering to meet at another location (e.g., local hospital or health department), talking with a family member to confirm the legitimacy of the study, letting the participant know that it is acceptable to have another person present for the interview, or reassuring them, when appropriate, that existing medical conditions are not barriers to participation (Moorman, Newman, Millikan, Tse, & Sandler, 1999).

  • Reaching out to the public. The U.S. Bureau of the Census and the Department of Health and Human Services Administration on Aging (AoA) coordinated in a government-wide outreach effort to encourage Americans, in particular older persons and their caregivers, to complete and return their census forms. They mounted education and information campaigns to get the word out on the importance of Census 2000 (FirstGov for Seniors, 2000).

  • Providing mode options. On the AHEAD study, persons 80 years of age or older are assigned to a personal interview, although they can be converted to a telephone interview at their request (Myers et al., 1997).

  • Converting refusers with a financial incentive.2 The Health and Retirement Survey uses an "end game" strategy of offering reluctant respondents a large financial bonus for participation ($100) and asking for an immediate "yes" or "no" decision (NESC, 2002).


  1. Nonresponse and Age in NSDUH

In NSDUH, response rates have generally decreased with an increase in respondent age. Gfroerer, Lessler, and Parsley (1997) noted that age in NSDUH was related to interview nonresponse, with refusal rates increasing as age increased. Ideally, response rates would be consistently high across all age groups. This would reduce the potential for bias in both age-specific and overall prevalence rates.

This section presents a detailed analysis of the relationship between nonresponse and age in NSDUH, with particular emphasis on changes in this relationship accompanying the introduction of several methodological changes in 2002. The goals of this section are to identify the particular ages and components of nonresponse contributing most to lower response rates among those aged 50+ and determine the impact that these nonrespondents have on the overall interview response rate (IRR).

We first consider response rates3 by age for the survey years from 1999 to 2004. The year 1999 is selected as the starting point to ensure comparability of results.4 As Figure 3.1 shows, IRRs were successively lower for each sampled age group (12 to 17, 18 to 25, 26 to 34, 35 to 49, and 50+) in each year.5 Table A.1 in Appendix A provides the values corresponding to this figure. Across all years, response rates were lowest for the 50+ age group and highest for the 12 to 17 age group. The difference in response rates between these two groups remained around 13 percent in 1999, 2000, and 2001 and was around 18 percent in 2002, 2003, and 2004. Response rates for each age group increased from 2001 to 2002, but less for the 50+ age group than for the other age groups. These increases were statistically significant for all age groups except for those aged 50+.

The increase in response rates between 2001 and 2002 correlates with several methodological changes introduced during this period (OAS, 2003):

  • The name of the survey was changed in 2002 from the National Household Survey on Drug Abuse (NHSDA).

  • A $30 incentive was given to interview respondents beginning in 2002.

  • Improved data collection quality control procedures were introduced in the survey during 2001 and 2002.

  • Population data used in NSDUH sample weighting procedures were based on the 2000 decennial census for the first time in the 2002 NSDUH.

  • The pair selection algorithm was changed in 2002 to increase the pairs selected in the 50+ age group.

Given the pattern of change in response rates by age between 2001 and 2002, it is likely that these methodological changes had a larger positive effect on the response propensity of younger respondents than older respondents, thereby creating even larger differences in response rates by age.

Figure 3.1 Weighted Interview Response Rate (IRR), by Age: 1999 to 2004

* Includes 66,706 CAI interviews and excludes 13,809 PAPI interviews.

    1. Components of Nonresponse

In this section, we examine response rates by age in more detail by disaggregating age into 5-year groups; disaggregating nonresponse into the components of noncontacts, refusals, and other incompletes; and narrowing the scope of the analysis to 2001 and 2002 (the year before and the year after significant methodological changes and an increase in the response rate differences between the 50+ and all other age groups). Patterns for 1999 and 2000 closely resemble those for 2001, and patterns for 2003 and 2004 closely resemble those for 2002. For ease of interpretation, we limit the analyses to 2001 and 2002.

Because the five traditional age categories used for sampling and analysis purposes on NSDUH are very wide, analysis at this level may mask some heterogeneity that exists within age groups. For example, it may be that sample members aged 50 to 59 respond at the same rate as those under 50, but sample members aged 60 or older respond at a much lower rate, thereby decreasing the overall 50+ response rate. It is important to understand whether this phenomenon is found across the 50+ age group or only in small subgroups to determine where the MIP should focus on improving response rates.

Figure 3.2 presents the weighted IRR for 2001 and 2002 by 5-year age groups.6 Table A.2 in Appendix A provides the values corresponding to this figure for 1999 to 2004. For both years, response rates decreased almost linearly with an increase in age. In 2001, response rates stayed about the same or decreased slightly between the ages of 20 and 69. Response rates for those 70 or older decreased sharply with age, with the lowest age-specific response rate of 59.1 percent occurring among the 80+. In 2002, response rates were about 6 to 10 percent higher for all age groups between 12 and 49, with the exception of the 30 to 34 age group, whose response rate increased only 3.6 percent from 2001. The response rate for all age groups 50 or older increased less than 3 percent, with the exception of the 70 to 74 age group, whose response rate increased by 6.5 percent. In general, response rates between 2001 and 2002 improved more among younger sample members than among older sample members.

To gain a better understanding of response rate differences by age, we next examine the components of nonresponse. To assess these possible differences, we present the weighted noncontact, refusal, and other incomplete rates7 by 5-year age groups for the 2001 and 2002 surveys. It is important to understand how these components factor into nonresponse if we are to effectively design and implement strategies to address them. For example, a strategy that is designed to reduce noncontacts may have no effect on refusals. Response rates may differ by age because certain sampled persons are more difficult to contact, are more likely to refuse participation, or do not complete the survey for other reasons. In other words, are response rates for the older age groups lower than for the younger age groups because older sample members are more likely to be unavailable or unlocatable at the time of the interview, do they simply refuse to participate in the interview more often, or are there other reasons?

Figure 3.2 Weighted IRR, by 5-Year Age Groups: 2001 and 2002



Figure 3.3 presents the 2001 nonresponse component rates for each 5-year age group. Table A.3 in Appendix A provides the values corresponding to this figure. Noncontacts increased from 2.7 percent for the 12 to 14 age group to 8.36 percent for the 20 to 24 age group. Noncontacts then decreased steadily with age, beginning with the 20 to 24 age group and ending in a noncontact rate of 1.7 percent for the 80+. Refusals were the most common reason for nonresponse for each age group except the 80+. Refusals accounted for 12.4 percent of the 12 to 14 age group and increased steadily with age. The highest refusal rate occurred among those aged 50 to 54 (19.3 percent). Refusals then generally declined to a rate of 17.3 percent among those 75 to 79, then dropped off sharply to 10.6 percent among the 80+. Nonresponse due to other incompletes remained below 4 percent for all age groups until those 60 to 64. The rate of other incompletes increased steadily from 3.3 percent among those aged 55 to 59 to 12.7 percent among those aged 75 to 79. The other incomplete rate rose dramatically to 28.8 percent for the 80+.8

Figure 3.3 Weighted Nonresponse Rates, by 5-Year Age Groups: 2001



Because we are examining components of nonresponse as they relate not only to age-specific IRRs, but also to the overall IRR, it is important to examine the impact each of these age groups has on the overall IRR. When examining Figure 3.3, keep in mind that the number of sample members in each group is not equivalent and that larger groups have a larger impact on overall rates. In general, the younger age groups have greater sample sizes than the older age groups. To account for this, Figure 3.4 takes the weighted sample size into account and presents the impact each of these age- and component-specific nonresponse rates had on the 2001 overall IRR. Table A.4 in Appendix A provides the values corresponding to this figure.

Nonresponse was highest for those aged 50+ in 2001, but Figure 3.4 shows that the overall IRR was affected more by nonresponse in the middle age categories. Each group between 15 and 59 decreased the overall IRR by an increment of more than 1 percent. The majority of this decrease was due to refusals. Noncontacts among the younger age groups had a larger overall impact on the IRR than noncontacts among the older groups. Other incompletes had a larger overall impact for the older group—especially for those aged 80+, for which they were responsible for a 0.7 percent decrease in the overall IRR.



Figure 3.4 Impact of Age-Specific Nonresponse on Overall IRR: 2001



In 2002, the pattern of nonresponse by age changed. As shown in Figure 3.5, nonresponse was below 25 percent for all age groups under 50 and above 25 percent for all age groups 50+. Table A.5 in Appendix A provides the values corresponding to this figure. The main reason for this difference was a higher rate of refusals among those aged 50+. In 2001, the 50+ age group represented about one fourth of all refusals, and in 2002 they represented about one third. Refusals for all ages below 50 in 2002 were below 15 percent, and refusals for all ages 50+ were above 15 percent. The relatively higher rate of other incompletes among the oldest sample members also contributed to higher nonresponse for those age groups.

Figure 3.6 shows the impact each of these age-specific nonresponse components had on the overall IRR in 2002 (also see Table A.6 in Appendix A). Whereas in 2001, refusals among those aged 12 to 54 accounted for a 8.9 percent drop in the overall response rate, the impact of these refusals in 2002 was only 6.8 percent. The effect of refusals among those aged 55+ actually increased between 2001 and 2002 from 3.0 to 3.5 percent.


Figure 3.5 Weighted Nonresponse Rates, by 5-Year Age Groups: 2002



Figure 3.6 Impact of Age-Specific Nonresponse on Overall IRR: 2002


We summarize these age- and component-specific rates in Figure 3.7, which presents the increase in IRR and improvement in nonresponse reduction for each 5-year age group. Table A.7 in Appendix A provides the values corresponding to this figure. The line connecting the bars indicates the change in IRR between 2001 and 2002, and the bars indicate the improvement in each nonresponse component. Bars falling below 0 percent in the x-axis indicate an increase in nonresponse and a negative impact on the response rate. We find that the reduction in refusals and, to a lesser extent, noncontacts among sample members younger than 55 in 2001 led to increases in age-specific and overall response rates. IRRs among those aged 55 to 59 decreased by almost 3 percent due to increases in refusals and noncontacts. Generally, response rates among those aged 60 to 69 and 80+ increased, but not as much as for the younger sample members. Although there were improvements in the rate of other incompletes among these groups, the percentage of refusals increased, preventing great improvements in response rates for these groups. Response rates among those aged 70 to 79 increased due to a reduction of other incompletes and, to a lesser extent, refusals.

Figure 3.7 Contribution of Nonresponse Components to Percentage Change in IRR: 2001 to 2002



It appears that lower response rates among older respondents were due to relatively higher refusal rates and, in the oldest age categories, other incompletes, as compared with younger sample members. With the introduction of methodological changes in 2002, improvements in refusal rates among younger respondents led to improvements in age-specific and overall response rates. Refusals among older respondents actually increased for sample members aged 55 to 69 and 80+.

In order to gain an understanding of why older respondents may be more likely to refuse participation and why the methodological improvements made in 2002 were unsuccessful in reducing their likelihood to refuse, we next examine other correlates of refusal among older respondents. If these other correlates help explain the differences in response propensity between younger and older respondents, we can address them by designing targeted methodological enhancements.



  1. Potential Explanations for Refusals among the 50+

Other than age, we know that several important factors relate to whether a sample member will respond or refuse participation (Groves & Couper, 1998). In this section, we consider several potential explanations for reduced response rates among older sample members that are measurable using NSDUH data. Specifically, we investigate the roles of the following:

  • reasons for refusal given by sample members,

  • age-specific reactions to incentives,

  • respondent and household characteristics,

  • environmental characteristics, and

  • FI characteristics.

    1. Respondents' Reasons for Refusal

The most obvious source from which to begin examining potential reasons for refusal in NSDUH are the reasons for refusal given by nonrespondents. These reasons are selected from a set of common refusal reasons by FIs on the screening device. The distribution of refusal reasons by age group for 2001 and 2002 are presented in Figure 4.1.9 Tables A.8 and A.9 in Appendix A provide the values corresponding to this figure. For both years, the most common reason for refusal among sample members of all age groups was "Nothing in it for me." The second and third most common reasons were "No time" and "Government/surveys too invasive," respectively. "House too messy/too ill" was given as a reason for refusal among less than 1 percent of cases for each age group in each year except for those aged 70+, for which it was mentioned in 1 to 2 percent of cases.

Comparing 2001 with 2002, the rates for "Nothing in it for me" decreased for sample members under 55 and those aged 70 to 79, but increased for sample members aged 55 to 69 and 80+. This suggests that the introduction of the $30 incentive in 2002 did not result in those aged 55 to 69 and 80+ seeing "something in it for them." The trend can be seen in Figure 4.2, which presents the reduction (positive number = reduction) in the weighted IRR by age and reason for refusal. The line connecting the bars represents the overall reduction in refusal rate by age group between the 2 years. The percentage of sample members refusing with the reason "No time" improved among the younger age groups, but showed an opposite trend for all sample members 55+. All reasons for refusal became more prevalent between 2001 and 2002 among sample members aged 80+.



Figure 4.1 Reasons for Refusal by Age: 2001 and 2002



Figure 4.2 Contribution of Refusal Reasons to Reduction in Refusal Rate: 2001 to 2002

The fact that the rate of refusal for almost all reasons for all age groups under 50 decreased between 2001 and 2002 is likely due in part to the introduction of the $30 incentive. Regardless of the reasons given for refusal, the incentive may have decreased the propensity to refuse for sample members under 50. The same cannot be said for the 50+ group. These facts suggest that the incentive may not be a convincing factor for the 50+ group overall. To further test this hypothesis, we next review age-specific results from the 2001 incentive experiment.

    1. Age-Specific Reactions to Incentives

In 2001, an experiment was conducted to evaluate the effectiveness of respondent incentives in improving response rates in NSDUH (RTI International, 2002). A randomized, split-sample, experimental design was included with the main study data collection to compare the impact of $20 and $40 incentive treatments with a $0 control on measures of respondent cooperation, data quality, survey costs, and population substance use estimates. The experiment found that, overall, refusal rates were significantly lower for the incentive groups than for the control group (19.6 percent for $0, 12.8 percent for $20, and 9.4 percent for $40). Refusal rates were not examined by age, but response rates were. Overall, the $40 incentive resulted in a significantly higher response rate than the $20 incentive, and the $20 incentive resulted in a significantly higher response rate than the $0 treatment. For respondents aged 18 to 25 and 26+, the increase in response rates due to the $40 incentive versus $20 was not significant, but the difference between $20 and $0 was. The results of the incentive experiment led NSDUH to begin offering all respondents a $30 cash incentive in 2002.

Table 4.1 reexamines the incentive experiment data by 5-year age groups and shows that for all age groups between 15 and 29, response rates increased significantly from the $0 to $20 treatments and also from $20 to $40. For those aged 50+, this pattern was only replicated among those aged 55 to 59. For the other sample members aged 50+, there was an increase in response from $0 to $20, although the increase was not significant for each age group. The increase from $20 to $40 did not correspond with a significant increase in response and actually correlated with a significant decrease in response for those aged 70 to 74 and 80+. These results suggest that an increase in the current $30 incentive for older respondents may not result in an increase in response rates.

    1. Additional Characteristics Influencing Refusal among the 50+

To assess the impact of important respondent, household, environmental, and interviewer characteristics on the likelihood of refusal among those aged 50+, we analyzed the available NSDUH data on the possible correlates of gender, race, ethnicity, number of respondents selected per household, household composition, population density, socioeconomic status (SES), region, and interviewer experience. This analysis was conducted on 2002 data because they represent the first complete year of results after the introduction of incentives and other methodological changes. Noncontacts and other incompletes are excluded from this analysis because their inclusion in the refusal rate denominator could result in misleading conclusions. Therefore, refusal rates for this analysis include only final refusals and final completes. Table 4.2 presents the definitions used for each of these measures in this analysis.

Table 4.1 Incentive Experiment Weighted IRR, by Age

Age

Incentive Amount

$0

$20

$40

12-14

81.1

92.9

a

94.9

a

15-19

76.6

89.0

a

93.2

a,b

20-24

71.2

85.8

a

91.8

a,b

25-29

72.3

80.0

a

92.6

a,b

30-34

66.1

70.7

 

87.4

a,b

35-39

70.1

81.3

a

84.7

a

40-44

71.2

74.9

 

85.1

a,b

45-49

73.8

75.1

 

76.0

 

50-54

61.3

77.1

a

76.9

a

55-59

69.5

77.7

a

84.5

a,b

60-64

68.0

67.5

 

69.5

 

65-69

74.7

81.6

 

86.0

a

70-74

62.1

79.4

a

65.5

b

75-79

40.3

85.7

a

82.5

a

80+

60.9

71.7

 

51.8

b

Total

69.2

 78.8

a 

83.3

a,b

a Significantly different from $0 at the 0.05 level.

b Significantly different from $20 at the 0.05 level.

Table 4.2 Respondent, Household, Environment, and FI Characteristics

Characteristic

Source of Data

Possible Values

Number of selected respondents

Screener

1, 2

Number of persons in household

Screener

1, 2, 3, 4, 5+

Presence of a minor (<18) in the household

Screener

Yes, no

Single parent household

Screener

Yes (only one 18+ and at least one 0-17 in household)
No (all other cases)

Population density

Census

MSA ≥ 1 million persons, MSA < 1 million persons

SES1

Census

High SES, low SES

Region

Census

Northeast / Midwest / South / West

FI and respondent gender

Screener / FI characteristics

All combinations of male / female

FI and respondent race/ethnicity

Screener / FI characteristics

All combinations of Hispanic / black / white or other

FI age

FI characteristics

Under 50 / 50+ / missing2

FI experience

FI characteristics

Inexperienced (no prior NSDUH experience) /

experienced (40-99 interviews in prior years) /
highly experienced (100 + interviews in prior years)

MSA = metropolitan statistical area.

1 In order to define SES, block-group-level median rents and property values were given a rank (1...5) based on State and MSA quintiles. The rent and value ranks were then averaged, weighted by the percentage of renter- and owner-occupied dwelling units, respectively. If the resulting score fell in the lower 25th percentile by State and MSA, the area was considered "low SES"; otherwise, it was considered "high SES."

2 Because FI age was not available for a significant percentage of cases, "missing" was included as a distinct category. No other variable had a significant percentage of missing values.

Table 4.3 presents the IRRs by age and the aforementioned additional characteristics for all sample members. Unless noted, all of the data in this table refer to the 50+ portion of the sample. The percentage of cases with these characteristics among refusals and completes is also presented. Among all sample members, refusals were most common for sample members with these characteristics:

  • in households in which one sample member was selected for the interview,

  • in two-person households,

  • in households with no members under age 18,

  • in non-single-parent households,

  • in MSAs with 1 million or more residents,

  • in high-SES segments,

  • in the Northeast region,

  • male being interviewed by a female FI,

  • white/other being interviewed by an Hispanic FI,

  • interviewed by FI under age 50,10 and

  • interviewed by inexperienced FI.

The correlates of refusal were similar for sample members aged 50+ compared with all sample members. There were few differences in terms of characteristics of sample members most likely to refuse. Sample members aged 50+ in households in which two respondents were selected refused more often than those in households in which one was selected. The FI/respondent race combination that had the highest rate of refusals was that in which the FI was black and the respondent was white/other. Refusal rates were significantly higher for the 50+ age group for most types of sample members; exceptions were households of more than five persons and some FI/respondent race combinations, as noted in Table 4.3.

To simultaneously test the effects of these measures, we ran logistic regression models in which the dependent measure is refusal (0 = complete, 1 = refusal) and the independent variables are the characteristics listed in Tables 4.2 and 4.3. We ran one model that included sample members of all ages and a second that included only those 50+. These models can measure the significance of age overall and for the 50+ while controlling for the other characteristics. By comparing the results of the two models, we can identify relationships that may be unique among sample members aged 50+.



Table 4.3 Weighted IRRs for the 50+ Age Group, by Characteristics: 2002

Characteristic

Un-weighted N

Percent of Refusals

Percent of Completes

Refusal to Complete Rate (All Rs)

Refusal to Complete Rate (12-49)

Refusal to Complete Rate (50+)

1 R selected

3,870

44.8

41.4

15.5

12.4

19.6b

2 Rs selected

2,815

55.3

58.6

13.7

11.3

21.2b

1-person household

1,518

12.7

10.8

16.5

11.4

19.8b

2-person household

3,519

38.1

28.7

18.4

13.7

22.1b

3-person household

936

19.2

19.8

14.1

12.3

19.3b

4-person household

415

16.7

20.6

12.1

11.5

16.7a

5+ person household

297

13.4

20.2

10.1

9.8

13.2

Any household members < 18

838

35.6

49.5

10.8

10.4

15.0b

No household member < 18

5,847

64.4

50.5

17.7

13.9

21.1b

Single-parent household

69

2.1

5.3

6.1

6.0

7.9

Not single-parent household

6,616

98.0

94.7

14.9

12.1

20.4b

MSA ≥ 1 million

2,222

48.4

43.7

15.8

12.7

22.8b

MSA < 1 million

4,463

51.7

56.3

13.3

10.8

18.5b

High SES segment

4,952

76.9

72.2

15.3

12.6

20.7b

Low SES segment

1,733

23.1

27.8

12.3

9.4

19.1b

Northeast

1,329

22.4

18.2

17.2

14.1

23.3b

Midwest

1,873

22.3

23.0

14.1

11.6

19.2b

South

2,095

31.8

36.6

12.8

10.0

18.5b

West

1,388

23.5

22.2

15.2

12.3

22.0b

FI male, R male

630

11.7

11.2

15.0

13.0

19.8b

FI male, R female

775

12.8

12.0

15.3

12.1

21.5b

FI female, R male

2,387

40.4

36.5

15.8

13.1

21.5b

FI female, R female

2,892

35.0

40.2

12.8

9.7

19.0b

FI Hispanic, R Hispanic

147

2.9

4.4

10.1

9.7

11.6

FI Hispanic, R black

19

0.8

0.6

17.4

13.1

30.8

FI Hispanic, R white/other

199

4.5

3.2

19.6

16.3

25.3a

FI black, R Hispanic

35

0.6

1.4

6.2

5.9

8.8

FI black, R black

182

2.9

3.6

11.8

8.5

20.5b

FI black, R white/other

337

5.8

5.1

16.2

11.1

25.8b

FI white/other, R Hispanic

194

4.7

6.9

10.3

9.0

16.6b

FI white/other, R black

321

4.3

7.5

8.8

8.0

11.1

FI white/other, R white/other

5,223

73.6

67.4

15.6

12.7

20.8b

FI aged < 50

2,504

39.8

37.7

15.2

11.8

22.4b

FI aged 50+

3,381

54.5

57.1

13.9

11.5

18.7b

FI age missing

350

5.7

5.2

15.6

12.4

22.9b

FI inexperienced

1,381

24.8

21.7

16.1

13.1

22.7b

FI experienced

1,666

26.6

27.0

14.3

11.4

20.5b

FI highly experienced

3,637

48.7

51.3

13.8

11.2

19.2b

a Significantly different from the 12-49 rate at the 0.05 level.

b Significantly different from the 12-49 rate at the 0.01 level.


Table 4.4 presents the results of these models. In Model 1, which contains all ages, we find that compared with the 50 to 54 age group, refusal was significantly less likely among all age groups under 50 except those aged 40 to 44. No 55+ age groups were significantly different from those 50 to 54 on refusal. Sample members in households of two and three individuals were significantly more likely to refuse than were sample members in households of five or more persons. Sample members in households containing a minor (under age 18) were significantly less likely to refuse than those in households without a minor. Those in single-parent households were also significantly less likely to refuse than those in non-single-parent households.

As for regional characteristics, sample members in densely populated and high-SES areas were significantly more likely to refuse than those not living in such areas. Sample members in the Northeast were significantly more likely and those living in the South were significantly less likely to refuse compared with those in the West.

The combination of FI and respondent gender was a significant correlate of refusal, with all combinations significantly more likely to refuse compared with the female FI/female respondent scenario. Compared to the scenario in which the FI and respondent were both white/other, refusal was significantly less likely when the respondent was Hispanic, regardless of FI race/ethnicity. When the FI was white/other and the respondent was black, refusal was also significantly less likely compared with the scenario in which both were white/other. Cases finalized by inexperienced FI were significantly more likely to result in a refusal than those worked by highly experienced FIs.

Although many of these relationships were also evidenced in Model 2 (50+ only), the results were not exactly the same. When one respondent in the household was selected for the interview, the 50+ sample members were significantly less likely to refuse than when two respondents were selected. This may be due to older respondents' not having time or not being willing to devote their collective available time to the survey. Households in which one member of the pair was aged 50+ may not view a household-level incentive of $60 ($30 for each respondent) as adequate compensation for their participation. This means that it is possible that the increase in the number of selected pairs containing an older person in 2002 may have had a detrimental effect on response rates among older sample members.

Compared with those under 50, the relationship between household size and refusal propensity was different among those aged 50+. Respondents in households with one or two members were significantly more likely to refuse than those in households with five or more members. However, the presence of a minor and single-parent status were not significant predictors among those aged 50+. This suggests that older sample members living in small households (one or two members) are much more likely to refuse, regardless of the age of the other household members. Although earlier analyses conducted under the MIP found that sample members aged 50+ were significantly less likely to refuse when a minor was in the household or when the household was headed by a single parent, the logistic models have the advantage of simultaneously controlling for other correlates.

Another difference between Models 1 and 2 is the absence of a significant relationship between SES and refusal propensity among those aged 50+. It may be that younger low-SES sample members are more attracted to the incentive compared with younger high-SES sample

Table 4.4 Logistic Regression Model Odds Ratios and Significance: 2002

Characteristic

Logistic Model 1 (All Rs)

Logistic Model 2 (50+)

Odds Ratio

P Value


Odds Ratio

P Value


Intercept

0.16

<0.0001

b

0.14

<0.0001

b

R 12-14

0.46

<0.0001

b

--

--

 

R 15-19

0.36

<0.0001

b

--

--

 

R 20-24

0.43

<0.0001

b

--

--

 

R 25-29

0.60

<0.0001

b

--

--

 

R 30-34

0.79

0.0180

a

--

--

 

R 35-39

0.71

0.0014

b

--

--

 

R 40-44

0.85

0.1065

 

--

--

 

R 45-49

0.78

0.0092

b

--

--

 

R 50-55

--

--

 

--

--

 

R 56-59

1.06

0.6344

 

1.05

0.6917

 

R 60-64

0.97

0.8200

 

0.94

0.6278

 

R 65-69

1.06

0.6809

 

1.02

0.9082

 

R 70-74

0.87

0.3582

 

0.85

0.2687

 

R 75-79

0.95

0.7488

 

0.91

0.5625

 

R 80+

1.33

0.0661

 

1.25

0.1512

 

1 R selected

0.94

0.2135

 

0.76

0.0052

b

2 Rs selected

--

--

 

--

--

 

1-person household

1.02

0.8810

 

1.76

0.0483

a

2-person household

1.25

0.0183

a

1.88

0.0210

a

3-person household

1.19

0.0139

a

1.51

0.0956

 

4-person household

1.12

0.1251

 

1.28

0.3832

 

5+ person household

--

--

 

--

--

 

Any household members <18

0.84

0.0153

a

0.89

0.5452

 

No household member < 18

--

--

 

--

--

 

Single-parent household

0.57

<0.0001

b

0.38

0.1102

 

Not single-parent household

--

--

 

--

--

 

MSA ≥ 1 million

1.21

0.0001

b

1.26

0.0080

b

MSA < 1 million

--

--

 

--

--

 

High SES segment

1.12

0.0492

a

0.97

0.7329

 

Low SES segment

--

--

 

--

--

 

Northeast

1.16

0.0453

a

1.16

0.2319

 

Midwest

0.94

0.3285

 

0.89

0.3470

 

South

0.87

0.0362

a

0.91

0.4343

 

West

--

--

 

--

--

 

FI male, R male

1.19

0.0092

b

1.06

0.6350

 

FI male, R female

1.21

0.0063

b

1.16

0.2091

 

FI female, R male

1.29

<0.0001

b

1.18

0.0252

a

FI female, R female

--

--

 

--

--

 

FI Hispanic, R Hispanic

0.66

0.0046

b

0.45

0.0235

a

FI Hispanic, R black

1.20

0.5018

 

1.57

0.4009

 

FI Hispanic, R white/other

1.12

0.3668

 

1.09

0.6943

 

FI black, R Hispanic

0.40

<0.0001

b

0.32

0.0703

 

FI black, R black

0.76

0.0901

 

0.87

0.6499

 

FI black, R white/other

0.91

0.3748

 

1.07

0.7027

 

FI white/other, R Hispanic

0.74

0.0036

b

0.79

0.3216

 

FI white/other, R black

0.63

<0.0001

b

0.48

0.0018

b

FI white/other, R white/other

--

--

 

--

--

 

FI aged < 50

1.09

0.0881

 

1.17

0.0796

 

FI aged 50+

--

--

 

--

--

 

FI age missing

1.10

0.3533

 

1.25

0.2226

 

FI inexperienced

1.25

0.0004

b

1.25

0.0368

a

FI experienced

1.09

0.0967

 

1.14

0.1519

 

FI highly experienced

--

--

 

--

--

 

a Estimate significant at the 0.05 level.

b Estimate significant at the 0.01 level.

members, but older persons do not differ in their attitudes toward the incentive. This idea is investigated further in Section 5.

In general, FI characteristics were not as often significant predictors of refusal among those aged 50+ as they were among the full sample. The scenario in which the FI was female and the respondent was male was significantly more likely to yield a refusal than when both were female. Situations in which the FI and respondent were both Hispanic or the FI was white/other and the respondent was black were significantly less likely to result in a refusal than when both were white/other.

    1. Need for Qualitative Information

The relationships revealed by the characteristic-specific refusal rates and logistic regression models present opportunities to better understand why some sample members participate while others refuse and how these reasons differ by age under the current NSDUH methodology. For instance, one can examine the relationship found for respondents aged 50+ in households in which two respondents were selected and postulate that these respondents do not feel that the positives associated with participation (e.g., service to the country, accurate prevalence estimates, $60 incentive for the household) outweigh perceived negatives (e.g., 2-hour household time commitment, invasion of privacy, risk of becoming the victim of a scam). Similarly, one could interpret the respondent-provided and interviewer-recorded reasons for refusal as the underlying causes for nonresponse. But these refusal reasons may, in part, represent convenient explanations for refusal that do not fully explain why older respondents are less likely to participate in the NSDUH. The data are limited in that they cannot directly answer these questions. However, such questions can be explored qualitatively, as described in Section 5.





  1. Focus Groups

    1. FI Focus Groups

Although statistical analyses show the correlation of age, household composition, and response, they do not offer explanations or evidence of causal links. The analyses indicate a need for qualitative information on the underlying causes for lower response propensity among sample members aged 50+. Because NSDUH FIs have the most direct contact and experience with respondents, their input was sought to address this issue. To draw from the experience of the FIs, three focus groups were conducted to explore the issue of nonresponse among those aged 50+ and ways to address it. This section summarizes the ideas and themes resulting from these focus groups. A detailed account of the focus group design and results has been provided by Murphy and Schwerin (2003).

Fears and Misperceptions. The most common reasons that respondents aged 50+ gave for refusing to participate in the NSDUH were "There's nothing in it for me," "I have no time," and "Government/surveys are too invasive." In most cases, these categories capture the true reasons for refusal. However, there may be additional information that is not captured in these general categories or additional reasons for nonparticipation that respondents aged 50+ do not overtly state. The FIs in these focus group interviews reported that many respondents aged 50+ refused due to certain fears and misperceptions concerning trust. A fear of scams among this group may lead to an aversion to inviting unknown persons into their households. Fear of and disdain for the government's motives also appear to be motivating factors for refusal among a subset of respondents. Sensitivity to the survey topic and fear of divulging private information to unknown individuals appear to be factors. Also, apprehension toward the handheld screening device (similar in size and appearance to a smart phone) or the idea of using the laptop audio computer-assisted self-interview (ACASI) may affect participation among those aged 50+. This is consistent with studies that have found that older adults have significantly higher computer anxiety than younger adults (Laguna & Babcock, 1997). Another commonly reported misperception among older respondents is that they have nothing to offer the study. FIs report that many respondents said, "I do not use drugs, so you don't need to interview me," or "My experiences are irrelevant to this study."

Children May Have a Positive Impact. As a result of preliminary analyses of data from selected pairs, it was found that sample members aged 50+ were less likely to refuse when the second selected sample member was aged 12 to 17. Looking more closely at the data, it was found that sample members aged 50+ were less likely to refuse when there was someone in the household under age 18, regardless of whether that person was sampled. Similar results were reported by Groves and Couper (1998), who found that respondents in households with no children were less likely to cooperate in a survey. They also found that sample members in households with more than one adult were more likely to cooperate than those with only one adult. As the logistic models in Section 4.3 show, this relationship is not statistically significant when controlling for other factors, but the focus groups gave us the opportunity to research this possible explanation in a different way.

More than half of the FIs said that they noticed that respondents aged 50+ were more likely to participate if a child aged 12 to17 from the household was also selected for the survey. These FIs believed that the survey provides an opportunity for parents or grandparents and children to communicate on the subject of drugs and provides a positive shared experience. FIs also reported that these respondents aged 50+ may be motivated by their concern for children and society in general. Some FIs mentioned that respondents aged 50+ sometimes prevent a teenager from responding because they are not comfortable having their children exposed to the topic of the survey. Nearly half of the FIs mentioned that they noticed increased participation among those aged 50+ when a child under 18 was present in the household but not selected as a respondent.

Taking It Slowly. All FIs agreed that in order to gain the cooperation of sample members aged 50+, a great deal of patience and friendly professionalism was needed. Gaining the trust of the respondent was an important first step that needed to be taken before attempting to complete a screener or interview. Often, this meant that the interviewer needed to approach the respondent in a casual, friendly manner and start a conversation of interest to the respondent (e.g., gardening, pets) before mentioning NSDUH. Public awareness of the study could be another important factor in encouraging participation. FIs reported that some potential respondents called the State and local public health departments as well as local police to verify the credentials of the NSDUH project and the NSDUH FIs. Increasing the public's awareness of the study through contact with local police and public health departments, as well as press releases to local newspapers, could help raise awareness among residents in the community and enhance the perceived legitimacy of the study.

More Money May Not Be the Answer. Although FIs reported that the $30 incentive was helpful in gaining the cooperation of most respondents, certain subsets of the population aged 50+ may not be as receptive. They reported that high-income respondents aged 50+ were less likely to be persuaded by the prospect of receiving $30 after completing the survey. They also reported that respondents aged 50+ on a fixed income and retirees were not as responsive to the incentive. FIs reported that money was not the prime motivator for this group and that $30 actually raised suspicions of fraud or scams. They reported that the prime motivator for this group was more likely to be community service. These findings are consistent with those from the Survey of Program Dynamics that showed that respondents who accepted incentives were less likely to have someone in the household aged 65+ (Kay, Boggess, Selavel, & McMahon, 2001).

    1. Potential Respondent Focus Groups

Input was also sought from potential respondents in the 50+ age group to provide a basis for viable methods to test and implement. Twelve focus groups were conducted to explore the issue of nonresponse among those aged 50+ and ways of addressing it. Four groups were conducted in Raleigh, North Carolina, four in Washington, DC, and four in Oakbrook, Illinois, a suburb of Chicago. A full report summarizing the focus group design and results was prepared for the Substance Abuse and Mental Health Services Administration (SAMHSA) by Murphy et al. (2005). Highlights from that report follow.

Topic Understanding and Interest. After hearing only the brief FI introduction and name of the survey, most participants did not have a clear understanding of the survey topics. After reading the lead letter, most participants still did not understand the topic of the survey and believed the study objective described in the letter was vague. Many focus group participants described their initial impression of NSDUH as a study of prescription drug benefits, prescription drugs, and health insurance. There were no discernible differences by geographic location, household size, or participant age in perceptions or misperceptions of the survey topic. Once the topic of the survey was fully explained to the focus group participants, almost all expressed the opinion that drug use and health is an important topic to research. All participants believed that interest in the topic would make them more likely to participate, but it was obvious that this would not be the sole deciding factor. Nearly all groups recommended that the Q&A brochure, or similar summary of the survey, be included in an advance mailing with the lead letter so that potential survey respondents would have advance knowledge of what the study is all about and what will be expected of them.

Courtesy and Flexibility. Across age groups, household size groups, and focus group sites, a number of participants felt that the interviewing process showed a lack of courtesy to the respondent. Many also felt that the survey recruitment process might benefit from a more flexible and accommodating approach. Focus group participants said almost uniformly that it was inappropriate for a stranger to make a personal visit to solicit time from a respondent without scheduling the visit in advance, whether by calling on the telephone, or through some other form of prior notification. A number of participants stated that they would be more likely to participate if an appointment was scheduled before the first in-person visit. Participants felt that it was impolite for an FI to appear at a respondent's doorstep unannounced and inconsiderate to expect the respondent to make time for the interview. In addition, many participants considered the repetitiveness of the screening interview questions to be a misuse of the respondent's time, particularly because the possibility existed that the respondent may not even be selected to participate in the survey. To address this, we could experiment with lead letter verbiage to provide respondents with a more precise range of time that an FI will be in their area. FI trainings could address the specific concerns of participants regarding the presumption of availability and the repetitiveness of the screener questions.

Selection Process. Participants from each focus group expressed confusion over the language used to describe the selection process or "qualification process," as some called it. There were many questions about the meaning of "random" selection of households. Respondents and participants wanted more information about this process to feel more comfortable with the study objectives and protocol. Most participants believed that RTI or the Federal Government also had access to their names and phone numbers. All participants in both age groups wanted the screening script and questions to get directly to the point. The repetition of the questions was a major issue, specifically for those in households of two or three where roster questions are asked for all household members. For some group members, the possibility of having another person in their household selected for the interview would make a difference. For others, it would not have made a difference to their participation. One recommendation would be to include a better explanation of the selection process in advance materials and the FI introduction and possibly add some text that explains that eligibility is based on who lives at that residence during a specific time period. Also, it should be made clear, as needed, that RTI and SAMHSA/U.S. Public Health Service/Federal Government do not have participants' names or phone numbers on file.

Descriptive Materials. Remarks about the project materials provided to the focus group participants mirrored the overall focus group feedback of desiring additional detailed information about the purpose and benefits of the research and information that would facilitate trust and legitimacy to the research organization and FI. The lead letter was seen as a good tool, and the information in the Q&A brochure addressed the issues being raised by the group. The newspaper articles received a mixed review. The refusal letter appeared to address many participants' concerns, but some said it would not have changed their minds. Several experiments may prove beneficial from the focus group feedback on materials. Researching a better way to address the recipient of the lead letter to be used in place of "Resident," and mailing the letters using a first class postage stamp in a higher-quality envelope with a pre-printed Research Triangle Institute return address with a logo may help get more people to open and read the letter.

Safety, Trust, and Confidentiality. Concerns were raised about the survey approach and physical safety, security of the household, and fear of "scams" or other uses of information for reasons other than what was specified by the FI or in the survey materials. Concerns about physical safety were most prevalent in the groups of participants living alone, especially among those aged 65+, and in the Oakbrook, Illinois, location. Suspicion was raised concerning some of the screening questions. Some participants were confused as to why the FI asked about separate residences on the property, convinced she was asking about separate entrances to the residence. More than physical or household safety, concern was raised over safety from scams and mistrust of the FI's intentions.

The importance of trusting the FI, the research organization, and the study purpose were expressed throughout all of the focus groups. The importance of the FI establishing rapport and creating a level of trust with the respondent was communicated by many participants. Trust in the research organization and legitimacy of the survey were special concerns of the 65+ age group.

Overall, confidentiality was not a major concern voiced by the participants. However, participants expressed major concerns about the questions being intrusive, invasive, and too personal. Experiments in streamlining the screening questions and adding purpose statements with specific questions, such as the "Missed DU" question on dwelling units (DUs), might lead to solutions that would alleviate some respondent concerns. Experiments with advance materials focused on maximizing the extent to which household members read and retain the information may shed light on improving methods for gaining trust, eliminating fear, and gaining the participation of respondents in this age group. Similarly, additional FI training focused on increasing respondent trust and familiarity with the purpose of the survey and questions, especially among respondents in this age group, could prove beneficial. Finally, research into or experimentation with alternative FI identification (ID cards rather than just badges or larger badges) could lead to increased trust and participation among this age group.

FI Issues and Training. On the whole, focus group participants said they would be more likely to respond to an FI who was prepared and polished, without being "slick." They expect FIs to perform their task in a professional manner, which includes being polite and positive, while displaying knowledge of the survey questions. Participants also expressed they would not respond well to an FI who was timid or who presented a weak approach. An approach issue that should be included in training is FIs need to be aware that they are "guests" to the respondent's property and understand how the respondent feels about someone unknown coming to their door. Training interviewers to be sensitive to these matters may improve their ability to build rapport with the respondent.

Incentives. In general, the offer of a $30 incentive was not seen as persuasive by the focus group participants. Very few mentioned they would be convinced to do the interview for that amount. In some cases, participants felt that being offered money by the government to complete the survey was inappropriate. Still others were suspicious of the $30 offer, thinking it was a trick, part of a sales pitch, or that something other than completing the survey would be expected in return. Most participants agreed that money, while potentially a persuasive tool, would not be a sufficient enough incentive on its own to gain their participation. Although no solid suggestions for noncash incentives were offered, these participants felt that the most important factors in deciding whether to participate was trust in the motives of the FI and survey and an understanding and appreciation for the topic and value of the data.



  1. Tailored Field Strategies for the 50+ Age Group

The evidence presented earlier in this report suggests that response patterns for the 50+ age group were systematically different from those of the younger age groups. In addition, the results of the focus groups suggest that the difference in response behavior with the population aged 50+ is probably a function, in part, of their perception of the interview process. This suggests that response rates for the 50+ age group could be improved with field protocols tailored to meet the specific concerns of this age group.

Several protocol changes and methodological enhancements have been considered to improve the response rates for the 50+ age group. These possible changes are summarized below.

  • Adjust existing training modules to better cover the concerns of the 50+ age group. A module for this group would draw on the information included in the focus group report, as well as additional information collected in any future focus groups on the topic. Such a module may provide field staff with the additional skills necessary to overcome barriers to cooperation that are unique to the 50+ age group.

  • Alter the lead letter and refusal conversion letter to emphasize concepts that are salient to the older population, such as civic duty or the problems of drug-related crime. These modifications could be made to all lead and refusal letters if we assume that the impact of the change would be positive or neutral for all age groups. Alternatively, these changes could be made to a special set of letters for distribution to only those segments with high concentrations of older people, if we assume the letter would encourage cooperation without introducing measurement error in those segments. Lab and field experimentation on the lead letter's contents and appearance is being considered, and a work plan is being prepared for SAMHSA review.

  • Develop alternative modes for interfacing with the ACASI interview, such as a larger keyboard, a keypad tailored to the instrument, or a touch screen. This may reduce the anxiety that older respondents have about using the laptop.

  • Conduct a public health communications campaign at the local level prior to data collection. This may lend legitimacy to the data collection effort. A national campaign might also be effective. An endorsement from the national American Association of Retired Persons (AARP), or at least a letter of acknowledgement regarding the NSDUH project, could help communicate the importance of the study to sample members aged 50+. FIs could be supplied with these letters to use when encountering these sample members.

  • Assess higher incentive amounts for older respondents. The FI focus groups provided little evidence that the incentive amount was a factor in the response propensity of older persons. We recommend continuing the qualitative research into the response behaviors of the 50+ age groups through additional focus groups or other methods, such as video vignettes with potential respondents in the 50+ age group and other age groups. These focus groups could be used to guide any changes in field protocols and to test the changes before they are implemented.

  • Stress helping the younger generation (i.e., grandchildren), so those aged 50+ leave a legacy.

  • Tailor a few brief video clips using individuals recognized by the general public and well-respected by the population aged 50+. These video clips would be loaded on the handheld screening device and could be shown by FIs when encountering reluctant respondents.

In general, we should be extremely careful when implementing any change that is unique to the 50+ age group in order to avoid a differential measurement error. Although we want to reduce nonresponse error differentially, we do not want to introduce additional measurement error to the 50+ age group through changes in the survey materials or in the interaction between the interviewer and the respondent that cause respondents to self-report differently. For this reason, we suggest that all changes be tested with an experiment prior to implementation on the full study.

    1. Assessment of the Impact on Prevalence Rates

The MIP specifies that once a problem is diagnosed, the possible influence of the problem on prevalence rates should be assessed. The question to consider here is, does differential nonresponse among the 50+ age group lead to positively or negatively biased prevalence rates for this group, or are responders and refusers similar in regard to substance use? Because prevalence data are not available for the sample members aged 50+ who refused the interview, this assessment cannot be directly supported by existing NSDUH data. Without direct measures of prevalence rates among older nonrespondents, we could not conclusively state whether nonresponse among this group may be resulting in biased estimates. However, an examination of indirect measures may be possible and will be explored as part of a forthcoming NSDUH study on nonresponse bias. The relationship between the number of call attempts and prevalence rates could provide indirect measures of nonresponse bias (Wang, Murphy, Baxter, & Aldworth, 2005).

A number of studies have used reluctant respondents as predictors of nonrespondents, with mixed success (Smith, 1984; Stinchcombe, Jones, & Sheatsley,1981). Preliminary analysis of NSDUH data shows that drug prevalence rates are slightly lower among respondents who ever refused to participate compared with those who never refused to participate. The opposite relationship is seen for certain substances in the younger age groups. But we cannot assume that respondents who have refused and then participated resemble those who refused and never participated regarding drug use. Lin and Schaeffer (1995) examined two different methods of using reluctant respondents and decided that both were flawed and more arbitrary than anticipated. Cohen et al. (2000) found that "reluctant respondents as a whole appear to be a distinctly separate group, sharing one set of characteristics with the cooperative respondent group, another set with those who refused during the second round of the survey, and a yet a third set of characteristics that are uniquely their own" (p. 131).

As demonstrated in the 2002 NSDUH's national findings report (OAS, 2003), the survey changes introduced in 2002 did not significantly increase the response rate among those aged 50+, but this group's prevalence rates did increase significantly. The upper bound analysis showed that the increase in prevalence rates could not be fully explained by the slightly higher response rate. Given this finding, there is reason to assume that changes to survey protocol can affect prevalence rates for this group, but there is no evidence that nonresponse bias is occurring. A forthcoming NSDUH methods study on nonresponse bias should provide some evidence of the presence or absence of nonresponse bias among the 50+ age group and other age groups. Any changes to survey protocol resulting from this report will be considered in this regard prior to experimentation, prior to implementation, and again after implementation by examining marginal prevalence rates in an upper bound analysis, as was done in the 2002 national findings report.


References

Blake, M., Korovessis, C., & Pickering, K. (2002, January). National Adult Learning Survey 2001 technical report (Report P2034, prepared for the Department of Education and Skills, United Kingdom). Retrieved from http://www.esds.ac.uk/doc/4455/mrdoc/pdf/a4455uab.pdf

Chiu, P., Riddick, H., & Hardy, A. (2001). A comparison of characteristics between late/difficult and non-late/difficult interviews in the National Health Interview Survey. Presented at the annual meeting of the American Statistical Association, Atlanta, GA.

Chromy, J. R., Bowman, K. R., Crump, C. J., Packer, L. E., & Penne, M. A. (1999). Population coverage in the National Household Survey on Drug Abuse. In 1999 Joint Statistical Meetings, American Statistical Association, Survey Research Methods Section, Baltimore, MD (pp. 576-580). Alexandria, VA: American Statistical Association.

Cohen, S., Machlin, J., & Branscome, J. (2000). Patterns of survey attrition and reluctant response in the 1996 Medical Expenditure Panel Survey. Health Services & Outcomes Research Methodology, 1(2), 131‑148.

Comstock, G. W., & Helsing, K. J. (1973). Characteristics of respondents and nonrespondents to a questionnaire for estimating community mood. American Journal of Epidemiology, 97, 233-239.

DeMaio, T. (1980). Refusals: Who, where and why. Public Opinion Quarterly, 44, 223-233.

Eyerman, J., Odom, D., Wu, S., & Butler, D. (2002). Nonresponse in the 1999 NHSDA. In J. Gfroerer, J. Eyerman, & J. Chromy (Eds.), Redesigning an ongoing national household survey: Methodological issues (HHS Publication No. SMA 03-3768, pp. 23-51). Rockville, MD: Substance Abuse and Mental Health Services Administration, Office of Applied Studies.

FirstGov for Seniors. (2000). Census 2000 – Older Americans count. http://www.seniors.gov/articles/0300/older_americans_census.html

Forbes, D. A., & Neufeld, A. (1997). Strategies to address the methodological challenges of client-satisfaction research in home care. Canadian Journal of Nursing Research, 29(2), 69-77.

Gfroerer, J., Lessler, J., & Parsley, T. (1997). Studies of nonresponse and measurement error in the National Household Survey on Drug Abuse. In L. Harrison & A. Hughes (Eds.), The validity of self-reported drug use: Improving the accuracy of survey estimates (NIH Publication No. 97-4147, NIDA Research Monograph 167, pp. 273-295). Rockville, MD: National Institute on Drug Abuse.

Groves, R. M., & Couper, M. (1998). Nonresponse in household interview surveys. New York, NY: John Wiley & Sons, Inc..

Herzog, A. R., & Rodgers, W. (1988). Age and response rates to interview sample surveys. Journal of Gerontology, 43(6), S200-S205.

Hoinville, G. (1983). Carrying out surveys among the elderly. Journal of the Market Research Society, 25, 223-237.

Jobe, J., Keller, D., & Smith A. (1996). Cognitive techniques in interviewing older people. In Schwartz, N., & Sudman, S. (Eds.), Answering questions: Methodology for determining cognitive and communicative process in survey research. San Francisco, CA: Jossey-Bass.

Kautter, J., Khatutsky, G., Pope, G., & Chromy, J. (2003). Impact of nonresponse on MCBS estimates final report (prepared by RTI for the Centers for Medicare & Medicaid Services). Research Triangle Park, NC: RTI International.

Kay, W., Boggess, W.S., Selavel, K., & McMahon, M. (2001). The use of targeted incentives to reluctant respondents on response rate and data quality. Presented at the annual meeting of the American Statistical Association, Atlanta, GA.

Kirsch, I., Yamamoto, K., Norris, N., Rock, D., Jungeblut, A., O'Reilly, P., Campbell, A., Jenkins, L., Kolstad, A., Berlin, M., Mohadjer, L., Waksberg, J., Goksel, H., Burke, J., Rieger, S., Green, J., Klein, M., Mosenthal, P., & Baldi, S. (2001). Technical report and data file user's manual for the 1992 National Adult Literacy Survey. Retrieved from http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2001457

Laguna, K., & Babcock, R. L. (1997). Computer anxiety in young and older adults: Implications for human-computer interactions in older populations. Computers in Human Behavior, 13(3), 317-326.

Lin, I.-F., & Schaeffer, N. C. (1995). Using survey participants to estimate the impact of nonparticipation. Public Opinion Quarterly, 59, 236-258.

McQuillan, G. M., Khare, M., Karon, J. M., Schable, C., & Vlahov, D. (1997). Update on the seroepidemiology of human immunodeficiency virus in the United States household population: NHANES III, 1988-1994. Journal of Acquired Immune Deficiency Syndromes, 14, 355-360.

Moorman, P. G., Newman, B., Millikan, R. C., Tse, C. K., & Sandler, D. P. (1999). Participation rates in a case-control study: The impact of age, race, and race of interviewer. Annals of Epidemiology, 9(3), 188‑195.

Murphy, J., & Schwerin, M. (2003). Nonresponse among respondents aged 50 and older: NSDUH field interviewer focus group report (prepared by RTI for the Substance Abuse and Mental Health Services Administration). Research Triangle Park, NC: RTI International.

Murphy, J., Schwerin, M., Hewitt, D., & Safir, A. (2005). Nonresponse among respondents aged 50 and older potential respondents focus group report (prepared by RTI for the Substance Abuse and Mental Health Services Administration). Research Triangle Park, NC: RTI International.

Myers, G. C., Juster, F. T., & Suzman, R. M. (1997). Asset and Health Dynamics Among the Oldest Old (AHEAD): Introduction. Journals of Gerontology, 52B, v-viii.

National Center for Health Statistics. (2002). 1994-2000 Second Longitudinal Study of Aging: Wave 3 survivor file, codebook version SF 2.1. Retrieved from ftp://ftp.cdc.gov/pub/Health_statistics/NCHs/lsoa2/survivor_interview/w3sf/codebooks/W3_SF_CB_SF21.pdf

Nehring, V., & Geach, B. (1973). Patients' evaluation of their care: Why they don't complain. Nursing Outlook, 21(5), 317-321.

New England States Consortium. (2002). Getting feedback from frail elders and people with disabilities: Factors to consider when selecting a method, an instrument, a vendor. Retrieved August 31, 2005, from http://www.hhp.umd.edu/AGING/MMIP/TApapers/TApaper13.pdf

Office of Applied Studies. (2003). Results from the 2002 National Survey on Drug Use and Health: National findings (HHS Publication No. SMA 03-3836, NHSDA Series H-22). Rockville, MD: Substance Abuse and Mental Health Services Administration.

Paul, E., & Lawes, M. (1982). Characteristics of respondent and non-respondent households in the Canadian Labor Force Survey. Survey Methodology, 8, 48-85.

Redpath, B., & Elliot, D. (1988). National Food Survey: A second study of differential response, comparing census characteristics of NFS respondents and non-respondents; also a comparison of NFS and FSs response bias. Statistical News, 80, 6-10.

Rodgers, W., & Herzog, R. (1987). Interviewing older adults: The accuracy of factual information. Journal of Gerontology, 42, 387-394.

RTI International (2002). 2001 National Household Survey on Drug Abuse incentive experiment (prepared for the Substance Abuse and Mental Health Services Administration). Research Triangle Park, NC: Author.

Smith, T. W. (1984). Estimating nonresponse bias with temporary refusals. Sociological Perspectives, 27, 473-489.

Smith, T. W. (1995). Trends in non-response rates. International Journal of Public Opinion Research, 7, 157-171.

Stinchcombe, A. L., Jones, C., & Sheatsley, P. (1981). Nonresponse bias for attitude questions. Public Opinion Quarterly, 45, 359-375.

Sudman, S., & Bradburn, N. (1982). Asking questions: A practical guide to questionnaire design. San Francisco, CA: Jossey-Bass.

Trembaly, A., & Moore III, T. F. (1995). Nonresponse issues of the National Survey of College Graduates. In Proceedings of the Section on Survey Research Methods, American Statistical Association (pp. 459-464). Alexandria, VA: American Statistical Association.

Wang, K., Murphy, J., Baxter, R., & Aldworth, J. (2005). Are two feet in the door better than one? Using process data to examine interviewer effort and nonresponse bias. To be presented at the 2005 Conference of the Federal Committee on Statistical Methodology. Washington, DC.

Zimowski, M., Tourangeau, R., Ghadialy, R., & Pedlow, S. (1997) Nonresponse in household travel surveys (FHWA/PD-98/029, prepared for Federal Highway Administration). Chicago, IL: NORC.





Appendix A: Tables Relating to Figures in the Report



Table A.1. Weighted Interview Response Rate (IRR), by Age: 1999 to 2004

Age

Year

1999*

2000

2001

2002

2003

2004

12-17

78.07%

82.58%

82.18%

89.99%

89.83%

88.86%

18-25

71.21%

77.34%

75.51%

85.16%

83.73%

83.96%

26-34

69.45%

74.92%

74.82%

79.41%

78.72%

79.05%

35-49

67.75%

73.89%

72.38%

78.95%

77.40%

76.54%

50+

64.63%

69.53%

69.92%

71.54%

71.22%

71.66%

Total

68.60%

73.93%

73.31%

78.56%

77.67%

77.51%

* Includes 66,706 CAI interviews and excludes 13,809 PAPI interviews.

Table A.2. Weighted IRR by 5-Year Age Groups: 1999 to 2004

Age

Year

1999*

2000

2001

2002

2003

2004

12-14

78.96%

83.82%

83.24%

89.97%

89.90%

89.09%

15-19

75.85%

81.21%

80.20%

89.37%

88.75%

88.05%

20-24

70.74%

77.33%

74.42%

84.57%

82.85%

83.66%

25-29

70.58%

75.78%

75.15%

81.24%

79.49%

80.88%

30-34

69.27%

75.25%

74.81%

78.38%

78.75%

77.51%

35-39

69.56%

74.86%

74.29%

80.03%

78.91%

78.10%

40-44

68.56%

74.30%

72.20%

77.93%

77.25%

76.27%

45-49

67.34%

73.10%

71.63%

79.48%

76.52%

75.37%

50-54

65.69%

71.76%

71.09%

74.06%

73.43%

75.16%

55-59

67.80%

72.54%

75.02%

73.16%

71.83%

73.94%

60-64

65.57%

72.18%

71.94%

74.55%

75.25%

72.74%

65-69

67.87%

73.98%

72.80%

73.87%

75.61%

72.47%

70-74

68.46%

73.46%

68.56%

75.10%

70.29%

72.57%

75-79

64.55%

71.09%

68.24%

72.94%

66.53%

69.23%

80+

57.21%

59.86%

59.14%

56.02%

58.40%

58.56%

Total

68.60%

73.93%

73.31%

78.56%

77.67%

77.51%

* Includes 66,706 CAI interviews and excludes 13,809 PAPI interviews.


Table A.3. Weighted Nonresponse Rates, by 5-Year Age Groups: 2001

Age

Noncontacts

Refusals

Other Incompletes

All Nonresponse

12-14

2.73%

12.41%

1.62%

16.76%

15-19

4.42%

13.42%

1.96%

19.80%

20-24

8.36%

14.26%

2.96%

25.58%

25-29

7.95%

14.15%

2.75%

24.85%

30-34

6.98%

15.41%

2.81%

25.19%

35-39

5.80%

17.16%

2.76%

25.71%

40-44

6.37%

18.16%

3.28%

27.80%

45-49

5.60%

19.23%

3.54%

28.37%

50-54

5.95%

19.26%

3.70%

28.91%

55-59

4.00%

17.73%

3.25%

24.98%

60-64

4.32%

16.60%

7.14%

28.06%

65-69

2.80%

16.29%

8.11%

27.20%

70-74

1.84%

17.52%

12.08%

31.44%

75-79

1.71%

17.33%

12.73%

31.76%

80+

1.42%

10.62%

28.82%

40.86%



Table A.4. Impact of Age-Specific Nonresponse on Overall IRR: 2001

Age

Noncontacts

Refusals

Other Incompletes

All Nonresponse

12-14

-0.10%

-0.46%

-0.06%

-0.62%

15-19

-0.28%

-0.85%

-0.12%

-1.25%

20-24

-0.48%

-0.83%

-0.17%

-1.48%

25-29

-0.43%

-0.78%

-0.15%

-1.36%

30-34

-0.44%

-0.97%

-0.17%

-1.58%

35-39

-0.40%

-1.20%

-0.19%

-1.79%

40-44

-0.45%

-1.30%

-0.23%

-1.98%

45-49

-0.37%

-1.29%

-0.23%

-1.89%

50-54

-0.37%

-1.22%

-0.23%

-1.82%

55-59

-0.18%

-0.81%

-0.14%

-1.13%

60-64

-0.14%

-0.54%

-0.23%

-0.91%

65-69

-0.09%

-0.54%

-0.27%

-0.90%

70-74

-0.05%

-0.48%

-0.33%

-0.86%

75-79

-0.03%

-0.38%

-0.28%

-0.69%

80+

-0.03%

-0.25%

-0.68%

-0.96%

Total

-3.84%

-11.90%

-3.48%

-19.22%


Table A.5. Weighted Nonresponse Rates, by 5-Year Age Groups: 2002

Age

Noncontacts

Refusals

Other Incompletes

All Nonresponse

12-14

1.57%

7.27%

1.19%

10.03%

15-19

3.03%

6.23%

1.37%

10.63%

20-24

5.31%

7.99%

2.13%

15.43%

25-29

6.20%

10.42%

2.13%

18.75%

30-34

6.40%

12.66%

2.57%

21.62%

35-39

5.75%

11.58%

2.64%

19.97%

40-44

5.39%

13.94%

2.73%

22.07%

45-49

4.70%

13.80%

2.02%

20.52%

50-54

5.21%

17.31%

3.42%

25.94%

55-59

5.59%

18.78%

2.47%

26.84%

60-64

3.50%

18.09%

3.87%

25.45%

65-69

1.87%

19.12%

5.13%

26.13%

70-74

1.57%

15.85%

7.48%

24.90%

75-79

1.73%

16.71%

8.62%

27.06%

80+

1.58%

16.92%

25.49%

43.98%



Table A.6. Impact of Age-Specific Nonresponse on Overall IRR: 2002

Age

Noncontacts

Refusals

Other Incompletes

All Nonresponse

12-14

-0.07%

-0.31%

-0.05%

-0.43%

15-19

-0.21%

-0.43%

-0.09%

-0.73%

20-24

-0.34%

-0.52%

-0.14%

-1.00%

25-29

-0.38%

-0.65%

-0.13%

-1.16%

30-34

-0.43%

-0.85%

-0.17%

-1.45%

35-39

-0.42%

-0.85%

-0.19%

-1.46%

40-44

-0.41%

-1.08%

-0.21%

-1.70%

45-49

-0.33%

-0.99%

-0.14%

-1.46%

50-54

-0.33%

-1.11%

-0.22%

-1.66%

55-59

-0.28%

-0.95%

-0.12%

-1.35%

60-64

-0.13%

-0.69%

-0.15%

-0.97%

65-69

-0.06%

-0.57%

-0.15%

-0.78%

70-74

-0.04%

-0.45%

-0.21%

-0.70%

75-79

-0.04%

-0.39%

-0.20%

-0.63%

80+

-0.04%

-0.45%

-0.68%

-1.17%

Total

-3.51%

-10.29%

-2.85%

-16.65%


Table A.7. Contribution of Nonresponse Components to Percentage Change in IRR: 2001 to 2002

Age

Noncontacts

Refusals

Other Incompletes

All Nonresponse

12-14

1.16%

5.14%

0.43%

6.73%

15-19

1.39%

7.20%

0.59%

9.17%

20-24

3.06%

6.27%

0.83%

10.16%

25-29

1.75%

3.73%

0.62%

6.10%

30-34

0.58%

2.75%

0.24%

3.57%

35-39

0.05%

5.58%

0.12%

5.74%

40-44

0.97%

4.21%

0.55%

5.73%

45-49

0.91%

5.43%

1.51%

7.85%

50-54

0.74%

1.95%

0.28%

2.97%

55-59

-1.59%

-1.05%

0.78%

-1.86%

60-64

0.82%

-1.49%

3.28%

2.60%

65-69

0.92%

-2.83%

2.98%

1.07%

70-74

0.27%

1.67%

4.61%

6.54%

75-79

-0.02%

0.62%

4.10%

4.70%

80+

-0.16%

-6.30%

3.33%

-3.13%


Table A.8. Reasons for Refusal, by Age: 2001

Age

Nothing in it for me

No time

Government/ Surveys too invasive

Gatekeeper/ Household member won't allow participation

Confidentiality or survey legitimacy concerns

House too messy/ Too ill

Other

Missing

12-17

1.80%

1.02%

0.46%

0.48%

0.02%

0.02%

0.21%

0.13%

18-25

6.34%

4.28%

1.11%

0.89%

0.23%

0.07%

0.56%

0.50%

26-34

6.71%

5.29%

1.24%

0.28%

0.36%

0.10%

0.66%

0.42%

35-49

7.82%

6.24%

2.26%

0.35%

0.34%

0.16%

0.84%

0.42%

50-54

9.69%

5.60%

2.06%

0.36%

0.44%

0.09%

0.71%

0.31%

55-59

7.80%

5.04%

3.03%

0.30%

0.62%

0.30%

0.37%

0.26%

60-64

8.98%

2.63%

2.57%

0.26%

0.41%

0.60%

0.78%

0.39%

65-69

7.08%

3.12%

3.21%

0.23%

0.50%

0.41%

1.10%

0.63%

70-74

8.00%

3.46%

3.04%

0.17%

0.70%

1.45%

0.46%

0.24%

75-79

7.27%

2.55%

2.42%

0.42%

1.45%

1.64%

1.57%

0.00%

80+

5.63%

1.58%

1.02%

0.42%

0.27%

0.85%

0.86%

0.00%


Table A.9. Reasons for Refusal, by Age: 2002

Age

Nothing in it for me

No time

Government/ Surveys too invasive

Gatekeeper/ Household member won't allow participation

Confidentiality or survey legitimacy concerns

House too messy/ Too ill

Other

Missing

12-17

0.93%

0.33%

0.18%

0.26%

0.01%

0.00%

0.04%

0.05%

18-25

3.30%

2.00%

0.63%

0.94%

0.15%

0.03%

0.33%

0.18%

26-34

4.89%

4.74%

0.87%

0.55%

0.26%

0.05%

0.43%

0.16%

35-49

5.57%

4.46%

1.50%

0.53%

0.45%

0.08%

0.46%

0.21%

50-54

6.72%

4.94%

2.65%

0.59%

0.81%

0.45%

0.99%

0.15%

55-59

8.49%

6.31%

1.93%

0.44%

0.53%

0.40%

0.58%

0.12%

60-64

10.06%

4.78%

1.43%

0.47%

0.51%

0.50%

0.34%

0.00%

65-69

9.83%

4.72%

2.16%

0.40%

0.45%

0.57%

0.48%

0.50%

70-74

6.79%

3.54%

2.33%

0.46%

0.81%

1.12%

0.47%

0.33%

75-79

7.14%

3.10%

2.51%

0.66%

1.20%

0.83%

1.16%

0.12%

80+

6.07%

2.57%

2.67%

1.40%

0.56%

1.90%

1.34%

0.42%


Table A.10. Contribution of Refusal Reasons to Reduction in Refusal Rate: 2001 to 2002

Age

Nothing in it for me

No time

Government/ Surveys too invasive

Gatekeeper/ Household member won't allow participation

Confidentiality or survey legitimacy concerns

House too messy/ Too ill

Other

Missing

12-17

0.86%

0.69%

0.28%

0.22%

0.01%

0.02%

0.17%

0.08%

18-25

3.04%

2.28%

0.48%

-0.05%

0.08%

0.04%

0.23%

0.32%

26-34

1.82%

0.54%

0.37%

-0.27%

0.10%

0.05%

0.23%

0.27%

35-49

2.26%

1.78%

0.76%

-0.18%

-0.11%

0.08%

0.38%

0.21%

50-54

2.96%

0.65%

-0.59%

-0.23%

-0.37%

-0.36%

-0.28%

0.16%

55-59

-0.69%

-1.26%

1.10%

-0.14%

0.09%

-0.09%

-0.21%

0.15%

60-64

-1.08%

-2.16%

1.13%

-0.21%

-0.10%

0.10%

0.44%

0.39%

65-69

-2.75%

-1.60%

1.05%

-0.16%

0.04%

-0.16%

0.62%

0.13%

70-74

1.21%

-0.08%

0.71%

-0.29%

-0.12%

0.33%

-0.01%

-0.09%

75-79

0.13%

-0.54%

-0.08%

-0.23%

0.25%

0.81%

0.41%

-0.12%

80+

-0.44%

-0.99%

-1.64%

-0.98%

-0.30%

-1.05%

-0.48%

-0.42%




1 Prior to 2002, NSDUH was called the National Household Survey on Drug Abuse (NHSDA).

2 There are questions about the ethics and fairness of the use of targeted incentives for certain subgroups of interest or for refusal conversion (Groves & Couper, 1998).

3 Unless otherwise specified, all rates presented in this report are calculated using weighted data that do not include nonresponse or population adjustments.

4 The redesign of NSDUH in 1999 resulted in major changes in data collection procedures. In particular, the sample size was increased, there was a transition from a paper to a computerized instrument, and there was a transition from paper-and-pencil interviewing (PAPI) to computer-assisted interviewing (CAI). An analysis of the split PAPI/CAI sample in 1999 showed that the transition from PAPI to CAI had a positive effect on response rates, but the increase did not occur evenly across age groups, with the smallest increase occurring for respondents 35 years old or older (Eyerman, Odom, Wu, & Butler, 2002).

5 Because age for all sample members is not collected until the screening interview, screening response rates by age are not available.

6 Five-year age groups were chosen because they provide sufficient detail to detect differences by age and sufficient cases to present reliable measures. The lower bound for each age group ends in 0 or 5 for ease of interpretation, with the exception of the 12 to 14 group, which represents only 3 years but contains a great number of cases due to the oversampling of 12 to 17 year olds. Because sample sizes for those aged 80 or older are small, all persons 80 or older were included in the 80+ category. Exact age was not obtained for a small percentage of cases. In 2001, the weighted IRR for the 419 cases missing an exact age was 36.6 percent. In 2002, the weighted IRR for the 188 cases missing an exact age was 25.0 percent.

7 Noncontacts include cases in which no one was at the housing unit after repeated visits, the respondent was unavailable after repeated visits, and access to the building was denied. Refusals include final respondent refusals and final parental refusals (for respondents aged 12 to 17). Other incompletes include all other cases not finalized as completes, noncontacts, or refusals: physical/mental incapability, language barriers, and other noninterviews.

8 Nearly one fourth of the 80+ sample members did not complete an interview because they were physically or mentally incapable.

9 Because the focus here is on respondents aged 50+, that age group has been split into 5-year categories. Age categories for those under 50 follow the standard NSDUH groupings (12 to 17, 18 to 25, 26 to 34, and 35 to 49).

10 The highest refusal rate was actually found most often where the FI's age was unavailable, but this rate was very close to that for FIs under 50.

File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title2003 National Survey on Drug Use and Health: Nonresponse among Sample Members 50 or Older in NSDUH: Methodological Improvement P
SubjectNSDUH nonresponse among 50 or older
AuthorRTI International for SAMHSA Center for Behavioral Health Statis
File Modified0000-00-00
File Created2023-07-31
© 2024 OMB.report | Privacy Policy