Section B 2127_0646 renewal

Table of Contents

Supporting Statement

B.1. Describe the potential respondent universe and any sampling or other respondent selection to be used.

B.2. Describe the procedures for the collection of information.

B.3. Describe methods to maximize response rates.

B.4. Describe any tests of procedure or methods to be undertaken.

B.5 Provide the names and telephone numbers of individuals consulted on statistical aspects of the design.

Supporting Statement

B. Collections of Information Employing Statistical Methods

The proposed information collection will employ statistical methods to analyze the data collected from respondents. The following sections describe the procedures for respondent sampling and data tabulation.

B.1. Describe the potential respondent universe and any sampling or other respondent selection to be used.

The potential universe for these surveys is all non-institutionalized individuals age 18 or older in telephone households within the United States. The target population is 18 years and older because the modest size of the proposed samples would result in too few drivers age 16 and 17 to justify additional procedures for including child respondents.

This information collection will encompass multiple national and more localized (Regional, State, or community) samples.

National Surveys

National telephone survey samples will be required for both seat belt and alcohol-impaired driving Mobilization/Crackdown interventions. Up to two National Seat Belt Mobilizations and two National Alcohol-Impaired Driving Crackdowns will be evaluated each year, with each survey drawing respondents from all 50 States and the District of Columbia. For each Mobilization/Crackdown, a pre-test (baseline) national sample and a post-test national sample of approximately 1500 respondents will be surveyed. This will consist of a representative sample of 1200 respondents age 18 and older with an additional over-sampling of approximately 250 respondents ages 18 to 34. The younger age group will be over-sampled because they are a particular focus of the media campaigns for the Mobilizations and Crackdowns. The field period for each survey wave will necessarily be limited to less than one month in order to maintain proximity to the interventions. Due to concurrent interviewing and planned callbacks at the time the 1450^th interview is completed, a handful of additional interviews will be completed by the time of field closure, resulting in somewhere between 1450 and 1500 completed interviews. Over the projected upcoming 3-year period, this would be:

(1500 respondents pre-mobilization + 1500 respondents post-mobilization) X

4 mobilizations (2 Alcohol-Impaired Driving Crackdowns + 2 Seat Belt mobilizations) X 3 Years = 36,000 respondents

The surveys will utilize a dual-frame design, with landline and cell phone samples. The NHTSA contractors will apply to landline and cell phone banks systematic selection procedures that include Random Digit Dial (RDD) sampling techniques in order to select a probability sample of respondents, that is, a sample of respondents where every person with a telephone (either landline or cell phone) has a known probability of being selected for the study. The samples will be stratified according to four Census Regions: Northeast, Midwest, South, and West. The respondents will be asked an initial set of screening questions to identify those who meet the eligibility criteria for participating in the survey. For the cell phone sample, the respondents will first be asked one or more questions under procedures approved by an Institutional Review Board (IRB) to assure that it is safe for them to respond. Both the Alcohol-Impaired Driving Crackdowns and Click It or Ticket Mobilizations will screen for age criteria (18 or older). The Alcohol-Impaired Driving Crackdown surveys will also screen respondents on driver status and alcohol use in the past 12 months. In the landline samples, if there are multiple persons residing in a household eligible to participate, NHTSA may disproportionately select the younger/male eligible household member as the sole survey respondent. Weighting procedures will adjust for any disproportionate sampling. For the cell phone sample, the survey will treat the cell phone as a single user device.

The percentage of the national samples composed of cell phone interviews will be determined by coverage and cost. Early estimates for the first half of 2010 from the National Health Interview Survey indicated that 26.6% of households had only wireless telephones.¹ While a proportional allocation of cell phone only respondents within the sample based on this number would be statistically powerful, it also would be expensive since it costs more to interview cell only respondents compared to conducting landline interviews. But allocating too little sample to cell only will result in extreme weight differential that usually has a negative impact on the variance of survey estimates. A middle ground maximizes the effective sample size for a fixed cost. For the National Alcohol-Impaired Driving Surveys, optimal allocation will be 15 percent (180) cell only. This assumes a 2:1 cost ratio of a cell phone survey to a landline survey. To improve efficiency and include cell-mostly respondents who may be under-represented in the landline sample, dual users (respondents having both a landline and cell phone) will be interviewed from the cell phone sample. A total of 450 cell phone interviews (including the 180 or 40% cell only) will be conducted for the 1200 cross-sectional sample, and the remaining 750 interviews will be conducted with individuals on landlines. Thus for the cell sample, 2.5 people would need to be screened to get one cell only respondent. Assuming that 65% of the cost per completed interview goes to contacting and screening a cell phone respondent, the relative cost per interview for cell only will be 2.5 * 65% + 1 * 35% = 1.975. Combining the general cell phone ratio cost ratio (2) with the screening cost ratio (1.975) gives the final cost ratio for cell only versus landline: 2 * 1.975 = 3.95

Localized Mobilization/Crackdown Surveys

NHTSA may on occasion have a need to assess a special Regional, State, or Community seat belt/anti-DWI intervention activity occurring as part of the National Mobilization or Crackdown. If this were to occur, NHTSA would apply the same pre- post design, and same sampling and interviewing procedures, as conducted for the national surveys. Differences from the national surveys would be that the eligible phone banks against which the systematic sampling procedures would be applied would be limited to a particular geographic area (with no geographic stratification), and the sample size per survey wave would typically be 500. There may be disproportionate selection to bring a greater number of younger persons and males into the sample if those groups are higher priority targets of the special intervention activity. Weighting procedures will adjust for any disproportionate sampling.

Demonstration Project Surveys

For each Demonstration project, a pre-test (baseline) sample and a post-test sample per designated site will be surveyed. As with the localized Mobilization/Crackdown surveys, systematic procedures would be applied to selection of sample from telephone exchanges in landline and cell phone banks. Sampling sizes will vary according to the analytic needs of the demonstration project (e.g., the level of subgroup analyses needed). Where the project is concerned with detecting changes in attitude, awareness, etc. within the general community, then the sample size will typically be 500. If the demonstration project demands subgroup analyses, then sample sizes will be larger, most likely approximately 1200. There may be disproportionate selection to bring a greater number of younger persons and males into the sample if those groups are higher priority targets within the demonstration project. Weighting procedures will adjust for any disproportionate sampling.

The Demonstration Project surveys will address high priority intervention approaches for NHTSA. High priority areas include techniques to improve enforcement of traffic safety laws, and approaches to improving traffic safety among high risk populations.

Sampling Error

The techniques described above will result in a complex sampling design. The complex sampling design would need to be taken into account while computing sampling error (or precision) of estimates. However, computation in previous NHTSA surveys of the precision of estimates in a complex design show confidence intervals that do not differ much from those produced through simple random sampling². Formulas for simple random sampling therefore are a convenient means of computing sampling error for the sample sizes being proposed under this clearance since those estimates will be reasonably close to estimates from complex designs despite the above-described variations in methods across surveys.

The confidence interval for sample estimates of population proportions, using simple random sampling without replacement, is calculated by the following formula:

Where:

se (x) = the standard error of the sample estimate for a proportion

p = some proportion of the sample displaying a certain

characteristic or attribute

q = (1 ‑ p)

n = the size of the sample

z = the standardized normal variable, given a specified

confidence level (1.96 for 95% confidence interval)

To test whether or not a difference between two sample proportions is statistically significant, a rather simple calculation can be made. The maximum expected sampling error of the first sample is designated s1 and the maximum expected sampling error of the second sample is s2. The sampling error of the difference between these estimates is sd and is calculated as:

• For comparison of two samples of 500 each, a difference would have to exceed 6.2 percentage points to be statistically significant (with the conservative estimate of p=q).

• For comparison of two samples of 1200 each, a difference would have to exceed 4.0 percentage points to be statistically significant (with the conservative estimate of p=q).

• For comparison of two samples of 1500 each, a difference would have to exceed 3.6 percentage points (with the conservative estimate of p=q).

We believe that this is sufficient for our purpose to assess the impact of the Mobilizations/Crackdowns.

B.2. Describe the procedures for the collection of information.

The proposed surveys will be administered using a pre-/post-test design to examine the changes that occur as a result of specific seat belt and impaired driving interventions. The National and localized Mobilization and Crackdown surveys will be conducted on a schedule corresponding with the fixed annual dates for those national campaigns. The demonstration project surveys will be conducted on a more variable schedule that will depend on the timing and sequencing of the components of each demonstration project.

National Mobilization/Crackdown Surveys

At the national level, data collected from random samples of approximately 1500 people before and after the Mobilizations/Crackdowns will be compared to examine changes in awareness, attitudes and self-reported behavior. Each survey wave will be composed of a cross-sectional sample of 1200 randomly selected people 18 and older, and an independent randomly selected sample of 250 people 18 through 34.

The proposed national survey samples are based on a modified stratified random digit dialing method, using a geographically stratified RDD sample rather than a single-stage/RDD sample. There are several important advantages to using a geographically stratified base for the RDD sample: (1) it draws the sample proportionate to the geographic distribution of the target population rather than the geographic distribution of telephone households, which is vital to constructing unbiased population estimates from telephone surveys; (2) it allows greater geographic stratification of the sample to control for known geographic differences in non-response rates; and (3) it facilitates the use of Census estimates of population characteristics to weight the computed sample to correct for other forms of non-sampling bias

The initial stage of the sample construction process requires the development of a national area probability sample based upon the distribution of the target population for this study, i.e. the non-institutionalized population age 18 and older in the United States. Once the sample has been geographically stratified with sample allocation proportionate to population distribution, a sample of assigned telephone banks will be randomly selected from an enumeration of the Working Residential Hundreds Blocks of the active telephone exchanges within the region. The Working Hundreds Blocks are defined as each block of 100 potential telephone numbers within an exchange that includes 3 or more residential listings. (Exchanges with one or two listings are excluded because in most cases such listings represent errors in the published listings.). The random samples of cell phone numbers will be selected from the dedicated exchanges (exchanges containing only cell phone numbers) of that region.

In the third stage sample, a two-digit number is randomly generated by computer for each Working Residential Hundreds Block selected in the second stage sample. This third stage sampling technique is known as random digit dialing (RDD). Every telephone number within the Hundreds Block has an equal probability of selection, regardless of whether it is listed or unlisted. The use of RDD sampling eliminates the otherwise serious problem of unlisted telephone numbers.

For the landline sample, the fourth stage of sampling will involve a procedure once contact is made with the household to select one designated respondent for each household sampled. An algorithm or system of prioritization may be inserted into the initial screening to disproportionately select younger and male respondents if the household has more than one person eligible to participate in the survey. Otherwise, the “most recent/next birthday method” will be used for within household selection among multiple eligible respondents. Salmon and Nichols (1983³) proposed the birthday selection method as a less obtrusive method of selection than the traditional grid selections of Kish, et al. In theory, birthday selection methods represent true random selection (Lavrakas, 1987⁴). Empirical studies indicate that the birthday method produces shorter interviews with higher response rates than grid selection (Tarnai, Rosa and Scott, 1987⁵).

For the cell phone sample, no selection procedure for multiple eligible respondents will be needed as the surveys will treat the cell phone as a single user device. However, immediately upon contact, cell phone respondents will be asked one or more questions according to IRB-approved procedures to assure that it is safe for them to respond.

As regards the over-samples, the selection procedures will be the same as those used for the cross-sectional samples. But rather than screen for anyone 18 and older, they will screen only for respondents 18 through 34. The interviews will be conducted on cell phones due to the large percentage of individuals in that age range who live in wireless only households: 39.9% age 18-24, 51.3% age 25-29, and 40.4% age 30-34 according to NHIS data for January – June 2010.

Upon contacting the landline household, interviewers will briefly state the purpose of their call (including noting the anonymity of the interview), and then request to speak to the person in the household meeting the requisite screening criteria. If the person who answered the phone is the selected respondent, then the interviewer will proceed with the interview. If the selected respondent is someone else who then comes to the phone, then the interviewer will again introduce the survey (with anonymity statement) and proceed with the interview. If the selected respondent is not available, then the interviewer will arrange a callback. Cell phone respondents will be read the same project introduction, asked if they are in a situation where it is safe for them to respond at that time, and if so, proceed with the interview. If the interviewer finds that the prospective respondent is driving or in some other situation that could compromise safety, then the interviewer will immediately hang up while saying s/he will call back at a more convenient time.

Localized/Demonstration Surveys

The major differences between the national and localized/demonstration sample selection procedures will be sample size, the absence in most cases of Stage 1 distribution of sample by geographic stratification, and lesser use of over-sampling or disproportionate selection of respondent groups. Otherwise, sampling procedures will mirror the same procedures described above for selecting the national samples (i.e., Stages 2-4). Like the national surveys, they will include interviews with respondents on cell phones. Because the cell phone exchange dialed for a respondent may not match where the respondent actually resides, screening will need to include a question asking if the respondent currently lives in the geographic location targeted by the intervention.

Demonstration projects will typically be directed towards a community, a county or a media market composed of multiple counties. The telephone exchanges covering the geographic area undergoing the intervention will be determined, and a systematic procedure for randomly selecting telephone numbers to call will be implemented. Demonstration project surveys may require more screening criteria than those mentioned previously as interventions may be directed at very specific subgroups within the community. When contacting households having multiple members eligible to participate in the survey, in-house selection methods will be conducted that obtain scientifically valid random samples.

Data Collection Procedures across Samples

Data collection will be conducted by trained interviewers working in telephone research centers that utilize a computer-assisted telephone interviewing (CATI) network. The CATI network will have capability for silently monitoring the performance of interviewers. Monitoring will be conducted by supervisory staff during all interview shifts to determine the quality of interviewer’s performance in terms of:

1. Initial contact and recruitment procedures;

2. Reading the questions, fully and completely as written;

3. Reading response categories, fully and completely, (or not reading them) according to the study specifications;

4. Whether or not open-ended questions are properly probed;

5. Whether or not the interviewer enters the correct code, number, or verbatim response to the question;

6. Whether or not ambiguous or confused responses are clarified

7. How well questions from the respondent are handled without alienating the respondent or biasing his/her response;

8. Avoiding bias by either comments or vocal inflection;

9. Ability to persuade wavering, disinterested or hostile respondents to continue the interview; and

10. General professional conduct throughout the interview.

Initial telephone contact will be attempted during the hours of the day and days of the week that have the greatest probability of respondent contact. This means that the primary interviewing period will typically be conducted between 5:30 p.m. and 10:00 p.m. on weekdays; between 9:00 a.m. and 10:00 p.m. on Saturdays; and between 10:00 a.m. and 10:00 p.m. on Sundays. If the interview cannot be conducted at the time of initial contact, the interviewer will reschedule the interview at a time convenient to the respondent. Although interviews will be conducted on evenings and weekends whenever possible, daytime interviews will be scheduled whenever necessary.

Statistical Analysis

As specified above, NHTSA will employ dual-frame designs in conducting its telephone surveys. For community level samples of 500, the basic approach to weighting will be to adjust the achieved samples so that they approximate key demographics of the selected sites according to Census figures (sex, age). Since demographic data would not be available for potential eligibility criteria such as driver status, the general population figures that are available would be used. If there are indications of substantial differences between the breakouts of the samples by phone status versus what would be expected within the current environment, then further adjustment may be applied. However, there are no data to provide a precise barometer of what the phone status distribution should be at the community level, and available State figures would likely be too old to use for weighting purposes and may not be representative of the selected communities anyway. Therefore, it would take an obvious discrepancy with State and national figures to consider weighting on this factor.

For larger samples drawn from national sampling frames, more detailed procedures will be employed to adjust the samples to represent their respective populations. For example, for the National Alcohol-Impaired Driving Crackdown surveys, the NHTSA contractor will use benchmark data from the National Health Interview Survey to adjust the landline sample to reflect the population distribution of landline-only, landline-mostly, true-dual, and cell-mostly. Then the contractor will adjust the cell sample to reflect the population distribution of landline-mostly, true-dual, cell-mostly, and cell-only. Then the contractor will adjust the over-lapping groups to adjust for the fact that they are represented in both samples and will be over-represented relative to the landline-only and the cell-only groups.

Chi square or difference of proportions tests will be applied to final data to compare results from survey waves. Specifically, statistical tests will be used to determine if there are statistically significant differences between pre and post waves. Additional statistics may be calculated if NHTSA sees a need for more refined analyses. Appropriate software will be used to account for the complex sampling designs when analyzing data (e.g., SUDAAN).

B.3. Describe methods to maximize response rates.

The National Mobilizations/Crackdowns and demonstration projects will include a minimum five call attempts and seven callbacks during the field periods. However, the limited field periods will require that the surveys place particular emphasis on contact scripts and the training/monitoring of interviewers. The initial contact script has been carefully developed and refined to be persuasive and appealing to the respondents. The interviewing will be conducted only by thoroughly trained and experienced interviewers who are highly motivated and carefully monitored. All interviewers will have had training on how to overcome initial reluctance, disinterest or hostility during the contact phase of the interview. There will be maintenance and regular review of field outcome data in the sample reporting file, derived from both the sample control and CATI files, so that patterns and problems in both response rate and production rates can be detected and analyzed. Periodic meetings will be held with the interviewing and field supervisory staff and the study management staff to discuss problems with contact and interviewing procedures and to share methods of successful persuasion and conversion.

B.4. Describe any tests of procedures or methods to be undertaken.

The proposed Mobilization and Crackdown surveys are a continuation of seat belt and impaired driving mobilization surveys conducted in previous years. As such, they will utilize questionnaires nearly identical to those utilized previously, and follow methods that have been previously implemented and found successful.

B.5. Provide the name and telephone number of individuals consulted on statistical aspects of the design

The following individuals consulted on statistical aspects of the study design:

Richard Compton, PhD

Director, Office of Behavioral Safety Research

DOT/National Highway Safety Administration

1200 New Jersey Ave, SE

Washington, DC 20590

(202) 366-2699

Maria Vegega, PhD

Chief, Behavioral Research Division

Office of Behavioral Safety Research

DOT/National Highway Safety Administration

1200 New Jersey Ave, SE

Washington, DC 20590

(202) 366-2668

Linda Cosgrove, PhD

Chief, Injury Prevention Research Division

Office of Behavioral Safety Research

DOT/National Highway Safety Administration

1200 New Jersey Ave, SE

Washington, DC 20590

(202) 366-5592

Alan Block, MA

Office of Behavioral Safety Research

DOT/National Highway Safety Administration

1200 New Jersey Ave, SE

Washington, DC 20590

(202) 366-6401

1 Blumberg, S.J. and Luke, J.V. Wireless Substitution: Early Release of Estimates From the National Health Interview Survey, January – June 2010. Released 12/21/2010. Available from http://www.cdc.gov/nchs/data/nhis/earlyrelease/wireless201012.htm

2 Such as the 2008 National Survey of Drinking and Driving Attitudes and Behavior, which also utilized a dual frame design of landline and cell phones. DOT HS 811 344. August 2010. Pages 14-17.

3 Salmon, C. and Nichols, J. The Next-Birthday Method of Respondent Selection. Public Opinion Quarterly, 1983, Vol. 47, pp. 270-276.

4 Lavrakas, P. Telephone Survey Methods: Sampling, Selection and Supervision. Beverly Hills: Sage Publications, 1987.

5 Tarnai, J., Rosa, E. and Scott, L. An Empirical Comparison of the Kish and the Most Recent Birthday Method for Selecting a Random Household Respondent in Telephone Surveys. Presented at the Annual Meeting of the American Association for Public Opinion Research. Hershey, PA, 1987.

10