Justification Section B

Table of Contents

Supporting Statement

Justification

B.1 Describe the potential respondent universe and any sampling or other respondent selection to be used.

B.2 Describe the procedures for the collection of information.

B.3 Describe methods to maximize response rates.

B.4 Describe any tests of procedures or methods to be undertaken.

B.5 Provide the name and telephone number of individuals consulted on statistical aspects of the design.

Supporting Statement

B. Collections of Information Employing Statistical Methods

The proposed information collection will employ statistical methods to analyze the data collected from respondents. The following sections describe the procedures for respondent sampling and data tabulation.

B.1. Describe the potential respondent universe and any sampling or other respondent selection to be used.

Information collection will occur at five sites that will be conducting different levels of Highly Visible Enforcement (HVE) over the course of a one-year period. Sites may be counties, a combination of counties, a city, or other jurisdiction having a minimum population size of approximately 200,000. Each site will be termed a community by this project. The sites will vary in the amount of HVE they conduct. This project has defined three different levels of HVE as follows: (1) an integrated program that conducts HVE throughout the entire year; (2) an intermediate program that conducts 6 waves of HVE over the course of the year; and (3) a low level program that conducts HVE primarily during the two National Alcohol Crackdowns. The specific sites are yet to be determined.

Telephone surveys will be conducted at each site to gauge community-level awareness of the HVE activity and community perceptions of the likelihood of an alcohol-impaired driver being stopped by law enforcement officers. The respondent universe will be residents of each selected community who are drivers, age 18 and older, have access to a residential landline telephone and/or a personal cell phone, and have consumed alcohol in the past year. Respondents 18 through 34 will be over-sampled.

Drivers more at risk of driving at illegal Blood Alcohol Concentrations (BACs) are a particular target for the intervention programs. Thus the telephone surveys will contain questions intended to differentiate those high risk drivers from other drivers. These will include CAGE questions that have been validated as predictors of drinking problems, and questions pertaining to usual drinking quantity that likewise have shown a relationship to problem behavior.

Data from NHTSA’s 2008 National Survey of Alcohol Attitudes and Behavior suggest that at a minimum close to 10 percent of the telephone sample will be categorized as high risk. Since site selection criteria will include consideration of whether a site has an alcohol issue, the likelihood is that the percentage of high risk drivers will be somewhat above that figure. But that still provides a fairly small number of cases to assess the impression that the HVE activity is making on high risk drivers. Therefore, surveys will also be conducted at a venue where higher percentages of high risk drivers can be found. More specifically, project staff will go to bars to briefly interview patrons and take breath samples. This will be scheduled to occur during the same times of the year as the telephone survey waves. While the bar surveys will be composed of convenience samples, methods for selecting the samples will be identical across sites and across survey waves to allow comparisons. Moreover, selected questions common to both the telephone and bar survey instruments will allow comparison of the convenience sample to the probability-based sample of high risk drivers on program awareness and perceived risk measures.

B.1.a Telephone Survey

B.1.a.1 Sampling

There will be three telephone survey waves. The surveys will use a dual frame design in which both a landline sample and cell phone sample are drawn and subsequently combined. Both samples will be obtained from Survey Sampling International (SSI). The landline sample will use a random digit dial (RDD) list-assisted method for sampling telephone numbers in defined geographic areas (i.e., the targeted communities). This single-stage, unclustered sampling method selects a probability sample from all telephone numbers that are in 100-banks that have at least one residential listed telephone number. The telephone exchanges from which the 100-banks will be sampled will be those assigned to the targeted geographic areas. The last two numbers within the sampled 100-banks will be randomly generated to produce the telephone numbers for the households that will comprise the landline sample. The Contractor will call these numbers to determine eligibility to take the survey. If the number is confirmed as belonging to a household and the Contractor finds that there is just a single household member who meets the eligibility criteria, then the Contractor will seek to conduct the survey with that individual. If multiple household members are eligible to participate, then the Contractor will randomly select one member from among those eligible using the next birthday method and seek to conduct the survey with that randomly selected household member.

The cell phone sample will be selected from 1,000-series blocks in the U.S. dedicated to wireless service. Geographic information for the telephone exchanges will be used to identify those associated with the targeted geographic areas. RDD will be used to select the cell phone numbers that will be in the sample, and interviewers will manually dial the numbers to reach a respondent. This study will treat the cell phone as a single user device, so no within-household selection methods akin to those used for the landline sample will be needed. However, interviewers will be required to immediately establish that the respondent is in a situation where it is safe to carry out the interview. If the respondent is driving or in some other situation that poses risk, the interviewer will stop the call and tell the respondent s/he will call back at another time.

To be eligible to participate in the telephone survey, respondents must be drivers who are at least 18 years of age, and they must have consumed alcohol in the past year. These screening criteria will be applied at the onset of the survey. Because younger adults tend to be heavier drinkers, and therefore a particular interest for alcohol-impaired driving interventions, the telephone surveys will be designed to obtain a sufficient number of interviews with respondents ages 18 through 34 per site per survey wave in order to conduct analyses of this age group. This will require over-sampling due to the low response rate of individuals in this age range.

The sample size will be 1,200 respondents per site per survey wave. This will be composed of a cross-sectional sample of 1,000 and an over-sample of 200. Average interview time will be 10 minutes. The cross-sectional sample will be screened for eligibility for the following criteria: age 18 or older, a driver, and a consumer of alcohol in the past year. Of these 1,000 interviews, 28 percent will be conducted with individuals on cell phones who have been screened as cell phone only. This percentage is comparable to the estimated percentage of adults living in wireless-only households according to published figures for the last half of 2010. ¹ The remaining 72 percent of cross-sectional interviews will be conducted with individuals on landlines.

The 2007 Motor Vehicle Occupant Safety Survey found respondents ages 18 through 34 to compose just 18.7 percent of an 18 and older national landline cross-sectional sample as opposed to Census Bureau population estimates of approximately 31 percent. The ratio is about .6. Applying the 18.7 percent figure to the 720 landline interviews results in an estimate of 135 interviews with respondents that are ages 18 through 34 from the landline sample. The cell phone sample will yield proportionally more respondents in this younger age range as approximately one-half (52.5%) of cell only adults are ages 18 through 34. But the lower response rate of the younger age group will lead to fewer cell phone interviews with individuals 18 through 34 than 52.5 percent of the total cell sample of 280 (which would be 147 interviews). Applying the .6 ratio cited above to the 147 cell interviews (52.5% of the cell sample) yields 88 cell interviews with people 18 through 34. Combined with the landline interviews, this would sum to 223 interviews with people 18 through 34.

The preceding discussion shows that the cross-sectional sample will produce a relatively small number of interviews with respondents 18-to-34, and will yield a percentage well below their population prevalence. Moreover, it’s impossible to say how closely the numbers achieved from community-level samples will match these estimates. Therefore, an additional 200 interviews will be conducted with an independent over-sample screened to be conducted exclusively with respondents in the 18-34 age range. This will guarantee a sufficient number of interviews for analyses of this age group.

B.1.a.2 Sampling Error

Core analyses for this project will compare telephone survey responses across sites, and across survey waves. The five program sites will be conducting demonstrably different levels of HVE activity. For the extra effort to reach the higher levels of HVE to be considered worthwhile, detected differences must be more than just a few percentage points. A threshold difference of 6 percentage points to reject the null hypothesis is considered appropriate for this study.

If simple random sampling was being used to conduct the survey, the confidence interval for sample estimates of population proportions would be calculated by the following formula:

Where:

se (x) = the standard error of the sample estimate for a proportion

p = some proportion of the sample displaying a certain

characteristic or attribute

q = (1 ‑ p)

n = the size of the sample

z = the standardized normal variable, given a specified

confidence level (1.96 for 95% confidence interval)

Testing whether or not a difference between two sample proportions is statistically significant would involve a rather simple calculation. The maximum expected sampling error of the first sample is designated s1 and the maximum expected sampling error of the second sample is s2. The sampling error of the difference between these estimates is sd and is calculated as:

For comparison of two samples of 1200 each using the above formula and assuming simple random samples, a difference would have to exceed 4.0 percentage points to be statistically significant (with the conservative estimate of p=q). However, the proposed methods for this study involve a complex sampling design using a dual frame and over-sampling of younger persons. This means that tests of statistical significance will need to take into account design effects. It’s uncertain what the design effect will be for prospective community surveys. But data on selected items from a 2008 dual frame national alcohol telephone survey indicated confidence intervals that were, on average, about 1.5 times greater than what the confidence intervals would be for a simple random design². Applying a multiplier of 1.5 to the 4.0 percentage point difference required for statistical significance under a simple random design results in the 6 percentage point threshold difference specified in the first paragraph of this section. Using the same multiplier for subgroup comparisons involving sample sizes of approximately 400 (i.e., respondents ages 18-34) would require a difference of just over 10 percentage points in order to be statistically significant.

B.1.b Bar Survey

B.1.b.1. Sampling

NHTSA plans to augment the information it collects in the telephone surveys with information collected from patrons at bars. The purpose is to get information on intervention awareness and perceptions of risk from key targets of the intervention activity (i.e., individuals at greater risk of driving at BACs above the legal limit) whose numbers in the telephone survey will be relatively small. Bars offer a location where this higher risk group will be more concentrated (studies show that about 50 percent of drivers arrested for DWI had their last drink at a bar or restaurant). BAC samples can also be collected with the self-report survey information, which would allow correlating BAC with awareness and perceived risk.

The Contractor will conduct short interviews and obtain breath samples from 400 bar patrons per site per survey wave. The approach that will be used is the Portal Survey Technique, where individuals are interviewed both upon their entry and exit from the bar. During the entry stage, survey staff will approach naturally occurring groups of 2 to 8 individuals approaching the bar with at least one member aged 21-34. Interviewers will recruit respondents from roughly 9:00 pm to 12:30 am, although these survey times may be adjusted due to jurisdiction-specific factors. At entry, generally, a line forms outside the bar as the bar gets busier and as the bar’s security check IDs to ensure patrons are of legal drinking age. The interviewers will approach people in this line, or if there is no line, they will approach the next group approaching the bar’s entrance. If a line has formed, the interviewers will be trained to approach a systematically selected person in line (e.g., the fifth person in line). The group affiliated (for example) with that fifth person is the selected group. The group will be included in the study if that group: (a) has a participant between the ages of 21 and 34, (b) plans to stay in the bar for 45 minutes or more (this allows ample time for patrons to have at least one drink in the bar), and (c) has at least one member who plans to drink alcohol. The Contractor plans to recruit groups because they have found from past experience that it is easier to recruit participants when the group is intact (most members do not like to split from their group). Procedures for approaching potential participants are adapted from those used for many years during roadside breath-test surveys of motorists (Voas, Wells, Lestina, Williams, & Greene, 1998³). Participants will be offered an incentive to participate. The incentive will be a $10 gift card they receive upon completion of both components of the portal survey (Entry and Exit). The breath test results will not be available on site but will be downloaded later. Those who agree to participate will be identified with coded wristbands and asked to return to the interviewers upon exit (outside the bar) to respond to a few remaining survey questions, provide another breath sample, and receive their incentive. An identification number on the bracelet will be used to link the exit questions and breath-tests to the entry data.

The duration of information collection for the respondents will be 10 minutes. During entry, 2 minutes will be spent obtaining consent, 4 minutes to conduct the interview, and 1 minute to obtain the breath sample. On exit, the bar patrons will spend 2 minutes responding to a shorter interview and 1 minute providing a final breath sample.

B.1.b.2 Sampling Error

Stroup’s (1999) simulation method was used to estimate power for the bar patron surveys. We estimated statistical power for tests of differences between any two Site x Wave samples (n = 400 each), whether comparing the same site across two waves, or comparing two separate sites within the same wave. Assuming α = .05 (two-tailed), we would have adequate power (.80) to detect changes as small as 9 percentage points. We would have power of over .90 to detect changes of 11 percentage points.

These power calculations are conservative. First, we currently do not have estimates of the variability (in drinking) among sites within Waves or within sites across Waves, and cannot model these sources of variation in our computations. Statistical power should be greater when these sources are included in our statistical model. Second, our power estimates also do not include covariates. Data on the demographic characteristics of individual bar patrons will be collected, and these data will be included in analyses. Past research suggests that individual characteristics account for considerable variability in drinking levels, therefore including these variables in our analytic models should increase power further. Finally, we have estimated power assuming two-tailed hypothesis tests. Adopting a strategy of using a directional test would grant us statistical power to detect as statistically significant even smaller differences than those previously stated.

B.2. Describe the procedures for the collection of information.

1. Telephone Surveys

For the landline interviews, a sample of assigned telephone banks will be randomly selected from an enumeration of the Working Residential Hundreds Blocks of the active telephone exchanges within the defined geographic area for each project community site. A two-digit number is then randomly generated by computer for each Working Residential Hundreds Block selected, a technique known as random digit dialing (RDD). The next stage involves calling the number and selecting an eligible household member to participate in the survey. The interviewers will ask several screening questions to determine if there are any drivers within the contacted household who are in the required age range and occasionally drink alcohol. If only one household member is found eligible, then the interviewer will ask to proceed with the interview with that person. If more than one household member is eligible, then the interviewer will ask to speak to the eligible household member who will have the next birthday. If an interview can’t be conducted with the selected household member at the time of the call, then the interviewer will ask for a convenient time when a callback can be scheduled. If the selected household member refuses to participate in the interview, but it is a soft refusal, then the interviewer will place the sampled household number in a queue for refusal conversion efforts to be conducted one-to-two weeks after the refusal. However, if it was a hard refusal, then no refusal conversion effort will be conducted.

The cell phone sample will be selected from 1,000-series blocks in the U.S. dedicated to wireless service. All calls placed to cell phone numbers will be hand-dialed. The interviewers will first confirm the respondent is on a cell phone and make sure that the respondent is not in some situation that could put him or her at risk if the interview proceeded. If a respondent is driving or is in some other situation where there could be a danger, then the interview will be terminated and attempted at another time. If a prospective respondent refuses to participate or prefers to participate at a later time, then the Contractor will apply the same refusal conversion and callback procedures used for the landline sample.

The methods for interviewing the over-sample will be the same as those for the landline and cell phone cross-sectional samples. The sole difference is that the age screening will be for 18 through 34 rather than 18 and older. The over-sample will be composed of interviews of people on both landline phones and cell phones.

Data collection will be conducted by trained interviewers working in telephone research centers that utilize a computer-assisted telephone interviewing (CATI) network. Initial telephone contact will be attempted during the hours of the day and days of the week that have the greatest probability of respondent contact. If the interview cannot be conducted at the time of initial contact, the interviewer will reschedule the interview at a time convenient to the respondent. Although interviews will be conducted on evenings and weekends whenever possible, daytime interviews will be scheduled whenever necessary.

2. Bar Surveys

The bar surveys will be conducted using teams comprised of one supervisor and six to eight trained interviewers. The interviewers will use a Full Portal Survey Technique which is an approach that the Contractor (PIRE) has tested for the past 7 years and become proficient at administering (Miller, Holder, & Voas, 2009⁴). Here, surveys will be administered at two points in time. As patrons enter the bar, the interviewers obtain consent from the bar patrons for their participation, and then administer a brief entry survey upon completion of each individual’s informed consent. After participants complete the entry interview, an initial breath alcohol test will be administered. When those same participants exit the bar, a brief exit interview will be administered. Participants will also be asked to provide another breath sample at that time. Participant code numbers assigned during entry will allow all measures to be linked to a particular individual while maintaining his or her anonymity.

Each survey weekend, each team will survey for two nights. Five to seven staff will recruit and conduct the survey/interviews and one staff person will count the number of males and females exiting the bar by the hour. This count will provide information on the crowd level in the establishment and will be used to track the proportion of patrons that responded to the survey. The supervisor will manage the interviewers and any set-up arrangements with bar personnel. The survey hours of operation will be from 9 p.m. to 2 a.m. (generally closing time, however this will be adjusted based on the jurisdiction). Between 8:30 p.m. and 9 p.m., the survey supervisor will brief each interviewer on the evening’s tasks in recruiting survey participants. The survey supervisor will pair the interviewers into teams of two and provide each with the necessary supplies (alcohol breath test devices, iPod touch® devices, wristbands, clipboards, mouthpieces, pens, etc.). The supervisor will also assign each team of two interviewers a series of group numbers. Additionally, the survey supervisor will confirm with the study’s bar liaison and the bar manager where to stage the interviewers for participant recruitment. Due to the physical limitation of space outside any bar, and due to law enforcement restrictions on setting up any table on public pedestrian thoroughfares, the team will remain completely mobile, processing recruited participants in any space reasonably unsaturated with persons using the sidewalk or entering the bar.

The interviewers will recruit individuals into the study from 9 p.m. to 12:30 a.m. and conduct Exit Surveys when the participants exit the bar. The bars usually have a last call for drinks at 1:30 a.m., and the security staff generally empties the bar and locks the doors by 1:45 a.m., hence the reason Entry Surveys will stop at 12:30 a.m. However, as noted above, these survey times may need to be adjusted to the jurisdiction. At entry, generally, a line forms outside the bar as the bar gets busier and as the bar’s security check IDs to ensure patrons are of legal age to enter the bar. The interviewers will approach people in this line, or if there is no line, they will approach the next group approaching the bar’s entrance. Whole groups are recruited because logistically it is difficult to gather information from just one individual in a group while the others are waiting. If a line has formed, the interviewers will be trained to approach a systematically selected person in line. The group affiliated with that person is the selected group. The group will be included in the study if that group (a) has a participant between the ages of 21and 34, (b) plans to stay in the bar for 45 minutes, and (c) has at least one member who plans to drink alcohol. If they meet these criteria, they will be asked to participate in a voluntary and anonymous survey at entry to and exit from the bar, for which they will each be paid $10. The interviewer will assign the group unique study identifiers, keyed onto a wristband that participants will wear until they return for the Exit interview. When participants exit the bar, they will seek out an interviewer to complete their exit survey. The exit survey will consist of a follow-up one-on-one interview, a breath sample, and receipt of a $10 reloadable gift card.

B.3. Describe methods to maximize response rates.

The telephone interviewing will be conducted only by thoroughly trained and experienced interviewers who are highly motivated and carefully monitored. The CATI network they use will have capability for silently monitoring the performance of the interviewers. Monitoring will be conducted by supervisory staff to determine the quality of interviewer’s performance in terms of:

1. Initial contact and recruitment procedures;

2. Reading the questions, fully and completely as written;

3. Reading response categories, fully and completely, (or not reading them) according to the study specifications;

4. Whether or not open-ended questions are properly probed;

5. Whether or not the interviewer enters the correct code, number, or verbatim response to the question;

6. Whether or not ambiguous or confused responses are clarified

7. How well questions from the respondent are handled without alienating the respondent or biasing his/her response;

8. Avoiding bias by either comments or vocal inflection;

9. Ability to persuade wavering, disinterested or hostile respondents to continue the interview; and

10. General professional conduct throughout the interview.

All telephone interviewers will have had training on how to overcome initial reluctance, disinterest or hostility during the contact phase of the interview. There will be regular review of field outcome data so that patterns and problems in both response rate and production rates can be detected and analyzed. Periodic meetings will be held with the interviewing and field supervisory staff and the study management staff to discuss problems with contact and interviewing procedures and to share methods of successful persuasion and conversion.

As regards the bar surveys, the Contractor that will be conducting them is highly experienced in this form of data collection. Their survey managers have been conducting surveys at bars and elsewhere for 10 years. They have become proficient at recruiting, obtaining consent, and surveying participants quickly and efficiently. The interviewers that will be used on this project will be selected for maturity, ability to rapidly establish rapport with strangers, and attention to detail. Any newly hired interviewers for the project will undergo a training program covering general interviewing skills, the purpose of the survey, the importance of random selection, the importance of confidentiality, how to use data-collection instruments and equipment, and how to accurately record data using electronic and/or paper-and-pen surveys. Additionally, they will undergo 8 hours of classroom training involving hands-on use of the equipment and the data-collection instruments, how to approach and introduce the study, how to determine qualifying selection criteria, and how to implement the survey procedures and additional safety protocols, such as the “Impaired Driver Protocol” that will be used (Appendix H). Before going into the field, they will conduct a series of mock interviews; once in the field, they will be paired with a supervisor (a coach) to observe and then conduct surveys while being coached. Additionally, they will have to have successfully completed a Human Subjects course on the proper treatment of human subjects in research projects. The field director will conduct periodic quality control evaluations for adherence to protocol.

B.4. Describe any tests of procedures or methods to be undertaken.

For the telephone surveys, there will be automated testing of the CATI programming to assure that it operates as planned. The telephone interviewers will receive project-specific training that will include mock interviews to enhance their familiarity with the survey instrument and provide them with practice in administering project-specific protocols.

The bar surveys will utilize methods that the Contractor, PIRE, has conducted on various projects over the years. PIRE has published articles on the use of portal survey techniques, including those listed in Appendix I.

B.5. Provide the name and telephone number of individuals consulted on statistical aspects of the design

The following individuals consulted on statistical aspects of the study design:

Alan Block, MA
Office of Behavioral Safety Research
DOT/National Highway Safety Administration
1200 New Jersey Ave, SE
Washington, DC 20590
(202) 366-6401

John H. Lacey, M.P.H.
Director
Alcohol, Policy, and Safety Research Center
Pacific Institute for Research and Evaluation
11710 Beltsville Drive, Suite 300
Calverton, MD 20705

Julie Yao, Ph.D.

Associate Research Scientist

Alcohol, Policy, and Safety Research Center

Pacific Institute for Research and Evaluation

11710 Beltsville Drive, Suite 300

Calverton, MD 20705

Jerry Karson

Vice-President, Business Development

American Directions Group

1350 Connecticut Avenue, NW

Suite 1102

Washington, DC 20036

1 Blumberg, SJ & Luke, JV. Wireless Substitution: Early Release of Estimates From the National Health Interview Survey, July–December 2010. Centers for Disease Prevention and Control, Division of Health Interview Statistics, National Center for Health Statistics. Released 06/08/2011. Available at:

http://www.cdc.gov/nchs/data/nhis/earlyrelease/wireless201106.pdf

2 See the 2008 National Survey of Drinking and Driving Attitudes and Behavior, Volume 3 Methodology Report, DOT HS 811 344. August 2010, Page 17.

3 Voas, R. B., Wells, J., Lestina, D., Williams, A., & Greene, M. (1998). Drinking and driving in the United States: The 1996 National Roadside Survey. Accident Analysis and Prevention, 30(2),

267‐275.

4 Miller, B.A., Holder, H.D., & Voas, R.B. (2009). Environmental strategies for prevention of drug use and risks in clubs. Journal of Substance Use, 14(1), 19-38.

11