Part B - Evaluation of Impaired Riding Interventions Supporting Statement 6-10-12

Evaluation of Impaired Riding Interventions

Supporting Statement for Information Collection Request

TABLE OF CONTENTS

SUPPORTING STATEMENT

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe the potential respondent universe and any sampling or other respondent

selection method to be used................................................................................................1

2. Describe the procedures for the collection of information.................................................5

3. Describe methods to maximize response rates...................................................................7

4. Describe any tests of procedures or methods to be undertaken..........................................8

5. Provide the name and telephone number of individuals consulted on statistical

aspects of the design...........................................................................................................8

B. Collections of Information Employing Statistical Methods

The National Highway Traffic Safety Administration (NHTSA) and evaluation contractor Pacific Institute for Research and Evaluation (PIRE) plan conduct intercept surveys to examine the attitudes, awareness, and self-reported behaviors of motorcycle riders associated with high visibility enforcement campaigns with the goal of deterring alcohol-impaired motorcycle riding. The high visibility enforcement campaigns will occur in conjunction with motorcycle events that draw a large number of motorcycle riders to an area and that typically feature drinking by attendees, so that the intervention can most effectively reach its target audience of motorcycle riders who drink alcohol.

During the campaigns, law enforcement officers will conduct heightened enforcement of impaired driving in the form of saturation patrols, roving patrols, and sobriety checkpoints on roads leading to the motorcycle events. Media will publicize the increased enforcement of impaired motorcycle riding. This media will include radio advertisements with the tagline “Ride Sober or Get Pulled Over,” and may also include advertisements on the website for the event and on local motorcycling websites, print media displayed in publications read by and at locations frequented by event attendees, news stories, and variable message boards on the roads with increased enforcement activity.

The proposed information collection will employ statistical methods to analyze the data collected from respondents. The following sections describe the procedures for respondent sampling and data tabulation.

B.1. Describe the potential respondent universe and any sampling or other respondent selection to be used.

NHTSA and PIRE plan to evaluate the campaign efforts in four sites: two sites in Indiana, and two sites in Pennsylvania. We plan to evaluate one site in each State that will conduct its campaign in conjunction with local bike nights, which are gatherings of motorcycle riders that occur weekly throughout the motorcycle riding season at bars and restaurants. We additionally plan to evaluate a second site in each State that will conduct its campaign in conjunction with a one-time motorcycle rally, which is a large multi-day gathering of motorcycle enthusiasts. Sites with bike nights will conduct one wave of campaign activity during the summer of 2012. Sites with motorcycle rallies will conduct one wave of activity during the event, also during the summer of 2012. Enforcement activity for bike nights will occur over a one-week period for each wave (i.e., during several consecutive evenings on which there are bike nights), and enforcement activity for motorcycle rallies will occur on days riders are traveling to the event and during the event itself. Media publicizing the enforcement will occur one to two weeks before the events and during the events.

NHTSA and PIRE plan to conduct intercept surveys at bike nights before and after the two waves of activity, for a total of two surveys per site (Pre and Post). Intercept surveys are planned at rallies during the events themselves, and thus one survey will be conducted at each rally. Intercept surveys will additionally be collected at appropriate comparison bike nights and rallies that will not receive an intervention. For bike night sites, pretest measures will be compared to posttest measures to examine if attitudes, awareness, and self-reported behaviors of motorcycle riders that attend bike nights change from before the campaign to after the campaign. For rallies, attitudes, awareness, and self-reported behaviors of motorcycle riders attending the events will be compared to those attending comparison rallies that did not receive an intervention. Because interviews will occur at the events around which the interventions are targeted (i.e., at the bike nights and motorcycle rallies), the potential respondent universe consists of motorcycle riders that attend these bike nights and motorcycle rallies.

In-person intercept surveys were chosen over more traditional methods such as phone and mail surveys for two reasons. First, a very small percentage of Americans are motorcycle riders, and thus it would not be practical to use random-digit dialing to landline phones or with a cell phone sample to reach the intervention’s target audience. Second, the enforcement campaigns will be closely coupled with events that motorcycle riders may travel a distance to attend. Because these will be event-focused interventions rather than community-level interventions, mail surveys sent to registered motorcycle owners who live in the communities surrounding the events may not reach many or even most members of the target audience.

The planned event sites and their planned comparison event sites are listed in the table below.

Planned Sites
	Rally (One Time Event)		Bike Nights (Recurring)
	Indiana	Pennsylvania	Indiana	Pennsylvania
Intervention	ABATE Boogie Springville, IN July 19-22, 2012	White Rose Thunder York, PA September 27-30, 2012	Marion County, IN July-August, 2012	Allegheny County, PA July- August, 2012
Comparison	Annual Thunder Beach Motorcycle Rally Panama City Beach, FL October 10-14, 2012	Ray Price Capital City Bikefest Raleigh, NC September 21-23, 2012	Knoxville, TN July-August 2012

The comparison rallies are expected to have slightly higher attendance than their corresponding intervention rallies. Attendance at the ABATE Boogie is expected to be 10,000-12,000 motorcycle riders, and White Rose Thunder expected 35,000 attendees; the Thunder Beach Motorcycle rally expects 65,000 attendees, and the Ray Price Capital City Bikefest expects 85,000 attendees. Crucially, however, each intervention rally is of a similar character and attracts similar types of motorcycle riders to its planned comparison rally, which we feel are the most critical similarities for the rallies to have. The ABATE Boogie and the Thunder Beach Motorcycle Rally both have a party atmosphere, while White Rose Thunder and the Ray Price Capital City Bikefest both take place in a city’s downtown area and expect to attract families of enthusiasts. All four rallies feature bands, organized motorcycle rides, vendors, and other events. The comparison rallies are geographically distant from the intervention rallies to ensure that riders at the comparison rallies are not exposed to the media for the intervention, as part of the media strategy involves advertising on the internet to motorcycle riders who live in the states surrounding the intervention rallies. Bike nights in the planned intervention and comparison sites typically occur weekly at numerous bars and restaurants in the areas, during several nights of the week. Large bike nights may attract several thousand motorcycle riders.

Pennsylvania and Indiana both have a demonstrated motorcycle safety problem, and the summer months during which the high visibility enforcement campaigns are planned are when the largest numbers of crashes are seen in both States. In 2010, there were 109 fatal motorcycle crashes in Indiana and 212 fatal motorcycle crashes in Pennsylvania (NHTSA, 2012). Indiana had the 12^th-highest number of fatal motorcycle crashes in the Nation, and Pennsylvania had the 4^th-highest number of fatal motorcycle crashes. Sixty-one percent of the fatal motorcycle crashes in Indiana occurred from June-September, and 63% of the fatal motorcycle crashes in Pennsylvania occurred from June-September. In 2009, 34% of fatally injured motorcycle riders in Indiana had a blood alcohol concentration (BAC) of .08 and above, which is per se evidence of alcohol impairment, and 31% of fatally injured motorcycle riders in Pennsylvania had a BAC of .08 and above (NHTSA, 2009); both States had higher rates of fatally injured motorcycle riders with a BAC of .08 and above than the National average of 30%.

Data on the demographic characteristics of motorcycle riders that attend these events, and on the local level for riders that live in the communities where these events occur, are not available. However, demographic characteristics of American motorcycle riders (i.e., people who reported riding a motorcycle in the past month) can be estimated from the 2009 National Household Travel Survey (Federal Highway Administration, 2009). These estimates and corresponding estimates of the number of riders in the sample that will fall into each age range appear in the table below.

Expected Population and Sample Distribution by Age
	Percentage of Motorcycle Riders	Expected Distribution of Sample (n)
Total		5,040
< 21	2.4%	121
21-35	19.2%	968
36-45	29.6%	1,492
46-55	27.6%	1,391
56-65	17.6%	887
66+	3.6%	181

Of individuals who reported riding a motorcycle in the past month in the 2009 National Household Travel Survey, 89.4% were male and 10.6% were female (Federal Highway Administration, 2009). If similar gender proportions were found in the sample for this study, 4,506 male riders and 534 female riders would be interviewed.

We will not attempt to characterize the larger population of American motorcycle riders from these results, and thus we do not plan to apply weighting to the sample. Resulting publications will include a caveat that data were collected from a convenience sample of volunteers, at specific events in specific locations, and that the results cannot be generalized to the population of American motorcycle riders. We plan instead to use the findings to describe a snapshot of attitudes, awareness, and self-reported behaviors from the sample of motorcycle riders who attend the specific events at which the high visibility enforcement campaigns will be directed.

Response Rates:

Response rates in past in-person intercept interview studies have been high. Billheimer (1996) performed in-person interviews with motorcycle riders in sites where riders congregate, such as at motorcycle trade shows, with an 88% response rate. Similar response rates were found in prior intercept interview studies of college students on their drinking behavior (88% of students approached by Foss, Marchetti, & Holladay, 2001; 92% of party hosts approached by Clapp, Holmes, Reed, Shillington, Freisthler, & Lange, 2007), of electronic music dance event attendees on drug use (82 to 95%: Gripenberg-Abdon et al., 2012; Voas et al., 2006), of young adults returning to the US after drinking in Mexico on their drinking behavior (90%), and of partygoers on risky behavior (78%; Ramchand, Becker, Ruder, & Fisher, 2011). Several of these studies provided small incentives worth $5-25 to respondents in order to achieve these high response rates (Gripenberg-Abdon et al., 2012; Ramchand et al., 2011; Voas et al., 2006), which is similar to the incentive we plan to offer to respondents to ensure a comparable response rate.

Total Sampling Needs:

Overall, the total sample needs for the interviews are 5,040 respondents over the course of one year. Because the events at which we plan to conduct surveys have very high motorcycle traffic (see page 2), it will be possible to complete the necessary number of interviews. Since the interviews are estimated to be 5-10 minutes in length, with an average of 8 minutes in length, the estimated annual time is 672 hours. The breakdown of the total number of respondents appears in the tables that follow.

Sample Size, Broken Down By Planned Site
Pilot Testing
			40
Bike Nights
	Wave
	Pre	Post 1	Total
IN Bike Nights (Intervention)	500	500	1,000
PA Bike Nights (Intervention)	500	500	1,000
Bike Nights Comparison	500	500	1,000
Rallies
		During Event
IN Motorcycle Rally (Intervention)		500	500
Motorcycle Rally Comparison #1		500	500
PA Motorcycle Rally (Intervention)		500	500
Motorcycle Rally Comparison #2		500	500
Total			5,040

Total Burden
	Annual Sample	Minutes	Total Hours
Total	5,040	x 8	672

B.2. Describe the procedures for the collection of information.

The intercept surveys will be used in a Non-Equivalent Groups Design (NEGD), which is frequently used by NHTSA evaluators to evaluate high visibility enforcement campaigns, such as the Click It or Ticket seat belt campaign, the Over the Limit Under Arrest and Drive Sober or Get Pulled Over impaired driving campaigns, and the Phone In One Hand, Ticket In the Other distracted driving campaign. For planned sites with recurring bike nights the design is structured like a pretest-posttest randomized experiment, but lacks random assignment. The intercept interviews will examine the changes that occur as a result of specific impaired riding interventions. The interviews will be conducted on a schedule corresponding with the fixed dates for the impaired riding campaigns that will depend on the timing and sequencing of the components of each demonstration project. The pretest survey will be conducted prior to the commencement of campaign activities, and the posttest survey will occur after the enforcement wave. Pretest and posttest responses from the bike night intervention and comparison sites will be compared to determine if there are any significant differences that can be attributed to the impaired riding demonstration program activities.

For planned sites with one-time motorcycle rallies, it will not be possible to collect pre-event interview measures. Riders at these events may travel to attend and cannot be located prior to the event, and the events occur infrequently enough (annually) that we cannot collect baseline data at a prior year’s event before the HVE campaign that will launch this summer. Thus, interviews in these sites will follow a test-comparison design instead of a pre-post design. Interviews will be conducted at the motorcycle rallies, and results will be compared to those from interviews conducted at appropriate comparison rallies that did not experience interventions. These comparison rallies are matched to the intervention rallies by size and character of the events (e.g., types of activities that occur during the events). In these cases, we will not be measuring the change in behavior, awareness, and perception before and after the event, but rather will be comparing a snapshot of riders at a site that did experience an intervention to a snapshot of riders at a site that did not.

A sample of N = 500 motorcycle riders will be recruited from each intervention and comparison site for each survey administration. Previous NHTSA research indicates that this sample size has sufficient power to detect pre-post changes in intervention awareness surveys.

For bike night sites, each intercept survey administration will occur over three consecutive nights when bike nights occur (e.g., Wednesday, Thursday, and Friday nights). Data collection will occur outside of bars and restaurants during the establishments’ bike nights. Permission will be gained from property managers or owners of the establishments. NHTSA and PIRE will use several methods to obtain the cooperation of these establishment managers and owners. To the extent possible, we will elicit help from local agencies so that we can emphasize that we are working with local people when approaching establishments. Additionally, we plan to use gift certificates or gift cards to the establishments as incentives for participation to survey respondents. We will work to find ways to prevent the incentive from being used for the purchase of alcohol (e.g., obtaining gift certificates good for food only).

Data collection for motorcycle rally sites will occur during the events themselves, and will take place inside of the events. We will attempt to gain cooperation of the event organizers with the same methods used for bike nights, which include approaching event organizers along with local community representatives, and using gift certificates or gift cards to vendors that will be at the event as incentives to survey respondents. If it is not possible to gain the cooperation of event organizers to survey at the restaurants and bars that conduct bike nights and at the motorcycle rallies, alternative strategies for data collection include collecting data outside of or otherwise nearby motorcycle rallies instead of inside of them, and/or collecting data at other locations in the area where large number of motorcycle riders are expected to congregate (e.g., motorcycle dealerships and accessory shops, gas stations, alternative restaurants and bars).

We will ensure that data collection occurs at similar locations in intervention sites and comparison sites; that is, if data needs to be collected at establishments outside of the rally for a rally site, we will collect data at similar establishments in the associated comparison site. Representatives from NHTSA and PIRE will visit bike night locations prior to conducting the pretest surveys to assess motorcycle traffic and other characteristics of the establishments that may affect data collection.

Survey Procedure

Five data collectors will conduct surveys in seven-hour shifts over three consecutive days in each site. Bike night data collection will occur concurrently with the hours of the bike nights, which will take place in the evening hours (e.g., 5 p.m. to midnight). Rally data collection hours will be more flexible because the events last all day; however, data collection hours will be the same at intervention rallies and their associated comparison rallies. For bike nights, the data collectors will collect data at a minimum of three establishments per survey administration (i.e., a different establishment each night). If it is determined that the motorcycle traffic at a single bike night would not be sufficient for the necessary number of completed surveys, data collectors will plan to attend multiple bike nights in a single evening.

The data collectors will employ a multi- step process to survey riders: (1) interception, (2) recruitment, (3) determining eligibility, and (4) completion of interview. Data collectors stationed at each selected site will be responsible for approaching and holding the initial contact with potential participants. Interviewers will be stationed in a single location throughout their shift, which is planned to be outside of bars and restaurants for bike nights, and a location inside the event for rallies. The interviewers will intercept individuals passing the position where they are stationed. To obtain 500 completed interviews during each survey administration, the goal of each interviewer will be to complete a minimum of 5 interviews per hour (3 days x 7 hours x 5 interviewers x 5 interviews per hour = 525 interviews).

Upon approaching a potential participant, the interviewer will introduce him or herself and provide a brief explanation of the study. Following the initial interception the interviewer will administer a set of one or two screening questions (see Attachment C—Survey). The objectives of these screening questions are to determine eligibility. Each question will be read to the participant and the interviewer will record the participant’s responses on the questionnaire form. A rider will be eligible to complete the interview if (1) they have operated a motorcycle on the road in the past 30 days, (2) there are no language barriers, and (3) they are 18 and older, so that they can give proper consent. For locations in which interventions occur in conjunction with a special event, a rider must also have attended the event to be eligible.

Thus, one screening question (“Are you 18 or over, and have you operated a motorcycle on the road in the past 30 days?”) will be administered to riders in program locations in which the intervention does not occur in conjunction with a special event, and in corresponding comparison locations. Two screening questions (“Are you 18 or over, and have you operated a motorcycle on the road in the past 30 days?” and “Are you attending [name of event]?”) will be administered to riders interviewed in locations in which the impaired riding intervention occurs in conjunction with a special event, and in corresponding comparison locations.

Regardless of the eligibility determination, a “disposition” code will be entered onto each form to indicate the results of the screening. Examples of disposition codes are:

• Ineligible Due to Interview Responses

• Eligible Due to Interview Responses

• Refused

• Ineligible Due to Language Barrier

• Ineligible Due to Other (specify other)

Once the eligibility of the rider has been determined, the data collector will administer the remainder of the interview to eligible participants, or thank the respondent and end the interview with ineligible participants. The remainder of the interview will follow a pre-determined script (provided in Attachment C--Survey).

B.3. Describe methods to maximize response rates.

PIRE will hire interviewers who are also motorcycle riders, and who will wear motorcycle gear when conducting interviews. This should maximize response rates by making potential respondents comfortable with the interviewers.

Respondents will also be provided with a small incentive for their participation worth approximately $10 to bolster response rates. When possible, the incentive will be a gift card or gift certificate to the bar where the survey will occur, or to a vendor at the motorcycle rally where the survey will occur. We will ensure that we will only distribute gift certificates that cannot be used to purchase alcohol (e.g., gift certificate good for food only if for a restaurant).

Because we are conducting surveys at locations where motorcycle riders congregate to enjoy themselves, such as bars and motorcycle rallies, we acknowledge that potential respondents may need a greater incentive to draw themselves away from social activities and participate than they might if the surveys were conducted in a different, less social environment. The value of the incentive we propose is analogous to those used to bolster response rates in other similar intercept interview surveys conducted at social venues. For example, previous surveys lasting 5-15 minutes that interviewed individuals who were attending or traveling to events such as bars, dance clubs, and festivals have offered incentives worth $5-$25 for participation (Gripenberg-Abdon et al., 2012; Johnson & Clapp, 2011; Kelley-Baker, Mumford, Vishnuvajjala, Voas, & Romano, 2008; Ramchand et al., 2011; Voas et al., 2006).

NHTSA does not intend to assess non-response bias in this evaluation project. Generally, only a small proportion of non-respondents are willing to participate in a non-response follow up interview. We will track the number of refusals in each site and the rider’s approximate age and gender by observation. As noted in section B.1., response rates in past similar intercept interview studies with motorcycle riders and of drinking and drug use behavior have been above 80%.

B.4. Describe any tests of procedure or methods to be undertaken.

The length of the survey was tested with several employees of the contractor conducting the study, PIRE. PIRE employees took between 5-10 minutes to complete the interview.

NHTSA plans to conduct a pilot of the interview methodology and questions with approximately 40 motorcycle riders. We plan to conduct the pilot test at a bike night in one bar in the Washington, DC area in either the last week of June or first week of July 2012, before the survey is debuted in the field in Indiana and Pennsylvania on July 10^th. Any problems encountered during the pilot testing of the interviews will be resolved before they are put into the field.

B.5 Provide the names and telephone numbers of individuals consulted on statistical aspects of the design.

Jessica Cicchino, Ph.D.

Research Psychologist

National Highway Traffic Safety Administration

1200 New Jersey Ave SE, W46-491

Washington, DC 20590

(202) 366-2752

[email protected]

Alan Block, M. A.

Research Psychologist

National Highway Traffic Safety Administration

1200 New Jersey Ave SE, W46-499

Washington, DC 20590

(202) 366-6401

[email protected]

Maria Vegega, Ph.D.

Chief, Occupant Protection Division (former Chief, Behavioral Research Division)

National Highway Traffic Safety Administration

1200 New Jersey Ave SE, W44-302

Washington, DC 20590

(202) 366-2668

[email protected]

John Lacey, MPH

Center Director, Alcohol, Policy, and Safety Research Center

Pacific Institute for Research and Evaluation

11720 Beltsville Drive, Suite 900

Calverton, MD 20705

(301) 755-2779

[email protected]

Richard Blomberg, M.S.

President

Dunlap and Associates, Inc. (subcontractor to Pacific Institute for Research and Evaluation)

110 Lenox Avenue

Stamford, CT 06906

(203) 323-8464

[email protected]

ii