Section B OMB 2127-0665 Application 2 27 2014

Table of Contents

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe the potential respondent universe and any sampling or other respondent

selection method to be used...............................................................................................2

2. Describe the procedures for the collection of information............................................... 10

3. Describe methods to maximize response rates.................................................................15

4. Describe any tests of procedures or methods to be undertaken.........................................16

5. Provide the name and telephone number of individuals consulted on statistical

aspects of the design..........................................................................................................17

SECTION B

B) Collections of Information Employing Statistical Methods

The National Highway Traffic Safety Administration (NHTSA) is seeking approval for the Distracted Driving Intercept Survey (DDIS) to measure public awareness, attitudes, and self-reported behaviors regarding distracted driving and interventions to reduce distracted driving.

The Distracted Driving Intercept Survey (DDIS) will measure attitudes and awareness of distracted driving messaging and enforcement before and after specific local high-visibility enforcement mobilizations, during which enforcement officers will employ new strategies to enforce texting bans. The high-visibility enforcement programs will use earned media, targeted enforcement, and visible enforcement on the roads to increase drivers’ perceived risk of being pulled over and ticketed for texting while driving.

This survey will be conducted in program and control sites with samples of 500 or 1,000 licensed drivers, aged 18 years and older. The sample size will depend on the measurement period. To increase statistical power, more surveys will be conducted for the very first and last measurements than for the remaining measurements (i.e., 1,000 for the first and last measurements and 500 for the remaining measurement periods). Participants will be recruited at driver licensing offices in Connecticut and Massachusetts. The paper and pencil survey will be administered while drivers are waiting to have the photograph developed at the last stage of a license renewal procedure.

One reason why Connecticut and Massachusetts were chosen as sites was because they have primary enforcement laws banning texting for all drivers, a prerequisite for participation in the enforcement campaign. Both States also have a variation of a primary hand-held ban. Connecticut has a primary hand-held ban for all drivers, while Massachusetts has a primary hand-held ban for junior operators.

The program and control locations for each State are as follows:

B.1 Describe the potential respondent universe and any sampling or other respondent selection method to be used.

NHTSA plans to use the Distracted Driving Intercept Survey (DDIS) to measure changes in public awareness before and after a distracted driving high-visibility enforcement program. The potential response universe for the DDIS is licensed drivers 18 years and older drawn from people visiting the Department of Motor Vehicles (DMV) offices in the program and control communities in the participating States, Connecticut and Massachusetts.

As typical with program evaluations of this nature, survey administration will follow a Non-Equivalent Groups Design. The intention of this design is to measure the effect of a program by taking a yardstick measurement before and after the program to determine change. With strong consistency in measurement protocol across measurement periods and similarity across the program and control samples to limit extraneous influences on the results, this design can produce a non-biased and reliable indication of change.

Utility of this evaluation does not necessitate administering to a probability based sample and generating representative results. The objective of the evaluation is to take a reliable measure of change in public awareness, not to identify what represents the community as a whole.

The design has strong consistency in measurement and sampling protocol across measurement periods and includes built in methods for addressing any inherent differences between the program and control samples. The Non-Equivalent Groups Design is susceptible to the threat of internal validity, as the group of respondents in the program and control areas may have been characteristically different prior to the intervention, possibly contributing to differences in response independent of the intervention effect. Ideally with Non-Equivalent Groups Designs, the two groups (program and control) would be characteristically similar prior to the intervention, just differ by intervention exposure, to produce results that reflect the influence of the intervention. However, with some programs, the evaluator has less control over site selection and must adapt to the realities of the situation, including any differences in the program and control samples. For the current project, any inherent differences across the program and control samples will be accounted for in the error term and included as covariates in the analyses.

While Non-Equivalent Groups Design lacks random assignment, inherent characteristics of the sample universe and rigorous protocols can reduce bias in the sample. In both States, all licensed drivers must visit the DMV periodically for license renewal. The date of license renewal is based on birthday; therefore, it is reasonable to treat customers renewing their license on a given day as a random sample of the population of all available respondents (i.e., licensed drivers in State). Younger drivers (obtaining a license for the first time) may be overrepresented, but the vast majority of these will be under 18 years-old and excluded from participation. In most cases all visitors to the office will be approached. In cases where the office is too busy to approach all customers, data collectors will apply a systematic sampling interval to approach either every 2^nd or 4^th visitor, depending on the estimated office traffic flow and target sample for the day.

The systematic sampling interval (every 2^nd or 4^th) was determined using the formula:

N (population of people who visit the DMV per day)

n (survey sample per day)

The Contractor visited the DMVs to get a feeling for the traffic flow in the participating offices. They estimated that about 400 people visit the DMVs per day. Our n will vary across the waves, being 1,000 for the very first and last measurements and 500 for the remaining measurements. The surveys for each measurement period will be administered over a 5 day period. For the measurement periods with 1,000 surveys, 200 surveys will be administered per day and for the measurement periods with 500 surveys, 100 will be administered per day.

For the measurement periods administering 200 surveys per day:

400 (N) / 200 (n) = a systematic sampling interval of 2

For the measurement periods administering 100 surveys per day:

400 (N) / 100 (n) = a systematic sampling interval of 4

Table 1. Demographics of Test and Control Sites.¹

Table 2. DDIS Sample Size

	WINTER 2014		SUMMER 2014		Total
Connecticut	Wave 1		Wave 2
	Pre	Post	Pre	Post
Program Site	1,000	500	500	1,000	3,000
Control Site	1,000	500	500	1,000	3,000
Total	2,000	1,000	1,000	2,000	6,000

	SUMMER 2014		FALL 2014		Total
Massachusetts	Wave 1		Wave 2
	Pre	Post	Pre	Post
Program Site	1,000	500	500	1,000	3,000
Control Site	1,000	500	500	1,000	3,000
Total	2,000	1,000	1,000	2,000	6,000

Connecticut

The New London County area was chosen as the control site because 1) it is demographically similar to the Fairfield County area, which helps control for response differences due to the population rather than exposure to the intervention, 2) it is in a separate media market, which helps reduce media bleed into the control area, and 3) they agreed to participate.

While this program will only have earned media and not paid, it is still valuable to consider potential media bleed into the control area in the event that the earned media makes it over the media channels. There are four major media markets in Connecticut: Fairfield, New Haven, Middlesex, and New London. The program area is in the Fairfield media market and the control area is in the New London media market, so those in the control area will likely not be exposed to the earned media.

Figure 1. Connecticut: The program area in Fairfield County (left) and control area in New London County (right)

Program Area

Control Area

NHTSA proposes to administer surveys at the motor vehicle division offices in the Fairfield County (program) and New London County (control) areas. As is identified by the blue pinpoints in Figure 1, this includes one office in each area. While there is only one office in each area, an examination of the availability of offices in the surrounding areas suggests that most people in the area will need to travel to these offices for services. In addition, our evaluation contractor contacted these offices to inquire about the magnitude of typical office traffic. The office staff reported that typical office traffic is high, providing further evidence of local dependence on these offices for services.

To ensure our respondents are from the participating areas, we are including a question on the awareness survey asking respondents to identify their town or city of residence. By making this determination, we will be able to analyze respondents who are from the areas of interest separately from those who are from other areas.

While Connecticut does offer some services online, all drivers have to visit one of the offices every 6 years to renew their license and update their driver license photo.² Given this requirement, all licensed drivers in the participating locations have a chance to be surveyed (assuming the license renewal dates are randomly distributed throughout the year based upon birthdate).

Massachusetts

The Hampden County area was chosen as the control site because they 1) are geographically distant from the Essex and Middlesex areas, 2) are in a separate media market, and 3) agreed to participate. Having the program and control areas in separate media markets helps control for media bleed into the control area. While this program will only have earned media and not paid, it is still valuable to consider potential media bleed into the control area in the event that the earned media makes it over the media channels. There are two major media markets in Massachusetts, one surrounding the Boston area and one surrounding the Springfield area. The program area is in the Boston media market and the control area is in the Springfield media market, so those in the control area would not likely be exposed to the earned media.

Figure 2. Massachusetts: The program area in Essex and Middlesex Counties (right) and control area in Hampden County (left)

Program Area

Control Area

NHTSA proposes to administer surveys at all of the Massachusetts registry of motor vehicle (RMV) offices in the participating areas, including three in the Essex and Middlesex areas (program) and two in the Hampden County area (control). Office locations are indicated by blue pinpoints in Figure 2. In both cases, the offices are spread across the participating areas making it less likely for the sample to be biased. This distribution helps achieve external validation of the measures because any differences across sites can be observed.

While Massachusetts does offer some services online, all licensed drivers have to visit one of the offices every 10 years to renew their license and update their driver license photo.³ Note that drivers in Massachusetts must renew their license every 5 years, but may do so online every other year, only requiring them to go into the office every 10 years. Given that we are proposing to administer surveys at all offices in the participating areas, all licensed drivers in those locations have a chance to be surveyed (assuming the license renewal dates are randomly distributed throughout the year based upon birthdate).

B.1.1 Total Sampling Needs

Overall, the total sample needs for the interviews are 12,000 respondents over the course of two program waves. The interviews are estimated to run 5 minutes⁴ in length, which would be a total public burden of 1,000 hours (see Table 3).

Table 3: Total Sample and Burden

	Total Sample (2 waves)	Minutes	Hours
Connecticut	6,000	x 5	500
Massachusetts	6,000	x 5	500
Total	12,000		1,000

B.2. Describe the procedures for the collection of information.

The surveys will be used in a Non-Equivalent Groups Design (NEGD), which is frequently used by NHTSA evaluators to evaluate special media and enforcement mobilizations, such as Click It or Ticket. The design is structured like a pretest-posttest randomized experiment, but lacks random assignment. The surveys will be administered before and after each program wave (see Figure 3). The surveys will be administered simultaneously in the program and control areas.

Figure 3: Illustration of survey placement to measure change.

Wave 1

Wave 2

The research questions for the evaluation of the Distracted Driving Demonstration program, which the DDIS data collection seeks to answer, include the following:

Research Question 1: Did drivers see and hear NHTSA’s High Visibility Enforcement messages?

• 16. Have you recently read, seen or heard any messages about the enforcement of texting laws in Massachusetts? [] Yes [] No

  • If yes, where did you see or hear about it? (Check all that apply):

[] Newspaper [] Radio [] TV [] Billboards [] Brochure [] Online [] Police Enforcement [] Other

If yes, what did it say? _____________________________________

• 17. Do you know the name of any distracted driving program(s) in [State]? (check all that apply)

  • List of distracted driving slogans will be provided

Research Question 2: Did the program change drivers’ perceptions or awareness of distracted driving enforcement or their perceived risk of getting a ticket for driving distracted?

• 10. Do you think that it is important for police to enforce distracted driving laws?

  • [] Yes [] No

• 11. What do you think the chances are of getting a ticket if you compose, send, or read an electronic message on a mobile telephone or other hand-held device while driving?

  • [] Very likely [] Somewhat likely [] Neutral [] Somewhat unlikely [] Very unlikely

• 12. Do you think the texting law in [State] is enforced?

  • [] Very strictly [] Somewhat strictly [] Neither strictly nor loosely [] Somewhat loosely [] Very loosely

• 13. Have you received a ticket for composing, sending, or reading an electronic message on a mobile telephone or other hand-held device while driving?

  • Ever: [] Yes [] No In the past month: [] Yes [] No

• 14. Have you received a ticket for distracted or inattentive driving?

  • Ever: [] Yes [] No In the past month: [] Yes [] No

• 15. In the PAST MONTH, have you seen police on the roads you normally drive?

  • [] More than usual [] About the same [] Less than usual [] Never see them

Research Question 3: Did self-reported cell phone use and texting while driving change?

• 8. How often do you talk on a hand-held cellular phone when you drive?

  • [] Always [] Nearly always [] Sometimes [] Seldom [] Never

• 9. How often do you compose, send, or read electronic messages on a mobile telephone or other hand-held device when you drive?

  • [] Always [] Nearly always [] Sometimes [] Seldom [] Never

Logistic regression will be used to compare pre and post intervention responses from the program and control sites to determine if there are any significant differences that can be attributed to the distracted driving demonstration program activities. For questions with only two response choices, the two choices will act as the binary dependent variable in the analysis. For questions that have multiple responses, responses will be re-coded into two categories: 1) response for the first selection choice, or the top-box (e.g., very likely, always) will be coded as “1” and 2) response for remaining selection options will be coded as “0.” For example, if the respondent selected “always” as their response to question 9, they would be coded as a “1,” but if they selected any of the remaining responses, such as “nearly always,” they would be coded as a “0.” This approach will predict the probability of the participant responding either “very strictly” or “always” to any of the other responses. For all analyses the independent variable will be intervention exposure (exposed to program activity or not exposed to program activity).

B.2.1. DDIS Project Samples

A sample of 1,000 drivers will be recruited from Department of Motor Vehicle (DMV) offices for the first and last survey measurements in both the program and control sites (pre wave 1 and post wave 2). A sample of 500 drivers will be recruited for the remaining measurement periods for the program and control sites (post wave 1 and pre wave 2). For each survey question, we will examine the how the change across the intervention period differs between the program and control area, with the hypothesis that the program area respondents would exhibit a greater change in awareness than those in the control area. A power analysis confirms that the sample size will be sufficient to detect a change of 9.6 percentage points. Previous NHTSA research indicates that this sensitivity is sufficient.

Data Weighting

The data will be weighted to reflect the demographic makeup of each geographic location. The weighting process for this study essentially entails two major steps. In the first step, target population benchmarks will be created for computation of weight factors. For this purpose we will rely on public data sources such as Current Population Survey (CPS) or American Community Survey (ASC) as well as commercial sources such as Claritas to obtain demographic profiles of adults in each geographic location. In the second step, an iterative proportional fitting procedure will be used to balance the composition of respondents in each location to their respective demographic profiles obtained during the first step. It is anticipated that weight adjustments will include characteristics such as gender, age, race, and ethnicity.

Sampling Protocol

The Distracted Driving Intercept Survey (DDIS) will be administered at Department of Motor Vehicles (DMV) offices in the program and control areas in participating States, Connecticut and Massachusetts. When office traffic to too heavy to sample all visitors, we will apply a systematic sampling interval by sampling every 2^nd or 4^th visitor, depending on the target sample for the day (see Section B.1). This study uses a Non-Equivalent Groups Design and therefore does not have true random selection. However, the design is internally balanced and efforts will be made to correct for any non-response bias or any inherent differences across the program and control samples.

The DDIS data collectors will employ a multi- step process to survey drivers: (1) interception, (2) determining eligibility, (3) recruitment, (4) completion of questionnaire, and (5) on-site data quality procedures.

Upon approaching a potential participant, the screening interviewer will introduce him or herself and give a brief explanation of the study following a pre-determined script for this initial contact. Following the initial interception, the interviewer will administer a brief screener questionnaire (see Appendix C). The objectives of this questionnaire are to determine eligibility for participation in the survey. The screening interviewer will read each question to the participant and record the participant’s responses on the questionnaire form. Drivers will be eligible to complete the questionnaire if 1) they are a licensed driver, 2) they are 18 and older, and 3) there are no language barriers. A Spanish translation of the survey will be available.

Regardless of the eligibility determination, the interviewer will enter a “disposition” code onto each form to indicate the results of the screening. Examples of disposition codes are:

• Ineligible Due to Questionnaire Responses

• Eligible Due to Questionnaire Responses

• Refused Screener

• Eligible Due to Questionnaire Responses but Refused Main Questionnaire

• Ineligible Due to Language Barrier

• Ineligible Due to Other (specify other)

A tracking number will be assigned to each driver intercepted. This number will serve as the unique identifier that links the responses on the screener questionnaire to responses on the main questionnaire (see Appendix C).

Once the eligibility of the driver has been determined, the interviewer will endeavor to recruit eligible participants to complete the main questionnaire. In general, this will not be a scripted dialog, but the team member will cover key elements, which include additional details on the study, and an estimated time for completion (it is anticipated that the main questionnaire will take less than 5 minutes to complete). If the screening interviewer is successful in recruiting the driver, he or she will direct the participant to a questionnaire administrator waiting at a nearby table. One assistant will be fluent in Spanish to assist in interviewing Spanish-speaking participants.

The questionnaire administrator is responsible for all aspects related to the completion of the main questionnaire. This consists of: (1) receiving the eligible drivers identified by the Screening interviewer at a pre-determined location, (2) transferring the unique identifier from the screener form to a main questionnaire form, (3) directing the drivers on how to complete the form and what to do with it once they are finished, and (4) answering questions from participants in the process of completing a questionnaire.

After drivers complete the questionnaire, the interviewer will review the screening and questionnaire forms for completeness and accuracy. This review will ensure that the respondent did not inadvertently miss survey questions. If there are missing questions, the interviewer will attempt to question the respondent to obtain the response.

B.3) Describe methods to maximize response rates and to deal with issues of non-response.

The choice of driver licensing offices as the venue for the DDIS was to maximize response rates. Driver licensing offices are typically places where people have to wait and participants are likely to be free to take a survey, and perhaps would welcome the distraction. Furthermore, given that the criteria for selection includes being a licensed driver, we expect a higher percentage of individuals approached in this venue will be licensed drivers compared to other public venues such as shopping malls or movie theaters.

For a previous survey conducted at DMVs in Connecticut, NHTSA had a 69% response rate (i.e., 735 drivers were approached and 510 agreed to participate). Because NHTSA is administering this survey in Connecticut and Massachusetts, similar response rates may be present in the sample for this survey. NHTSA does not see any way to address non-response bias in this context, but will be weighting the data to the population to address any biases in the sample.

B.4) Describe any tests of procedures or methods to be undertaken.

NHTSA does not intend to conduct a pilot study for the DDIS and eligibility screener. The proposed intercept survey is identical to the intercept survey used in Hartford, Connecticut and Syracuse, New York in NHTSA’s distracted driving community level demonstration programs.⁵

B.5) Provide the name and telephone number of individuals consulted on statistical aspects of the design.

The following individuals have reviewed technical and statistical aspects of procedures that will be used to conduct the Distracted Driving Intercept Survey (listed alphabetically):

McKenzie Ballou, Ph.D. Co-Principal Investigator

(215) 790-8900 Ext. 126 M. Davis and Company

[email protected] 3000 Market St., Suite 202

Philadelphia, PA 19104

Richard Compton, Ph.D. Office of Behavioral Safety Research (NTI-130)

(202) 366-2699 National Highway Traffic Safety Administration

[email protected] 1200 New Jersey Ave SE, W46-304

Washington DC, 29590

Mansour Fahimi, Ph.D. Senior Statistician/Weighting

240-565-8711 13416 Bonnie Dale Drive

[email protected] North Potomac, MD 20878

Mary Hinch, M.A. Office of Behavioral Safety Research (NTI-132)

(202) 366-5595 National Highway Traffic Safety Administration

[email protected] 1200 New Jersey Ave SE, W46-466

Washington DC, 29590

Thomas R. Sexton, Ph.D. Senior Statistician

(631) 632-7181 Stony Brook University

[email protected] 317 Harriman Hall             

Stony Brook, NY 11794-3775

Maria Vegega, Ph.D. Office of Impaired Driving and Occupant Protection (NTI-112)

(202) 366-2668 National Highway Traffic Safety Administration

[email protected] 1200 New Jersey Ave SE, W44-302

Washington DC, 29590

1 Source: U.S. Census, http://factfinder2.census.gov/faces/nav/jsf/pages/searchresults.xhtml?refresh=t

2 See http://www.ct.gov/dmv/taxonomy/v4_taxonomy.asp?DLN=46899&dmvNav=|46899|

for list of available online services.

3 See http://www.massrmv.com/OnlineServices.aspx for list of available online services.

4 This is an estimate from a similar survey administered for the Hartford/Syracuse evaluation.

5 http://www.distraction.gov/download/research-pdf/508-research-note-dot-hs-811-845.pdf and http://www.distraction.gov/download/research-pdf/High-Visibility-Enforcement-Demo.pdf

1