Form 1127 Cognitive Testing Report

OMB NO. 2127-xxxx

Expiration date xx, xx, xxxx

Attachment F

Cognitive Testing Report

NHTSA National Survey of Speeding Attitudes and Behavior: findings from the cognitive interview pretest

1.0 Introduction

To pretest the NHTSA Speeding draft questionnaire, two experienced interviewers conducted nine cognitive interviews at Abt Associates’ Bethesda Cognitive Testing Laboratory (CTL) with licensed drivers. There was a mix of respondents by age, education, and gender. All respondents signed an informed consent form and were paid $75 in appreciation for their participation in the pretest.

The cognitive interview protocol consisted of a description of the cognitive interviews’ general objective to identify question flaws that may affect the validity or reliability of answers, instructions to the respondent, and guidelines for the cognitive interviewer. Respondents were asked to think aloud during the interview, saying what they were thinking as they answered the survey questions and to also volunteer any additional comments about the clarity or other aspects of the questions. In addition, the interviewers asked follow-ups to some of the survey questions to determine details about the response process, and to check on the presence of potential problems noted when reviewing the draft instrument.

2.0 Findings

This report describes the findings from the pretest. Section 2.1 discusses some issues that potentially affect more than one question in the survey. These issues came up in multiple cognitive interviews and should be addressed before finalizing the questionnaire. Section 2.2 is a question-by-question summary of findings. These provide additional information about the general findings and report on issues that may affect particular questions. Only those questions for which there is information to report are included. Questions which are identically structured as problematic questions are also excluded.

2.1 General findings

This section summarizes the main problems with the survey that should be addressed. For problems that affect multiple questions, additional detail is provided in the following question-by-question findings.

In general, respondents did not have major difficulties with the survey. The main debriefing comments concerned the interview length and some difficulties with the response scales. The only questions respondents thought that some people might not answer honestly concerned the use of cell phones while driving. The general issues that should be addressed concerned some definitions, the different views of what constitutes speeding, and some of the response scales.

2.1.1 Definition of a week

Some questions use a reference period of a week. Respondents came to different conclusions about whether those questions meant a seven-day calendar week or a five-day work week. As a result, some respondents reported using a work week, while others used a calendar week. Also, when uncertain which definition was intended, a few respondents seemed to settle on work week because it was easier to answer.

2.1.2 Types of roads

Respondents understand the differences between types of roads when a definition is provided. However, in the absence of a definition, they seem to reach different interpretations of what is intended. For example, respondents do not have well-defined ideas of what is a “city or town” street, which is understandable because having both city and town in the same description defines a very broad category. A city might be Rockville or Chicago, and towns may encompass anyplace from a small rural village or large suburban centers. If “city or town” is meant to exclude certain types of roads, it probably only excludes rural roads and highways outside any clearly urban area. Given this latitude, it is not surprising that almost all respondents answered “frequently” to the question about how often they drive on city or town streets.

In some instances, respondents felt that “suburban” would best describe the streets they drive on, but that choice was not available. Selecting from the options provided made some feel they were not accurately reporting the types of roads they had in mind.

2.1.3 Definition and general concept of speeding

The word speeding appears to include two concepts for many respondents. When defined in a question to mean simply exceeding the posted limit, they can accept that. When used more generally to describe a behavior, they often do not use that restricted definition, but think of speeding as not simply driving over the limit but also as an indicator of unsafe driving.

2.1.4 Response scales

The five-point response scales were sometimes difficult for respondents to keep in mind. This problem was exacerbated when the scale dimensions changed frequently from one question to another.

Another general finding is that the section on Speed Behavior on Various Road Types caused confusion in the first couple of interviews. It was our judgment that an introduction to the section that told respondents more about what questions were coming could reduce that problem. An alternative introduction was drafted and used to better effect in the remaining interviews.

We also tested a version of the questionnaire in which the order of questions in this same section was reversed. This did not have a noticeable effect on respondents’ question comprehension or ease of answering.

2.2 Cognitive Interview Pretest: Question-by-Question Findings

General Driving Information

3. Thinking about the roads you normally drive on, would you say that the roads where you drive most often are in areas that are . . .

1 More urban than rural

2 More rural than urban

3 (VOL) About the same

4 (VOL) Don’t Know

5 (VOL) Refused

There was some mention of the need for a “suburban” category, which for some respondents is a natural category between urban and rural. If “suburban” is technically considered to be a type of urban road, “suburban” could be offered as a response option and then re-coded for analysis.

Speed Behavior on Various Road Types

The Original introduction to this section was

For this next set of questions I am going to ask you about your driving behavior on Multi-Lane Highways. These roads include interstates, freeways and other highways.

Followed by:

For this next set of questions I am going to ask you about your driving behavior on Two-Lane Highways which are not divided. This means there is only one lane traveling in each direction and no median or barrier separating traffic traveling in opposite directions.

And then:

Now I am going to ask you about your driving behavior on City or Town streets. These streets are non-residential or neighborhood streets.

Alternative Introduction

In the early interviews, respondents had difficulty following the changes in the types of roads being asked about. We thought that an introduction to the section that was more explicit about the nature of the questions in the section might help prepare respondents for the shifts from asking about one type of road to another type.

This alternative introduction read:

We want to find out how people may change the way they drive on different types of roads, such as on highways, on city streets, or on rural roads.

These next questions are about how you drive on some of these different kinds of roads.

First, I’m going to ask about your driving on Multi-Lane Highways, such as interstates, freeways and other kinds of highways.

This alternative resulted in fewer questions, requests for clarification or mentions of confusion by respondents.

6a. How often do you drive on Multi-Lane Highways?

Do you drive on this type of road . . . ?

1 Frequently

2 Sometimes

3 Rarely

4 Never

5 (VOL) Don’t Know

6 (VOL) Refused

Respondents generally understood what kind of road the question is asking about. They generally gave correct descriptions or generic examples in response to probing.

6b. During a typical week, approximately how many miles do you drive on Multi-Lane Highways? [IF NEEDED: Your best guess is fine.]

_________ Miles

9998 Don’t Know

9999 Refused

Several respondents had difficulty deciding what was a “typical week.”

Some respondents also expressed uncertainty about whether the question meant a work week or not. It would help to make this clearer.

To make answering easier, some respondents apparently thought about the work week, Monday through Friday. Because work weeks are similar in terms of driving habits, it is easier to think about a “typical” week; and because a large part of their work-week driving is commuting to work, it was also easier to estimate miles.

It was mentioned that having response categories would make this easier to answer.

6c. What do you consider to be a safe speed limit for (most) Multi-Lane, Divided Highways in good weather on roads with no congestion during the day?

____MPH

This question has several conditions for respondents to keep in mind—safe speed, road type, weather, congestion, daytime. Most respondents seemed to focus on safe speed and weather, rather than congestion or time of day. Some of them may have started thinking about, or decided on, their answer before hearing the last part of the question.

6f. How often would you say you drive 10/15/20 miles an hour over the speed limit on Multi-Lane, Divided Highways?

10 miles over:

1 Always

2 Often

3 Sometimes

4 Rarely

5 Never

15 miles over:

1 Always

2 Often

3 Sometimes

4 Rarely

5 Never

20 miles over:

1 Always

2 Often

3 Sometimes

4 Rarely

5 Never

Some respondents wanted to interrupt the sequence and simply report how they drive on the particular type of road.

The “Always” category may not be a realistic choice; and fewer response options will make it easier for telephone survey respondents.

6g. How many miles per hour over the speed limit do you think you can drive on Multi-Lane, Divided Highways, before you will receive a ticket?

____MPH over the speed limit

998 (VOL) Don’t Know

999 (VOL) Refused

Respondents often answered based on remembering instances of being stopped or getting a ticket, rather than considering a general rule.

This question seemed somewhat unnecessary after having actual behavior from the previous question.

Next I am going to ask about your driving on Two-Lane Highways which are not divided. This means there is only one lane traveling in each direction and no median or barrier separating traffic traveling in opposite directions.

Two-Lane Highways, one lane in each direction

8a. How often do you drive on Two Lane Highways, one lane in each direction? Do you drive on this type of road . . . ?

1 Frequently

2 Sometimes

3 Rarely, or

4 Never

5 (VOL) Don’t Know

6 (VOL) Refused

When asked to define two-lane highways (or other types of roads) most respondents can do so, although some just repeat what they’ve just heard and others define the type of road by giving examples of specific roads which is not always sufficient information to determine understanding.

8b. During a typical week, approximately how many miles do you drive on Two Lane Highways, one lane in each direction? [IF NEEDED: Your best guess is fine.]

_________ Miles

9998 Don’t Know

9999 Refused

For types of roads used infrequently, asking in terms of a ‘typical week’ often produces an answer of ‘zero/none.’

8c. What do you consider to be a safe speed limit for (most) Two-Lane Highways, one lane in each direction in good weather during the day?

____MPH

Several respondents qualified their answers, e.g. “it depends on the highway” or “people should go at the posted speed.”

Most respondents seem to answer based on their experience. For respondents who seldom, if ever, travel on a particular type of road at all, asking the question in terms of “most” such roads may not be meaningful.

Now I am going to ask you about your driving on City or Town streets. These streets are non-residential or neighborhood streets.

When asked about their understanding of ‘city or town streets,’ most replies seemed reasonable: e.g. “many stop lights in a short area,” or “mix of business and residential;” but were often vague, e.g. “street in the city,” or by example, e.g. “streets in DC or Rockville.”

City or Town streets

9a. How often do you drive on City or Town streets? Do you drive on this type of road . . . ?

1 Frequently

2 Sometimes

3 Rarely, or

4 Never

5 (VOL) Don’t Know

6 (VOL) Refused

Most respondents answered “frequently.” This could be due to the areas from which cognitive interview respondents were recruited, but might also, as mentioned earlier, be due to the very inclusive description “city or town streets.” Very few people would not describe themselves as living in either a city or a town. There may not be much variance in the responses to this question as worded.

Norms/Factors on Speeding

10. People have different feelings about the speed at which they drive. Please tell me whether you strongly agree, somewhat agree, neither agree nor disagree, somewhat disagree, or strongly disagree with the following statements:

Respondents varied in their understanding of whether they should answer in terms of people generally or themselves personally. Some respondents said they answered in terms of both, e.g. “Both. What I do and what others should do.” The introduction to this section is not unambiguous. Some statements, e.g. a., refer to people generally, while others, e.g. g., seem more personal.

a. Everyone should obey the speed limits because it’s the law.

b. People should keep pace with the flow of traffic.

c. Speeding tickets have more to do with raising money than they do with reducing speeding.

d. Driving over the speed limit is not dangerous for skilled drivers.

e. There is no excuse to exceed the speed limits.

f. It is unacceptable to exceed speed limits more than 20mph.

g. If it is your time to die, you’ll die, so it doesn’t matter whether you speed.

h. Driving over the speed limit is not dangerous on multi-lane divided highways.

I. Driving over the speed limit is not dangerous on city streets.

Sometimes, in item I., the “not” was missed on an initial hearing.

1 Strongly agree

2 Somewhat agree

3 Neither

4 Somewhat disagree

5 Strongly disagree

6 (VOL) Don’t Know

7 (VOL) Refused

Respondents had some difficulty keeping the response scale in mind. The response options often had to be repeated. In person, there are sometimes visual signals that the respondent doesn’t remember the scale (and also more willingness to ask for it to be repeated); this is likely to be less true in a telephone interview.

11. For each of the following statements please tell me whether you strongly agree, somewhat agree, neither agree nor disagree, somewhat disagree, or strongly disagree.

It was clearer to respondents that they were to answer these items in terms of themselves.

a. I enjoy the feeling of driving fast.

b. The faster I drive, the more alert I am.

c. I often get impatient with slower drivers.

d. I try to get where I am going as fast as I can.

e. I worry a lot about having a crash.

f. I consider myself a risk taker.

Several respondents were not sure whether this meant risks in “life generally” or risk just in terms of their driving behavior. Some respondents said their answer would differ depending on the question’s intent.

g. Speeding is something I do without thinking.

In this last item, and in question 12, there is something of a disconnect between the literal definition of “speeding” and how some respondents think of speeding (see below).

1 Strongly agree

2 Somewhat agree

3 Neither agree nor disagree Middle alternative differs from previous question

4 Somewhat disagree

5 Strongly disagree

6 (VOL) Don’t Know

7 (VOL) Refused

12. People sometimes go faster than the speed limit for different reasons. On those occasions when you speed, what do you think are the main reasons you drive faster than the speed limit? Anything else?

Most respondents think of speeding as not simply driving over the posted speed limit, but also as a type of driving behavior, e.g. “driving erratically,” “going faster than the speed limit, aggressive and reckless driving.” For some respondents to “go faster than the speed limit” — in itself— does not logically mean “you speed.” For example a couple of respondents said they’d “speed” (meaning go over the posted limit) to “keep up with the traffic flow.” Another said “5 over [the limit] is not speeding.” These meet the literal definition of speeding, but not always how people think of speeding as a behavior.

MULTIPLE RECORD. DO NOT READ.

1 I’m late

2 I am unlikely to have a crash

3 It’s a habit

4 I’m alone in the car

5 Unlikely to get a ticket

6 People I am with encourage it

7 I’m comfortable driving fast

8 Other, Specify

9 (VOL) I never speed

10 (VOL) Don’t Know

11 (VOL) Refused

13. Please tell me whether you strongly agree, somewhat agree, neither agree or disagree, somewhat disagree, or strongly disagree with the following statements.

Driving at or near the speed limit . . .

a. Reduces my chances of an accident

b. Makes it difficult to keep up with traffic

c. Makes me feel annoyed

d. Takes me longer to reach my destination

e. Makes it easier to avoid dangerous situations

Some respondents felt item e. depended on circumstances and was difficult to answer generally, e.g. “if you are going the speed limit and are slower than the flow it could cause someone to make a dangerous move.”

f. Uses less fuel

1 Strongly agree

2 Somewhat agree

3 Neither

4 Somewhat disagree

5 Strongly disagree

6 (VOL) Don’t Know

7 (VOL) Refused

Attitudes Towards Enforcement

14. How important is it that something be done to reduce speeding on our nation’s roads?

Is it . . .?

1 Very important

2 Somewhat important

3 Not too important

4 Not at all important

5 (VOL) Don’t Know

6 (VOL) Refused

To some respondents the phrase “nation’s roads” meant all roads, but led others to think only about highways and interstates.

15. How often do you think police should enforce the speed limit?

Should they enforce it . . .

1 All the time

2 Often

3 Sometimes

4 Rarely, or

5 Never

6 (VOL) Don’t Know

7 (VOL) Refused

Respondents did not generally have problems with this question, but some focused on “police” more than “enforcing the speed limit.” Is the focus of the question on how often the speed limit should be enforced, regardless of the means; or more on how high a priority enforcing the limit should be for the police?

Automated Photo Enforcement Devices

The next questions are about speed cameras. Some areas have introduced speed cameras to identify vehicles that speed. This doesn’t require police to stop violators. Rather, information is captured by the camera. A traffic ticket is mailed to the owner of the vehicle along with a photograph and information about the location, time, date and speed of the vehicle.

18. Thinking about locations where speed cameras might be useful, would you find it acceptable to use them . . . ? READ AND ROTATE A-F

A. Where it could be hazardous to stop a driver

This caused confusion for some respondents. One person said “how would a camera stop a vehicle?” He thought that adding ‘hazardous for a police officer to stop a driver’ would be easier to understand. These reactions suggest that respondents sometimes missed the third sentence in the section introduction, which mentions police stopping violators. The introduction is quite long; a shorter introduction would probably make it less likely that respondents will miss key points.

B. Where stopping a vehicle could cause traffic congestion

C. Where there have been many accidents or crashes

The word “crash” generally meant something more serious than “accident” to most respondents.

“Crash” was associated with serious damage and/or injuries.

D. In a school zone

E. In a work zone OR In a construction zone

Most respondents did not see a difference between “work” or “construction” zones.

F. On all roads

1 Yes

2 No

3 (VOL) Don’t Know

4 (VOL) Refused

21. Please tell me whether you strongly agree, somewhat agree, neither agree or disagree, somewhat disagree or strongly disagree with the following statements.

a. Speed cameras are used mainly to prevent crashes

There was a question as to whether crashes included only cars or also pedestrians.

b. Speed cameras are used mainly to generate revenue

1 Strongly agree

2 Somewhat agree

3 Neither

4 Somewhat disagree

5 Strongly disagree

6 (VOL) Don’t Know

7 (VOL) Refused

Attitudes Toward Speeding Countermeasures

22. How would you feel about using the following measures in your community to reduce speeding? Would you strongly approve, somewhat approve, neither approve nor disapprove, somewhat disapprove, or strongly disapprove of (READ ITEM)?

a. More frequent ticketing for speeding

b. Issuing higher fines for speeding tickets

c. Increasing public awareness of the risks of speeding

d. Road design changes, like speed humps and traffic circles, to slow down traffic

e. Increasing automobile insurance premiums for receiving speeding tickets

f. Electronic signs by the road that warn drivers that they are speeding and should slow down

1 Strongly approve

2 Somewhat approve

3 Neither approve or disapprove

4 Somewhat disapprove

5 Strongly disapprove

6 (VOL) Don’t Know

7 (VOL) Refused

Several respondents had difficulty remembering the response scale. The switch in scale from “agree” to “approve” also was a problem for a few respondents.

There are a number of new technologies in use to reduce the amount of speeding on our nation’s roads. These next questions ask what you think about the use of these technologies to reduce speeding.

23. How likely would you be to support a mandatory use of a speed governor, a device that limits the maximum speed of the vehicle for . . . Would you be very likely, somewhat likely, neither likely or unlikely, somewhat unlikely, or very unlikely?

a. Truck drivers

b. Drivers 18 years or younger

c. Drivers with multiple speeding tickets in one year

d. All drivers

1 Very likely

2 Somewhat likely

3 Neither

4 Somewhat unlikely

5 Very unlikely

6 (VOL) Don’t Know

7 (VOL) Refused

Again, the change in scale from “approve” to “likely to support” required some repeating of the questions and response options. It is not clear what “support” measures that is different from “approve.”

The meaning of “speed governor” was difficult for some respondents to grasp; it seemed to be the first time they had heard of it.

25. Please tell me how acceptable you would find having the following devices in your own vehicle(s). Would you find it very acceptable, somewhat acceptable, neither acceptable nor acceptable, somewhat acceptable, or very acceptable to have . . .

A. A device in your car that makes it impossible for you to drive faster than 10 miles over the posted speed limit.

The word “impossible” was too extreme for some respondents who could imagine situations where making it “impossible” to go over a certain speed would, in itself, be dangerous.

B. A device in your car that you can switch on or off, that prevents you from driving faster than the speed limit

C. A device in your car which allows parents to limit the maximum speed of the car, when the teenager drives the car

1 Very acceptable

2 Somewhat acceptable

3 Neither acceptable nor acceptable

4 Somewhat acceptable

5 Very acceptable

26. Variable speed limits are used to change the speed limit on a section of road based on traffic or weather conditions. The idea is to slow the traffic if the road ahead is congested area or is experiencing bad weather. The “variable” speed limit is typically shown by a digital signs on the side of the road.

Please tell me whether you think it is a good idea or a bad idea to use variable speed limits in the following situations: :

Respondents did not have a problem understanding the term “variable.” But some did have a problem with the length of the question. It often needed to be re-read.

Also, the positive effect of example given in the second sentence may bias responses in a positive direction.

A. Work zones

B. School zones

C. Bad weather

D. Congested Roadways

1 Good idea

2 Neither a good or bad idea

3 Bad idea

Crash Experience

28. Have you been in a traffic accident as a driver in the past FIVE years where speeding was a factor?

1 Yes

2 No

3 (VOL) Don’t Know

4 (VOL) Refused

It was unclear to several respondents whether this meant ‘they’ were speeding. There is some resistance to answering Yes, when the question seems to imply the respondent was speeding, when, in fact, it was the other driver who was speeding.

Use of Cell Phone Behaviors

41. When you get a call on the phone while you are driving, how often do you answer the call? Would you say you always, usually, seldom, or never answer a call while driving?

1 Always

2 Usually

3 Seldom

4 Never

5 (VOL) Turn it off/Put it on Silent Mode

8 (VOL) Don’t Know

9 (VOL) Refused

Several respondents had trouble answering because they felt the correct response was “sometimes.”

42. How often do you talk on the phone while you are driving? Would you say you talk on the phone while driving during .................?

1 All trips

2 Most trips

3 About half your trips

4 Fewer than half your trips, or

5 Never

8 (VOL) Don’t Know

9 (VOL) Refused

A couple of respondents did not like the scale going from “fewer than half” to “never.”

This may be because “Never” is a different dimension than “trips;” logically, option 5 should be “None of your trips.”

It is not clear that “trips” is the correct metric for measuring “how often.”

Respondents don’t all think of the same definition of a trip. Is commuting to work one trip or two; driving to the grocer, one or two?

44. When dialing the phone while driving, do you tend to dial the phone WHILE you are driving, do you tend to dial the phone while you are TEMPORARILY STOPPED, or do you tend to PULL OVER AND STOP the motor vehicle before dialing?

1 While driving

2 When temporarily stopped

3 Pull over and stop

4 (VOL) Never dial

8 (VOL) Don’t Know

9 (VOL) Refused

Most respondents did not have a problem with the term “dialing.”

45. How often do you READ text messages while you are driving and the vehicle is moving? Would you say you read text messages while driving during .................?

1 All trips

2 Most trips

3 About half your trips

4 Fewer than half your trips, or

5 Never

8 (VOL) Don’t Know

9 (VOL) Refused

46. How often do you SEND text messages while you are driving and the vehicle is

moving? Would you say you send text messages while driving during .................?

1 All trips

2 Most trips

3 About half your trips

4 Fewer than half your trips, or

5 Never

8 (VOL) Don’t Know

9 (VOL) Refused

For both these questions, some respondents mentioned they didn’t think some people would answer honestly.

NHTSA Form 1127 Page 1