PartB_Driver Interactions with Driver Assistance Technologies_6.4.20

DEPARTMENT OF TRANSPORTATION

INFORMATION COLLECTION SUPPORTING STATEMENT

TITLE OF INFORMATION COLLECTION: Driver Interactions with Driver Assistance Technologies

OMB CLEARANCE NUMBER: 2127-NEW

Part B. Statistical Methods

1. Describe potential respondent universe and any sampling selection method to be used.

The respondent universe consists of owners of vehicle models equipped with the advanced driver-assistance system (ADAS) features of interest in the study and individuals who do not have experience -- individuals solicited through print and online newspaper advertisements.

No statistical methods will be used in selecting study participants. Study participants will be selected based on a set of criteria that serve to ensure that participants will be generally representative of average U.S. drivers, have no recent criminal convictions, and hold no biases that may impact study results. The criteria state that participants must:

1. Be aged 25-65 years

2. Hold a valid U.S. driver’s license

3. Have no uncorrected vision or hearing problems

4. Be in good general health, able to drive a production passenger vehicle continuously and safely for a period of 3 hours and without the need for assistive devices

5. Drive at least 11,000 miles annually

6. Currently have no more than 2 points on their driving record

7. Have no recent criminal convictions

8. Self-report that they are able to read, write, speak, and understand English

9. Be willing to drive to NHTSA’s Vehicle Research and Test Center and spend approximately 4 hours participating in a research study

10. Not have anyone in their household who works for or is retired from any automotive manufacturer, which may constitute a conflict of interest with the research.

Criteria satisfaction is determined using a standard set of demographic, driving behavior, and general health questions. Specific questions were developed and reviewed through their use in multiple studies to ensure their wording is clear and only the minimum relevant information is obtained for the purposes of assessing individuals’ suitability for study participation. No health information will be stored; only a net response indicating whether a candidate participant “meets criteria” will be retained.

Study participants will be selected who meet the general criteria and best address the research focus. Recruitment will focus on three distinct subsets of licensed drivers characterized by vehicle ownership and technology use:

1) Ohio-registered owners of a model year 2018 or 2019 Cadillac CT6 equipped with General Motors’ (GM) Super Cruise system,

2) Ohio-registered owners of model year 2018 or 2019 Honda Odyssey minivans equipped with the Honda Sensing system, or

3) U.S. licensed drivers with no experience with advanced driver-assistance system technology.

Within each of those three categories, an attempt will be made to balance sex to the extent possible based on availability.

2. Describe procedures for collecting information including statistical methodology for stratification and sample selection, estimation procedures, degree of accuracy needed, and less than annual periodic data cycles.

No such statistical methods will be employed in sample selection.

Candidate participants will complete the Interest Response Form (hosted online on a secure website) if they view and choose to respond to the advertisement or mailed participation invitation. Individuals whose responses to the interest response form meet study participation criteria (described in Question 1 above) will be contacted and asked to complete the Recruitment Screening Form, which will be accessible via a secure website. Individuals, whose responses to the recruitment screening form meet study participation criteria, will be scheduled for study participation according to their availability and need for additional participants of their type. Interest Response Form and Recruitment Screening Form data will only be used for the purposes of identifying suitable study participants and will not be used in any other way.

Scheduled participants will be grouped into either the Inexperienced group or an Owner group. For the first part of data collection, participants designated in the Owner group will be grouped into Honda Odyssey Owners or Cadillac CT6 Owners, depending on their characteristics. Within the Inexperienced group, participants will be randomly assigned to drive either the Honda Odyssey or Cadillac CT6. Within both Owner groups, participants will be randomly assigned to whether they will drive the vehicle they currently own or the vehicle they do not currently own.

Participants will be asked to drive an instrumented, government-owned vehicle on a specified route over public roads. Vehicle instrumentation will include video cameras for capturing image and audio data, cabling for connecting to the vehicle’s Controller Area Network (CAN) bus, and a computer for recording the obtained data for later processing and analysis. Video data will be recorded at a rate of 30 Hz and used to provide information about participants’ eye glance behavior and hand position throughout their test drives, as well as vehicle instrument panel visual displays. Audio data will be used to obtain any verbal comments made in relation to ADAS feature use. Vehicle speed, steering angle, and pedal inputs will be obtained from CAN bus data and recorded at a rate of 20 Hz. Upon completion of the test drive, data will be transferred from the data acquisition computer to an encrypted portable external hard drive. Data files will then be copied to a limited-access local area network (LAN). Raw data files, including video image and audio data, will be stored on the LAN server, while data relating to eye glance, hand position, and driving performance measures will be extracted in .csv (comma-separate values; a text file) format and copied to a different limited-access network where research staff can access and analyze the data. Eye glance and hand data will be processed using neural networks to determine respective locations and durations. Vehicle instrument panel image data will be used to determine ADAS feature status (i.e., engaged, not engaged, presenting a visual alert to the driver). Vehicle motion and driving performance data will be synchronized with video and audio data to provide a means of examining drivers’ responses to the ADAS feature’s visual and/or auditory alerts indicating that the feature is disengaging (e.g., in the first part of data collection, GM Super Cruise only works on mapped highway segments, so when the end of a mapped segment is reached, the feature will alert the driver and then stop working).

Upon completion of the test drive, all participants will complete the Post-Drive Questionnaire pertaining to all drivers. Next, “Owners” will be asked to complete an additional questionnaire (“Post-Drive Questionnaire Owners”). Questions address the participants’ experience driving in the study and using the specified ADAS features, including their comfort in using the features and how well they understood the status of the features’ operation and any visual or auditory alerts provided. Post-drive questionnaire responses will be compared by group to assess whether the ADAS features differed in how well participants understood how the features worked and when they were engaged.

3. Describe methods to maximize response rate.

Members of the public will be recruited for participation through online and print advertisements. This will include candidates for participation in the “no experience” group. Additional targeted recruitment will be conducted based on Ohio Bureau of Motor Vehicles (BMV) data indicating ownership of the two specific vehicle models of interest. Owners of the specific vehicle models will be sent an invitation by mail and offered two response methods, online or postage-paid response card, so that individuals can choose the response method that is easiest for them to use.

Study participation likelihood will be increased by the provision of monetary compensation at an hourly rate of $50.00 as well as mileage reimbursement for travel to and from the test site. Monetary compensation is consistent with normal experimental practice and should encourage study participation.

Participant recruitment and testing will continue until 16 participants are completed for each group. We have determined that 96 total participants are necessary (16 per group) in order to provide sufficient statistical power to detect a mean difference between groups for take-over response times. The estimation that the means for take-over time will differ by 1.0 seconds is based on previous research from the VTTI Naturalistic Driving Study (NDS) study (Russell et al., 2018). Power analysis results indicated that a group sample size of 16 is required to detect a difference between means of 1.0 second with power equal to 0.8.

4. Describe tests of procedures or methods.

Part 1 of the data collection will employ a 3 (Driver Experience) x 2 (Vehicle) between-subjects design with Driver Experience and Vehicle serving as the independent variables. Driver experience will involve three levels: inexperienced, Honda Odyssey owners, and Cadillac CT6 owners. Vehicle will involve two levels: Honda Odyssey and Cadillac CT6. This will create six conditions: inexperienced drivers driving a Honda Odyssey, inexperienced drivers driving a Cadillac CT6, Honda Odyssey owners driving a Honda Odyssey, Honda Odyssey owners driving a Cadillac CT6, Cadillac CT6 owners driving a Honda Odyssey, and Cadillac CT6 owners driving a Cadillac CT6.

Part 2 of the data collection will involve the same experimental independent variables, but may have different numbers of levels depending on the vehicles selected.

Research questions being investigated include:

1. How often do drivers use ADAS features?

2. How much time does it take drivers to respond to disengagement notifications and achieve stable driving performance after resumption of manual control?

3. How does prior experience with ADAS features affect how drivers use ADAS features and takeover performance? Does this prior experience transfer when driving with a different system?

4. Does prior experience affect how drivers perceive the ease of use, trust and safety of the ADAS features?

Research Questions 1 and 3:

In order to investigate how often drivers use ADAS features, dependent measures related to the frequency and duration of ADAS will be collected. The frequency of how many times drivers engage and disengage ADAS technologies and the total duration of ADAS use throughout the drive will be examined. For the first part of data collection, how many times drivers engage and disengage Adaptive Cruise Control (ACC) and either Lane Keeping Assistance (LKA) or Lane Centering Assistance (LCA) and the total duration of ACC and LKA use throughout the drive will be calculated using ADAS status information obtained from the vehicle CAN and instrument panel camera. Data will be aggregated for all participants and the mean, median, and standard deviation of frequency and duration of ADAS use will be calculated. Separate 3 (Driver Experience) x 2 (Vehicle) between-subjects analysis of variance (ANOVAs) will be used to analyze whether the frequency and duration of ADAS use differs for drivers with different levels of experience with the system and whether prior experience with one system affects frequency and duration of use when driving with a new system. For this and all subsequent analyses, corrections will be applied for multiple comparisons.

Research Questions 2 and 3:

In order to investigate how long it takes drivers to respond to disengagement notifications and resume stable manual control over the longitudinal and latitudinal movement of the vehicle, dependent measures of take-over performance will be collected. Specifically, takeover response time will be measured as the time from system disengagement until the driver first resumes manual operation by either: pressing the brake pedal, pressing the accelerator pedal, or placing both hands on the steering wheel and moving the steering wheel. Eye glance and steering behavior will be used to characterize the time to regain stable vehicle control. The time required for drivers’ eye glances to return to the forward road and for steering wheel variability to return to stable following a system disengagement will be used to determine the time to regain stable control. Vehicle control metrics for calculating takeover response time such as brake application, accelerator application, and steering wheel variability will be obtained from the vehicle’s CAN. Eye glance and hand positions will be classified into areas of interest by a neural network using images collected by in-vehicle cameras. Eye glances will be classified into 6 areas of interest: forward road, instrument panel, center rearview mirror, left side rearview mirror, right side rearview mirror, and center console. Hand positions will be classified into 4 areas of interest: both hands on the steering wheel, one hand on the steering wheel, no hands on the steering wheel, and touching the center console. Data will be aggregated for all participants and the mean, median, and standard deviation of response times to disengagement notifications and time to achieve stable vehicle control will be calculated. Due to slight differences in how takeover requests are communicated to the driver in each vehicle, each vehicle will be analyzed separately. Separate one-way ANOVAs will be performed on the mean takeover time for the three experience levels for each vehicle. This will allow us to determine if differences in experience affect takeover performance, while controlling for differences in how the two vehicles communicate the need for a driver to resume manual operation.

Research Question 4:

Subjective impressions of the advanced driver-assistance system will be assessed following the drive. A post-drive questionnaire will be used to gather subjective ratings of ease of use, trust, and safety. Data will be aggregated for all participants and the mean, median, and standard deviation of ratings will be calculated for ease of use, trust, and safety. The non-parametric Kruskal-Wallis test will be used to analyze whether subjective ratings of ease of use, trust, and safety differ for drivers with different levels of experience with the system and whether prior experience with one system affects subjective ratings of a new system.

5. Provide the name and telephone number of individuals who were consulted on statistical aspects of the IC and who will actually collect and/or analyze the information.

Elizabeth Mazzae, BS & MSE in Human Factors Engineering, NHTSA, 937‐666‐4511

Scott Baldwin, BS & MSE in Human Factors Engineering, Transportation Research Center, Inc., 937‐666‐4511

Kelly Satterfield, PhD Psychology, Transportation Research Center, Inc., 937‐666‐4511

Isabella Skuce, MS Psychology, Transportation Research Center, Inc., 937‐666‐4511

6. Impact Of Less Frequent Collection Of Information. Describe the consequence to federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

This information collection effort will be conducted only once. If the collection were not conducted, NHTSA would lack important information needed to understand how drivers use ADAS systems, how well drivers understand the capabilities of the systems and how to use them properly, the tendency of drivers to misuse or abuse the features, and how all of these issues impact driving safety. This information will be important to inform NHTSA as we work through policy and Rulemaking issues relating to these systems.

1