Supporting Statement B for HF in ADS 03.10.23

_{Department of Transportation}

Federal Motor Carrier Safety Administration

SUPPORTING STATEMENT PART B

Human Factors Considerations in Commercial Motor Vehicle Automated Driving Systems and Advanced Driver Assistance Systems

INTRODUCTION

This is to request the Office of Management and Budget’s (OMB’s) review and approval of a new Federal Motor Carrier Safety Administration (FMCSA) information collection request (ICR) titled Human Factors Considerations in Commercial Motor Vehicle Automated Driving Systems and Advanced Driver Assistance Systems.

Part B. Collections of Information Employing Statistical Methods

1. DESCRIBE POTENTIAL RESPONDENT UNIVERSE AND ANY SAMPLING SELECTION METHOD TO BE USED.

1.1 Respondent universe

The inclusion criteria for this study are:

1. Possess a valid Class A or Class B commercial driver’s license (CDL),

2. not be prone to simulator/motion sickness,

3. be 21 years of age or older, and

4. be within one day drive of the Virginia Tech Transportation Institute (VTTI) ; as reported by the driver).

As of 2020, over 6.5 million commercial vehicle drivers were operating a large truck or bus in the United States.¹ Of these drivers, approximately 4.0 million have a valid CDL. Furthermore, it is estimated that approximately 1.9 million individuals hold a Class A CDL² and more than 645,000 individuals hold a Class B CDL.³ Approximately 20% or fewer are expected to be prone to simulator/motion sickness.^4,5 A previous VTTI simulator study had evidence of simulator sickness in 5%–12% of participating drivers.⁶ Drivers “within one day drive” includes drivers living within one day drive, as well as drivers who live farther than one day drive but whose working drive route brings them near VTTI. This is the potential respondent universe.

1.2 Sampling selection method

We anticipate a convenience sample of 100 drivers to participate in two study sessions. Multiple participant recruitment methods will be utilized:

1. VTTI will leverage its large database of Class A or Class B CDL drivers that have participated in previous research studies or have expressed interest in possible participation. A total of 959 Class A or Class B CDL drivers are currently in the system, which undergoes continuous updating. The database includes information on driver attributes, such as driver age, gender, and location.

2. Additionally, VTTI has existing relationships with numerous commercial fleets within a day’s drive of southwest Virginia. VTTI has conducted research on trucking for over 20 years. Many VTTI studies have involved working with local commercial fleets. Through these experiences, VTTI has developed positive working relationships with over 40 fleets within a four-hour drive of VTTI. These fleets have agreed to share VTTI research study information in their break rooms, next to time clocks, and via email.

3. Finally, VTTI will place recruitment ads in newsletters and on social media.

Through these methods, interested participants may be contacted by VTTI or may contact VTTI. We anticipate receiving a pool of potential participants, with the ability to include difficult to reach truck drivers such as females and minorities. However, there will be no required minimum number of female or minority drivers to be included in the study. VTTI will select the first 100 drivers who express interest in study participation through the contact methods above and who meet the study inclusion criteria.

A power analysis was performed to determine the appropriate sample size. The power analysis provides an estimated sample size for developing a strong study with sufficient data to answer all research questions. Current published research focused on simulator studies and driver readiness and performance was reviewed to identify possible values to expect in the current study. A study by Zhang et al.⁷ included 22 professional truck drivers in a simulator study assessing driver behavior in a truck platooning scenario. Truck platooning requires drivers to monitor the vehicle and to be prepared to take over vehicle control. In the study, drivers received training and then operated the simulator under different trials and conditions. The conditions included features planned in the current study, such as asking the driver to perform non-driving tasks and notifying the driver to take over the vehicle system controls. The researchers compared drivers’ perception reaction time (the time it takes for a driver to perceive a stimulus, cognitively process the situation, and decide on a response), movement reaction time (the time it takes the driver to perform the mitigation strategy), and the total reaction time (the total time for transfer of control from the vehicle to the driver). They found significant differences in reaction times among driver groups, ranging from 15% to 55%.⁷

The power analysis used a difference in reaction time of 15% as a baseline level for significant findings. The power analysis was performed in Minitab 20.3. The power analysis assumptions included a goal power of 0.80 (industry standard), alpha or significance level of 0.05 (industry standard), a 15% difference in driver reaction time, and an analysis of variance (ANOVA; the study analysis approach to assess reaction time will involve multiple covariates and control of multiple observations per driver). To obtain a power of 0.80, a sample size of 45 drivers will be required to detect a 15% statistically significant difference in driver reaction time. Each driver will likely contribute multiple data points for certain variables. Normally, this would lower the sample size necessary to achieve a power of 0.80; however, in cases where a driver will provide a single data point for the study, it will be necessary to retain a minimum of 45 drivers. To ensure a sample large enough to answer all research questions, a sample of 50 drivers that complete the study protocols is recommended for recruitment in the L2 and L3 studies. Figure 1 shows the power curve (in blue) generated based on the above stated assumptions as the estimated sample size changes from n = 5 to n = 60. The dashed orange line, which indicates power = 0.80, intersects with the power curve at approximately n = 45.

A sample size of 50 would raise the power to about 0.84.

Figure 1. Estimated sample size in L2 and L3 studies.

2. DESCRIBE PROCEDURES FOR COLLECTING INFORMATION, INCLUDING STATISTICAL METHODOLOGY FOR STRATIFICATION AND SAMPLE SELECTION, ESTIMATION PROCEDURES, DEGREE OF ACCURACY NEEDED, AND LESS THAN ANNUAL PERIODIC DATA CYCLES.

2.1 Procedures for collecting information

The following section describes the procedures for collecting information in terms of the study methodology. Following this information, additional sections will provide answers to the stratification, sample selection, estimation procedures, degree of accuracy needed, less than annual periodic data cycles, and analysis methodology.

The study includes data collection from a survey and a driving-simulator experiment. The collected survey data will support the simulator experiment data. The survey data will be used in two ways: (1) in the assessment of driving performance data as covariates in the model (to control for certain variables, such as age, gender, and experience) and (2) to answer a research question on the relationship between driver characteristics and driver readiness and performance. Data on driver readiness and performance will be collected from the simulator experiment. The planned experiment is explained in full below. The final experimental design will be informed by results of the literature review and an assessment of industry practices. These design components are specified in detail in the following sections. However, the plans for survey data collection will remain as stated in this document.

The study includes a Level 2 (L2) advanded driver assistance system (ADAS)-focused sub-study, a Level 3 (L3) automated driving system (ADS)-automation-focused sub-study, and a training-focused sub-study. Figure 2 below illustrates the three studies. The L2-focused and L3-focused studies will each include 50 participants, for a total of 100 participants. In the L2 and L3 studies, the participants will undergo researcher-directed non-driving task engagement when the ADAS/ADS technology is initiated. In both formats, participants will be presented with scenarios that may require a driver takeover. Each study will also include a simulator driving portion without researcher-directed, non-driving task engagement, where the participants can choose to engage in non-driving tasks at their own discretion. Survey data collection will be included in these studies.

The training study will include 100 participants. These participants will be recruited from the L2- and L3-focused studies. However, any remaining open spots will be filled by new participants. In the training study, participants will be assigned to an L2 or L3 ADAS/ADS technology and either (1) split driving or (2) trained driving. Each of these groups will include 25 drivers. Drivers in the split driving group will receive formal safety training midway through their simulator driving experience (resulting in periods of control/baseline driving and trained driving). Drivers in the trained driving group will receive formal safety training before their simulator driving experience (resulting in a single period of trained driving). Survey data collection will be included in the training study.

Figure 2. Brief summaries of the three sub-studies within the HF in ADAS/ADS Simulator study.

2.1.1 L2-Focused Study

The L2-focused study investigates the impact of driver internal/external distraction on driver readiness and driving behaviors during ADAS/ADS-activated driving. The study also includes an investigation of driver demographic factors and roadway conditions on driving performance and secondary task (also called non-driving task) engagement. To answer the research questions, the L2-focused study will require the collection of survey data and driving simulator experiment data. Participants in the driving simulator experiment will expect to spend approximately 4 hours in the research experiment (Figure 3) This time includes preparation for the study, survey tasks, and time conducting the simulator driving experiment. The simulator driving aspect of the experiment will include two key parts. The first portion of the simulator experiment will involve researcher-directed engagement in internally and externally distracting tasks. The second portion of the simulator experiment will be a non-directed engagement period of driving. During the second portion, drivers will experience multiple environmental and roadway conditions. The L2 system will be assumed to include two L2 technologies. Each technology will have a unique combination of auditory and visual alert notifications when the system requires participant takeover of the vehicle.

During the researcher-directed study portion, participants will be instructed to engage in six tasks. Non-driving tasks will be selected based on prior commercial motor vehicle (CMV) research identifying prevalence of non-driving secondary tasks⁸ and work in L2 and L3 operations.^9,10,11 The tasks will include at least one internally distracting task and one externally distracting task. Participants will also be given time for non-directed driving. The presentation order of tasks will be randomly assigned and counterbalanced across participants. Weather (clear weather) and lighting (daylight) will remain consistent for all participants. Each task will be prompted within 10 minutes of the previous task prompt. This part of the experiment will be conducted over approximately 1.5 hours. Each of the researcher-directed tasks will occur during a continuous drive (with short breaks occurring throughout as needed) instead of in separate scenarios. In addition, this portion will include three opportunities for the driver to take back control of the vehicle. Takeover opportunities will be signaled to the driver through an alert. Participants will experience a takeover request for two different L2 system technologies. This emulates previous research conducted by VTTI using a driver simulator to investigate driver distraction in ADAS/ADS-equipped vehicles. These previous experiences demonstrate that this approach is realistic.

The second portion of the study will involve a free-driving or “naturalistic” portion of driving, where participants are not directed to engage in tasks but instead will drive the CMV with the ADAS/ADS activated. During this portion, drivers can engage in secondary tasks, but they will not be explicitly instructed to do so. Environmental conditions will vary during this time, including changes to roadway type and weather. Only those scenarios where distraction/inattention during L2 is most prevalent and risky will be presented during this free-driving segment. Participants will receive one takeover request during this study portion. The participant sample will be split in half so that 25 participants will experience one of the two L2 system technology takeover requests. Data collected from this portion of the study will be used to investigate driver demographic features associated with driver readiness and performance and roadway conditions associated with driver engagement in secondary non-driving tasks. Figure 3 provides a summary of the L2 study.

Figure 3. L2-focused study participant plan.

2.1.2 L3-Focused Study

The L3-focused study investigates what factors affect successful takeover and what alert qualities have the most positive impact on successful takeover performance. The L3-focused study will also investigate what patterns of eye-glance behaviors are associated with successful takeover, especially in highly stimulating roadway environments, such as those with heavy traffic, vulnerable road users, or numerous signs, signals, and pavement markings. To answer the research questions, the L3-focused study will require survey data collection and a driving simulator experiment. Participants in the driving simulator experiment will expect to spend approximately 4 hours in the research experiment (Figure 4). This time includes preparation for the study, survey tasks, and time conducting the simulator driving experiment. During this study, participants will operate the simulator in two formats (Figure 4). The first part will include researcher-directed engagement with non-driving secondary tasks during L3 engagement. The second part will follow the “free” or “naturalistic” style discussed in the L2-focused study.

The researcher-directed task engagement portion of the study will involve instructing participants to perform non-driving secondary tasks while operating the driving simulator. The selected tasks will represent a variety of task types, such as internally distracting, externally distracting, high risk, or protective. The study will also include time for non-directed driving. The experiment will require driver takeover during engagement in non-driving tasks. Participants will be alerted through the vehicle that driver takeover is needed to maintain control of the vehicle. The alert format (haptic, visual, audible, or a combination) will be selected based on recommendations and current industry practice. However, the timing of the alerts will be varied during the experiment. Up to three alert timings will be assessed. The driving environment will include both a highly stimulating driving environment and a less-stimulating driving environment. This portion of the experiment will be conducted over approximately 2 hours. Timing of secondary task engagement and takeover requests will be calculated after determining the number of secondary task types, alert timings, and takeover requests included in the assessment. Data collected during this portion of the study will be used to understand how alert timing affects successful takeover of the vehicle when engaged in non-driving secondary tasks.

The second part of the L3 study will follow a “free” or “naturalistic” style, where participants will operate the vehicle with L3 ADS activated and without direction to engage in secondary tasks. During this study portion, drivers will be instructed to drive the vehicle and engage in non-driving tasks at their own discretion. Takeover requests will be included in this study portion as informed by current industry practice and operational design domains of L3 CMVs. The takeover request alerts will vary in alert timing, as in the researcher-directed study portion. All drivers will be exposed to highly stimulating and less-stimulating driving environments during this portion of the study. The study design will balance the takeover request alert timings and driving environment conditions across all participants. This portion of the study is expected to last approximately 1 hour. Data collected during this portion of the study will be used to understand what patterns of eye-glance behaviors are associated with successful takeover of the vehicle.

Figure 4. L3-focused study participant plan.

2.1.3 Training Study

The training-focused study will be used to assess how training affects problems with CMV driver distraction and driver readiness during L2 and L3 operation. Drivers will be recruited from the L2- and L3-focused sub-study participants, with additional drivers recruited if necessary, to meet 100 total participants in the training study. Like the L2- and L3-focused studies, the training study will require survey data collection and a driving simulator experiment. Participants in the training experiment will expect to spend approximately 4 hours in the research experiment. This time includes preparation for the study, survey tasks, and time conducting the simulator driving experiment.

It is anticipated that some participants in the first two sub-studies will not be able to participate in the training study. These participant slots will be filled by recruiting additional drivers. Participants that did not participate in the original L2 or L3 sub-studies will be randomly assigned to one ADAS/ADS level condition (L2 or L3).

Within the ADAS/ADS level condition groups, participants will either (1) receive training before operating the simulator or (2) receive training midway through the experiment after a baseline driving period (split group). The training will review the L2 or L3 technology capabilities and recommendations for safe driving while operating a vehicle with the technology. Data collected during the training-focused study will be compared to data from the naturalistic portions of the L2-focused and L3-focused studies. In addition, data collected during the baseline driving period will be compared to post-training driving.

It is possible there will be a subset of participants that only have data for the training study (the participants recruited to fill missing spots due to participant drop-out). The analysis will also include a comparison of data from returning participants and new participants, as these driver groups will vary in their experience with the simulator. Data from the new participants can be used as a control comparison to assess whether the training or additional simulator experience was more strongly associated with changes in driver behavior.

Figure 5 illustrates the experiment study groups, with participant counts, and experiment duration for the training-focused study. During the experiment, participants will drive for approximately 3 hours total. Drivers in the trained driver group will receive training before beginning the driving portion of the experiment. Drivers in the split group will complete the first driving session before receiving training. After training, the drivers will continue driving for the remaining study time in a second driving session. The figure uses an example split of 1 hour before training and 2 hours after training to illustrate one possible design. The final driving session duration will be determined based on the type and number of selected non-driving tasks to be performed. All participants in the split group will follow the same study design with consistent driving session durations.

The simulated drive will include various environmental and roadway conditions, which will be balanced across the study duration and over the participants. The selection of environmental and roadway conditions, to be based on current industry practice and operational design domains of L2 and L3 CMVs, will determine study timing and counterbalancing needs across participants. During the study, drivers will not be directed by researchers to engage in secondary tasks. Drivers will receive multiple takeover requests during their drive. Data collected from this study will be used to understand the impact of training on driver performance, driver willingness to engage in non-driving secondary tasks, and successful takeover of the vehicle.

Figure 5. Training-focused study participant counts and experiment duration by technology type and study groups.

2.2 Statistical Methodology for Stratification and Sample Selection

The sampling selection will be purposive in nature. The research team will recruit CMV drivers using the following techniques: contact Class A or Class B CDL drivers from VTTI’s large database of drivers that have participated in previous research studies or have expressed interest in possible participation; contact commercial fleets within one day’s drive of VTTI and ask the fleets to disseminate recruitment announcements describing the study and providing contact information; place recruitment ads in industry-relevant newsletters and social media accounts.

The sample selection method will follow a non-random, convenience sampling plan. Drivers who express interest in the study and meet study inclusion criteria will be selected on a first-come basis. Through the recruitment methods, we anticipate receiving a pool of potential participants that includes females, minorities, and varying driving experience levels. Although CMV carriers will be selected in a nonrandom fashion, this sampling methodology should not produce any bias in any of the key findings to be generated from the study. This stems from the fact that the research is not intended to produce national representative point estimates for any metric, but rather to better understand the impact of distraction and training on driver performance and behavior while operating a vehicle with L2 or L3 technologies.

Stratification by key variables, such as age, gender, or driving experience, will be included in the analysis approach. In the analysis models, drivers’ demographic characteristics, including age and gender, will be treated as blocking factors, as they may influence driving performance under the study conditions.

2.3 Degree of Accuracy Needed

The power analysis used a difference in reaction time of 15% as a baseline level for significant findings. The power analysis assumptions included a goal power of 0.80 (industry standard), alpha or significance level of 0.05 (industry standard), a 15% difference in driver reaction time, and an ANOVA (the study analysis approach to assess reaction time will involve multiple covariates and control of multiple observations per driver). Based on the power analysis, the recommended sample includes 50 drivers for the L2 sub-study, 50 drivers for the L3 sub-study, and a combined 100 drivers for the training study. The inclusion of 50 drivers per sub-study is expected to meet the accuracy needs for assessing driver performance and behavior under various study conditions. In addition, the power analysis used a previous study with similar metrics to guide assumptions. For the power analysis, the smaller difference in driver performance (15%) from the study was used as an assumption. If the proposed study were to find a larger difference, the power of the study would increase.

2.4 Less than Annual Periodic Data Cycles

The proposed study will include data collection from each participant at the time of each sub-study. The data collection is expected to be on less than annual periodic data cycles.

2.5 Analysis Methodology

The analysis methodology uses a multi-pronged approach to address all research questions. The principal statistical method for analyzing the data will include mixed models to account for multiple, correlated data points from a single participant. Details of the statistical methodology to be used in each sub-study are explained below.

2.5.1 L2 Study

In the L2 study, the team will complete several analyses, including an assessment of driver distraction and its effects on driver readiness and driving performance. Factors to be assessed include type of non-driving secondary task, driver characteristics, and driving conditions (weather, lighting, roadway). The L2 study will implement general linear mixed models (GLMMs) to answer the research questions. In the transportation safety field, GLMMs are often used to analyze driver behavior and assess relationships between driving scenarios and behaviors.¹² A GLMM is an extension of the generalized linear model, in which the linear predictor contains random effects in addition to the usual fixed effects. As participants will experience all experimental conditions, it will be necessary to use a model that includes driver-specific random effects.

In each research question, individual models will be built for the driver readiness and performance measures and experimental condition variables. For example, a model will be built comparing driver glance location as a measure of driver readiness in distracted driving and non-distracted driving. More general descriptions of the anticipated models are:

1. For driver performance or readiness measures with binary response options (e.g., eye state), the model will be a mixed-effect logistic regression model.

2. For categorical variables with more than two response options (e.g., driver state), a multinomial logistic regression model will be used.

3. For continuous response variables (e.g., speed or following distance), linear mixed models will be used to test for differences between experimental conditions.

4. Interaction terms will be included depending on the purpose of each test. For example, when modeling the effect of secondary task engagement on driver readiness in the L2 CMV, an interaction term between secondary task and L2 status will be considered, as drivers may adapt their engagement level under L2 automation.

5. Models may utilize a principal components or summary score approach to account for correlated metrics, as there is a risk of measured variables being highly correlated. The benefits of this approach will be assessed for each individual research question.

2.5.2 L3 Study

For the L3 study, GLMMs will again be utilized. Anticipated analysis methods include the following:

1. Mixed-effect Poisson or negative binomial regression models may be used to assess the alert rate.

2. Linear mixed models may be used to assess the time it takes to transfer control based on secondary task engagement. Secondary tasks to be compared will include both work-related and non-work-related tasks.

3. Mixed-effect logistic regression models may be used to model differences in takeover success. Two adjacent urgent levels of alerts will be compared to quantify the alert rate of different severities. Driver response time will be collected, and a minimum warning time will be recommended and informed by current industry practice.

4. Time-variant eye-glance pattern will be investigated by an attention score model, which provides a real-time indicator of a driver’s situation awareness.

2.5.3 Training Study

To evaluate the effectiveness of the training program, linear mixed models will be used with random intercepts. Driver random intercepts will account for participants’ correlated behaviors and expectations in the L2 or L3 system before and after training.

1. In the L2 analysis, the linear mixed model will explore any significant difference in observed driver readiness before and after training.

2. In the L3 analysis, the linear mixed model will be employed to measure the difference in driver ability to transfer control and driver performance before and after training.

3. In both models, drivers’ demographic characteristics, including age and gender, will be treated as blocking factors, as they may influence driving performance under the study conditions.

2.6 Considerations for the Study Design

The study designs for the L2, L3, and training assessments were created to best address the research questions within certain experiment bounds. The design maximizes data collection in scenarios of particular interest within the time constraints of a simulated driving session. The design also creates a simulated driving environment that closely resembles real-world driving. Drivers will experience limited surprise events and takeover requests, which will reduce participant distrust in the system and more closely mirror current system performance. The design also controls for simulator experience as a possible confounding variable. However, it is important to also highlight design limitations. One such limitation is the incomplete randomized order of study factors. Orders of environmental or roadway conditions will be counterbalanced across participants but will be limited to orders that most resemble real-world driving (e.g., drivers will not cycle through night driving, day driving, and again night driving within a short period of time). Takeover requests will be presented in limited scenarios. Therefore, drivers will not experience all orders of all factors and takeover requests in each factor combination. An additional limitation will be the use of a single alert type. Also, the study design expects drivers to return for the second session (training study), but limited participant dropout should be anticipated.

3. DESCRIBE METHODS TO MAXIMIZE RESPONSE RATE AND TO DEAL WITH THE ISSUES OF NON-RESPONSE.

3.1 Methods to Maximize Response Rate

Participants will be recruited from VTTI’s database of drivers who indicated interest in future studies and CMV drivers located within a day’s drive of VTTI headquarters in Blacksburg, Virginia. The database includes drivers who have previously completed a study with VTTI, and therefore have shown a commitment to study participation in past research. Considering the proximity of accessible, interested participants, VTTI expects to find drivers with a desire and ability to participate and complete the study. Drivers selected for the study will be reminded of the study participation date through phone calls and email the day before their scheduled participation. The phone calls and emails will include VTTI contact information, with multiple VTTI contact options.

The participants will be told their participation is voluntary, and they can terminate their participation at any point without prejudice or harm to them in any way. Participants will be told at recruitment that this is a two-part study and will have the opportunity to have any questions answered prior to deciding to participate. This should also increase the likelihood of their completing the entire study.

The research team will offer incentives to promote interest in participating in the study and to improve retention over the study period. The proposed incentives have been reviewed and approved by the Virginia Tech Institutional Review Board. Drivers will receive an additional incentive if they complete both study sessions. Incentives will be distributed via a rechargeable debit card, which the participant will receive at the initial meeting.

In simulator studies, participants may feel simulator sickness and excuse themselves from further participation. To minimize these simulator side effects, and thereby maximize responses, verbal health checks will be given periodically to allow for breaks and rests. For those who do not respond, or choose to withdraw from the study, new participants will be recruited to fill their spots. Non-response data points will not be included in the analysis. However, any data points collected prior to a participant’s withdrawal will be included.

3.2 Methods to Deal with Issues of Non-Response

During data collection, there will be multiple opportunities to correct missing data to lessen instances of non-response. For example, driver questionnaires filled out at the study start will be reviewed by a researcher during the simulator portion of the study. If the questionnaire is not fully complete, participants can be asked at study completion if they would like to provide a response to missed questions or keep responses as originally submitted. During the simulator portion of the study, drivers will be given significant time between study tasks. This will provide the participant multiple opportunities to provide answers to study questions scheduled for the time window. Researchers will address any follow-up questions participants may have regarding the study question.

There will be several additional strategies to deal with non-response in the data. These include:

1. Generalize to the respondents only. This strategy avoids making erroneous inferences about the larger population.

2. Compare data in hand on respondents and nonrespondents. If data (e.g., gender, age, race) is available, the composition of respondents will be compared with that of nonrespondents to see if there are any differences. The presence of differences indicates response bias and that caution is necessary in making inferences.

4. DESCRIBE TESTS OF PROCEDURES OR METHODS TO BE UNDERTAKEN.
   
   

All data collection methods proposed in the current study have been assessed in prior VTTI studies. The demographic questions have been successfully used and tested in various prior VTTI studies.¹³ Alertness questions included in the demographics survey were cited from the well-validated Karolinska Sleepiness Scale.¹³ The driver behavior questions pertaining to signs of aggressive driving on the roadway use the Dula Dangerous Driving index, a well-known metric for assessing driving characteristics.¹⁴ Moreover, the simulator sickness procedures and questionnaires are all materials administered in previous VTTI simulator studies to minimize participant risk of negative side effects.^{6,15, 16}

The study will include a pilot test for each of the L2, L3, and training studies, for a total of three pilot tests. The pilot tests will use the protocol drafted for the full tests. Each pilot test will include up to three participants, for a total of less than nine participants across the L2, L3, and training pilot tests. Participants in the pilot tests will operate the simulator, as directed in the full tests. Results from the pilot test and feedback from the participating CMV drivers will be used to identify areas for improvement and refine study protocols for the full L2, L3, and training test runs.

Pilot test data will be summarized using graphs, plots, tables, and summary statistics. Data collected in individual questionnaires and during the simulator study portion will be assessed by source. The pilot test data exploration will be used to identify questionnaire items that need further clarification in the full test and simulator test protocol aspects that would benefit from revision.

5. PROVIDE NAME AND TELEPHONE NUMBER OF INDIVIDUALS WHO WERE CONSULTED ON STATISTICAL ASPECTS OF THE INFORMATION COLLECTION AND WHO WILL ACTUALLY COLLECT AND/OR ANALYZE THE INFORMATION.

Project Leads for this ICR:

Susan Soccolich

Statistician

Virginia Tech Transportation Institute

3500 Transportation Research Plaza

Blacksburg, VA 24061

(540) 231-1032

[email protected]

1 Federal Motor Carrier Safety Administration. (2021). 2021 pocket guide to large truck and bus statistics. Federal Motor Carrier Safety Administration. https://www.fmcsa.dot.gov/sites/fmcsa.dot.gov/files/2022-01/FMCSA%20Pocket%20Guide%202021.pdf

2 Zippia. (2021). CDL Class A Driver: Demographics and statistics in the US. https://www.zippia.com/cdl-class-a-driver-jobs/demographics/

3 Zippia. (2021). Class B driver demographics and statistics in the US. https://www.zippia.com/class-b-driver-jobs/demographics/

4 Nelson, E. T., Kidd, D. G., & Cades, D. M. (2010). Examining patterns of simulator sickness

during increased exposure to a motion-base driving simulator over Time. Journal of the Washington Academy of Sciences, 96(3), 1–14. http://www.jstor.org/stable/24536258

5 Classen, S., Hwangbo, S.W., Mason, J., Wersal, J., Rogers, J., & Sisiopiku, V.P. (2021). Older drivers’ motion

and simulator sickness before and after automated vehicle exposure. Safety, 7(26). https://doi.org/10.3390/safety7020026

6 Morgan, J. F., Tidwell, S. A., Medina, A., Blanco, M., Hickman, J. S., & Hanowski, R. J . (2011). Commercial motor vehicle driving simulator validation study (SimVal): Phase II (Report No. FMCSA-RRR-11-014). Federal Motor Carrier Safety Administration.

7 Zhang, B., Wilschut, E. S., Willemsen, D. M. C., & Martens, M. H. (2019). Transitions to manual control from highly automated driving in non-critical truck platooning scenarios. Transportation Research Part F: Traffic Psychology and Behaviour, 64, 84-97.

8 Hammond, R. L., Soccolich, S. A.; Han, S., Guo, F., Glenn, T. L., & Hanowski, R. J. (2021). Analysis of naturalistic driving data to assess distraction and drowsiness in drivers of commercial motor vehicles (Report No. FMCSA-RRR-20-003). Federal Motor Carrier and Safety Administration.

9 Dunn, N. J., Dingus, T. A., & Soccolich, S. (2019). Understanding the impact of technology: Do advanced driver assistance and semi-automated vehicle systems lead to improper driving behavior? American Automobile Association Foundation for Traffic Safety.

10 Dunn, N. J., Dingus, T. A., Soccolich, S., & Horrey, W. J. (2019). Investigating the impact of driving automation systems on distracted driving behaviors. Accident Analysis and Prevention, 1567, 106152.

11 Noble, A. M., Miles, M., Perez, M. A., Guo, F., & Klauer, S. G. (2021). Evaluating driver eye glance behavior and secondary task engagement while using driving automation systems. Accident Analysis and Prevention, 15, 105959.

12 Guo, F. (2019). Statistical methods for naturalistic driving studies. Annual Review of Statistics and Its Application, 6, 309–328.

13 Kaida, M., Takahashi, T., Åkerstedt, A., Nakata, Y., Otsuka, T., Haratani, K., et al. (2006). Validation of the Karolinska sleepiness scale against performance and EEG variables. Clinical Neurophysiology, 117, 1574–81.

14 Dula, C. S., & Ballard, M. E. (2003). Development and evaluation of a measure of dangerous, aggressive, negative emotional, and risky driving 1. Journal of Applied Social Psychology, 33(2), 263-282.

15 Robin, J. L., Knipling, R. R., Tidwell, S. A., McFann, J., Derrickson, M. L., & Antonik, C. (2005). FMCSA Commercial Truck Simulation Validation Study Phase I Pilot Test: Driving scenario definition and development. Driving Simulation Conference North America 2005, Orlando, FL.

16 Morgan, J. F., Tidwell, S. A., Blanco, M., Medina-Flinstch, A., & Hanowski, R. J. (2013). Commercial truck driver performance in emergency maneuvers and extreme roadway conditions presented in a driving simulator. Washington Academy of Sciences, 99, 25-37.

8