PRA_DADSS_FOT_Part B_Final

DEPARTMENT OF TRANSPORTATION

INFORMATION COLLECTION SUPPORTING STATEMENT

TITLE OF INFORMATION COLLECTION: Driver Alcohol Detection System for Safety Field Operational Test

OMB CONTROL NUMBER: 2127-0734

Abstract:¹ This is a request to the Office of Management and Budget (OMB) for review and approval of a renewal with modification of currently approved information collection request (ICR) titled “Driver Alcohol Detection System for Safety - Field Operational Test (DADSS-FOT).” This ICR is for a voluntary information collection to evaluate touch- and breath-based sensor technologies that are integrated into research vehicles. The purpose of the information collection is to collect information to provide a greater understanding of the performance of both breath- and touch-based sensors with actual dosed passengers using the technology under varying environmental conditions. Although the sensors will undergo significant laboratory testing, it is necessary to evaluate their function in real-world environmental conditions to ensure that they will be operational for all possible conditions. This ICR is for the collection of data from human subjects to allow NHTSA to perform real-world testing. The primary component of this information collection is the collection of sensor data during a Field Operational Test (FOT) involving human subjects; however, demographic information about participants and post-test information will be collected as well. The objectives of the FOT are to: (1) Determine the effectiveness of the DADSS sensors in a real-world driving environment; (2) Analyze DADSS breath- and touch-based sensors in real-world driving scenarios; and (3) Obtain technical data to further refine the DADSS Performance Specifications that will ultimately be used for system design and product development. Respondents are voluntary participants that will be accepted for inclusion based on a pre-screening interview to ensure they are able to consume alcohol. For the study, NHTSA intends to collect data from a total of 480 uniquely individual participants to collect a minimum of 312,000 data points through their participative rides/drives. NHTSA has already collected data from 62 participants and will need to collect data from an additional 418 individuals. The respondent selection interviewing is a one-time data collection and the respondents are able to participate in the FOT up to 60 times within the study; however, for the 62 individuals that have participated, they have participated an average of 2.13 times. In accordance with DOT policy on research involving human subjects, this study has been reviewed and approved by a Health and Human Services-approved Institutional Review Board before data collection began. Recipients of the respondent information are the data analysts and researchers and the data are used initially to assess whether to include the respondent in the study and later to assess the functionality of the in-vehicle equipment. During the FOT, the only data collected from the human subjects will be measurement data from sensors on the vehicle, which will be recorded and used by the data analysts and researchers. This information collection is under request for extension and both the annual burden hours and the total annual burden cost have been revised based on the study experience up to August 31, 2021. The original ICR reported a total 2-year burden estimate of 115,830 hours (annual burden of 57,915 hours) and a total 2-year cost of $2,256,847.50 (annual cost of $1,128,423.75). These were estimates of maximum burden to the public based on the assumption that each individual participant would choose to participate the maximum 60 times. With adjustments made to time, response rate, and participation level, burden calculations have been updated. The burden is therefore no longer based on maximum participation of 60 times per individual participant, rather the average number of times individuals have chosen to participate thus far in data collection. The updated burden estimate is an annual burden of 3,249 hours. The annual burden cost associated with this ICR extension is zero other than the time spent participating. In addition, the respondents are compensated at an hourly rate above the opportunity cost.

Part B. Collections of Information Employing Statistical Methods

1. Describe potential respondent universe and any sampling selection method to be used.

The Driver Alcohol Detection System for Safety - Field Operational Test (DADSS-FOT) is designed to collect both subjective and objective data regarding an in-vehicle, non-invasive alcohol detection system using both touch- and breath-based sensors. Even as technology moves towards automating automobiles, the realization of full automation may be years away and humans will still be in the loop of operations, requiring their unimpaired attention. The objectives of the data collection are to:

• Determine the effectiveness of the DADSS sensors in a real-world driving environment.

• Analyze DADSS touch- and breath-based sensors in real-world driving scenarios

• Obtain technical data to further refine the DADSS Performance Specifications for the DADSS System that will ultimately be used for vehicle design and development.

The criteria for participation are broad as we do not require a nationally representative sample, nor do we intend to study target behaviors for a particular demographic. Therefore, a convenience sample that meets specific criteria are sufficient for this study. Even so, we will keep track of demographics during recruitment to ensure that we do not have a skewed sample. The following criteria and recruiting methods were used in the EURO-FOT that the research team participated in and had success with recruiting efforts (this study and its details are proprietary and not available for public review).

Participants must:

• Be at least 21 years of age

• Hold a valid U.S. or Canadian driver’s license

• Have no more than one (1) driving infraction and/or conviction on their driving record for the previous three years (participants who self-report their status will be required to submit to a background check for confirmation prior to participation in the study)

• Be free of any criminal conviction in their past including criminal driving offenses (participants who self-report their status will be required to submit to a background check for confirmation prior to participation in the study)

• Be willing to work at least five (5) days per week for 12 consecutive weeks during a three-month data collection cycle

• Meet self-reported health criteria:

  1. Cannot have a substance abuse condition including alcoholism

  2. Cannot have a history of neck or back conditions which still limit their ability to participate in certain activities.

  3. Cannot have a history of brain damage from stroke, tumor, head injury, recent concussion, or disease or infection of the brain

  4. Cannot have a current heart condition which limits their ability to participate in certain activities

  5. Cannot have current uncontrolled respiratory disorders or disorders requiring oxygen

1. Cannot have had epileptic seizures or lapses of consciousness within the last 12 months

2. Cannot have chronic migraines or tension headaches (no more than one per month during the past 12 months).

3. Cannot have current problems with motion sickness, inner ear problems, dizziness, vertigo, or balance problems

4. Cannot have uncontrolled diabetes (have they been recently diagnosed, or have they been hospitalized for this condition, or any changes in their insulin prescription during the past 3 months)

5. Must not have had any major surgery within the past 6 months (including eye procedures).

6. Cannot currently be taking any medication or supplements that may interfere with driving ability (i.e., cause drowsiness or impair motor abilities).

7. Must not be pregnant or planning to become pregnant (Pregnancy tests are also run on biological females during orientation and prior to any administration of alcohol).

8. Must be vaccinated for COVID-19 (Rapid COVID tests are also run for all participants during orientation).

• Have normal (or corrected-to-normal) hearing and vision.

• Self-report that they are able to read, write, speak and understand English.

• Be excluded if anyone in their household works in or is retired from any of the following businesses, occupations, or industries, which may constitute a conflict of interest with the DADSS-FOT:

  1. The police force or another law enforcement agency, working as a police officer, corrections officer, or probation officer

  2. A newspaper, magazine, radio or television station, or related website or online news site

  3. An advertising, marketing, or public relations agency

  4. A market or public opinion research company

  5. The automobile or automotive industry

  6. Liquor sales or hospitality, such as bartending

  7. Law, such as a lawyer or attorney, or working at a law firm, or in the legal profession

  8. The federal, state, or county Departments of Transportation

• Be excluded if anyone in their immediate family has been a victim of drunk driving, or if they personally know someone that has been a victim.

A total of 480 unique test participants is needed to complete data collection for this study. This is a key measure of necessity for the study. Based on the Intraclass Correlation Coefficient (ICC) statistic (for more detail, see Part B item 2), our research team estimates that the required number of data points needed to achieve power of 96% where there is 95% agreement between readings from two data sources (touch- or breath-based sensor and the reference sensor) is 2000. Because this is the first real-world test of these sensors and the exact failure rate associated with the environmental conditions in which we plan to conduct the tests is unknown, the research team is proposing to obtain 30% more data than the statistical analysis requires to account for the potential of sensor failure. Therefore, the research team will collect 2600 data points for each condition, for each technology tested and for three (3) alcohol conditions (no alcohol, one drink and two drinks). There are four data points to be collected per test: two (2) breath-based, one (1) touch-based and one (1) reference.

The study conditions are determined by the different geographic regions and temperature conditions from which we plan to sample. There are five different geographic regions: Northern, Midwestern, Pacific, Southwestern and Southeastern. There are two different temperature levels: cold and hot. Five geographic regions and four temperature levels per region yields 10 unique conditions. To study the covariate relationships, humidity and altitude will also be measured in these 10 different conditions. This yields 312,000 required data points required for the study. This minimum of 312,000 data points is the other key measure for the study design. Thus, the two essential components for data collection are the 480 unique test participants and the collection of 312,000 data points across the study. An extension to this information collection does not change these essential components.

The original study design was set to utilize 30 in-service test vehicles over the 24-month study period. Due to delays from COVID shutdowns and development and implementation of safety precautions, the time for study completion needs to be extended. The research team originally anticipated recruiting two (2) participants per car per three-month period which 480 test participants over the course of the study. Thus, arriving at the 24-months. COVID delays and recruitment rates necessitate an increase in the study time-period.

Table 1. Summary of Data Collection Requirements

Attribute	Count
Statistical Power Targeted	96%
Data Points Needed Per Technology Per Region Type	2000
Data Points Needed Per Technology Per Region Type (including anticipated 30% error)	2600
Data Points Collected Per Test	4 (2 breath, 1 touch, 1 reference)
Number of Test Conditions (Geographic Site x Temperature Category)	10
Number of Data Points	312,000
Count of Participants	480
Study Months	52 (high estimate)

During the original information collection, the research team considered an individual would have the opportunity to participate as many as 60 times in the FOT. Burden hours and costs were developed to establish a maximum burden using that 60-time participation rate. Further breaking down the information collection, 480 participants at a maximum of 60 rides/drivers would result in 28,800 rides/drives. Thus, in order to collect the necessary 312,000 data points, the research team would need to collect a minimum of 11 data points per ride/driver. Prior to initiation of the study, breath sampling rates from the participants were estimates and gathering a minimum of 11 data points from the breath samples was deemed feasible. This is the information that was presented in the original information collection.

During the information collection up through August 31, 2021, the research team has gained experience with the study and is able to refine the process for data collection within the existing study design and thus refine calculations for burden. The essential components of the study have not changed and remain at 480 unique test participants and a minimum of 312,000 data points. The research team has gained experience to adjust in-vehicle breath sampling rates while remaining within the estimated ride/drive time in the FOT and maintain the ability to retrieve the 312,000 data points.

The opportunity to participate in the ride/drive up to 60 times is still available to participants. However, experience with participants thus far in the study shows participants have not chosen to participate that many times. The participants have averaged 2.13 times per person. This will not affect the statistical power of the study, though, as the power comes from the 480 uniquely individual participants and the 312,000 data points collected as previously stated. Through the course of the data collection, the research team has been determined that they are able to increase the breath sampling rate, as stated previously. Therefore, as the average rate of participation has decreased, the breath sampling rate can increase. The research team will continue analysis of the number of participants, the average participation rate, and the number of data points collected and adjust the breath sampling rate during the ride/drive. The number of breath samples collected per participant and ride/drive can be adjusted as the study moves forward to ensure that the necessary number of data points is collected across the 480 participants (multiple, but varying numbers of, data points are retrieved from each breath sample). In order to maintain the statistical power for the study, the research requires a minimum of one FOT (ride/drive) participant; however, that minimum is not expected to yield 312,000 data points. This will be managed through the continued analysis of the research time regarding the number of participants, the average participation rate, and the number of data points collected in order to adjust the breath sampling rate.

2. Describe procedures for collecting information, including statistical methodology for

stratification and sample selection, estimation procedures, degree of accuracy needed,

and less than annual periodic data cycles.

Overview

NHTSA and its research team believe that, to meet its research objectives, this study collects data at the lowest frequency possible within a single ICR approval based on the methods of the study described below and through statistical estimation of the data required to address the goals of the study. To ensure that the research team collects the least amount of data possible, the Intraclass Correlation Coefficient (ICC, a value between 0 and 1) can be computed to analyze the validity of breath- or touch-based sensors versus a reference instrument. The ICC indicates how closely the instruments correlate in their measurements, where 0 = no correlation and 1 = full correlation. Based on ICC calculations including an inflation factor for sensor failure, approximately 2600 observations per test condition per technology across all test participants are required. This study will utilize 30 test vehicles and 480 test participants. This study is expected to be completed within a 52-month period. The duration of the study has been increased since the original collection to account for delays due to COVID as well as recruitment challenges.

Recruitment & Orientation

To recruit participants, KEA uses a web-based advertisement strategy to reach potential participants. When interested participants contact KEA, a screening questionnaire is conducted over the phone to determine eligibility for participation in the full study. Once deemed eligible through self-report and a verified background check (driving and criminal), participants are scheduled for a study orientation. (Although not required by DOT, a background check is needed due to the nature of the study.) Upon arrival to the facility, orientation is conducted by DADSS research team members at each selected locale. Orientation is conducted in groups to the extent possible or on an individual basis.

The study purpose, approach and goals are openly communicated to participants. Each participant is briefed on the purpose of the study, given the opportunity ask questions and given the opportunity to willfully consent to participate in the study. Security measures in place for data transmission are explained. Once informed consent has been obtained, test participants are oriented to their test vehicle and DADSS sensor functions and operations. The vehicle orientation includes basics on features and safety technologies within each vehicle, what to do in the event of an emergency, crash involvement, etc.

A research team member provides an overview of the DADSS Alcohol detection subsystems and their purpose. The operation of each is demonstrated with the opportunity for questions. Procedures for bypass of the ignition interlock system in the event of technology failure is explained. Driving rules are covered including the following high-level requirements:

• Alcohol must not be present in the physiological system of any of the participants at the beginning of the test day (i.e., BAC 0.00%) as measured by ACS Alcolock L3 Interlock. Test drivers, specifically, are not permitted to consume any alcohol on the test day. Any alcohol detected in the drivers during the study or operation of the vehicle is grounds for removal from the study. Driving participants who do not comply with the no drinking rule and whose BrAC levels are above the legal limit while operating a test vehicle are subject to the prevailing laws of the region where the infraction took place.

• Paid drivers are expected to obey all local laws including safe driving speeds. In the event of a violation, ticket or evidence of unsafe operation of the vehicle (such data is automatically captured by the DAS), drivers will be immediately removed from the study.

Test Day Procedures

At the start of each test day, drivers and test passengers are expected to meet with DADSS study coordinators at a predetermined location to receive keys and check in for the day. Each day, participants are provided route information in the form of waypoints. This series of waypoints is provided in a format that is easy to follow using in-vehicle navigation. Instructions include a count of the minimum number of successful alcohol tests required for the day and the location and sequence of tests expected. Participants must successfully complete the minimum number of requested tests unless a permanent system failure occurs. Participants are given contact information for a DADSS research team member in the event of a complete system failure to receive instructions for returning the test vehicle and assignment to another vehicle.

At designated test spots, drivers and test passengers are asked to:

• Safely park vehicles

• Perform tests within a predefined number of minutes after turning the ignition off (in some cases an immediate restart will occur or a predefined cool down/shut down period will be required)

• Perform four (4) tests (two (2) for the breath-based DADSS system, one (1) for the touch-based DADSS system, and one (1) for the breath-based reference sensor test)

At the completion of the driving shift, participants return the vehicle to a designated location. After each test day, drivers are asked if they had any issues with the performance of the DADSS sensors, problems experienced with the vehicle or DADSS subsystems or other relevant information (see post-test questionnaire).

Equipment Validation

This study is intended to test reliability and technical function of the DADSS alcohol detection systems during normal vehicle operation and in geographic areas of interest. Each quarter, DADSS program staff will remove the DADSS subsystems from each test vehicle and send the device to KEA labs for validation tests to assess any system level changes that have occurred.

Statistical Estimation and Procedures

Estimation model: A Hierarchical Linear Model (HLM) will be applied to this multilevel data to explain dependencies and compute relationships within each group. The dependent variable will be observations of alcohol level (Level 2), and the independent variables including Level 1 units (i.e. devices) and Level 2 units (i.e. drivers). Driver level covariates will include time, altitude, temperature, humidity, and the amount of alcohol intake per occupant. The resulting model is a mixed model including fixed effects plus the random effects.

In this study, the main hypotheses are: 1. Temperature, humidity and altitude will impact alcohol readings per device, after controlling for driver weight, alcohol consumed and time. 2. The device type (breath, touch or reference) are related to observed alcohol level, after controlling for temperature, humidity and altitude. 3. The device moderates the relationship between observed alcohol level and temperature, observed alcohol level and humidity, and observed alcohol level and altitude.

Variance estimation and inference methods. In this three-level hierarchical model, there are time points nested within each of drivers, in turn nested within each of devices. At level 1, the outcome for time t within driver level unit and device level unit is represented as

Level 1

Level 2

Level 3

Where are level-1 coefficients, with the corresponding ’s the level-1 predictors. are the level-1 random effect, with the assumption that . Without device level coefficients, will be used to account the difference between devices. are driver level coefficients and the are driver level predictors including driver’s weight. are time level coefficients, are time level predictors including temperature, humidity and altitude, and are time level random effects. Taken as a vector, the u’s are assumed to have a multivariate normal distribution with a mean vector of 0 and a covariance matrix , with maximum dimension .

Validity and reliability will be analyzed by comparing breath and touch instruments to a reference instrument. The Intraclass Correlation Coefficient (ICC, a value between 0 and 1) will be computed to analyze the validity and reliability of breath- or touch-based sensors versus the reference category (see Table 1). The ICC indicates how closely the instruments correlate in their measurement of each subject, where 0 = no correlation and 1 = full correlation.

Table 2. Intraclass Correlation Coefficient Table

ICC	Required Data Points Per Technology	POWER
0.90	1000	49%
0.90	2000	61%
0.90	3000	76%
0.95	1000	77%
0.95	2000	96%
0.95	3000	99%

To ensure that the research team obtains observations for an adequate number of trips in the aforementioned geographic regions as well as the targeted environmental conditions we are targeting power of 96% and an ICC of 0.95. As described in Part B item 1 of this document, this requires 312,000 data points given the target number of test conditions, the technologies being tested and the anticipated sensor failure rates. As stated previously, during data collection thus far, the research team has identified that they are able to increase breath sampling rates in a given ride/drive and consequently increase the number of data points collected in that ride/drive. The research team will continue to monitor the number of participants, the data points collected in a given ride/drive, and adjust breath sampling rates in a given ride/drive in order to collect the necessary data points for the study design. If there is an indication that the research team is unable to collect the minimum number of data points per condition, then additional FOTs would be run for that particular condition. This will be monitored throughout the study. The study design does not require a minimum number of unique participants for each condition, rather the design lends itself well to have those unique participants diversified over conditions.

The impact of data collection in the proposed study are expected to be stable over time and will not require annual data collection cycles. Data collection from all participants only happens over the phone during the recruiting interview and general feedback at the conclusion of each test day. Data collection from the test vehicles is automatic and does not require any action from the participants. The data is collected and transmitted securely and wirelessly from the test vehicles’ data acquisition system (DAS).

The procedure for the collection of information for this research is summarized as follows:

• Participant pool is defined.

• Recruitment agency recruit participants from five (5) geographic regions using existing participant databases of the recruitment agency.

• Personnel from KEA’s recruitment team go through the eligibility questionnaire over the phone to determine if the person is eligible to participate (criteria listed above).

• Qualified participants are then scheduled for an orientation at a location in the appropriate geographic region.

• Upon arrival, the participants show the researcher their valid driver’s license and researchers will obtain consent. Then the participants will complete the test orientation. It will take approximately 30 minutes to complete the demographic interview and one (1) hour to complete the orientation session.

• A total sample size of up to 480 unique individual participants will be collected. Each participant has the opportunity to ride/drive 60 times; however, the average thus far is 2.13 rides/drives per participant.

• The interviews will only be conducted in English.

3. Describe methods to maximize response rate.

To minimize burden to public and study costs, participants are screened over the phone prior to their participation to ensure eligibility for full participation. The research team has given careful consideration to factors that might prevent individuals from participating in this study and/or successfully completing it and have included these factors in the screening criteria. The research team also uses popular online job boards (e.g., Indeed) to reach potential participants. Also, eligible participants are compensated for their time to maximize the likelihood that participants will complete the study.

3. Describe tests of procedures or methods

Demographic data is collected during a verbal interview. Data processing consists of tabulation of quantitative information using descriptive statistics. The data collection protocol has not been distributed to anyone who is outside of this research team. The demographic/eligibility scripts have been distributed to the research team members (less than ten individuals) for validation.

The DADSS system and test vehicles have been extensively evaluated in bench tests to assess the validity and reliability of the system and its subsystems. Prior to the execution of the study, two DADSS test cars were made available to research team members (less than 10 individuals) for system tests of the interface as well as extensive testing of the DAS and data storage/transfer systems. The purpose of these system tests was to identify any trouble with the system and subsystems so that the issues could be remedied prior to study commencement.

5. Provide name and telephone number of individuals who were consulted on statistical

aspects of the IC and who will actually collect and/or analyze the information.

The following individuals are primarily responsible for data collection and analysis:

Dr. Abdullatif (Bud) Zaouk	508-658-9420	KEA President & Principal Investigator
Dr. George Bahouth	410-988-4107	Statistical Design, Data Collection, Warehousing & Analysis
Dr. Kelly Ozdemir	508-658-9425	Research Coordinator
Ms. Kianna Pirooz	508-658-9710	Recruitment & Screening
Mr. Tim Allen	508-658-9533	Data Analysis

Dr. Zaouk, Dr. Ozdemir, Ms. Pirooz, and Mr. Allen are located at KEA Technologies, Inc. headquartered in Marlborough, Massachusetts. Dr. Bahouth is located at Impact Research in Columbia, Maryland.

1 The Abstract must include the following information: (1) whether responding to the collection is mandatory, voluntary, or required to obtain or retain a benefit; (2) a description of the entities who must respond; (3) whether the collection is reporting (indicate if a survey), recordkeeping, and/or disclosure; (4) the frequency of the collection (e.g., bi-annual, annual, monthly, weekly, as needed); (5) a description of the information that would be reported, maintained in records, or disclosed; (6) a description of who would receive the information; (7) if the information collection involves approval by an institutional review board, include a statement to that effect; (8) the purpose of the collection; and (9) if a revision, a description of the revision and the change in burden.

6