PRA-2126NEW.OBMS.SS.PartA 121611.Rev

Department of Transportation

Office of the Chief Information Officer

SUPPORTING STATEMENT

Evaluating the Safety Benefits of an On-Board Monitoring System (OBMS) in Commercial Vehicle Operations: Independent Evaluation and Data Analysis

This Supporting Statement is developed to request the Office of Management and Budget’s (OMB) review and approval of a new information collection request (ICR) entitled, “Evaluating the Safety Benefits of an On-Board Monitoring System in Commercial Vehicle Operations: Independent Evaluation and Data Analysis”

Section A. Justification:

1. Circumstances that make collection of information necessary.

The mission of the Federal Motor Carrier Safety Administration (FMCSA) is to reduce the number and severity of crashes on U.S. highways. In direct support of this mission, the Onboard Monitoring System (OBMS) Field Operational Test (FOT) and safety research program is being conducted by FMCSA’s Research Division. The primary goal of this safety research FOT is to determine whether on-board monitoring and feedback will reduce risky behavior among commercial motor vehicle (CMV) drivers and improve driver safety performance. The term feedback refers to both real time feedback from the OBMS and feedback from Supervisor feedback. The OBMS will record (through snippets of video and other performance/kinematic measures) unsafe driving behaviors and provide real time feedback to drivers. In addition, recorded driver events are also transmitted to, and reviewed by, the driver’s fleet safety manager. Depending on the judgment of the fleet safety manager, the recorded incident can then be shown to the driver in a coaching session with the goal of pinpointing the problematic behavior and providing instruction on how to avoid that problem in the future.

In studies of CMV drivers safety, fatigue/sleepiness and distraction/inattention have long been considered crash risk factors (Bunn et al., 2005; Hanowski et al., 2005). In a 2009 study, FMCSA also estimated that improper lane-keeping and speeding are the top two driver-related factors leading to large truck fatal crashes, accounting for 11.6 percent and 8.3 percent of total fatal crashes respectively. Following too closely was also identified as a significant risk factor and accounted for 2.0 percent of fatal crashes. Strategies and technologies that can mitigate these driver-related risk factors are needed to improve CMV driver safety and reduce crashes. The project will determine if recording and reporting of safety-critical events, such as distraction, fatigue, improper lane-keeping behavior, or speeding, followed by driver coaching from safety supervisors, reduces the number of these events. The OBMS utilized for this FOT combines several safety technologies and applications into a single system. These include driver behavior monitoring, lane/roadway departure warning, forward collision warning, fatigue monitoring, alcohol detection, electronic on-board recording (EOBR) of driver hours of service (HOS), and continuous naturalistic (occurring in a real world working environment) data collection.

University of Washington, as the independent evaluation team, will be administering surveys to CMV drivers participating in the OBMS FOT to assess their attitudes toward OBMS and coaching they receive based on the OBMS data. This information collection supports the DOT’s Strategic Goal of Safety because it contributes to efforts to improve CMV driver safety on our nation’s highways.

The statutory authority to conduct studies pertaining to commercial motor vehicle safety and to require motor carriers to maintain driver files is derived from 49 U.S.C. §§ 504 (Attachment A), 31133 (Attachment B), 31136 (Attachment C), and 31502 (Attachment D), and 49 CFR § 1.73 (Attachment E).

2. How, by whom, how frequently, and for what purpose the information is used.

The focus of this information collection is to understand the participating fleets’ CMV drivers’ expectations, attitudes and acceptance of OBMS. Driver perception of the system is an important determinant of its ability to improve driver behavior. For example, negative driver perceptions might coincide with unresponsiveness to coaching feedback, and would limit the systems’ effectiveness in improving safety performance. It is possible that there could be a mismatch between drivers’ actual use of the system and their reported experience of the system. Also, an examination of drivers’ opinions on OBMS in conjunction with their actual behavior might reveal subgroups of drivers with characteristics that indicate their propensities to have more or fewer safety critical events. This could support targeting OBMS only at certain types of drivers. Data for this project would be collected via mail and interviews. The results will be summarized and integrated into the rest of the FOT study report that evaluates the effectiveness of the OBMS in improving safety and driver performance.

All the CMV drivers participating in the FOT will be asked to complete surveys. Drivers will be randomly assigned into a comparison group (group 1), long-term feedback group (group 2) and short-term feedback group (group 3). The number of drivers to be randomly assigned to each group is as follows:

Group 1 drivers (no feedback): Drivers who are in Group 1 (30 devices, up to 125 drivers) will use the OBMS in their vehicle for an 18-month period without any feedback provided.

Group 2 drivers (long-term feedback): Drivers who are in Group 2 (210 devices, 250 drivers, long-term adaptation group) will experience a data collection period of an approximate 2-month Baseline phase, 14-month Intervention phase, and 2-month Withdrawal phase.

Group 3 drivers (short-term feedback): Drivers who are in Group 3 (30 devices, up to 125 drivers, short-term adaptation group) will experience a data collection period of an approximate 2-month Baseline phase, 7-month Intervention phase, and up to 9-month Withdrawal phase.

Drivers will be asked to complete four unique questionnaires over the course of 18 months and one in-person interview at the end of the study. Drivers regardless of study group will receive questionnaires at month 0 (start of study) and at the ends of months 2, 4, 9, 16, and 18. After completing the 18-month study, an exit interview will be conducted in-person to assess in greater detail participants’ opinions on OBMS functions and safety climate change due to implementation of the system.

This ICR has been divided into four ICRs as follow:

1. IC-1, Form MCSA-5852, OBMS Study Pre-Study Questionnaire. The Form MCSA-5852 will be used to assess drivers’ initial understanding and attitudes toward the system prior to system use. The questionnaire includes questions on usefulness, ease of use of specific components, as well as general system use. The pre-study questionnaire also includes demographic questions.

2. IC-2, Form MCSA-5853, OBMS Study Intervention Questionnaire. The Form MCSA-5853 will be used to assess drivers’ attitudes toward the OBMS after experiencing it. The questionnaire will have questions on system usefulness and ease of use similar to those on the pre-study questionnaire (Form MCSA-5852).

3. IC-3, Form MCSA- 5854, OBMS Study Withdrawal Questionnaire. The Form MCSA-5854 will be used to assess drivers’ attitudes toward the OBMS after the system is no longer in use. It will capture their perceptions and use based on their memory of past experience with the system and current driving experience without the system.

4. IC-4, Form MCSA-5855, OBMS Study Exit Interview. The Form MCSA-5855 will be used at the end of the study when drivers have completed all portions. It is used to assess drivers’ overall likes and dislikes about the OBMS, influence and change on safety climate, and to consider suggestions for improvement to the on-board monitoring system.

In addition to the information collection effort described above, in the broader OBMS FOT, the following types of data would be captured automatically by the OBMS in its normal commercial driver safety product operations: safety epochs, performance events, trip summary data, EOBR data and continuous naturalistic data.

Safety Epochs: Safety epochs are captured by the OBMS when algorithms determine that a likely safety-critical event has occurred. Specifically the following situations will result in an epoch capture: spiked acceleration (probable impact); hard braking; large lateral acceleration; swerve; low time-to-collision when headway is short; repeated glances to an in-cab location combined with degraded lane keeping; forward collision avoidance warning message; lane departure warning message and a driver initiated epoch capture. The exact threshold value(s) for each of these trigger conditions is parameterized so that they may be adjusted to optimize valid data capture while reducing the amount of invalid epochs collected. In addition to video and audio segments after any safety epoch trigger, the following safety epoch data elements are also captured by the OBMS:

GPS time; GPS longitude; GPS latitude; GPS speed; GPS heading; GPS altitude; network speed; network throttle; network time; lateral acceleration; longitudinal acceleration; vertical acceleration; horizontal turn rate (gyro); headway to lead vehicle (if lead vehicle present); lane offset; right tire to right lane boundary; left tire to left lane boundary; driver head position x, y, z; driver head rotation; alcohol presence (0-100); turn signal state and other data as available from vehicle network (depends on make, model, and age of truck).

Performance Events: The OBMS also monitors for a number of events that provide additional insight into driver behavior but do not typically require video validation. As these events occur and are detected by the OBMS algorithms, data are captured describing each event and they are then recorded in the trip file. Various data elements are captured for the following performance events: lane change aborts; pre-shutdown idle event; operational idle (>5 min); speeding event; engine overspeed event; hard brake event; hard corner event; coasting event; backing event and warm-up/ idle event.

Trip Summary Data: During a trip, the OBMS system accumulates performance metrics on a variety of measures and at the conclusion of the trip they are written as summary items in to the trip file. Data are collected for the flowing summary trip categories: GPS position fixes; general trip data; fuel usage; idling; speed selection; custom speed bands; cruise gear selection; cruise control usage; engine usage; rpms; headway and relation to traffic flow.

Continuous Naturalistic Data: These are video and sensor data that are being captured for FMCSA to use in future research. Total data volume for the study is expected to exceed a half petabyte. The naturalistic data being collected include:

• Continuous video files from when vehicle is in motion until 3 minutes after vehicle is stopped during each and every drive.

• Continuous sensor data from all sensors captured at 10 Hz over same period.

The subjective assessment data collected via the survey effort and the objective data elements collected directly by the OBMS system will be utilized by FMCSA to answer the following research questions:

• Does individual driving performance improve over time with OBMS feedback?

• How does the OBMS and feedback program improve safety?

• How do the driver’s opinions and attitudes towards the OBMS and program change over time?

• Can the OBMS system accurately distinguish “good” drivers from “at-risk” drivers?

• If driving performance improves, does it remain improved over time?

• What are the fleet safety manager’s opinions and attitudes about the OBMS?

• What is the business case for implementing an OBMS program?

• What are the user differences between the EOBR that is integrated into the OBMS and the previous methods for recording HOS data?

The research questions will be grouped into four categories: driver performance and safety, attitudes toward OBMS, business case, and EOBR comparison. Descriptive statistics (mean, medians, box plots, etc) will be provided for all dependent variables. Correlations will also be conducted on the dependent variables to identify any significant relationships that may need to be accounted for in the regression models. Inferential statistics will then be conducted to address each research question.

3. Extent of automated information collection.

To increase the response rate, study questionnaires will also be available on-line. The on-line questionnaires will be available to respondents from a University of Washington operated, secure (password protected) website. Each questionnaire will have a unique web link and only research team members and participants from this study will have access to the passwords. The electronic version will help reduce cost and time for the respondent, funding agency, and researcher. In addition, the web questionnaire is easier and faster to fill out for participants (e.g., easy to change answers, built-in error prevention, etc.). The on-line questionnaires will also reduce the burden to the respondents by removing the need to locate a mail drop facility, stamping, and entry corrections. However, it is expected that only about 7 percent of the questionnaire responses will be completed online, because many CMV drivers involved in this study have limited computer/internet experience and access (Murray et al., 2008).

Responses from paper-questionnaires will be manually entered into a database with the same format as the web-version.

4. Efforts to identify duplication.

Earlier FOTs have conducted surveys to assess driver acceptance of similar but less complicated systems, such as Volvo’s collision warning system with Adaptive Cruise Control (ACC) and Electronically Controlled Brake System (ECBS), Freightliner’s roll advisor and control system, and Mack trucker advisor and Lane Departure Warning (LDW) systems. However, no study has ever been conducted for an OBMS system with multiple integrated technological features such as driver behavior monitoring, lane/roadway departure warning, forward collision warning, fatigue monitoring, alcohol detection, and Electronic On-board Recorder (EOBR). Given the differences in the complexity among systems, it is expected that different attitudes and perceptions will occur. In addition, none of these studies involved complicated experimental settings, data analysis, or a large sample size. The proposed study questionnaire should help FMCSA obtain a more comprehensive understanding of CMV drivers’ acceptance and attitudes toward onboard monitoring systems.

5. Efforts to minimize burden on small businesses.

The intended response population for this information collection is a limited number of CMV drivers from three large carrier organizations. No small business will be asked to participate in the survey.

6. Impact of less frequent collection of information.

The four ICs proposed in this supporting statement comprise a single study that is planned to be undertaken once in conjunction with the FOT for OBMS. However, information is collected at multiple times before, during and after the study. Drivers’ opinions about the OBMS may change over time as they become more familiar with it. Therefore, multiple data collections spread over 18 months are necessary to measure these changes.

7. Special circumstances.

There are no special circumstances related to this information collection.

8. Compliance with 5 CFR 1320.8.

FMCSA published a notice in the Federal Register with a 60-day public comment period to announce this proposed information collection on March 29, 2011(76 FR 17474) (see Attachment F). One comment was received in response to this notice (see Attachment G). For FMCSA’s response to the sole comment received see Attachment H.

FMCSA published a second notice in the Federal Register on June 24, 2011 (76 FR 37167) with a 30-day public comment period that announced this information would be sent to OMB for approval (see Attachment I).

9. Payment or gifts to respondents.

The drivers responding to the surveys are employees of the carriers participating in the FOT. Given that the drivers are responding to actual conditions in their work environments, they have a greater incentive to participate and respond to the survey questionnaires. However, because multiple survey questionnaires will be administered over the course of 18 months, incentive payments will be made to ensure that respondents stay interested and continue to provide thoughtful responses through the end.

To maximize the response rate, participants will be paid $10 after completing each questionnaire. In addition, they will be compensated with $40 for completing the study and participating in the exit interview. Payments for each questionnaire will be mailed to the participant’s address after questionnaire responses are received by the study team. Payment for the last questionnaire and exit interview will be handed to the participant immediately after the interview. Participants will have a choice of receiving a check.

Similar studies of this target population include the Commercial Motor Vehicle Driver Risk Factor Study, Annual Commercial Motor Vehicle Driver Survey: Work and Compensation, and National Survey of U.S. Long Haul Truck Driver Injury and Health (all three were federally funded studies). The compensations provided and the expected response rates in the three studies are listed as below.

Commercial Motor Vehicle Driver Risk Factor Study:

The compensation rates are $20 for completing the initial telephone interview (20 minutes or less to complete) and $30 for completing a paper-and-pencil (or online) questionnaire (30 minutes or less to complete). The compensations were paid in checks. The estimated response rate for this study was about 50%.

Annual Commercial Motor Vehicle Driver Survey: Work and Compensation:

The compensation rate was a $10 gift card for completing the survey (15 minutes to complete). The survey was distributed in one of two ways: via postal mail or in-person. The estimated response rates were 30% for mail-in survey and 80% for in-person.

National Survey of U.S. Long Haul Truck Driver Injury and Health:

The compensation was a $25 gift card to those who completed the main interview and $2 to those that completed the non-respondent interview. The expected response rate was 80%.

10. Assurances of Confidentiality.

This collection will be kept private to the extent possible under law. Data will be treated in a secure manner and will not be disclosed, unless the FMCSA is otherwise compelled by law. Respondents’ identifying information will not be included on study materials. A unique study ID will be assigned to each participant to link their responses for the paper/online questionnaire and in-person interview. After an individual is finished with the project, his or her personal information will be deleted from the tracking file. To further protect the privacy of respondents, this project’s data collection and analysis plan is subject to approval by the University of Washington’s Institutional Review Board (IRB). The primary purpose of the IRB is to protect the rights and welfare of the people involved in research. The IRB’s approval form has been attached as Attachment J.

11. Justification for collection of sensitive information.

This information collection involves some data of a sensitive nature, such as demographic data, health information, and crash history. These questions are important to linking driver factors to safety and perceptions of OBMS and are therefore necessary to meet research goals. Additionally, participants will be informed that they do not have to answer any questions that they consider to be of a sensitive nature.

12. Estimates of burden hours for information requested.

The OBMS FOT survey is composed of four unique surveys administered six times over 18 months. The pre-study (IC 1) and exit interview (IC 4) will be administered one time and are estimated to take 20 minutes (0.33 hours) to complete. The intervention questionnaire (IC 2) and withdrawal questionnaire will be administered five times each and are estimated to take 15 minutes each (0.25 hours) to complete. The response times were determined through pre-testing of an initial survey on four CMV drivers. The information collection seeks responses from 500 drivers via mail-in/website survey and the same 500 drivers via in-person interviews. All information will be collected in years 1 and 2 of the information collection. Total hourly burden for the entire 18-month period is shown in the table below.

Total Burden of OBMS FOT CMV Driver Survey

	IC-1 Pre-study questionnaire (mail-in/website) Form MCSA-5852	IC-2 Intervention questionnaire (mail-in/website) Form MCSA-5853	IC-3 Withdrawal questionnaire (mail-in/website) Form MCSA-5854	IC-4 Exit interview Form MCSA-5855	Total for ICs 1-4
Respondents	500	500	500	500
Time per Response	20 minutes	15 minutes	15 minutes	20 minutes
Year 1 Responses per Respondents	1	3	3	0	7
Year 1 Total Responses	500	1,500	1,500	0	3,500
Year 1 Total Hourly Burden	167	375	375	0	917
Year 2 Responses per Respondents	0	2	2	1	5
Year 2 Total Responses	0	1,000	1,000	500	2,500
Year 2 Total Hourly Burden		250	250	167	667
Average Annual Responses	250	1,250	1,250	250	3,000
Average Annual Hourly Burden	83.3	312.5	312.5	83.3	792

Estimate of Average Annual Burden: 792 hours [250 responses x 20 minutes/60 minutes to complete for IC 1 + 1,250 responses x 15 minutes/60 minutes to complete for IC 2 + 1,250 responses x 15 minutes/60 minutes to complete for IC 3 + 250 responses x 20 minutes/60 minutes to complete= 791.6 rounded to 792 ].

Annual Number of Respondents: 500 drivers.

Average Annual Number of Responses: 3,000 responses [3,500 responses for Year 1 + 2,500 responses for Year 2/2Years = 3,000].

13. Estimates of total annual costs to respondents.

There are no costs to respondents beyond those associated with the annual hourly burden (not to be included here).

14. Estimate of cost to the Federal government.

The total Federal government budget for this information collection is $541,150 and itemized in the table below. The hours and costs noted were based on agreed upon labor hours and rates between US DOT – FMCSA and the University of Washington. The period of performance for data collection is 24 months.

COST TO FEDERAL GOVERNMENT

Tasks	Hours by tasks	Other expenses (travel, postage, printing)	Total costs by task
Prepare Institutional Review Board (IRB) materials	450		$16,700
Development of survey design and protocol	1140		$44,800
Pilot test and pre-testing of survey instrument	129.6	$20,950	$28,600
Acquisition and conduct of survey via driver interviews	1758	$192,466	$258,400
Acquisition and conduct of survey via mail-in	2358	$35,937	$122,000
Data and statistical analysis	1812		$70,650
Total costs, all tasks			$541,150

15. Explanation of program changes or adjustments.

This is a new information collection.

16. Publication of results of data collection.

The results of the surveys would be analyzed and integrated into the main FOT study report. Data collection will be completed in 18 months, followed by statistical analysis. Both descriptive and analytical methods will be employed during the data analysis. Statistical methods, such as chi-square, odds ratio, cluster analysis, and regression will be adopted when appropriate. The results of the study would be documented in a technical report to be delivered to and maintained by FMCSA. This report would be available to the public at the FMCSA web site at www.fmcsa.dot.gov.

17. Approval for not displaying the expiration date of OMB Approval.

FMCSA is not seeking an exemption from displaying the expiration date on the information collection questionnaires.

18. Exceptions to certification statement.

None

References:

Bunn, T. L., Slavova, S., Struttmann, T. W., & Browning, S. R. (2005). Sleepiness/fatigue and distraction/inattention as factors for fatal versus non fatal commercial motor vehicle driver injuries. Accident Analysis and Prevention, 37, 862-869.

FMCSA(2009). Large truck and bus crash facts 2007 (No. FMCSA-RI-02-011). Washington, DC: US Department of Transportation.

Hanowski, R. J., Perez, M. A., & Dingus, T. A. (2005). Driver distraction in long-haul truck drivers. Transportation Research Part F, 8, 441-458.

Murray, D. C., McDonald, W., Hickman, J., & Bergoffen, G. (2008). CTBSSP Synthesis 16: Safety impacts of speed limiter device installations on commercial trucks and buses. A synthesis of safety practice. Washington, D.C: Transportation Research Board.

 Attachments:

1. Title 49 U.S.C. § 504 titled, “Reports and records.”

2. Title 49 U.S.C. § 31133 titled, “General powers of the Secretary of Transportation.”

3. Title 49 U.S.C. § 31136 titled, “United States Government regulations.”

4. Title 49 U.S.C. § 31502 titled, “Requirements for qualification, hours-of-service, safety, and equipment standards.”

5. Title 49 CFR § 1.73 titled, “Delegation to the Administrator of the Federal Motor Carrier

Safety Administration.”

6. 60-day Comments Request Federal Register Notice (76 FR 17474), March 29, 2011.

7. Comments Received from 60-day Federal Register Notice

8. FMCSA’s Response to Comments Received from 60-day Federal Register Notice

9. 30-day Comments Request Federal Register Notice (76 FR 37167), June 24, 2011.

10. University of Washington Institutional Review Board (IRB) Approval.

11. Informed Consent Form

12. Questionnaire cover letters, follow-up letters, thanks letters, and telephone follow-up reminders.

10