Nafmp Ssb_7.31.2020

Information will be collected to assess the North American Fatigue Management Program (NAFMP), an educational program developed collaboratively by the Federal Motor Carrier Safety Administration (FMCSA), an agency within the U.S. Department of Transportation; Transport Canada (a department within the Government of Canada); and other entities.

We will begin by recruiting 2 medium-to-large commercial motor vehicle (CMV) carriers (each employing >500 drivers) based on the following selection criteria:

[preferred] One carrier based in the U.S. and one carrier based in Canada; alternatively, 2 U.S.-based carriers with operations in both the U.S. and Canada
[required] Mix of long-haul and regional operations
[required] Availability of fleet vehicles currently equipped with an industry standard electronic logging device (ELD) and carrier willingness to permit temporary installation of additional study-related information collection devices
[required] Sufficient number of drivers to support study recruitment goals and willingness to support the participation of drivers in NAFMP training and study-related data collection
[required] Capability and willingness to support involvement of management in appropriate NAFMP training, and study-related data collection

Roles and responsibilities for each CMV will be outlined in a written participation agreement (Attachment E). Although the participating CMV carriers will be selected in a nonrandom fashion, this sampling methodology should not produce any bias in any of the key findings to be generated from the study. This stems from the fact that the research is not intended to produce nationally representative point estimates for any metric, but rather will focus on measuring the relationship between real-time fatigue detection and safety events and the impact of the implementation of the NAFMP (before and after) among long-haul and regional truck drivers.

The potential respondent universe is truck drivers employed at the participating carriers. Those who are employed by carrier will need permission from the carrier because of the data requirements for the study (defined in Part A). The research team will determine each driver’s eligibility for the program based on responses to the online application. Driver eligibility requirements are as follows:

Must be employed at one of the participating carriers for at least one year
Must have a valid Class A commercial driver’s license (CDL)
Must have a valid medical examination card (MEC)
Speak English
Participation in the study is voluntary but drivers must have company approval for participation (if applicable)
Must be “married” to their truck (i.e., drive the same truck). Slip seat (i.e., two or more drivers rotating the same truck, but never in the truck at the same time) and team drivers (i.e., two or more drivers rotating driving duties in the same truck) can only participate if both drivers agree to participate.

We anticipate receiving a pool of potential participants, with the ability to include difficult to reach truck drivers, such as females and minorities. However, there will be no required minimum number of female or minority drivers to be included in the study.

Based on our power calculation and the stated purpose of the study, we will need to study at least 90 drivers, which, given the size of the respondent universe, should pose no problem for recruitment. As drivers will inevitably be lost to follow-up by leaving their companies or dropping out of the study, we expect an attrition rate of at least 40% and will thus need to recruit a convenience sample of up to 180 drivers. For the study to capture a variety of driver types, we will recruit sub-populations from the following categories: 90 drivers from long-haul operations and 90 drivers from regional operations.

In statistical terms, our sample will consist of drivers who, from the perspective of the investigators, form a statistically random sample by the categories listed above. We will recruit from each stratum until we have reached the required quota. The research team estimates that approximately 300 drivers will need to complete the application in order to recruit sufficient drivers for the study.

The number of participants needed was evaluated by the primary outcome metrics: safety events (fatigue/distraction events, crashes, near-crashes, and crash-relevant conflicts) recorded by the real-time fatigue detection system. According to the proposed vendor of the real-time fatigue detection system, a driver typically experiences 10 to 20 such events/month. A Poisson regression model will be used for evaluating the outcome difference and the corresponding Poisson sample size evaluation was used. The basic setup is shown in Table 1: here we model only 90 drivers as the power calculations for all scenarios using 180 drivers were all greater than 95%.

Table 1. Assume N1=N2 (same participants for before and after).

	Before period	After period
Observation Time (Month)	T1=3	T2=5
Sample size	N1 = 90	N2 = 90
Event rate	= 10, 15, 20

Testing hypothesis¹^,²:

Shape3

Where Shape4 Shape5 is the event rate ratio between the baseline (before) and after intervention phases. Set Shape6 Shape7 as 0.8, 0.85, 0.90, 0.95, 0.96, (5% to 20% reduction in event rate), the corresponding sample size is in shown in Figure 1.

Shape8

Figure 1. Sample Size Calculations for Various Power

Table 2 shows the numeric results from the power analysis. As displayed in Table 2, most of the scenarios achieve greater than 80% power to detect a significant difference. The only scenario that did not achieve this level of power was an average of 10 events/month/driver with an estimated rate reduction of 5%. Thus, it is clear that any scenario using 180 drivers will achieve 95% or greater power; however, some scenarios using 90 drivers (depending on attrition) may not achieve sufficient power. We did not present any hypotheses related to differences between regional drivers and long-haul drivers; however, given the results in the power analysis, we may be able to nest the analyses based on this grouping variable.

Table 2. Numeric Results for Two-Sample Poisson Test.

Events/Month/Driver (λ1)	% Reduction in Rate Ratio ( )	Power
10	5	0.6659
10	10	0.9945
10	15	1.00
10	20	1.00
15	5	0.8145
15	10	0.9998
15	15	1.00
15	20	1.00
20	5	0.9010
20	10	1.00
20	15	1.00
20	20	1.00

2. PROCEDURES FOR THE COLLECTION OF INFORMATION

The evaluation is based on a prospective, observational before-and-after study design. Carrier management, CMV drivers, and other personnel will complete their respective training modules. Primary study outcomes focus on drivers’ sleep and driving performance. Information relevant to these outcomes will be collected during a 3-month baseline observation period followed by drivers’ completion of NAFMP Modules 3 and 8, and additional data collection during a 5-month post-training observation period. Drivers will self-report information about their satisfaction with the training, sleep patterns, and behaviors and attitudes that correlate with fatigue. Each driver will report information to the study team primarily using a dedicated Smartphone with pre-installed apps (study-specific questionnaires). The study team will also collect objective information about drivers’ anthropometric measurements, as well as real-time data about sleep patterns, hours on duty, and driving performance. The majority of objective data will be sourced from an actigraph (a device worn by the driver); from specialized equipment installed on the driver’s vehicle (a standard electronic log device, plus study-specific equipment); and from safety, crash, and administrative data reports obtained from the carrier. Finally, the study team will collect information from carriers about management practices relevant to fatigue reduction, and the costs of implementing the NAFMP.

Information collection procedures will be divided into the following phases:

An online application (Attachment I) will be used for driver recruitment. This will be open to all company drivers interested in participating in the study. The research team estimates that approximately 300 drivers will need to complete the application in order to recruit sufficient drivers for the study. The application form will be used to determine drivers’ eligibility for the study.
An onsite briefing session will be conducted by the research team with eligible drivers. During the briefing session, drivers will review the informed consent form, have the opportunity to ask questions, and complete the Background Questionnaire (Attachment J) on-site.
As part of the field study, drivers will participate in a data collection period for up to 8 consecutive months. Information collection will involve:

A real-time fatigue detection system for monitoring driver fatigue and safety performance. The detection system will be installed by the device vendor. Participating truck company will agree to allow the installation of these devices in their trucks.
A smartphone app (see Attachment L), including:

Psychomotor vigilance test (PVT) (a 3-min serial reaction time task); see Attachment M
Karolinska sleepiness scale (KSS) (a self-report of fatigue)
A sleep log (for reporting sleep and wake times, sleeper berth use, flexible sleeper berth exemption use, and sleep quality)
A fatigue scale (for reporting subjective fatigue)
A stress scale (for reporting subjective stress)
A difficulty of drive scale (for reporting subjective difficulty in daily drive)
A driving hazards scale (for reporting experiences of driving hazards)

ELD (for measurement of duty and driving times). Trucking companies are required to have ELD in their truck by Federal Motor Carrier Safety Administration.
Wrist actigraphy (non-invasive measure of sleep and wake activity). Actigraphy is a minimally obtrusive, validated approach to assessing sleep/wake patterns^3,4,5, similar to a fitbit (see Attachment K). Wrist activity, analyzed in conjunction with the records from a sleep log, is a valid and sensitive methodology for measuring sleep and is the methodology of choice for field studies because it is non-invasive.
Monthly telephone briefing sessions with each driver will be conducted by the project team at Virginia Tech Transportation Institute.

Several repeated questionnaires, including the Exercise and Food Consumption Questionnaire (Attachment N), Quality of Life Short Form 36 version-2 Questionnaire (SF-36v2) (Attachment O), Family Interactions Questionnaire (Attachment P), and the Job Descriptive Index (Attachment Q). These will be online surveys.
Company records such as number of crashes, number of moving violations, ELD data, number of workers’ compensation claims, amount of workers’ compensation payment, number of sick leave days, vehicle miles traveled by vehicle, fuel costs by vehicle, vehicle maintenance costs, number of hours that were used by drivers for NAFMP training, and number of hours that were used to implement NAFMP program by manager will be retrieved and transferred from company management to the project team monthly (see Attachments F-1 and F-2).
During a debriefing session (at the end of this study), over-the-phone or web-based, drivers will complete a final debriefing questionnaire (Post-Study Questionnaire) concerning their involvement in the study (see Attachment R, Attachment S, and Attachment U).

For a summary of the timing and frequency of these activities, see Attachment T, Overview of Key Activities and Data Collection.

Information collection procedures may be adapted to incorporate precautions for minimizing the risk of transmitting or contracting COVID-19.

3. METHODS TO MAXIMIZE RESPONSE RATES AND DEAL WITH NO RESPONSE

Participants will be recruited based on their employment status at one of the participating carriers that have volunteered their support for this study. The participants will be told their participation is voluntary, and they can terminate their participation at any point without prejudice or harm to them in any way. They will be informed their participation will not affect their employment. The opportunity to fully understand that this is an 8-month study, and to have any questions answered prior to deciding to participate, should increase the likelihood that participants complete the entire study.

To increase response rate in the initial online application, companies supporting the study will be asked to distribute brochures or other study-related recruitment information. Once drivers are enrolled in the study, an intensive approach will be adopted to maximize participant retention in this longitudinal study and compliance with study protocols. Most study measures will be collected in near-real-time and reviewed daily to detect missing, spurious, or corrupt data, or device hardware or software failures. When a protocol deviation is detected, a member of the study team will contact the participant by phone to understand the source of the problem and provide corrective feedback. Immediate feedback is key to setting consistent demand characteristics throughout the entire data collection phase of the study.

We will offer incentives to promote interest in participating in the study and retention over the study period. The proposed incentives have been reviewed and approved by the Virginia Tech Institutional Review Board (IRB) and include incentives for the completion of individual portions of the study, and care for and return of study equipment. Drivers will receive an incentive for attending the briefing session, for each day of participation, and for the return of the study equipment. Incentives will be distributed via rechargeable debit cards which the participants will receive at the initial project briefing.

During the study recruitment and data collection phases, investigators involved in data collection will prepare written reports on carrier recruitment and data collection, including the number of drivers participating in the study and the status of the data generated by each driver, and provide these reports to NIOSH on a weekly basis. The research team will perform data quality assessments on a daily basis and will have monthly telephone conversations with each participating driver to answer questions and discuss study progress. During their monthly briefing, participants will receive feedback relative to adherence with study protocols and compliance with study measures. More frequent contact will be conducted for those participants that do not adhere to study protocols. The research team will inquire about, document, and seek to rectify any data anomalies detected during the quality control process. Based on previous experience by the research team, this frequent communication between drivers and researchers helps build a rapport that is beneficial for driver retention.

For subjects enrolled in the study, we expect that 50% to 60% will complete the entire study. Thus, of the 180 who begin the study, 90 to 100 will complete the full study. We will evaluate the partial data from those who do not complete the full study and compare their data to those who have completed the full study. We will present the implications of any potential non-completion bias in published study results.

4. TESTS OF PROCEDURES OR METHODS TO BE UNDERTAKEN

The data collection equipment and smartphone apps have been successfully used in prior studies with truck drivers.^{3, 4} Part of the NAFMP was pilot tested by the developer during its development stage and the pilot test suggested positive trends in sleep duration, sleep efficiency, and reduced critical events after the implementation of the driver module.⁵ However, the observation period in the pilot test was too short (9 days before and 9 days after) and included too few participants (77) to adequately evaluate if the NAFMP was effective in reducing fatigue.⁵

A pilot test was also conducted on the questionnaires that were developed for the proposed study, including the Background Questionnaire, Exercise and Food Questionnaire, Family Interactions, and Post-Study Questionnaire. This pilot test only included new questionnaires that had not been tested in the field and/or validated through prior use. The pilot test included nine truck drivers (six males and three females). The responses on these questionnaires were not recorded; however, we recorded data on the driver’s opinions and thoughts regarding the items in the questionnaires. These issues included incorrect sequencing, cognitively burdensome questions, or response options that were not mutually exclusive. The questionnaires were revised accordingly. Additional information is available in the Pilot Test Report (Attachment U).

5. INDIVIDUALS CONSULTED ON STATISTICAL ASPECTS AND INDIVIDUALS COLLECTING AND/OR ANALYZING DATA

Project Leads for this information collection request:

Guang X Chen, MD Jim Collins, Ph.D.

Epidemiologist Branch Chief

NIOSH NIOSH

Division of Safety Research Division of Safety Research

(304) 285-5995 (304) 285-5998

[email protected] [email protected]

Project officer and also responsible for Contributed to development of

receiving and approving contract deliverables the project

W. Karl Sieber, Ph.D. Tim Bushnell, Ph.D.

Research Health Scientist Economist

NIOSH NIOSH

Division of Office of Director

Field Studies and Engineering (513) 458-7108

(513) 841-4231 [email protected]

[email protected]

Contribute to development of Contribute to development of

The project the project

Contractor Contractor

Jeffrey Hickman, Ph.D. Feng Guo, Ph.D.

Research Scientist Statistician

VTTI VTTI

3500 Transportation Research Plaza 3500 Transportation Research Plaza Blacksburg, VA 24061 Blacksburg, VA 24061

(540) 231-1542 (540) 231-1038

[email protected] [email protected]

Contributed to data collection design Will assist in data analysis

Will collect the data and assist in

data analysis

Consultant Consultant

Theresa Hallquist Pierre Thiffault, Ph.D.

USDOT Project Manager Senior Research Analyst

Federal Motor Carrier Safety Administration Transport Canada

1200 New Jersey Ave SE 330 Sparks

Washington, DC 20590 Ottawa, Ontario, K1A 0N5

(202) 366-1064 (613) 993-8552

[email protected] [email protected]

Consulted on data collection design Consulted on data collection design

1 Gu, K., Ng, H.K.T., Tang, M.L., and Schucany, W. (2008). Testing the Ratio of Two Poisson Rates. Biometrical Journal, 50(2), 283-298.

2 Huffman, M. (1984). An Improved Approximate Two-Sample Poisson Test. Applied Statistics, 33(2), 224-226.

³Van Dongen, H., & Mollicone, D.J. (2014). Field Study on the Efficacy of the New Restart Provision for Hours of Service (No. RRR-13-058).

⁴Hanowski, R.J., Hickman, J.S., Fumero, M.C., Olson, R.L., & Dingus, T.A. (2007). The sleep of commercial motor vehicle drivers under the 2003 hours-of-service regulations. Accident Analysis and Prevention 39:1140-1145.

⁵Dinges, D.F., Maislin, G. Brewster, R.M., Krueger, G.P., & Carroll, R.J. (2005). Pilot test of fatigue management technologies. Transportation Research Record: Journal of the Transportation Research Board No. 1922, Transportation Research Board of the National Academies, Washington, DC, 175-182.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	Department of Transportation Office of the Chief Information Officer
Author	herman.dogan
File Modified	0000-00-00
File Created	2021-01-13