Supporting Statement B Human Error Study

Department of Transportation

Federal Railroad Administration

INFORMATION COLLECTION SUPPORTING STATEMENT B

Experimental Investigation of Automation-induced Human Error in the Locomotive Cab

OMB CONTROL NUMBER 2130-XXXX

1. Description of sampling method to be used.

Not applicable. The survey method of research is not being used. This research is not a survey but a simulator-research laboratory study. There is no sampling method for this research. Human subjects will be selected on the basis of locomotive operating experience.

2. Description of procedures for information collection, including statistical methodology for stratification and sample selection.

Conduct Experimental Scenarios: Each crew in the study will participate in 3 scenarios of approximately 30 minutes each. These scenarios will be run on a 17-mile section of simulated track.

The first scenario is a practice run to get the engineers familiar with the dynamics of the train and the section of track and will be run completely in manual mode, without TripOptimizer(TO) (fuels savings software system) or Positive Train Control(PTC)(safety failsafe software system). The engineers will adhere to speed restrictions required in the scenario.

The second scenario will include the use of TO or PTC and it will include the same speed restrictions from the manual condition in the first scenario, but in this scenario, the speed restrictions will be programmed into TO or PTC. There will be no additional events in this scenario.

The third scenario will include speed restrictions, and three additional events, one event will be a workzone, with information provided to the crew before the run. The other two events will be “unexpected” and provided by a call from an acting dispatcher to the crew. These events will be a temporary speed restriction and a stop and protect instruction to the crew. In this proposed study, the information provided by the dispatcher will be programmed into the PTC or TO by the dispatcher after the dispatcher provides the verbal information to the crew.

In the study, the call from dispatch regarding the unexpected speed restriction will be timed to occur at a maximally disruptive point, just prior to the TO request for information regarding the track (or a similarly disruptive point for PTC interactions, described below). This will insure that all crews receive the disruption at the same time, and all experience the potential interference with noticing the change in automation. When the dispatcher enters information on the temporary speed restriction, this will match the verbal instruction given to the crew (correct location, correct speed restriction).

The second dispatcher call will refer to the stop and protect. Here, the dispatcher will enter the data regarding the upcoming stop and protect into TO or PTC, but he will select an incorrect location. The details regarding this error will depend on features associated with the track segment and simulator programming capabilities, but we expect to have the dispatcher make a very common type of error – transposing digits (or another similar error). If the dispatcher says “stop and protect at the grade crossing at MP 95.7” but incorrectly enters MP 97.5, will the crew notice the mistake? If there are grade crossings at both locations, this mistake might well be difficult to detect.

1. The call from dispatch will arrive just prior to the TO request for information from the engineer. This will cause all engineers to be subjected to the high communication workload at the critical moment the TO requests track information.

2. To provide an interruption just prior to interaction with automation in the PTC scenarios, the dispatcher will contact the PTC crews just after they contact with the workcrew foreman. PTC allows engineers to input when (if) they have received verbal permission from the foreman. Once this occurs, the workzone restriction in PTC is “lifted” or removed. It seems less likely that engineers will forget this step, compared with engineers neglecting to input track information into TO, as 1) there is a visual reminder in PTC that the workzone is still present, 2) PTC will stop the train if the engineer does not enter “permission granted to enter the workzone,” and 3) the dispatcher interruption will occur during another ongoing communication, so the crew may be more likely to defer that communication until they have finished interacting with the foreman.

3. The dispatcher will give instructions regarding the upcoming speed restriction, and will program this information correctly into TO or PTC while (or just after) speaking with the engineer or crew.

4. Include the stop and protect event, but in this scenario the dispatcher enters this information into TO or PTC as (or just after) he gives the instruction to the engineer or crew. However, the dispatcher states, e.g., 95.7 (correct) while entering 97.5. It is important that the true stop and protect is before the one programmed into the automation (TO or PTC). Both MP values will correspond to grade crossings, increasing the likelihood that the actual stop and protect will be passed.

Data collection: Data collected will include event logs from the simulator (automatically sampled at 1 Hz frequency and videos of the crew (overview of the cab, from behind), the engineer (view of the face), the TO or PTC display screens and the out the window view of the simulated environment. In addition, we will collect eye-tracking data from the crew which requires facial view. Further, experimenters will sit near (but outside of) the CTIL to observe and take notes on the crews’ performance during the scenarios.

Participants: Thirty subjects will participate in the study. Fifteen of these will be designated TO engineer crews, and 15 will be designated PTC engineer crews. The TO engineers will have the track data to enter (point 1 in the list above) and the PTC engineers will have the workzone permission to enter (point 2 in the list above). Both types of crews will experience the automation programming error regarding the stop and protect location.

Analyses: The analyses will include progressively detailed assessments.

• One set of analyses will investigate the errors regarding noticing and responding to automation requests.

  • We will tally the times that crews failed to notice the TO request for track information and subsequent display changes, or failed to respond to the PTC request regarding the workzone.

  • We will identify the time between the first request for information (TO – track information; PTC – permission to enter the workzone) and the time at which the engineer entered that information or noticed the change to manual mode.

  • Any cases of unusual behavior (rapid or slow detection) will be evaluated qualitatively to identify possible explanations of compensating behaviors or interfering conditions.

• Another set of analyses will investigate the stop and protect error. We will identify how many crews noticed and how many failed to notice the dispatcher’s data entry error.

  • For those that noticed the error, we will determine the time that elapsed between when they first heard the discrepant information until they indicated that they suspected an error.

  • For those that failed to notice the error, we will investigate and identify where were they looking and what they were doing when the dispatcher stated the discrepant milepost.

• Further, we will remain alert for the occurrence of other errors that might emerge in the scenarios, and will examine them in detail.

3. Description of methods to maximize response rate and to deal with non-response issues.

Not applicable

4. Describe any test procedures for procedures or methods to be undertaken.

Not applicable

5. Provide name and phone number of individuals consulted on statistical aspects of study design and other persons who will collect/analyze information for agency.

None consulted beyond original research proposer.

Point of Contact for the study:

Michael E. Jones

US Department of Transportation

Federal Railroad Administration

Human Factors Division (RPD-34)

Washington, DC. 20594

[email protected]

202-493-6106

Page 1