Ssb

Generic Clearance for CDC/ATSDR

Formative Research and Tool Development

Title: Characterization of Haul Truck Health and Safety Issues

Supporting Statement B

February 12^th, 2020

Contact Information:

Jennica L. Bellanca

CDC/NIOSH/PMRD/HFB

(412) 386-6445

[email protected]

Table of Contents

Section

B. Supporting Statement B

1. Respondent Universe and Sampling Methods

2. Procedures for the Collection of Information

3. Methods to Maximize Response Rates and Deal with No Response

4. Tests of Procedures or Methods to be Undertaken

5. Individuals Consulted on Statistical Aspects/Individuals Collecting and/or Analyzing Data

Supporting Statement B

1. Respondent Universe and Sampling Methods

The proposed study will involve collecting data from two samples—haul truck operators and their supervisors, managers, and safety professionals or other individuals who work in a position that supports/oversees haul truck operations. These individuals will be recruited from surface metal, nonmetal, stone, and sand and gravel mines in the U.S. All participants must be 18 years of age or older. All participants must be able to speak and read English. There are no restrictions for participation based on gender, race, ethnic community, socioeconomic status. Haul Truck operators will be targeted based on their experience with a challenging scenario such as a serious incident or near miss related to operation of a haul truck as well as their knowledge and familiarity with the task requirements of operating a haul truck. The managers and safety professionals will be recruited based on their relation to the haul truck operator and the mine.

For this effort, researchers will be aiming to collect data from six mines, with two from each of the flowing size categories to gain a perspective on size related differences: small (≤ 25 employees), medium (26-100 employees), and large (> 100 employees).

In the U.S., according to the Bureau of Labor Statistics (BLS) May 2018 Occupational Employment Statistics data, there are approximately 38,140 workers employed as heavy or industrial truck drivers and 8,410 heavy vehicle and mobile equipment service technicians and mechanics in mining or in support of the mining industry [1]. These workers are presumably employed at one of the 13,046 active mines, where 213 are surface metal, 865 are surface nonmetal, 4,242 are surface stone, and 6,299 are surface sand and gravel [2].

Researchers will utilize convenience and snowball sampling methods for this study. Mining companies will first be contacted through existing professional relationships to determine interest in participating in the research study. If a mining company expresses interest, NIOSH researchers will work with the mine site management or safety professional to identify potential study candidates. The mine site management or safety professional will be provided with information about the study that can be shared with potential participants. Once data collection begins, researchers will ask participants to recommend others that meet the inclusion criteria and may have a unique serious incident or near miss conducive with the study goals.

2. Procedures for the Collection of Information

Information collected for this project will not require rigorous statistical analysis as the main goal of this information collection is to identify themes or patterns in the data that describe the decisions, behaviors, and expectations of haul truck operators to inform future phases of research.

Up to 120 semi-structured interviews across six surface mine sites will be conducted with individuals about their experiences related to haul truck operations, maintenance, and safety. To capture a variety of perspectives, various members of the mine organization will be invited to participate in the interviews. Participants from each mine site may include haul truck operators, maintenance crew members, operations and maintenance supervisors, and manager/safety professionals. To gain an understanding of both the baseline task requirements for haul truck operators and how operators respond to challenging or non-routine scenarios, two interview instruments will be used. The first interview instrument (see Attachment B) will use cognitive task analysis (CTA) methods to gain understanding of the goals, skills, challenges, and task requirements for haul truck operators. The second interview instrument (see Attachment C) will use critical decision methods (CDM) to gain an understanding of how haul truck operators respond to challenging or non-routine scenarios and gain a greater depth of knowledge about these scenarios. Researchers will ask operators to talk through a particularly challenging scenario, or event, from their own experience. An example of a challenging event may be a near miss, a collision involving property damage, or loss of control due to environmental conditions.

Each interview instrument has questions tailored for the participant’s level within the organization (i.e. operator/crew member, supervisor, management/safety professional). Table 1 shows the number of participants anticipated for each participant group. Note that each participant will be asked for one interview using one interview instrument method (i.e. CTA or CDM, not both). Each participant will be asked for approximately 1 hour of their time to complete the interview.

Table 1: Number of Participants by Group

Participant	Interview Instrument	Number of Participants per Mine Site	Total Number of Participants in Study
Haul Truck Operator	CTA or CDM	Up to 16	Up to 96
Operations/Maintenance Supervisor	CTA	Up to 2	Up to 12
Management/Safety Professional	CTA	Up to 2	Up to 12
Total	-	Up to 20	Up to 120

The interviews will occur in locations of convenience for the mining company and participants. Possible locations include in the mine office, maintenance shop, training center, or a neutral location (e.g. hotel lobby). The observations will occur at their location of work or as close as possible. Two researchers will attend each observation or interview unless prohibited by the mine site or participant. One researcher will moderate, while the other will take notes. The interviews will also be audio recorded as a requirement for participation. Prior to the interview, participants will be read a verbal informed consent and asked to give verbal affirmation that they agree to participate in the study. Any participants that do not wish to participate will be given the opportunity to leave. Copies of the informed consent will be available to all participants.

Researchers may conduct interviews with mine site management or safety professionals over the phone. In these cases, researchers will provide a copy of the informed consent forms via email or fax prior to the interview. At the start of a phone call interview, participants will be read a verbal informed consent and asked to give verbal affirmation that they agree to participate in the study. Any participants that do not wish to participate will be given the opportunity to leave the call.

3. Methods to Maximize Response Rates and Deal with No Response

Within the U.S. mining sector, haul-truck-related accidents continue to be one of the leading health and safety concerns; therefore, it is anticipated that the response rate will naturally be high. Based on the consent rate from previous studies involving a similar sample, a response rate greater than 80% is expected.

To maximize the response rate, NIOSH researchers will work to (1) clearly communicate expectations with respondents, (2) offer convenient times and locations for respondents to participate, and (3) ensure the interviews do not exceed the allotted amount of time to minimize the overall burden.

4. Tests of Procedures or Methods to be Undertaken

For this study, CDC/NIOSH behavioral research scientists internally reviewed the interview protocols and data analysis plan to ensure readability and relevance to the U.S mining sector.

Research Instruments

Cognitive Task Analysis (CTA) is a common method used to identify and understand the cognitive processes, communication, and situational awareness requirements of a task [3]. Various forms of CTAs have been employed to support designs in many domains including military [4], aviation [5], driving [6], and manufacturing [7]. CTA methods help us extract, organize, and represent the knowledge that operators in various roles with varying levels of experience and expertise rely on to execute tasks. In general, CTA methods help elicit the following information:

• How do expert operators differ from novice operators?

• What do they pay attention to?

• How do they know what is important?

• How do they diagnose problems?

• How do they make sense of and mentally organize their work?

For this project, using CTA, the researchers are primarily interested not just how operations should be carried out, but how they actually are carried out, during both routine and non-routine situations. While CTA employs many different methods, for this project, the researchers will focus on two of them: 1) Semi-Structured Interviews, and 2) Critical Decision Method. The Semi-Structured Interview provides us a general understanding of an operator’s mental models, revealing both successful and vulnerable aspects of daily operations. The Critical Decision Method provides a more detailed map of the intuitive mental models applied in atypical situations, revealing how expertise is challenged when routine practices breakdown, and what strategies were successful and unsuccessful in coping with real world challenges [8].

Cognitive Task Analysis (CTA) Semi-Structured Interview

The purpose of the CTA interview (Attachment B) is to gain a general understanding of their perception of goals, skills, challenges, performance evaluation and current training for haul truck operators and supervisors. Researchers are looking specifically for what differentiates success and failure, both in terms of skills needed for the job, achieving performance targets and then common and uncommon hazards and safety incidents and their root causes.

Critical Decision Method (CDM) Interview

The Critical Decision Method (CDM) interview (Attachment C) is designed to elicit intuitive knowledge that applies to atypical or unexpected situations. Many aspects of decision making are not easy to consciously articulate. Using the CDM method, the researchers help elicit operators’ intuitive (tacit) knowledge by asking them to talk us through a particularly challenging event from their own experience. They describe the event on a timeline, highlight critical decision points, and identify cues that helped them make sense of and respond (effectively or ineffectively) to the situation. The CDM revolves around events that were not only difficult, but ones that challenge the mental models they employ during normal operations.

Data Analysis

Interview data will be analyzed using a grounded theory approach to qualitative data analysis. The grounded theory approach allows theories or hypotheses about behavioral phenomena to emerge from collected data, rather than applying an existing framework prior to data collection [9]. Grounded theory builds from raw data to codes. Coded data is then organized into concepts and categories that build up a larger framework to understand a phenomenon under examination [10]. For this project, the thematic content regarding decisions, behaviors, expectations, prior training and other antecedents to adverse haul truck events will emerge from the interview data to help us build an overall framework. Coding schemes and thematic analysis are developed and expanded upon to determine cognitive and behavioral patterns leading to adverse haul truck incidents.

Working deductively, the researchers will primarily utilize thematic analysis to identify themes and then give those themes codes. A theme is “a pattern found in the information that at minimum describes and organizes the possible observations and at maximum interprets aspects of the phenomenon” [11]. Similarly, Rubin and Rubin [12] described a theme as a reoccurring explanation or summary of what is going on with the data set. Once a theme is identified in the data, the researchers will give it a code. A good thematic code includes “a definition of what the code concerns,” “a description of how to know when it occurs,” and “examples, both positive and negative, to eliminate possible confusion when looking for the code” [11]. Once the researcher observes that a theme is occurring in the data and then describes the theme by giving it a code, this code becomes part of a codebook.

While synthesizing the data, two methods are applied that give greater validation to emergent codes and frameworks: triangulation and disconfirmation. Triangulation aims to validate emergent thematic content through the convergence of multiple sources of data, which may include adverse incident statistics, data from outside resources, corroboration among operators and supervisors, and consistency across knowledge elicitation methods (in this case, between the interviews and the CDM). Triangulation also refers to a process of interrater reliability; wherein coding schemes are applied by different investigators independently and then compared to achieve convergence of interpretation. The researchers will use interrater reliability by having multiple investigators independently code data for emergent themes, and then compare instances of overlap or disagreement, seeking to resolve differences in interpretation. Disconfirmation describes a process by which, once general themes and codes are established, the research team actively seeks disconfirming or negative cases in the data that do not fit the hypothesized code or theme. Disconfirming cases challenge the logic of a code or theme and the code or theme must be revised to include the case or must provide a reasonable explanation for the anomaly. This applies primarily to the stage in which causal links or larger patterns are established.

5. Individuals Consulted on Statistical Aspects/Individuals Collecting and/or Analyzing Data

This information collection request does not employ statistical methods. However, qualitative analysis methods will be used. Interview data will be analyzed using a grounded theory approach to qualitative data analysis. The grounded theory approach allows theories or hypotheses about behavioral phenomena to emerge from collected data, rather than applying an existing framework prior to data collection [9].

The following individuals will assist with leading study design, data collection, and/or data analysis. If additional assistance or guidance is required regarding data collection, analysis, or management, other internal resources are available through teams within the project staff’s branch.

Table 2. Study Personnel

Personnel	Title	Organization	Email
Jennica Bellanca, MS	Mechanical Engineer	NIOSH/PMRD	[email protected]
Whit Missildine, PhD	Research Behavioral Scientist (Contractor)	Viion, Ltd.	[email protected]
Jonathan Hrica, MS	Associate Service Fellow / Mining Engineer	NIOSH/PMRD	[email protected]
Timothy Orr, BS	Lead Computer Engineer	NIOSH/PMRD	[email protected]
Dana Willmer, PhD	Branch Chief / Research Behavioral Scientist	NIOSH/PMRD	[email protected]

References

[1]	Bureau of Labor Statistics, Occupational Employment Statistics Data, Washinton, DC: Department of Labor, 2018.
[2]	NIOSH. Number of active surface mines by sector, 2018. Accessed 2 Feb 2020. https://wwwn.cdc.gov/NIOSH-Mining/MMWC/Mine.
[3]	M. R. Endsley and D. G. Jones, Designing for Situational Awareness: An Approach to User-Centered Design, 2nd ed., Boca Raton, FL: CRC Press, 2012.
[4]	J. M. Riley, M. R. Endsley, C. A. Bolstad and H. M. Cuevas, "Collaborative Planning and Situation Awareness in Army Command and Control," Ergonomics, vol. 49, no. 12, p. 1139 – 1153, 2006.
[5]	M. R. Endsley and R. R. Robertson, "Situation Awareness in Aircraft Maintenance Teams," International Journal of Industrial Ergonomics, vol. 26, no. 2, pp. 301-325, 2002.
[6]	G. Walker, N. Stanton and M. Young, "Hierarchical Task Analysis of Driving: A New Research Tool," in Contemporary Ergonomics, M. Hanson, Ed., London, Taylor & Francis Ltd., 2001, pp. 435-440.
[7]	J. M. Usher and D. B. Kaber, "Establishing Information Requirements for Supervisory Controllers in a Flexible Manufacturing System Using GTA," Human Factors and Ergonomics in Manufacturing, vol. 10, no. 4, p. 431–452, 2000.
[8]	Crandall, B., Klein, G.A. & Hoffman, R.R. (2006). Working Minds: A Practitioner’s Guide to Cognitive Task Analysis. A Bradford Book.
[9]	Glaser BG, Strauss A. Discovery of Grounded Theory. Strategies for Qualitative Research. Sociology Press, 1967
[10]	Glaser BG. The Grounded Theory Perspective III: Theoretical coding. Sociology Press, 2005.
[11]	Boyatzis, R. E. (1998). Transforming qualitative information: Thematic analysis and code development. Sage.
[12]	Rubin, H. J., & Rubin, I. S. (2005). Qualitative interviewing: The art of hearing data (2nd ed.). Thousand Oaks, CA: Sage.