SectionB_MinerFirst

5

Information Collection Request for

“Assessing and Evaluating Human Systems Integration Needs in Mining”

Brianna M. Eiter, Ph.D.

Cognitive Psychologist

Tel. 412.386.4954

[email protected]

June 5, 2013

Part B: Collections of Information Employing Statistical Methods

Table of Contents

1. Respondent Universe and Sampling Methods…………………………………………………….

2. Procedures for the Collection of Information……………………………………………………..

3. Methods to Maximize Response Rates and Deal with Nonresponse……………………………...

4. Tests of Procedures or Methods to be Undertaken………………………………………………..

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data..

References

B. Collections of Information Employing Statistical Methods

1. Respondent Universe and Sampling Method

According to the National Mining Association slightly more than 50,000 individuals currently work in underground coal mines in the U.S. (NMA, 2011). A description of the mines where data is collected will be provided (e.g., size, location, height of coal seam, type of extraction method, etc.). Because NIOSH cannot control which mines agree to participate in the study, the data collected cannot be considered representative of the entire population of miners working at U.S. coal mines. It should be assumed that the findings of this study are generalizable to other groups of coal miners.

The subjects will be employees from three participating underground coal mines. Mines will be recruited via stakeholder meetings and previous research contacts. The participating mines will be recruited with an attempt to get mines to participate that vary in the number of employees on the continuum of size. An attempt will also be made to recruit miners from both unionized and non-unionized mines. It is expected that the employees of the participating mines will vary along a number of variables including age, gender, and experience. All employees will be given the option of declining individual participation in the study.

It is expected that no more than 300 individuals will participate in data collection, between 150 and 200 underground coal miners, no more than 50 mine safety professionals, and no more than 50 NIOSH employees. A variety of data collection methods will be used during the life of the project. These methods include direct observation, task and cognitive task analyses, a research questionnaire, focus groups as well as through laboratory based experimental methods. Because of the nature, scope, and complexity of the project, data collection is expected to last approximately 36 months. Data will only be collected once with each participating miner; this is not a longitudinal study.

The sampling of participants will be a convenience sampling. Purposive sampling will be used to ensure that the research team gets a broad sample of individuals from the participating mine organizations that represent specific job categories. There are many job types within an underground coal mine. While it is important to understand the tasks and complexities associated with each, the current study will focus on those positions that are the most physically and cognitively demanding to perform. Thus, purposive sampling will be used. Convenience sampling will occur based on the mines and miners and NIOSH employees available to participate at the point in time when assessments are conducted.

The data collected for this study will be both qualitative and quantitative in nature. The data will be examined for correlations between items as well as the reliability of the entire scale. In addition, descriptive statistics (e.g., variability and mean scores for each item across the participants) and tests for statistical significance of group differences will be performed. Demographic information will be collected to describe the sampled group (through means and ranges). Qualitative data will be coded by researchers based on specific themes presented by respondents.

2. Procedures for the Collection of Information

Data will be collected both in the field and in the laboratory. The field and laboratory studies are non-intrusive. For the field studies, no variables will be manipulated. Data will be collected with regards to the tasks and subtasks that are completed while miners perform specific job categories in an underground coal mine. There are many job categories within an underground coal mine. Because of this large number, we will target miners who we believe are in the most physically and cognitively demanding positions during data collection and use this information to guide the research program. In addition to field studies, laboratory studies will also be conducted. These studies will also be non-intrusive. Variables within these studies, however, will be manipulated as we will attempt to establish causation. The data collected will be used to formulate Human Systems Interface (HSI) guidelines that are specific to the mining community.

The design for this study uses six different methods to collect information related to the topic of situation awareness and cognitive workload. These methods are analysis of the Mine Safety and Health Administration (MSHA) database, direct observation, task and cognitive task analysis, research questionnaire, focus groups and experimental research strategy. These different methods complement one another and provide a way to collect both quantitative and qualitative data on the variables of interest. Of particular importance is the degree to which data collected across various methods converge, allowing greater confidence in the conclusions drawn from the data collection. The same methods will be used at all mines where data collection will take place; therefore, all miners taking part in the study will receive the same data collection instruments. This will allow for comparative analyses to be applied to the data if necessary.

Seven methods will be used to collect data throughout the research project. These methods are:

• Analysis of the MSHA Database: will occur prior to the collection of data in the field. Information gained through this analysis will be related to accidents and injuries that occurred within underground coal mines, we are specifically interested in the type of information miners had available at the time of the event and what information was not present. This information will be used to determine which job categories are the most physically and cognitively demanding. Information gained during this analysis will also be used to guide the development of questions for the Research Questionnaire and Focus Groups.

• Direct Observation: will take place at an underground coal mine. Members of the research team will observe an underground coal miner while he performs his job. The goal of this task is to observe the tasks and subtasks that occur while specific jobs within the mine are performed. This task gives the research team a unique opportunity to gain first-hand mining knowledge. Observation notes will be taken during the observation process (see Appendix C1 and Appendix C2).

• Task and Cognitive Task Analysis: is a research technique used to describe the physical tasks and cognitive plans that are required to accomplish a particular work goal. For this task, miners will be asked to sequentially describe the steps taken to perform their job. This task will not be completed while the miner is performing his job; therefore, the description of the task will be from memory. While describing the steps, miners will also be asked to think about what tasks are physically or cognitively challenging and why they are challenging. A pre-determined series of questions will be used during the task and cognitive task analyses. These questions will be finalized after the completion of the MSHA database analysis and the shadowing task (see Appendix D).

• Research Questionnaires: paper-and-pencil questionnaires administered to respondents. These questionnaires provide a quantitative and qualitative means to collect information about tasks and subtasks performed by miner’s working in specific job categories. It also allows for the assessment of the importance of providing different types of information to miners working in an underground coal mine. Questions on the questionnaires will be a combination of close and open-ended questions. Close-ended questions will require respondents to describe their opinions using choices that are provided or by using a Likert-type scale. Open-ended questions will give the participant the opportunity to provide his or her opinion regarding a certain topic. This tool is effective in collecting data from a larger and broader sample of individuals than can be contacted through focus groups and observations alone. Five research questionnaires will be used. Below is a brief description of each questionnaire.

  • General Preference Questionnaire: the goal of this questionnaire is to determine how and when miners working in an underground coal mine prefer to have information about their work environment, the location of themselves, others and equipment communicated to them while they are working. (See Appendix G).

  • Subject Matter Expert Questionnaire: the goal of this questionnaire is to determine how subject matter experts (e.g., experienced continuous miner operators) prefer to have information about their work environment, the location of themselves, others and equipment communicated to them while they are working. Questions focus on the everyday use of information as well as the use of information during an emergency situation. (See Appendix F).

  • Safety Director Questionnaire: the goal of this questionnaire is to determine what machinery and equipment is currently being used within the underground coal mining environment. Questions focus on make and model, number and the people who use the equipment. (See Appendix H)

  • Roof bolter Questionnaire: (See Appendix I): the goal of this questionnaire is to assess and document the opinions and experiences of roof bolter operators both before and after a visual feedback (light) intervention is implemented. Additional information as to the need for area and additional functional lighting will be assessed.

  • Vest Usability Testing: To examine the effectiveness and viability of physically integrating equipment, several groups of miners will be asked to wear mining vests. Various designs and materials for mining vests will be chosen and distributed to miners across job titles. The miners will be asked to wear the vests during their normal working hours over a month. Pre and post opinions concerning the concepts and designs of mining vests and the comfort and usability of the tested vest will be collected using research questionnaires (Appendix J).

• Focus Groups: consists of questions related to the interface design, usability and wearability of the integrated design interface. Questions included will likely focus on miners’ perceptions of the design of the device, its usability as well as its wearability. The same questions will be asked across a representative sample of miners at various mine locations. Focus groups will last approximately an hour. There will be two researchers involved in each focus group, one to lead the discussion while the other is responsible for note taking. Approximately 10 miners will be asked to take part in each focus group. A set of pre-scripted questions serves as the basis for the focus group. The researchers can use discretion in terms of follow-up and additional questions. Focus groups will not be conducted until the end of the second year of the project and the development of the focus group questions depends on the completion of year one work as well as the initial development of the integrated design interface. Therefore, we will submit the focus group questions at the end of the first year of the project.

• Experimental Research Studies: A series of experimental research studies will be conducted to test for usability, changes in cognitive workload, and situational awareness. At this point, these studies have not been designed because the design, manipulated variables, and measured variables will depend on the information gathered from the analysis of the MSHA database and the results of the other tasks. All experimental research studies will be designed by members of the research team and data collection will take place primarily at the NIOSH research facility in Bruceton, PA. There is a possibility that experimental data will also be collected at mine sites.

Members of the research team will enter the information from the survey into a password protected computer database using the Statistical Package for the Social Sciences (SPSS), a word processor program, and a database spreadsheet. All electronically collected data will be stored in a secure location at the NIOSH research facility in Bruceton, PA. There will be no identifying markers placed on individual data forms or within any electronic data files.

3. Methods to Maximize Response Rates and Deal with Nonresponse

It is anticipated that at least 90% of individuals selected to participate in the evaluations will participate – this includes miners who are asked to take part in the direct observation task, the task and cognitive task analyses, the research questionnaire and focus groups. Underground coal miners and NIOSH employees will be asked to take part in the experimental research studies.

The research questionnaires and focus groups will be administered in group sessions and should help to ensure a higher response rate. All other tasks will be conducted one-on-one with a smaller number of participants. Due to normal absences from work, a few miners may be unavailable on the particular days that the data collection activities are conducted.

4. Tests of Procedures or Methods to be Undertaken

Information gathered from the analysis of the MSHA database as well as information from previous research studies (Bureau of Mines, 1977; Steiner & Burgess-Limerick, 2007) were used to guide the development of questions included in direct observation and task and cognitive task analyses. The questions included on the five research questionnaires (General Preference, Subject Matter Expert, Safety Director, Roof Bolter and Cognitive Lighting) have been created based on Human Systems Integration (HSI) research from other fields (e.g., the Army’s HSI standards – MANPRINT). The questions used in the focus groups will be fully developed following an analysis and synthesis of data collected during the direct observation, cognitive task analysis and administration of the research questionnaires.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The persons who will collect and/or analyze the data are listed below.

Project Staff:

Brianna Eiter, Ph.D., Cognitive Psychologist, NIOSH Office of Mine Safety and Health Research, 412.386.4954, [email protected]

Jennica Bellanca, M.S., Biomechanical Engineer, NIOSH Office of Mine Safety and Health Research, 412.386.6445, [email protected]

Justin Patts, B.S., Mechanical Engineer, NIOSH Office of Mine Safety and Health Research, 412.386.6835, [email protected]

Lisa Steiner, MS, CPE, Team Leader, Cognitive Engineering, NIOSH Office of Mine Safety and Health Research, 412.386.6446, [email protected]

References

Bureau of Mines. (1977). Industrial Engineering Study of Continuous Mining Systems. (USBM Contract NO J0357096). McLean, VA: J.J. Davis Associates.

Hedrick, T. E., Leonard, B., & Rog, D. J. (1993). Applied Research Design: A Practical Guide. London: Sage Publications.

Steiner, L., & Burgess-Limerick, R. (2007). Equipment Related Issues and Controls in the USA Underground Mining Industry. Ergonomics Australia Journal, 21(2), 4-11.

Tvaryanas, A.P. (2006). Human systems integration in remotely piloted aircraft operations. Aviation, space, and environmental medicine, 77(12), 1278-1282.