Supporting Statement A_07082014

This is a revision information collection request (ICR) from the National Institute for Occupational Safety and Health (NIOSH), Centers for Disease Control and Prevention. The revisions to the collection instruments are included in this submission. As stated previously in the original ICR justification, research activities involving rank-and-file underground coal miners who participate in the mine escape training may occur at either the Mine Rescue and Escape Training (MRET) Lab or in an off-site classroom or other typical instructional setting either at an above-ground mine safety training facility, mine administration building, or a university or academic environment (hereinto referenced as the “classroom setting”). Having these two subsamples allows us to better assess uses for virtual reality (VR) training applications, determine the potential additive value of training provided in the MRET Lab, and the potential benefits of adapting simulation-based mine emergency training to a broader audience. To accommodate an appropriate amount of mine escape participants for both the MRET Lab modules and classroom settings, we are requesting adding 60 more participants to our 150 participant data collection cap, which would ideally leave us with 30 BG4 participants, 60 mine rescue participants (MRET Lab), 60 mine escape participants (MRET Lab), and 60 mine escape participants (classroom setting), for a new grand total of 210 participants. The overall burden hours are 15 hours more than in the currently approved ICR (32).

A. Justification

1. Circumstances Making the Collection of Information Necessary

This information collection request (ICR) is a revision requesting 3 years of approval from OMB. This collection request describes data collection tasks under the project entitled “Virtual Reality to Train and Assess Emergency Responders” (0920-0975, expiration 7/31/2016). This study is being conducted by the National Institute for Occupational Safety and Health. NIOSH, under P.L. 91-173 as amended by PL 95 -164 (Federal Mine Safety and Health Act of 1977), and PL 109-236 (Mine Improvement and New Emergency Response Act of 2006) (See Attachment A) has the responsibility to conduct research to improve working conditions and to prevent accidents and occupational diseases in underground coal and metal/nonmetal mines in the U.S..

The turn of the 21st century started with much promise for the coal mining industry. Because there was only one underground disaster in the 1990s (South Mountain Coal Company No. 3 Mine in 1992, 8 fatalities), it seemed that emergency response in the United States no longer needed to be a top research priority. Then came disasters at Jim Walter Resources’ No. 5 Mine (Brookwood) in 2001 (13 fatalities); followed by Wolf Run Mining Company’s Sago Mine (2006, 12 fatalities), Kentucky Darby LLC’s Darby No. 1 Mine (2006, 5 fatalities), Genwal Resources’ Crandall Canyon Mine (2007, 6 fatalities), and Performance Coal Company’s Upper Big Branch Mine (2010, 29 fatalities). Those events highlighted the critical need to balance investments to reduce low probability/high severity events with those that focus on frequent, but less severe injuries and illnesses.

One result of the mine disasters of the past decade has been the passing of significant federal and state mine safety and health legislation. These activities directly and indirectly required changes to the training provided to all miners. The MINER Act of 2006 specifically addresses and increases training requirements. Additionally congress passed an emergency supplemental appropriation for NIOSH to accelerate the diffusion of critical safety technologies into the mines. As technologies are developed, training on their introduction, maintenance, and use will be needed.

As we enter the second decade of the 21st century, mine safety and health research is poised to make major strides in improving mine emergency preparedness. One way NIOSH can contribute to this progress is by continuing to conduct research on the latest technologies and methods for miner training. "Training is one of the most important components for preparing miners, rescue teams and other personnel to respond to mine emergencies." (Joseph A. Main, Assistant Secretary of Labor for Mine Safety and Health, (2011) http://www.dol.gov/opa/media/press/MSHA/MSHA20101308.htm) (last accessed 22 March 2012).

The present research project seeks to determine optimal use of virtual reality (VR) technologies for training and assessing mine emergency responders using the Mine Rescue and Escape Training (MRET) Laboratory. Responders include specially trained individuals, such as mine rescue or fire brigade team members, and also managers and miners who may be called upon to respond to an emergency situation. Self-protective actions are also considered responses.

Privacy Impact Assessment

No IIF (“Information in Identifiable Form,” otherwise known as personally identifiable information) is being collected. Information might be collected electronically or on paper (depending on the individual information collection request). Electronic means include handheld devices which allow subjects to interface with the virtual environment. Computer or paper based instruments will be used to collect information from trainees on their perception of the efficacy of VR environments and in measuring trainee learning.

Overview of the Data Collection System

The research under this project focuses on determining optimal use of virtual reality (VR) technologies for training and assessing mine emergency responders. Activities will be conducted in two areas: 1) training assessment, and 2) training development.

To accomplish the goals of the project, researchers will assess participants’ learning as a result of training, as well as perceptions of the efficacy of VR technologies for training mine emergency responders. The portions of the project investigating the use of virtual reality and the MRET Lab in mine emergency training will be conducted with a descriptive research design. This strategy is appropriate when the goal is to “generate ideas and to embark on a field of inquiry that is relatively unknown” and when the purpose is to “learn more about the nature of the program and to identify the lines of inquiry that may be productive to pursue in more focused follow-up studies.” (Hedrick, et el, 1993) For training assessment, quasi-experimental designs will be used. Randomization of subjects and treatments is often not possible when working in applied settings like safety and health training. In a quasi-experimental design, pre/post data can be gathered on the variable of interest while other potentially intervening variables are held as constant as possible.

Training Assessment

The MRET Lab will provide environments where training materials and trainees can be evaluated. Methods for conducting those assessments will be designed and refined with a focus on the following specific aims and related research questions. While exploring ways to use virtual reality environments for training assessment, the research team will be evaluating the initial four modules developed for the lab. The results of these analyses will guide the development of future training.

One aspect of training assessment is to evaluate four mine emergency training modules delivered under the MRET Laboratory development contract. Researchers are seeking answers to the following questions.

Do trainees’ objective-related knowledge and/or skill improve post training?

Written assessments of objective-related knowledge have been developed and will be tested for each module during the first year of the project. The assessment tools were developed in-house and reviewed by subject matter experts. Demographic information will be used to explore the relationship between performance and expertise.

Data will be gathered using the completed assessment tools. Subjects with a range of expertise, from new miners to experienced mine rescue team members, will be included in the participant pool. At least five training sessions will be run for each module. After the resulting data is analyzed, the need for sessions to address specific patterns or anomalies will be determined.

What is participant reaction to each module?

A written questionnaire was developed to capture the opinions of trainees of the four training modules. It will be administered post-training at the same time as the post-training knowledge test. The data will be analyzed to explore differences based on trainee characteristics and also differences across the four training modules.

Next, NIOSH researchers wish to evaluate participant reactions to training experiences provided in the MRET Lab. For this aspect of the research, scientists are looking to answer these questions:

Under what conditions do participants report learning from MRET Lab experiences?

The answer to this research question will be sought through analyses of trainee responses to the questionnaire discussed under No. 2 (above) in relation to characteristics of the training context. Variables such as whether the trainees were a group prior to training (such as a mine rescue team), if the trainees had prior experience with virtual reality training, and the range of expertise of the trainees in the session will be explored.

Under what conditions do participants enjoy MRET Lab training experiences?

The question will be studied as explained in 3.0 above.

To what degree is simulator (or motion) sickness an obstacle to training?

Items from the Simulator Sickness Questionnaire (development by Kennedy, et al., 1993) will be administered post training. Resulting data will be analyzed in relationship to trainee performance and opinion data to determine if sickness influences either variable. If initial training sessions show sickness to impact trainee assessments, then counter measures will be explored. For example, Robert M. Stern, Ph.D (2010) reports “dried ginger root in capsule form helps with the nausea, and eating a high-protein snack every two hours may also minimize symptoms”.

Training Development

In-house training development will target training on benching mine rescue breathing apparatus and will focus on the use of 3D technologies to develop a prototype equipment maintenance training module for a mine rescue closed-circuit breathing apparatus (CCBA). In this arena, researchers intend to answer the follow questions.

How can 3D stereoscopic technologies enhance a training module for benching a Dräger BG4 CCBA?

Will mine rescue benchmen and trainers accept the developed training module?
Will benchmen who have completed the training have more confidence in their knowledge and/or skills?

Post-training questionnaires will be used to gather reaction and task self-efficacy data to answer these questions. Benchmen (mine rescue team members responsible for service and maintenance of CCBAs) will be asked their opinions about the training and to self-assess improvements in confidence and skill.

Will the performance of benchmen improve after completing the training?

Performance data (e.g., speed, accuracy, number of correct responses) will be gathered during training in year 2 of the project. Comparing initial attempts with later attempts will give an indication of training-based improvements. Subjects will be taught to use interaction devices prior to training so their comfort with these tools will not confound the data. Trainee performance at mine rescue contests will also be tracked before and after training. While changes in contest performance will not be able to be directly attributed to training because of other variables that cannot be controlled for the study, the information can provide some indication of the impact of training and contribute to the overall training evaluation.

Items of Information to be Collected

The list of information items to be collected includes, but is not limited to:

Age
Years mining experience
Years of mine emergency experience
Years of mine rescue experience
Pre-training attitudes concerning mine safety and emergency preparedness
Team efficacy
Team performance (measured in self-report questionnaires and through speed/accuracy metrics as part of the simulation data output)
Trainee and trainer responses to questions about the utility of the training specifically and using VR in general for training critical emergency response skills
Trainee responses, pre-and post-training to measure learning
Trainee responses using VR environment interaction devices

No individually identifiable information is being collected.

Identification of Website(s) and Website Content Directed at Children Under 13 Years of Age

Web-based data collection methods will not be used. In addition, respondents will not be referred to websites. No participants will be under 13 years of age.

2. Purpose and Use of Information Collection

Since mining is a hazardous environment, it is extremely important for NIOSH to collect this information. The information will be used by NIOSH researchers to assess the efficacy of using Virtual Reality immersive environments for teaching critical mine emergency response skills and for understanding trainees’ reactions to VR environments. Results from this effort will guide further research in the use of VR environments for teaching other mine safety skills and assess the value of using VR environments for other types of mine safety and health issues including hazard recognition, mine ventilation, and ground control. This project and data collection has already been fully funded by the NIOSH Office of Mine Safety and Health Research.

As mentioned earlier, the information to be collected will aid in improving the safety and health of underground coal miners by assessing the efficacy of immersive virtual mining environments for teaching critical mine safety and health skills. These data is not available from any other sources. It is essential for assessing virtual environments for mine safety and health training and in making recommendations for use of this technology by mine safety and health practitioners.

Privacy Impact Assessment Information

No IIF is being collected.

3. Use of Improved Information Technology and Burden Reduction

Approximately 80% of the information collected via data collection instruments will require respondents to fill out a paper document. Data collection related to the use of technologies and devices within the VR environment will be collected via technology based strategies. In order to reduce burden to the miners, data collection will occur either at the MRET laboratory facility or at a surface mine safety training facility.

4. Efforts to Identify Duplication and Use of Similar Information

Data related to the use of VR environments for teaching critical skills in other populations exists, such as studies by the Department of Defense on soldiers in the military and studies on the use of VR technologies in the medical community. Simulators using partial immersive environment concepts have been developed for teaching underground and surface miners how to operate equipment such as underground continuous miners and surface mine haul trucks. Fully immersive environments are being used in Australia for training mine rescue teams. However fully immersive VR environments have never been used before in the U.S. for teaching and assessing critical mine safety skills. As such there are no similar data available for the population of underground coal miners, mine rescue team members, and mine safety and health trainers in the United States since VR technologies and immersive environments are just being introduced to the U.S. mining community via this NIOSH project.

5. Impact on Small Businesses or Other Small Entities

Approximately 210 trainees will be participating in the series of studies under this project. They will include rank-and-file miners, members of mine rescue teams, and mine health and safety professionals. Many of those participating will be from large mining operations. The number of trainees, who are employed by small mines (operations with 20 or fewer employees), will be minimal. Those who will be participating in this series of studies and data collection efforts will be doing so as part of regular, mandated mine safety and/or mine rescue training. The number of survey questions will be held to the absolute minimum required for the intended use of the data. It is critical that the same number of questions be asked at each of the mines in the sample. Otherwise, it will be impossible to make valid comparisons and to formulate appropriate conclusions and recommendations. Data collection instruments will take less than 15 minutes to complete.

6. Consequences of Collecting the Information Less Frequently

NIOSH’s Mine Rescue and Escape Training Laboratory is the only one of its kind in the United States specifically dedicated to the development and testing of virtual reality based mine safety and health training. If NIOSH does not conduct the subject research, it is doubtful the mining industry, academia or enforcement agencies will conduct such an extensive study to assess the efficacy of virtual reality environments for mine safety training. With the development of any training modules, testing of the material is paramount to ensure the training is appropriately presenting the content. As virtual reality technology continues to evolve, it is essential that research be conducted to assess the usefulness of this technology for teaching critical mine safety and health skills. If this research is not conducted, assessments and subsequent recommendations about how to utilize VR technologies for mine safety training will not be disseminated to coal industry personnel.

There are no legal obstacles to reduce the burden..

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This request fully complies with the regulation 5 CFR 1320.5.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A. A 60-day Federal Register Notice was published in the Federal Register May 16, 2014, Vol. 79, No. 95, pp. 28515-28516 (see Attachment B). No comments were received.

B. An extensive literature review was conducted. There were no personal consults outside NIOSH.

9. Explanation of Any Payment or Gift to Respondents

Respondents will not receive any form of payment or gifts.

10. Assurance of Confidentiality Provided to Respondents

Trainers and trainees will not provide any form of identifying information (e.g., name or SSN); therefore no IIF will be included in the data records. No IIF will be collected as part of the data collection processes. All participants will be assigned a number which will not be linked with a name or other identifying information. All information provided by respondents will be maintained by CDC/NIOSH researchers in a secure manner unless compelled otherwise by law. The data files will be analyzed in the aggregate and no individual respondents will be identified.

This data collection has been approved by the NIOSH Human Subjects Review Board (HSRB) (Attachment E).

Privacy Impact Assessment Information

10-A. The privacy act does not apply to this submission as no information in identifying format will be collected.

10-B. In terms of physical controls, the completed data collection instruments will be stored in a locked file cabinet at NIOSH’s Office of Mine Safety and Health Research (OMSHR) Pittsburgh Office. This is a secure, gated facility with 24 hour security guard service. Only personnel with proper identification badges are allowed access to the site. All of the data will be entered and combined into data files that will be stored with technical safeguards in a secure, password protected location on the OMSHR computer network. This computer network is only accessible to NIOSH employees. All networks at NIOSH are firewall protected and utilize a virtual private network. Access to this information will be restricted to researchers directly involved with the study and who need to view the data. A training session will be conducted for all researchers about the data collection and how the data will be stored. At this training session, all researchers will be made aware of their responsibilities for protecting information being collected and maintained. At the end of the data collection, the surveys will be destroyed.

Technical Controls

Passwords

Firewall

Virtual Private Network (VPN)

Physical Controls

Security Guards

Identification Badges

Office and File Cabinet Locks

Administrative Controls

Access to data limited to researchers directly associated with task

Training Session

10-C. Study participants will be read a waiver of signed informed consent (see Attachment D). The consent form describes the study, the conditions of the study, and the use of information collected from the study. Respondents who agree to participate in interviews and/or focus groups state, “I agree to participate in this study.” Respondents who decline to participate in the interview state, “I decline.” The study researcher who reads the consent document signs it to certify that they have accurately described the study to the respondent.

10-D. Survey respondents are advised in the waiver of signed informed consent that their participation is voluntary (see Attachment D).

No IFF is being collected.

11. Justification for Sensitive Questions

Respondents will not be asked any questions of a sensitive nature.

12. Estimates of Annualized Burden Hours and Costs

The respondents targeted for this study include rank-and-file miners as well as mine safety and health trainers. A sample of 210 individual will be collected from various mining operations and mine rescue teams which have agreed to participate. The amount of time to complete a data collection instrument will be about 15 minutes. Data collection will be done with various data collection instruments. The data collection instruments listed in the table below can best be understood by the training exercises under which they will be used: benching simulation participants, mine rescue module participants, and mine escape module participants. Benching simulation participants will only take a single post-training questionnaire, which should take approximately 3 minutes to complete. Mine rescue module participants will take a pre-training questionnaire and a pre-training knowledge test before the training simulation, a post-simulation questionnaire after the training simulation, and a post-training questionnaire and a post-training knowledge test after the training is complete. The knowledge tests will either have longwall or continuous mining content, depending on which of the two training modules the participants will experience. Each survey should take approximately 3 minutes to complete, for a total of 15 minutes of surveying per mine rescue module participant. Mine escape module participants will undergo the same exact survey sequence as the mine rescue participants, only the mine escape participants will take the mine escape questionnaires instead of the mine rescue questionnaires. Like the mine rescue participants, each survey should take approximately 3 minutes to complete, for a total of 15 minutes of surveying per mine escape module participant.

The following table provides an estimate of the annualized burden hours. The estimates are based on the researcher’s previous experience conducting similar methods of data collection.

Estimated Annualized Burden Hours

Type of Respondent	Form Name	No. of Respondents	No. Responses per Respondent	Average Burden per Response (in hours)	Total Burden Hours
Dräger BG4 participants (i.e., closed circuit breathing apparatus training participants)	Post-Training Questionnaire	30	1	3/60	2
Mine Rescue participants	Pre-Training Questionnaire	60	1	3/60	3
	Post-Simulation Questionnaire	60	1	3/60	3
	Post-Training Questionnaire	60	1	3/60	3
Mine Escape participants	Pre-Training Questionnaire	120	1	3/60	6
	Post-Simulation Questionnaire (MRET Lab version)	60	1	3/60	3
	Post-Simulation Questionnaire (Field Test Version)	60	1	3/60	3
	Post-Training Questionnaire	120	1	3/60	6
Mine Escape/Longwall Mining participants	Pre/Post-Training Knowledge Test	60	1	6/60	6
Mine Escape/Continuous Mining participants	Pre/Post-Training Knowledge Test	60	1	6/60	6
Mine Rescue/Longwall Mining participants	Pre/Post-Training Knowledge Test	30	1	6/60	3
Mine Rescue/Continuous Mining participants	Pre/Post-Training Knowledge Test	30	1	6/60	3
Total					47

B. The estimated total cost for this information collection is $1181.11.

Estimated Annualized Burden Costs

Type of Respondent	Total Burden Hours	Hourly Wage Rate	Total Respondent Costs
Mine Employee	47	$25.13	$1181.11

The value assigned for the hourly wage rate is based on the average U.S. hourly wage rate for coal miners available in the following information: Bureau of Labor Statistics, U.S. Department of Labor, May 2011 National Industry-Specific Occupational Employment and Wage Estimates

NAICS 212100 - Coal Mining, on the Internet at http://www.bls.gov/oes/current/naics4_212100.htm (visited September 4, 2012).

13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

None.

14. Annualized Cost to the Government

Data will be collected four years. The estimated annual cost to the Federal Government is $108,580. This includes data collection by CDC/NIOSH employees, data analysis, and report writing. The hours designated for government staff were calculated as shown in the table below. The total cost average for a four year period is $434,320.

	Hours	Hourly Rate	Cost at Hourly Rate	Other Costs (data collection, etc.)	Total
Federal Government Employee	8320	$51.00	$424,320	$10,000	$434,320

15. Explanation for Program Changes or Adjustments

16. Plans for Tabulation and Publication and Project Time Schedule

Data analyses will be conducted over the life of the project. The project schedules below provide an estimate of data collection activities, analysis, and dissemination.

Project Schedule – Training Assessment:

Subjects	Non-NIOSH government employees (formative evaluation/pilot testing), mine rescue team members, miner trainees, mine safety and health professionals
Data Collection Techniques	Pre/Post-training knowledge test, observation, pre- and post-training questionnaires
Training Materials	4 modules delivered as part of MRET Lab contract (2 mine rescue modules and 2 mine emergency evacuation modules)
Timeline	Year 1&2: Data gathering instrument development and testing; human subjects and Office of Management and Budget approvals Years 3&4: Training model evaluation; MRET Lab use evaluation; MRET Lab use guidelines report; training assessment best practices document

Project Schedule – Training Development:

Subjects	Mine rescue team members and trainers, mine safety and health professionals
Data Collection Techniques	Pre/Post-training knowledge test, observation, post-training questionnaire, interviews
Training Materials	BG4 benching simulation training
Timeline	Year 1&2: Data gathering instrument development and testing; approvals Year 3&4: BG4 benching training evaluation; VR use guidelines reported

17. Reason(s) Display of OMB Expiration Date is Inappropriate

Not applicable. The OMB expiration date will be displayed.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Kelly MeNelis
File Modified	0000-00-00
File Created	2021-01-27

Supporting Statement A_07082014

Supporting Statement A_07082014.docx

Virtual Reality to Train and Assess Emergency Responders

OMB: 0920-0975

Table of Contents

1. Circumstances Making the Collection of Information Necessary