Supporting Statement A

Generic Clearance for CDC/ATSDR

Formative Research and Tool Development

Formative Evaluation of an Immersive VRMine Rescue Contest Simulation Exercise

0920-1154

October 26, 2020

Contact Information :

Cassandra Hoebbel

NIOSH - PMRD

Centers for Disease Control and Prevention (CDC)

412-386-6133

[email protected]

Table of Contents

Section

A. Justification

1. Circumstances Making the Collection of Information Necessary

2. Purpose and Use of the Information Collection

3. Use of Improved Information Technology and Burden Reduction

4. Efforts to Identify Duplication and Use of Similar Information

5. Impact on Small Businesses or Other Small Entities

6. Consequences of Collecting the Information Less Frequently

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

9. Explanation of Any Payment or Gift to Respondents

10. Protection of the Privacy and Confidentiality of Information Provided to Respondents

11. Institutional Review Board (IRB) and Justification for Sensitive Questions

12. Estimates of Annualized Burden Hours and Costs

13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

14. Annualized Cost to the Federal Government

15. Explanation for Program Changes or Adjustments

16. Plans for Tabulation and Publication and Project Time Schedule

17. Reason(s) Display of OMB Expiration Date is Inappropriate

18. Exceptions to Certification for Paperwork Reduction Act Submissions

Attachments

Attachment A – Authorizing Legislation (Federal Mine Health and Safety Act 1977)

Attachment B – Exploration Team Survey

Attachment C – Briefing Officer Survey

Attachment D – Team Trainer Interview Guide

Attachment E – Recruitment Script

Attachment F – Recruitment Script

Attachment G – Informed Consent

Attachment H Federal Register Notice

A. JUSTIFICATION

1. Circumstances Making the Collection of Information Necessary

The Centers for Disease Control and Prevention (CDC), National Institute for Occupational Safety and Health (NIOSH), Pittsburgh Mining Research Division (PMRD) seeks approval from the Office of Management and Budget (OMB) to collect feedback from mine rescue teams on the fidelity, usability, and acceptability of an immersive VRMine simulation for mine rescue team contest exercises.

This results from this formative research will be used to build upon an existing technology (VRMine) as an intervention tool to reduce the risk of mine disasters and improve post-disaster survivability of mineworkers by improving the knowledge, skills, and abilities of underground mine rescue teams. The information collection is designed to help CDC NIOSH’s Mining Program understand the specific interests and needs of mine rescue teams in optimally preparing to respond to a mine emergency. Specifically, this formative research activity will be beneficial in ensuring that VRMine is both feasible and acceptable to the underground mine rescue community before it is launched as a substitute for, or supplement to existing mandated mine rescue contests (30 CFR § 49.18).

There is a substantial body of evidence that suggests that the formative quantitative [Brook 1986; Hart 1988; Whitmer & Singer1998] and qualitative [Hix et al. 1999] information collections proposed in this study play complementary roles in iterative design cycles to inform the development and/or improvement of acceptable and useable technological interventions.

Background

According to the National Mine Rescue Association Update for 2020 [Hoebbel 2020], there are 262 underground mine rescue teams (125 coal and 137 metal/nonmetal) in the United States, with a total of approximately 2500 individuals serving (~10 per team) across 32 states. While all miners must prepare for emergencies, these mine rescue teams form the core of the on-the-ground response when a large-scale event occurs, and team members must be prepared to safely enter the complex and dynamic environment that exists in the aftermath of a catastrophic event.

The Federal Mine Safety and Health Act of 1977 (Public Law 91-173, as amended by Public Law 95-164) required mine rescue teams for all underground mines in the United States and was strengthened when the MINER Act of 2006 (Public Law 109-236) mandates minimum training requirements for mine rescue teams. In addition to minimum hours-based training, coal mine rescue teams must participate in a minimum number of local mine rescue contests throughout the year. Due to risks associated with underground mine emergencies, these contests have traditionally been performed using live exercises in above-ground facilities or open fields. Oftentimes these contests utilize low fidelity environments (e.g., stanchion ropes to simulate entries and crosscuts, printed paper placards to represent environmental conditions and downed miners) and participants must adhere to strict rules and are evaluated by a vast team of contest judges. Not only have miners and subject matter experts emphasized the need for more realistic and engaging training environments to enhance the learning experiences of mine rescue teams, diminishing resources across the mining industry have led to reduced participation in training activities beyond what is minimally required.

For these reasons and to meet other programmatic goals and objectives, NIOSH Pittsburgh Mining Research Division’s Virtual Immersion and Simulation Lab (VISLab) has developed a fully immersive, and dynamic virtual training environment (VRMine) for mine safety and health training, which can be modified in terms of both content and portability [Bellanca 2019]. To date, a number of NIOSH virtual reality environments have been used and evaluated as supplemental mine emergency response training activities [e.g., Hoebbel et al. 2015; Bauerle et al. 2016; Connor et al. 2016], but the VISLab’s portable and modifiable immersive technologies have yet to be introduced to the industry.

Through the work proposed within this protocol, NIOSH seeks to capitalize on an invitation from MSHA to partner in putting this technology into the hands of mine rescue teams from around the world while soliciting feedback on its fidelity, usability, and acceptability as an enhancement to traditional mine rescue team training activities. Ultimately, this high-fidelity virtual environment could be utilized for competitive training purposes requiring minimal space and potentially reducing the need for teams to travel great distances for participation beyond the required local contests. As a supplement to current minimal training, this technology could also reduce or eliminate the need for physical participation of field judges through use of its spectator view feature and system generated data capabilities, making competitive drills both more feasible and accessible.

The work supports NIOSH Mining Program Strategic Goal 3 to “reduce the risk of mine disasters and improve post-disaster survivability of mineworkers” and Intermediate Goal 3.4, “workplace solutions are adopted to improve miner self-escape, rescue, and post-disaster survival” [NIOSH 2019a]. Of critical importance is that successful completion of this work is expected to provide opportunities for future efforts that can be designed in response to the NIOSH’s review [NIOSH 2019b] of the Mining Program’s Evidence Package for 2008 - 2018 [NIOSH 2019c] for 2008–2018, which states:

Further human factors research is needed to assist mine rescue teams in the management of a disaster. The tension surrounding one of these events requires a unified approach as to how teams are managed and deployed. As of now, MSHA has the authority over any recovery action, but clear lines of command with an understanding of the responsibilities for all elements of the operation are imperative. Properly handling these complex interactions can mean the difference between success and failure in disaster response [p. 8].

Acknowledging the merit of such an endeavor, the Mining Program’s initial response to this recommendation cited a current lack of adequate resources, which would likely include a robust human factors assessment team, vast physical space, close coordination with MSHA, and the involvement of multiple private and state mine rescue teams and all who had the time, interest, and resources to travel to the study location to participate. The Immersive VRMine can offer unique opportunities to explore how teams are currently managed and deployed at the scene of an emergency while reducing or eliminating the need for a number of the extensive resources previously believed to present insurmountable challenges. In addition to the enhancement these technologies bring to our human factors research capabilities, increased accessibility can also create opportunities for teams to interact more frequently, thereby strengthening individual training experiences as well as team processes believed so critical to successful team performance [Hoebbel et al. 2015].

The information collection activities are limited to formative work that will result in the improvement of tools and methodologies for preparing underground mine rescue teams respond to mine emergencies. The types of information collection activities included in this generic package are:

1. Qualitative interviewing will use volunteer respondents for formative research to improve upon VRMine for use in mine rescue training activities. Interviews may be conducted out in person with mine rescue team trainers. Results of qualitative interviews will be used to improve VRMine’s acceptability to mine rescue team trainers for future use in mine rescue team training activities.

2. Usability testing of VRMine will be conducted with end-users (mine rescue team members) who are the targeted consumers of this technology. This testing will use qualitative and quantitative data collection methods with volunteer respondents to assess the design and use of VRMine.

3. Feasibility testing of this portable technology for mine rescue team training activities will be conducted with observations related to its deployment in the field and user interactions. Information from field testing will be used, as appropriate, to improve methods, materials, instructions to users, technical documentation, and user interactions to enhance the benefits of engaging with the technology and to reduce the burden of future use.

The information collected will not include personally identifiable information such as name, address, medical information, referred individuals, etc. and convenience sampling will be used. Team identities will be coded, and the key will be stored in a location separate from the data and accessible only to the project-specific research staff. Under no circumstances will an individual be identified using a combination of variables such as gender, race, birth date, and/or other descriptors.

2. Purpose and Use of Information Collection

The purpose of this information collection request is to enable CDC to conduct formative research to develop new tools and methodologies supporting the NIOSH Mining Program’s Strategic Goal 3 to “reduce the risk of mine disasters and improve post-disaster survivability of mineworkers” and Intermediate Goal 3.4, “workplace solutions are adopted to improve miner self-escape, rescue, and post-disaster survival” [NIOSH 2019a]. This information collection will include cross-sectional quantitative and qualitative research techniques to inform the development of a population-appropriate intervention that is up-to-date with respect to terminology, present practice and industry need.

Specifically, NIOSH is seeking to gather technological design recommendations based on end-user feedback related to needs and preferences. Because of the NIOSH Mining Program’s need to respond to changes in stakeholder priorities through the development of new projects and subsequent interventions, it is necessary that a combination of formative research methods is undertaken. Key stakeholders and end-users of VRMine are required to perform specific activities that will provide necessary information to provide reactions and make recommendations to ensure that the intervention is acceptable to the target audience. Crucial to this development and/or improvement of VRMine, information collection from members of the target population is required to assure success in intervention development. Reactions of both survey respondents and interviewees will be used to improve VRMine for the purposes of improving the knowledge, skills, and abilities of mine rescue teams, specifically.

This work supports current and future research of health and safety issues related to mine emergency response by characterizing the fidelity, usability and acceptance of this intervention from the perspective of end users (mine rescue teams), with the specific aims of (1) evaluating the technological feasibility of a portable multi-player VR mine rescue simulation activity through interaction with and observation of mine rescue team participants, and (2) evaluate the fidelity, usability, and acceptability of a portable multi-player VR mine rescue simulation activity for mine rescue team training activities from the perspective of the participating teams (i.e. the exploration team, the briefing officer, and the team trainer).

To accomplish Aim 2, it is necessary to collect information from volunteer mine rescue team members after their participation in the activity. Up to 30 mine rescue teams (n=180) will convene for the 2020 IMRC in Beckley, WV (currently postponed until Spring 2021). Volunteer teams will participate in an immersive VRMine rescue scenario utilizing multi-player head-mounted display technology. An orientation to the technology, a mine rescue problem briefing, participation in the exercise, and debriefing will be followed by the evaluation activity, which is expected to take no longer than 30 minutes. Individual volunteer team members and trainers will be asked to participate in the data collection described below. Specific to this information clearance request, up to 5 US-based mine rescue teams (n = 50) will participate in the activity and subsequent information collection.

To gain a holistic view of multiple user types and allow for triangulation of data, participant reactions regarding the fidelity, usability, and/or acceptability of this technology for mine rescue team contest and/or other training activities will be obtained through written surveys to be completed by the exploration team (Attachment B) and the team’s briefing officer (Attachment C) and interviews with the team trainers (Attachment D). Questions are designed to elicit specific information from the participants regarding their experience with the simulation technology utilized in the contest activity and include but are not limited to established and validated scales such as the System Usability Scale [Brook 1986], the NASA Task Load Index [Hart 1988], and Whitmer & Singer’s [1998] Presence questionnaire. Survey instruments will be translated into the participant team’s primary language prior to administration and translators will be used to assist in the facilitation of both surveys and interviews, as available. At least one translator will accompany each non-English speaking team to the competition. Data collection for each participant is expected to last no longer than 30 minutes.

Utilizing the observation guide (Attachment E), researchers will also observe participant interactions with one another and with the VR technology during the activity. Data that may emerge as a result of informal conversations, unplanned activities, significant stoppage in the exercise, other distractions, and anomalies and surrounding circumstances will be recorded. Time-stamped data will also be generated by the VRMine system for later analysis in conjunction with observational data. This observation will present no additional burden to participants.

By using formative research methods, NIOSH and MSHA will develop a greater understanding of mine rescue team needs and preferences as they relate to meeting mine rescue team training requirement. Obtaining this data will be an integral part of concerted efforts to design and provide efficient interventions to this community and will be used with other information to develop the most appropriate intervention. The work proposed does not intend to produce results that can be generalized beyond the scope of the target at-risk population.

3. Use of Improved Information Technology and Burden Reduction

The data collection instruments related to this information collection request include paper and pencil surveys and semi-structured interviews with built-in skip patterns. By nature of these instruments and the in-person data collection setting, the use of technology is minimal. However, the results of the information collection will be combined with time-stamped data generated by the VRMine system, for triangulation and later analyses. The use of this technology will enrich the data generated through human subjects and would be difficult or impossible to otherwise collect.

The use of existing and validated parsimonious construct scales and purposeful streamlining by research partners to produce the fewest number of additional questions practicable will reduce burden to participants. Further reduction in burden will be accomplished as a result of convenience sampling efforts which will target individuals already assembled for the mine rescue team competition following their participation in the VRMine activity.

4. Efforts to Identify Duplication and Use of Similar Information

This formative evaluation is specific to the use of portable VRMine for mine rescue team training activities within the context of underground mine rescue team competitions. Since this technology has never been used nor evaluated within this context, there will be no duplication and similar information is not available.

5. Impact on Small Businesses or Other Small Entities

No small businesses will be involved in the data collection activities.

6. Consequences of Collecting the Information Less Frequently

Individual data collections will be time-limited, occur only once, and be conducted during a one week period. Time-limited, cross-sectional data collection using an adequate convenience sample from the population of interest is both useful and appropriate for formative evaluation data. There are no legal obstacles to minimizing the burden.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This request fully complies with the regulation 5 CFR 1320.5.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agencies

The Federal Register notice was published for this collection on August 23, 2019, Vol. 84, No. 164, pp. 44308. (See Attachment H). No public comments were received

9. Explanation of Any Payment or Gift to Respondents

No payments or gifts will be offered as incentives to participate in this study.

10. Protection of the Privacy and Confidentiality of Information Provided by Respondents.

Data collection activities do not require respondents to provide personally identifying information to project staff or to answer any sensitive questions. The Privacy Act does not apply. This project has been determined to be a non-research activity and IRB approval is not required, however the same human subjects protections will be imparted to the respondents.

Participation in this formative evaluation activity is strictly voluntary. Participants will be informed of the purpose of the study, specific procedures that will be conducted, and a description of protections for their privacy and confidentiality. Respondents will provide verbal informed consent prior to the start of information collection and will be allowed to ask questions about the project before deciding whether to participate or not.

Persons participating in this project will be informed that all data will be maintained in a secure manner, and that the data will only be used for purposes stated during the consent process. Although the identities of respondents may be known to project personnel who conduct interviews and interact with respondents, no sensitive topics will be addressed and no personally identifying information will be collected or stored. Only authorized project staff will have access to study information and all information will be kept on a secured server or in a locked cabinet and/or locked office with limited access.

11. Institutional Review Board (IRB) and Justification for Sensitive Questions

IRB Approval

An appropriate CDC official has determined that this data/information collection is not research involving human subjects and IRB approval is not required. This data/information collection does not require participants to answer questions of a sensitive nature.

A.12. Estimates of Annualized Burden Hours and Costs

The annualized response burden is estimated at 17.5 hours.

The proposed research study involves conducting one-time, semi-structured interviews with up to five mine rescue team trainers, five briefing officers, and 25 exploration team members. Each respondent will be asked for approximately 30 minutes of their time to complete data collection. Estimated burden for recruitment and informed consent of each respondent is shown in Exhibit A.12.A below.

Exhibit A.12.AEstimated Annualized Burden Hours

Category of Respondent	Form Name	Number of Respondents	Number of Responses per Respondent	Average Hours Per Response	Total Response Burden (Hours)
Individual	VRMine Rescue Competition Exercise Evaluation Survey: Exploration Team Members	25	1	30/60	12.5
Individual	VRMine Rescue Competition Exercise Evaluation Survey: Briefing Officers	5	1	30/60	2.5
Individual	VRMine Rescue Competition Exercise Evaluation Interview: Team Trainers	5	1	30/60	2.5
Total		35			17.5

A.12.B Estimated Annualized Costs

The annualized cost to the respondent is segmented accordingly in Exhibit A.12.B. The United States Department of Labor, Bureau of Labor Statistics, May, 2018 (https://www.bls.gov/oes/2018/may/naics3_212000.htm) data were used to estimate the median hourly wage rate for occupations within the occupational code NAICS 212000 – Mining (except Oil and Gas).

Exhibit A.12.B. Annualized Cost to Respondents

Activity	Total Burden Hours	Hourly Wage Rate	Total Respondent Cost
Data collection	17.5	$20.80	$364

A.13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

CDC does not anticipate providing start up or other related costs to private entities.

A.14. Annualized Costs to the Government

Data collection is time-limited and conducted only once. Hourly rates for CDC/NIOSH project personnel were obtained from the 2020 General Schedule Pay Table (https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2020/GS_h.pdf). An estimated cost per individual activity is listed below

Expense Type	Expense Explanation	Hours	Hourly Rate	Annual Costs (dollars)
Data Collection, Analysis, and Writing Costs
	Industrial Engineer (GS-9)	16	$21.86	$349.76
	Behavioral Researcher Scientist (GS-12)	80	31.70	2,536
	Mechanical Engineer (GS-12)	120	31.70	3,804‬
	Computer Engineer (GS-12)	60	31.70	1,902
	Computer Engineer (GS-12)	60	31.70	1,902‬
	Graphic Designer (GS-12)	60	31.70	1,902‬
	Computer Engineer (GS-13)	120	37.70	4,524
	TOTAL COST TO THE GOVERNMENT			$16,919.76

A.15. Explanation for Program Changes or Adjustments

Not applicable. This is a new information collection.

A.16. Plans for Tabulation and Publication and Project Time Schedule

This activity is expected to be completed within 12 months and is currently scheduled to occur during the month of April 2021. The activity may be rescheduled within the one year of clearance being requested.

A.17. Reason(s) Display of OMB Expiration Date is Inappropriate

The display of the OMB expiration date is not inappropriate.

A.18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification.

REFERENCES

Bauerle T, Brnich MJ, Navoyski J [2016a]. Exploring virtual mental practice in maintenance task training. J Work Learn 28(5):294–306.

Brooke JB [1996]. SUS - a quick and dirty usability scale. In: ‘Usability Evaluation in Industry’, Jordan P, Thomas B, Weerdmeester B, and McLelland I (eds). Taylor and Francis: London.

Connor BP, Brnich MJ, Mallett LG, Orr TJ [2016]. Effective group training with computer-based virtual environments. Coal Age 121(6):44–49.

Hart SG & Staveland, LE [1988]. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Advances in psychology,52, 139-183.

Hoebbel C, Bauerle T, Macdonald B, Mallett L [2015]. Assessing the effects of virtual emergency training on mine rescue team efficacy. Conference Proceeding, Interservice/Industry Training, Simulation and Education Conference (I/ITSEC), Orlando, FL, November 30–December 4, 2015.

Hoebbel C [2020]. Email communication with Wes Kenneweg, National Mine Rescue Association, NIOSH, April 7.

NIOSH [2019a]. NIOSH Mining Program: Strategic plan FYs 2019-2023: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health. https://www.cdc.gov/niosh/mining/UserFiles/researchprogram/strategicplan/StrategicPlan11-10-2019_508-1.pdf

NIOSH [2019b]. NIOSH Mining Program Review available at https://www.cdc.gov/niosh/programs/review/pdfs/MiningProgramEvidencePackage-508compliant.pdf

NIOSH [2019c]. NIOSH Mining Program: Evidence package for 2008–2018. Pittsburgh, PA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health. https://www.cdc.gov/niosh/programs/review/pdfs/MiningProgramEvidencePackage-508compliant.pdf

Witmer BG & Singer MJ [1998]. Measuring presence in virtual environments: a presence questionnaire. Presence, 7, 225-240.

10