Ssa

Keeping Cool: Evaluating NIOSH’s Heat Stress Training Module to Inform Resource Development and Research

CDC/ATSDR Formative Research and Tool Development

0920-1154

Supporting Statement A

May 20, 2024

Kristin Yeoman, MD, MPH
Medical Epidemiologist
CDC/NIOSH
315 E. Montgomery Ave.

Spokane, WA 99207

509-354-8067

[email protected]

Table of Contents

Section

1. Justification

1. Circumstances Making Collection of Information Necessary

2. Purpose and Use of the Information Collection

3. Use of Improved Information Technology and Burden Reduction

4. Efforts to Identify Duplication and Use of Similar Information

5. Impact on Small Businesses or Other Small Entities

6. Consequences of Collecting the Information Less Frequently

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

9. Explanation of Any Payment or Gift to Respondents

10. Protection of the Privacy and Confidentiality of Information Provided to Respondents

11. Justification for Sensitive Questions

12. Estimates of Annualized Burden Hours and Costs

13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

14. Annualized Cost to the Federal Government

15. Explanation for Program Changes or Adjustments

16. Plans for Tabulation and Publication and Project Time Schedule

17. Reason(s) Display of OMB Expiration Date is Inappropriate

18. Exceptions to Certification for Paperwork Reduction Act Submissions

Attachments

Attachment A: Federal Mine Safety and Health Act

Attachment B1: Initial Interview with OSH Managers and Trainers Script

Attachment B2: Post-Preparation Interview with OSH Managers and Trainers Script

Attachment B3: Post-Training Interview with OSH Managers and Trainers Script

Attachment C: Employee Survey

Attachment D: Employee Focus Group Script

Attachment E: Heat Stress Training Log

Attachment F: IRB Exemption

Attachment G: Consent form

Goal of the study: The goal of this study is to evaluate the usability of National Institute for Occupational Safety and Health (NIOSH) heat stress training module. NIOSH will conduct a process evaluation to 1) Identify how trainers/occupational safety and health (OSH) managers prepare to provide the training to workers, 2) Evaluate how, when, and why trainers/OSH managers use the training module, 3) Determine whether trainers/OSH managers are satisfied with the training, and 4) Determine whether workers are satisfied with the training.

Intended use of the resulting data: Data will be used to inform improvements to the training module and development of additional training resources, such as an implementation guide. The study will also further NIOSH understanding of what additional resources are needed to advance workplace heat stress education and prevention.

Methods to be used to collect data: The research study will be conducted at up to four U.S. mining or aggregate employers. Up to 20 OSH managers and trainers will be interviewed to evaluate goals of training, usefulness of the guide within the training module, whether trainers are adequately prepared and satisfied with training materials, and their impressions of how well training sessions went. Up to 200 workers who participate in the trainings will complete brief surveys to provide feedback on the training module, and up to 84 workers (included in the 200 workers) will additionally participate in focus groups to provide training module feedback.

Subpopulation to be studied: Data collection will focus on two subpopulations:

1. OSH managers and trainers who make decisions on training needs and who prepare and deliver heat stress trainings.

2. Workers who participate in heat stress trainings.

How data will be analyzed: Interview and focus group results will be analyzed using Hamilton’s Rapid Assessment Process (RAP). If investigators determine that continuing to traditional analysis will be helpful after the RAP, investigators will develop a coding system by independently coding a sample of transcripts and then coming together to refine codes, operational definitions, and coding rules. Data will be examined for within and cross-case differences and findings will be integrated across data collection techniques to identify cross-method themes, contradictions, and lessons learned. For quantitative survey data, investigators will conduct descriptive analyses to summarize the Likert items. If appropriate, investigators will assess group differences within employers as well as multi-level analysis to identify similarities and differences across trainers and employers. Finally, integrative techniques will be used to triangulate the findings and better understand employer and worker perspectives on the heat stress training module. The findings will also inform NIOSH product development and research to improve employer heat stress prevention.

Supporting Statement A

1. JUSTIFICATION

1. Circumstances Making the Collection of Information Necessary

The Centers for Disease Control and Prevention (CDC), National Institute for Occupational Safety and Health (NIOSH) requests OMB approval of a new generic information collection (GenIC) under the Generic Clearance for CDC/ATSDR Formative Research and Tool Development (OMB No. 0920-1154). The purpose of this GenIC is to enable CDC/ATSDR to conduct formative research for developing new tools and methodologies supporting CDC/ATSDR’s research, surveillance, and program evaluation activities. NIOSH seeks approval to conduct an evaluation of a heat strain training module developed for the mining industry. The evaluation will include interviews, focus groups, and surveys of occupational safety and health (OSH) managers, trainers, and miners who take the training. This study is being conducted by the National Institute for Occupational Safety and Health. NIOSH, under P.L. 91-173 as amended by P.L. 95 -164 (Federal Mine Safety and Health Act of 1977, Appendix A), has the responsibility to conduct research to improve working conditions and to prevent accidents and occupational diseases in underground coal and metal/nonmetal mines in the U.S.

Heat strain is an increasing problem in mining. As mines expand into deeper and hotter environments, heat strain among underground miners is likely to increase. Furthermore, heat waves are expected to increase in frequency, length, and intensity, posing a substantial risk to surface miners. Heat strain can lead to a number of adverse health conditions of varying severity (e.g., heat syncope, heat rash, heat exhaustion, heat stroke). Additionally, studies have demonstrated associations between heat exposure and work injuries. Because of these factors, increased risk of heat strain among both underground and surface mines can negatively affect the health and safety of mine workers.

In addition to mines establishing heat management plans to protect their workers, mineworkers need to understand their risk factors for heat strain and how to mitigate them. To assist the mining industry in training workers on heat strain mitigation, NIOSH developed an online heat stress training module specifically designed for use by managers or workers regardless of their previous knowledge of heat strain. The goal is to provide a training module that can be used by anyone conducting a discussion-based training on heat strain. Resources for the trainer to prepare for and conduct the training are provided in the training module. However, it is unknown whether these resources provide sufficient information and preparation for trainers (who are not experts in heat strain) to effectively facilitate the discussion-based training for miners. To ensure that the NIOSH heat strain training module and supportive resources are usable, adaptable, and beneficial to mining employers and workers, it is important to evaluate how the module is used by trainers and received by workers.

2. Purpose and Use of Information Collection

The purpose of this formative, exploratory process evaluation of NIOSH’s heat stress training module is to inform improvements to the module and development of additional training resources, such as an implementation guide. The study will also further NIOSH understanding of what additional research is needed to advance workplace heat strain education and prevention.

To ensure that the NIOSH heat stress training module meets the needs of employers and workers, it is important to evaluate how the module is implemented by end users (i.e., OSH managers and trainers) in mining-related workplaces. While the training module was designed to provide workplace trainers all the materials needed to provide the training, it is important to assess whether trainers feel adequately supported and prepared to facilitate the training. Further, while the online training module does not currently include an independent, downloadable implementation guide, employers may benefit from a “how to get started” user guide. It is important that the development of this guide be informed by field work in collaboration with end users to promote its useability, acceptance, fit, integration, and effectiveness.

To inform module improvements and additional implementation and prevention resources, the process evaluation includes these objectives:

• Identify how trainers/OSH managers prepare to provide the training to workers.

• Evaluate how, when, and why trainers/OSH managers use the training module.

• Determine whether trainers/OSH managers are satisfied with the training.

• Determine whether workers are satisfied with the training.

The study will include interviews with OSH managers and trainers, as well as focus groups and surveys with workers who take the training. NIOSH will conduct up to five interviews with each OSH manager or trainer (N=20) at the following time points: 1) one initial interview at the outset of the study (30 minutes), 2) one post-preparation interview after the trainer prepares for delivery of the training module but prior to the initial training session (20 minutes), and 3) up to three interviews after training has been completed (i.e., immediately following the first, second, and last training sessions; 30–40 minutes). The number of post-training interviews will depend on the number of times a trainer provides a training session. If they provide only one training session, then only one post-training interview will be conducted. During these interviews, OSH managers and trainers will be asked about their goals related to heat strain training and prevention, how they used the NIOSH resources to prepare to provide the training, how they used the training module, and whether they were satisfied with the training. Appendix B contains sample scripts for the three interviews (i.e., initial, post-preparation, and post-training interviews).

The evaluation protocol also aims to collect feedback from workers about the training, its benefits, and possible improvements. All workers who participate in training will be asked to complete a brief survey (5 minutes) immediately after the conclusion of the training session (i.e., before leaving). The survey includes eight Likert-scale items and three open-ended questions. In addition, NIOSH personnel will facilitate one or more focus groups (approximately 30 minutes) at each mine operation. Across all operations (i.e., employers), NIOSH will conduct a maximum of 12 focus groups including up to approximately 84 workers in total.) Some workers (maximum N=84) may provide feedback through survey and focus groups. Refer to Appendix C for the employee survey and Appendix D for the focus group script.

NIOSH will also ask participating OSH managers/trainers to complete a brief log (Appendix E) capturing information about each training session conducted as part of the study. The log is expected to take no more than 5 minutes at the end of each training, and investigators estimate that each trainer will complete the log no more than 10 times. The log will capture the following information about each training: date, number of participants and their work roles, and a brief explanation of why they were selected for training. NIOSH will request a general description of employer operations to document the operational context in which the evaluation takes place (e.g., number of sites, site size, company size). This log is not intended to capture worker-specific data but rather general information on the types of workers being trained. This information will provide important context in assessing the implementation and effectiveness of the training module.

In sum, this study will provide NIOSH with important feedback that will be used to update the training module and develop mine-specific heat strain resources such as an employer guide to integrating heat training into their heat risk mitigation strategies holistically. The study will also further NIOSH understanding of what additional research and resources are needed to advance workplace heat strain education and prevention.

3. Use of Information Technology and Burden Reduction

To minimize respondent burden, interviews with OSH managers and trainers will be recorded to ensure accurate data collection, minimizing the need of NIOSH investigators to write responses, which would considerably slow down the interviews. Focus groups with employees will also be recorded, allowing respondents to speak freely without the risk of comments being missed if investigators were to try taking notes manually. Surveys will be brief, with 8 questions using easy-to-use Likert scale responses, as well as 3 questions with free text responses. Surveys are paper based because of their brevity, limiting the amount of time required and eliminating connectivity problems that can sometimes occur with online surveys. Some mines do not have internet connection, so using paper surveys will ensure that surveys can be completed quickly and efficiently. Employee surveys will be completed at the end of the training session, which will eliminate the need to identify a time or location for future data collection. NIOSH will work with OSH managers to develop a focus group participation plan that minimizes burden on workers and employers (i.e., due to operational related needs).

4. Efforts to Identify Duplication and Use of Similar Information

This is a new training curriculum developed by NIOSH. No similar information has been gathered by or maintained by NIOSH or other federal agencies on this training module. Information is also not available from other sources.

5. Impact on Small Businesses or Other Small Entities

Workers from small businesses or other small entities may be involved in this effort, but NIOSH will minimize the burden on them by scheduling interviews, focus groups, and surveys on dates and times that are convenient for participants and minimizing the time required for each interview, focus group, or survey.

6. Consequences of Collecting the Information Less Frequently

The proposed data collection protocol provides an opportunity for workers to voice their opinions while minimizing respondent burden. The worker survey is very brief (i.e., 5 minutes). A smaller number of workers will be engaged in the focus groups. NIOSH will hold no more focus groups than required to reasonably sample worker feedback depending on the operation size, number of locations, and worker characteristics. Collecting less information increases risks that worker feedback is not adequately represented leading to study findings and subsequent resources lacking validity and reliability. Similarly, while up to five interviews may be conducted with OSH managers and trainers, these interviews are focused on different aspects of training (i.e., training needs, training preparation, and training delivery), all of which are vital to subsequently improve training and develop other heat strain resources. If interviews at various time points are not conducted (i.e., initial, post-preparation, and post-training), important information will be lost, thus preventing NIOSH from providing the most relevant and useful heat strain resources to the mining industry.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances with this information collection package. This request fully complies with the regulation 5 CFR 1320.5.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

a) A Federal Register Notice was published for the generic information collection. The Federal Register notice was published for this collection on July 22, 2022, Vol. 87, No.140, pp. 438360. No public comments were received.b) Outside consultation was requested during development of the training module. Galena mine (located near Wallace, ID and a part of Americas Gold and Silver Corporation) reviewed the module and provided recommendations to make it more useful for the industry. Additionally, the evaluation study concept was shared and discussed with Tilcon Connecticut, part of CRH Americas. The module was also presented to trainers attending the Mine Safety and Health Administration’s annual conference Training Resources Applied to Mining.

9. Explanation of Any Payment or Gift to Respondents

Participants will not be provided with payments or gifts.

10. Protection of the Privacy and Confidentiality of Information Provided by Respondents

“SSPO determined in conjunction with the CDC Privacy Office that Privacy Act is not applicable.” The project does not collect any PII from the data subjects. This data collection was determined to be research involving human subjects that is exempt from human subjects regulations by the CDC Human Research Protection Office.

The current study will not collect any directly identifying information nor use identifiers in the course of data collection. Individual interviews will be distinguished only by the broad demographic information of the mine that the interviewee represents (i.e., site size, company size, commodity, surface/underground mine, general US geographic quadrant location), the interviewee’s role within their company, and the content of the interview itself. Names of interviewees will not be logged or recorded anywhere in the data collected as part of the study. The only written record of participant names for interviews only will be in coordination activities (i.e., email exchanges to send/receive the consent form and setting up dates/times for the interviews). Participant names will not be collected at all for surveys or focus groups.

In-person interviews will take place in a designated area of the office identified by the participant as a place they are comfortable answering questions. The virtual interviews will take place only on Zoom for Government (ZoomGov), a FedRAMP Authorized Software-as-a-Service (SaaS). ZoomGov is similar to the “regular” commercial Zoom platform, but operates in a dedicated, secure infrastructure designed to meet the requirements of the FedRAMP Moderate baseline and DoD Impact Level 2. ZoomGov utilizes Amazon Web Services (AWS) GovCloud and U.S.-based co-located data centers and is managed by U.S. personnel only. The platform leverages FIPS 140-2 validated cryptographic solutions and 256-bit AES-GCM encryption. Interviews and focus groups will be recorded and transcribed. Identifying information will not be asked in the interviews or focus groups. After transcription, recordings will be destroyed. No personal identifiers will be asked of survey participants.

All study participants will be informed that participation is voluntary and that they can decline to answer any questions. Participants in interviews will be provided with an informed consent form prior to activities and will be allowed to ask questions, but a waiver of documentation of informed consent for interviews has been approved. The consent form provides information on purpose of the project, how the information will be used and protected, and that the information is voluntary, and they can stop participating at any time. Given the brief post-training survey request, a waiver of informed consent for workers taking the survey has been approved. Similarly, a waiver of informed consent for workers in the focus groups has been approved.

11. Justification for Sensitive Questions

No information will be collected that is of a personal or sensitive nature. Sensitive questions will not be asked of interviewees, focus group participants, and survey participants. Rather, NIOSH is only requesting feedback on perceptions related to the training module.

The proposed project was determined by the CDC Human Research Protections Office to be exempt from human subjects regulations. Please see the attached CDC Human Research Protections Office approval of exempt research protocol memo (Appendix F).

12. Estimates of Annualized Burden Hours and Costs

Response burden and cost for each type of data collection (interviews at 3 time points, focus groups, and surveys) are summarized in Table A12A and A12B. Note that NIOSH is not including the time required to conduct the training, as participating mine sites have already decided to conduct heat stress training, and the time required for training will be determined solely by trainers and OSH managers at participating mines. Up to 12 focus groups with up to 84 workers (average 7 workers per focus group) will be included. These 84 workers are part of the 200 total workers who will be offered the survey. We therefore include time to complete the focus group for only 84 workers, while the time to complete the survey will apply to the entire population of 200 workers.

The United States Department of Labor, Bureau of Labor Statistics, May, 2023 (May 2023 National Occupational Employment and Wage Estimates (bls.gov)) data were used to estimate the median hourly wage rate for miners. The United States Department of Labor, Bureau of Labor Statistics, May 2019 (Occupational Health and Safety Specialists (bls.gov)) data were used to estimate median hourly wage for occupational health and safety specialists.

Table A12A. Estimated Annualized Burden Hours

Type of Respondents	Form Name	No. of Respondents	No. of Responses per Respondent	Average Burden per Response (hours)	Total Burden Hours
OSH managers and trainers	Initial interview	10	1	30/60	5
OSH managers and trainers	Post-preparation interview	10	1	20/60	4
OSH managers and trainers	Post-training interview	10	3	40/60	20
OSH managers and trainers	Heat stress training log	10	10	5/60	9
Workers taking the training	Focus group	42	1	30/60	21
Workers taking the training	Survey	100	1	5/60	9
Total		182	-	-	68

Table A12A. Estimated Annualized Cost Hours

Type of Respondents	Form Name	No. of Respondents	No. of Responses per Respondent	Average Burden per Response (hours)	Total Burden Hours	Hourly Wage Rate	Total Respondent Costs
(OSH managers or trainers)	Initial interview	10	1	30/60	5	$36.68	$183.40
OSH managers or trainers	Post-preparation interview	10	1	20/60	4	$36.68	$146.72
OSH managers or trainers	Post-training interview	10	3	40/60	20	$36.68	$733.60
OSH managers or trainers	Heat stress training log	10	10	5/60	9	$36.68	$330.12
Workers taking the training	Focus group	42	1	30/60	21	$29.86	$627.06
Workers taking the training	Survey	100	1	5/60	9	$29.86	$268.74
Total		182			68		$2289.64

13. Annualized Cost to the Government

Data collection activities will be completed by 18 months after OMB approval, and data analysis and report writing are expected to be completed by two years after OMB approval. Table A13 shows the estimated annualized costs to the government. Hourly rates for CDC/NIOSH employees were obtained from the 2024 General Schedule Pay Table (Pay & Leave : Salaries & Wages - OPM.gov). Transcription costs were estimated using mean rates for North America and an average of 2 hours of total interview time per OSH manager/trainer (N=20) and an average of 30 minutes for 12 focus groups.

Table A13. Estimated Annualized Cost to the Government

Personnel	Hours	Hourly Rate	Data Collection, Analysis, and Writing Costs	Travel Costs	Total
Social Scientist (GS 14-1)	200	$58.55	$11,710	$6,000	$17,710
Social Scientist (GS 14-1)	100	$58.55	$5,585.50	$6,000	$11,585.50
Transcription	46	$120			$5,520
Total Annual Cost		--			$34,815.50
Total Project Cost (2 years)	--	--			$52,223.25

13. Explanation for Program Changes or Adjustments

This is a new data/information collection.

13. Plans for Tabulation and Publication and Project Time Schedule

Qualitative analysis: NIOSH investigators will qualitatively analyze the transcripts from the interviews and focus groups as well as our notes from the session observations individually using Excel or qualitative analysis software. We will use an iterative approach to analysis, concurrently with data collection, using multiple passes to establish first and second cycle codes, subsequently leading to identifications of patterns, examples, themes, and within and cross case analysis. Finally, we will integrate findings across data collection techniques to identify cross-method themes, contradictions, and lessons learned.

Quantitative analysis: We will use quantitative, multi-level analysis to examine survey data to identify similarities and differences across trainings and sites in a standardized manner.

Mixed-methods analysis: We will use integrative technique to triangulate the findings and better understand employer and worker perspectives. The findings also will inform NIOSH product development and research to improve employer heat stress prevention.

All reports derived from this study will emphasize that the findings cannot be generalized beyond the small participant sample.

Data collection activities for this study will be completed within 18 months of OMB approval, so 18 months of clearance is being requested for research activities. Table A15 outlines the project schedule.

Table A15 Project Timeline

Project Time Schedule
Activity	Time Schedule
Recruit mines to participate	1-2 months after OMB approval
Observe trainings and conduct data collection (interviews, focus groups, and surveys)	1-18 months after OMB approval (ongoing process)
Analyze data & generate interim and summary reports of findings	6-24 months after OMB approval

13. Reason(s) Display of OMB Expiration is Inappropriate

The OMB expiration date will be displayed on all data collection instruments.

13. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification.