Assessing and Evaluating the Effectiveness of Hazard Recognition and Risk Assessment Interventions

Attachment C

‘Are you Thinking Like an EXAMiner?’ Evaluation Interview Guide

Form Approved

OMB NO. 0920-0953

Expiration Date: 7/31/2018

[This Evaluation Interview Guide will be used with the person responsible for using and leading the ‘Are you Thinking like an EXAMiner?’ training. This person is likely to be a safety professional. The full interview will take place during Visit 1 and Visit 2].

Visit 1

Participant information

Participant #:______________ Date: ___________

Preparing to use the ‘Are you Thinking like an EXAMiner?’ module [Usability/Ease of Use]

Were you able to open the training module using the Intervention button?

In general, did you have any issues using this training module?

• Did the videos load and did they run smoothly?

• Were you able to see and use the pictures that were included in the module?

• Was the content intuitive? Did you require any additional information before you were able to use the module in your training class?

Presenting ‘Are you Thinking Like an EXAMiner?’ module [Usability/Ease of Use]

In total, how many times have you used the ‘Are you Thinking like an EXAMiner?’ training module? ____________

How large were the training groups you presented to?

Who was the primary audience?

Can you briefly describe how you used the training module? ‘

Are You Thinking like an EXAMiner?’ Content [Accuracy and Quality of Content/Reaction]

Did you cover:

• Workplace EXAMinations and Hazard Recognition Yes or No

• Are you Searching like an EXAMiner? Yes or No

• Taking it to the Field Yes or No

If you did not use all three sections of material, why did you choose not to?

Probes

• Was the content not relevant to your mine sites?

• Did you identify content that was not accurate?

• Was it a time issue, meaning did you have a limited amount of time for this material?

• Did you divide the material up and present it across multiple training session?

• Did you not think that your mineworkers would benefit from the information?

Public reporting burden of this collection of information is estimated to average 30 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to CDC/ATSDR Information Collection Review Office, 1600 Clifton Road NE, MS D-74, Atlanta, Georgia 30333; ATTN: PRA (0920-0953).

Overall, did the content included accurately reflect the training objectives? Yes or No, please explain:

Given that you have now used the training module:

• Can you provide an example of how the content applies to your workplace?

• How do you foresee your supervisors/mineworkers using this information on the job?

• Which sections are most relevant for your mineworkers?

• Which sections didn’t go over as well? Why do you think those worked/didn’t work?

Activities [Quality of Content]

Can you give me an example from one of your training sessions where a trainee was able to tie information from the thought activity to an experience from the workplace?

Job Aid [Reaction/Satisfaction]

In your opinion, does the job aid adequately reflect the content included in the training module?

How do you plan to have your frontline supervisors use the job aid?

What issues to do you foresee with your frontline supervisors implementing use of the job aid?

Future Use (Reaction/Satisfaction)

Are there any specific sections from the training module that you know you will present again in the future? If yes, what section(s) and what did your trainees gain from that material?

Would you recommend this product to other trainers? Why or why not?

What value do you see the training bringing to the mining industry?

Changes (Reaction/Satisfaction)

What changes can we make to improve the training module before we make it widely available to the industry?

‘Are you Thinking Like an EXAMiner?’ and Job Aid Evaluation Interview Guide

[This Evaluation Interview Guide will be used with the person responsible for using and leading the ‘Are you Thinking like an EXAMiner?’ training. This person is likely to be a safety professional. This interview will take place during Visit 2].

Visit 2

Participant information

Participant #:______________ Date: ___________

Questions aimed at mineworker performance

Have you used the training module or any of the content from the training since we last spoke?

If yes, what content have you used?

And, how have you used the content?

Can you provide examples of how your mineworkers applied the content of the training to their jobs?

Questions aimed at frontline supervisor performance

Did your frontline supervisors participate in using the job aids?

From a safety perspective, how did the job aid change the way frontline supervisors performed workplace examinations?

Did they identify more hazards?

Did they identify hazards they had missed in the past?

What were the most critical hazards identified through using the job aids?

And, what changes were implemented to address those hazards?

Who decided on the change?

Who was involved in making the change happen?

What action did you take to inform mineworkers in the area of the change?

From an organizational standpoint, how did you measure success for the changes that were made?

Did you see an increase in productivity?

Did you see a change in the number of incidents?

Did you see a change in the number of citations written during MSHA inspection?

Did you experience any push back from mineworkers or frontline supervisors with regards to the use of the training module or the job aid?

From an organizational perspective, what barriers did you observe frontline supervisors encountering as they used the job aid?

If you could change one thing about the job aid, what would you change?

How do you plan to use the training module and job aid in the future?