Information Collection Request for
“Assessing the Safety Culture of Underground Coal Mining”
Dr. Katherine Margolis
Health Communication Fellow
Tel. 412-386-4627
Fax. 412-386-6710
August 2009
Part B: Collection of Information Employing Statistical Methods
Table of Contents
Respondent Universe and Sampling Method 3
Procedures for the Collection of Information 4
Methods to Maximize Response Rates and Deal with Nonresponse 5
Tests of Procedures or Methods to be Undertaken 6
Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 6
References 7
1. Respondent Universe and Sampling Method
According to the National Mining Association approximately 46,000 people currently work in underground coal mines in the U.S. (NMA, 2008). A good description of the six mines where data is collected will be provided (e.g., size, location, height of coal seam, type of extraction method utilized, etc.). However, CDC does not claim that the mines selected for this study are statistically representative of the entire population of US coal mines, or of any subpopulation of mines or miners. It should not be assumed that the findings of this exploratory study are generalizable to other groups of coal miners.
The subjects will be employees from six participating mine organizations. Mines will be recruited via stakeholder meetings and previous research contacts. The participating mine organizations will be recruited with an attempt to get mines to participate that vary on the continuum of safety performance as well as in size and region. In addition, at least one unionized and non-unionized mine will be included in the sample. It is expected that the employees of the participating mines will also vary along a number of variables including age, gender, and experience. The number of employees selected for participation in the different data collection activities will depend on the size of each mine. All employees will be given the option of declining individual participation in the study.
Across the six mine organizations it is expected that a total of 900 participants will complete the survey, and 180 participants will be involved in the structured interviews with the associated Behavioral Anchored Rating Scales. Participant involvement in the Behavioral Observations will be determined once the study team travels to the individual sites and determines the types of work activities that might be appropriate for observation. The number of individuals that would be involved in these work activities is therefore unknown at this time. The types of work activities to be observed include daily production meetings, shift turnover meetings, preventive maintenance activities, and training sessions. Some of the same individuals at each mine site will participate in all of the data collection activities, while other individuals might only be involved in one or none of the data collection activities. All of the data across the six mine organizations will be collected over a period of approximately 24 months. For any one mine, data collection will take no less than one week and not more than two weeks depending upon the size of the organization. Data will only be collected one time at each mine, this is not a longitudinal study.
The sampling of participants will be a combination of convenience sampling and purposive sampling. Purposive sampling will be used to ensure that the research team gets a broad sample of individuals from each organization that represent both the vertical and horizontal groups in the organization (e.g., managers and non-managers; production and maintenance). Convenience sampling will occur based on the individuals available to participate at the point in time when the study is conducted.
The data collected for this study will be both quantitative and qualitative in nature. Tests for statistical significance of group differences will be performed only on the survey data collected in this study. All other data collected will be presented in terms of descriptive statistics (e.g., n-size, means) or qualitative summaries.
2. Procedures for the Collection of Information
This is a non-intrusive field study. No variables will be manipulated but rather data will be collected on behaviors of interest across 6 underground coal mines. The data collected will allow the understanding and definition of the existing safety culture in the underground coal mining industry. There will be an attempt to collect data at both larger and smaller sized underground coal mines in different regions of the country with at least one unionized and one non-unionized mines participating. In addition, organizations with a good safety performance record and those with a poor safety performance record will also be included in the study.
The design of this study uses 5 different methods to collect information related to the topic of organizational behaviors. These methods are: Functional Analysis, Structured Interview Protocol (see attachment C2), Behavioral Anchored Rating Scales (see attachment C3), Organizational Survey (see attachment C5), and Behavioral Observations (see attachment C4). These different methods complement one another and provide a way to collect both quantitative and qualitative data on the organizational behaviors of interest. Of particular importance is the degree to which data collected across various methods converge, allowing greater confidence in the conclusions drawn from the data collection. Further, the same methods will be used at each of the underground mines participating in the study in order to allow comparative analyses across the sites.
Seventeen different organizational behaviors have been identified to impact overall organizational and safety culture, based on research conducted in other high hazard industries. It is important to note that the behaviors or variables are not mutually exclusive. Because of this, it is not necessary to collect data on all 17 behaviors at each participating mine. Rather, data will be collected on approximately 10 of the behaviors at each of the 6 mine sites. This will insure that adequate data related to the concept of safety culture is collected at each site without making the data collection requirements too burdensome for the sites. The behaviors on which data will be collected at each site will be identified in part based on a review of relevant documentation from the site that is conducted as part of the functional analysis.
Five methods will be used at each of the 6 participating underground coal mines to collect data on the organizational behaviors of interest at that site. The methods are:
Functional Analysis (documentation review) – typically completed prior to commencing the assessment. Information gained is largely historical and consists of a review of safety-relevant documentation including organizational charts, recent assessment results, administrative procedures and training materials to understand a static representation of the functioning of the organization, especially with respect to safety.
Structured Interview Protocol – consists of a database of interview questions covering all of the organizational behaviors identified, from which a subset of questions is selected to provide a predefined focus to an interview. The same questions are asked across a representative sample of individuals in the organization at different levels, in different functional positions, and across locations, so that alignment and non-alignment with respect to the organizational behaviors can be identified. Interviews generally last about one hour and are in a face-to-face, one-on-one or two-on-one setting where the interviewer(s) poses a series of questions/topics to the interviewee and records information provided. A set of pre-scripted questions serves as the basis for the interview with interviewer discretion in terms of follow-up and additional questions. (See Attachment C2).
Behavioral Anchored Rating Scales (BARS) – are administered to those individuals who participate in the structured interviews at the conclusion of the interview. BARS are a type of quantitative survey scale that incorporates behavioral examples with general performance dimensions. The behavioral examples are designed to facilitate interpretation of poor, average, and high and act as “anchors” for defining the various levels of each behavior. The BARS provide an opportunity to quantitatively summarize qualitative data associated with the interviewee’s perceptions of the organizational behaviors that were discussed and are included in the time allocated for the interviews. (See Attachment C3).
Observations and Behavioral Checklists – Observations supplemented with the use of Behavioral Checklists provide an unobtrusive assessment of organizational behaviors related to processes critical for ensuring safety performance. Checklists have been developed for the critical organizational behaviors identified. Checklists are used while research team members are conducting observations of scheduled and unscheduled activities in the organization and provide quantitative and qualitative information about the organizational behaviors observed. (See Attachment C4).
Organizational and Safety Culture Survey – is a written paper-and-pencil questionnaire administered to respondents and provides a quantitative and objective way to collect information about the organizational behaviors and topics related to safety culture and safety conscious work environment. Questions are close-ended and require respondents to describe their opinions using a Likert-type scale. Topics covered on the survey include: Organizational Culture, Coordination of Work, Work Group Cohesion, Communications, Attention to Safety, Commitment, Hazardous Nature of Work, and Environment, Safety, and Health Issues. This tool is especially effective in reaching a larger and broader sample of individuals than can be contacted through using the interview and observation tools alone. (See Attachment C5).
Upon returning to the office, members of the research team will enter the information from the survey into a password protected computer database using the Statistical Package for the Social Sciences (SPSS), a word processor program, and a database spreadsheet. Any completed interview guides or individual data forms will then be destroyed.
3. Methods to Maximize Response Rates and Deal with Nonresponse
It is anticipated that the response rate will range between 75 and 95% depending on the method used. It is expected that 95% or more of the individuals selected to participate in the structured interview will participate. Survey response rates are typically lower (estimated at 75% or greater). However the surveys for this study will be administered in group sessions arranged at the individual mines which should help to ensure a higher response rate. Due to normal absences from work, a few miners may be unavailable on the particular days that the data collection activities are conducted at each mine. Extensive prior experiences at other types of organizations with this methodology suggest that the response rates will achieve the expected levels.
4. Tests of Procedures or Methods to be Undertaken
Extensive field testing of this methodology has occurred at over 40 different organizations, including over 40,000 individual respondents, across 5 countries in industries as diverse as nuclear power, fossil energy, research, mining, transportation, health care, and chemical reprocessing. This has led to a refinement of the questions and methods to ensure they are understandable and relevant across a broad population base.
5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
The persons who will collect and/or analyze the data are listed below.
Project Staff:
Katherine A. Margolis, Ph.D., Health Communication Fellow, NIOSH Pittsburgh Research Laboratory, 412-386-4627, [email protected]
Robert H. Peters, Manager, Human Factors Team, NIOSH Pittsburgh Research Laboratory, 412-386-6895, [email protected]
Catherine Y. Kingsley Westerman, Ph.D., Health Communication Fellow, NIOSH Pittsburgh Research Laboratory, 412-386-6673, [email protected]
Kelly L. McNelis, Ph.D., Associate Service Fellow, NIOSH Pittsburgh Research Laboratory, 412-386-6886, [email protected]
Gail McConnell, MPH DVM, Public Health Advisor, NIOSH Pittsburgh Research Laboratory, 412-386-4950, [email protected]
Sonja B. Haber, Ph.D., Senior Consultant, Human Performance Analysis, Corp., [email protected]
Deborah A. Shurberg, Ph.D., Consultant, Human Performance Analysis, Corp., [email protected]
NMA [2008]. Most Requested Statistics - U.S. Coal Industry. Washington, DC: National Mining Association. [http://www.nma.org/pdf/c_most_requested.pdf].
| File Type | application/msword |
| File Title | Table of Contents |
| Author | Christine Fagioletti |
| Last Modified By | Nelda Robinson |
| File Modified | 2009-08-31 |
| File Created | 2009-08-31 |