Request for Office of Management and Budget (OMB) Review and Approval
for a Federally Sponsored Data Collection
Part B: Collections of Information Employing Statistical Methods
Linda J. McWilliams, M.S.
Project Officer
National Institute for Occupational Safety and Health
Office of Mine Safety and Health Research
P.O. Box 18070
Pittsburgh, PA 15236-0070
412-386-6116 (tel)
412-386-6780 (fax)
March 24, 2010
Table of Contents
B. Collections of Information Employing Statistical Methods
B1. Respondent Universe and Sampling Methods
B2. Procedures for the Collection of Information
B3. Methods to Maximize Response Rates and Deal with Nonresponse
B4. Tests of Procedures or Methods to be Undertaken
B5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
References
Section B. Collections of Information Employing Statistical Methods
To estimate the rate of adoption and use of NIOSH mine safety interventions, identify barriers to adoption, and define the characteristics associated with successful (i.e., widely adopted) interventions, the NIOSH Pittsburgh Research Laboratory (PRL) is planning to conduct a survey of operators of active, underground coal mines. The major objectives of the survey will be to: (1) collect basic information about awareness and use of mine health and safety innovations/technologies developed by PRL, (2) identify perceived strengths and weaknesses of various innovations from the point of view of mine safety managers, and (3) determine the factors that could lead to more widespread adoption of these innovations.
B1. Respondent Universe and Sampling Methods
The “target population” that will be sampled is the entire set of mines about which the survey will be used to make inferences. The proposed “establishment” survey means that the effort is directed to collecting data that is valid at the mine-level, rather than at the level of the individual employee. The target population for this survey is defined as the universe of active, underground coal mines that are able to potentially use and benefit from the innovations developed by PRL.
The target population of mines will be limited to mines in current operation and producing coal. The survey is further restricted to operations that are covered under Title 30 of the U.S. Code, specifically mines whose mineral output is sold or used in commerce. These mining operations are required to file a Quarterly Mine Employment and Coal Production Report (MSHA Form 7000-2) (Appendix J) within 15 days after the end of each calendar quarter. Start-up mines with office staff and set-up people in place but not yet in direct production will be not be included in the survey.1 Based on MSHA data from 2006, there were approximately 515 underground coal mines in the U.S. for which some or all of NIOSH interventions potentially apply (Exhibit 9).
Exhibit 9: Universe of Active, Underground Coal Mines by State (2006) |
|||
State |
Frequency |
Percent |
Cumulative Percent |
Alabama |
7 |
1.4 |
1.4 |
Arkansas |
1 |
0.2 |
1.6 |
Colorado |
7 |
1.4 |
2.9 |
Illinois |
14 |
2.7 |
5.6 |
Indiana |
8 |
1.6 |
7.2 |
Kentucky |
171 |
33.2 |
40.4 |
Maryland |
3 |
0.6 |
41.0 |
Montana |
1 |
0.2 |
41.2 |
New Mexico |
1 |
0.2 |
41.4 |
Ohio |
11 |
2.1 |
43.5 |
Oklahoma |
2 |
0.4 |
43.9 |
Pennsylvania |
49 |
9.5 |
53.4 |
Tennessee |
6 |
1.2 |
54.6 |
Utah |
13 |
2.5 |
57.1 |
Virginia |
54 |
10.5 |
67.6 |
West Virginia |
166 |
32.2 |
99.8 |
Wyoming |
1 |
0.2 |
100.0 |
Total |
515 |
100.0 |
100.0 |
Source: Mine Safety and Health Administration (MSHA)
The sampling frame for a survey is the list or mechanism used to enumerate target population units for sample selection purposes. Because mines go in and out of operation periodically, it is impossible to determine the exact number of mines that will comprise the universe of active, underground coal mines when survey sampling begins. However, the universe will be comprised of those mines in operation during the most recent calendar quarter for which complete data are available. A sampling frame for this universe of mines will be constructed using the most recent release of the Form 7000-2 database prepared by MSHA. The “unit” on Form 7000-2 is the individual mine, which is designated by its unique MSHA Mine ID. The coverage of the target population of mines should be accurate, as completion of Form 7000-2 is mandated by law for all mines selling their output or using it in commerce. Inactive and abandoned operations are expected to be identifiable in the database, with the possible exception of recent closures not yet reported to MSHA. The mandatory reporting requirements suggest that reliable location information should be readily available for most frame units.
Additionally, because the survey contains a series of questions that pertain specifically to the unique safety issues present in mines operating longwall sections, the sample also will include a census of all such mines (approximately 50 mines), in order to provide the maximum possible power for statistical tests involving mines of this type. Mines with longwall sections are the only type which will not be included in sample selection because they will be identified prior to the development of the stratification matrix (discussed below) that will be used to draw the sample.
The ideal sampling plan for an establishment survey of this type is one that ensures equal chance of selection among mines in the sampling frame. Because the Survey of Coal Mine Safety Interventions is an establishment survey in which mines, not individual employees, are the relevant units of analysis, the sampling strategy must ensure that mines of different sizes are equally likely to be selected. ICF International (the survey contractor) will use the sampling frame described to draw a stratified random sample representative of the target population of active, underground coal mines. Stratified sampling is a technique that ensures the achieved sample reflects the diversity in the population with respect to important substantive characteristics, or strata. To draw a stratified random sample, the units in the sampling frame must first be organized within a matrix that assigns each mine to a cell defined by the strata being employed.
We estimate there will be about 500 underground mines in operation that do not operate a longwall section and would, therefore, be included in the sampling plan. Two strata will be used for sampling the mines to receive the Survey of Coal Mine Safety Interventions:
Mine size (defined by the number of employees)
Region (defined by geographic groups of states).
In the paragraphs below, we use 2006 MSHA data on the total number of employees working in active U.S. coals mines and their locations to highlight how we will prepare a matrix to facilitate stratified random sampling, once OMB approval is received and the actual universe of mines is enumerated.
Exhibit 10: Mine Size, by Number of Employees, Shown in Quartiles |
||||
Size |
Frequency |
Percent |
Cumulative Percent |
|
Small |
20 employees or less |
145 |
28.2 |
28.2 |
Medium |
21-40 employees |
128 |
24.9 |
53.0 |
Large |
41-80 employees |
123 |
23.9 |
76.9 |
Very Large |
More than 80 employees |
119 |
23.1 |
100.0 |
Total |
515 |
100.0 |
|
|
Exhibit 11 displays the distribution of the mines across the various States. Seventeen states contained active coal mine operations in 2006. As is well-known, the bulk of coal mines are concentrated in a small number of states in the Appalachian region (e.g., West Virginia, Kentucky). To ensure that mines outside Appalachia are not excluded from the survey through the process of random selection, our sampling strategy employs region as a strata to make sure they will be proportionately accounted for. Based on the distribution of mines by state (shown in Exhibit 8), the following 3-category regional stratification variable, based on the Department of Energy’s regional classification of coal mines, will be applied.2
Exhibit 11: Distribution by Region |
||
Region |
Frequency |
Percent |
Appalachia |
455 |
88 |
Interior |
37 |
7 |
Western |
23 |
5 |
Total |
515 |
100 |
Based on the results of the survey pretest (see section A), administering the survey through multiple methods (i.e., via Web and by mailed questionnaire), by employing the 6-step methodology outlined below in part B3, we expect to achieve an overall 80% response rate for the Survey of Coal Mine Safety Interventions.
To summarize, our respondent mines will include the following characteristics:
Underground coal mines
Producing commercial product
Active during the first calendar quarter after receiving OMB approval for which complete data are available
Randomly selected mines with all longwall mines (approximately 50) included
Representative in terms of size
Representative in terms of geographic region.
B2. Procedures for the Collection of Information
Because the total universe for this establishment survey is fairly small (i.e., less than 600 active mines), it is possible to sample a relatively large proportion of it. As a rule — and provided that the correct strata have been chosen and the sample is drawn randomly — sampling a large proportion of cases within the sampling frame reduces sample error, thereby increasing the confidence that the survey results will accurately reflect outcomes and differences in the population. To accomplish this objective, surveys will be sent to 250 randomly sampled mines, and to the census of mines that operate longwall sections (approximately 50 mines), for a total of 300 mines.
Exhibit 12 displays the distribution of active, U.S. underground coal mines in 2006, across both strata, as well as the proportional contribution of each cell to the total. Mines without longwall sections will be randomly sampled in direct proportion to the representation of their mine type in the population. For sampling purposes, there are 12 mine types, each defined by a cell in Exhibit 12 (e.g., Large, Interior mines). The number of mines that will be sampled at random from a given cell is calculated by multiplying the cell’s proportional contribution by the sample size (250).
For example, there are 78 “Very Large” mines located in the “Appalachian” region. These mines comprise 15.1% of the total sampling frame (0.151 in proportional terms). Of these 78 mines, 38 will be randomly sampled to receive the survey (0.151 * 250 = 37.75; or 38 after rounding). Exhibit 13 displays the number of mines to be randomly selected from each cell, for a total of 250. As noted, each of these sampled mines will receive a survey, plus each of the 50 mines operating longwall sections, for a total of 300.
Exhibit 12: Sampling Frame for Underground Coal Mines without Longwall Sections* (proportional contribution of each cell shown in parentheses) |
||||||
|
Region |
Population Total |
||||
Western |
Interior |
Appalachian |
||||
Size |
Small (20 employees or less) |
# of mines |
2 |
2 |
141 |
145 |
Proportion of total |
(.004) |
(.004) |
(.273) |
(.282) |
||
Medium (21-40 employees |
# of mines |
1 |
1 |
126 |
128 |
|
(Cells will be combined for sampling) |
||||||
Proportion of total |
(.004) |
(.245) |
(.249) |
|||
Large (41-80 employees) |
# of mines |
4 |
9 |
110 |
123 |
|
Proportion of total |
(.008) |
(.017) |
(.214) |
(.239) |
||
Very Large (More than 80 employees) |
# of mines |
16 |
25 |
78 |
119 |
|
Proportion of total |
(.031) |
(.049) |
(.151) |
(.231) |
||
Population Total |
# of mines |
23 |
37 |
455 |
515 |
|
Proportion of total |
(.045) |
(.072) |
(.883) |
1 |
||
* Mines with longwall sections excluded because all such mines will be surveyed.
Exhibit 13: Number of mines to be randomly selected from each cell (proportional contribution to sampling frame x 250) |
|||
Region Size |
Western
|
Interior |
Appalachian |
Small |
1 |
1 |
68 |
Medium |
1 |
61 |
|
Large |
2 |
4 |
54 |
Very Large |
8 |
12 |
38 |
Following OMB approval, survey materials displaying the OMB approval number and expiration date will be printed and prepared to mail out to the mines. The survey packet will contain a cover letter that explains the purpose of the survey and directions for accessing the Internet version, a professionally printed hard copy of the questionnaire, and a stamped, self-addressed return envelope. Each survey packet will be personalized to improve response and ensure that the correct mine is the recipient of the packet. The cover letter will be on NIOSH stationary, personalized to the respondent and signed by the Director of the Pittsburgh Research Laboratory. Each questionnaire will be labeled with an anonymous ID number.
The survey contractor will send the survey packet to the health and safety director or the manager of record for each mine. The director of health and safety has been selected as the target respondent for this survey because he/she is thought to have the greatest likelihood of knowing and supporting NIOSH and its mission, and have the most familiarity with the survey content. Approximately one week prior to mailing the survey packet, each sampled mining operation will be contacted by telephone to confirm the mine address information. This pre-mailing telephone call also will be used to confirm that the mine is still in operation. Once the target respondent is identified, the name, job title, and contact information of the person assigned to complete the questionnaire will be recorded in a sample tracking database, which will be used to prepare all subsequent correspondence.
The survey contactor, ICF, has more than 20 years experience in survey design and administration for federal agencies. ICF is well-versed in methods to minimize respondent burden and to maximize survey response rates. This section identifies the strategies to be employed to reach these objectives.
Using Web Technologies for Ease of Response
The NIOSH Survey of Coal Mine Safety Interventions will be administered via questionnaires mailed to the sampled mining operations and over the Web, allowing mine safety managers to use the method most convenient and comfortable for them. This should boost response and minimize perceived burden. The Web address on which the survey is hosted also will be included in the mailings. The Web administration will maximize the timeliness, efficiency, and response rate of data collection.
The on-line survey technology incorporates several features to maximize response rates and respondent usability. A password system will prevent any one person from completing more than one survey and allow respondents to begin the survey, and then come back at a later point to finish it. Other features include user-friendly drop-down boxes, internal links to the directions throughout the survey, and internal links to key terms and definitions.
Using Advance and Follow-Up Mailings to Publicize the Surveys and Encourage Response
A six-step process will be used to maximize response to the survey. These steps are:
Step 1: Phone call establishing initial contact with mine representatives
Step 2: Pre-notification letter
Step 3: Survey packet mail out
Step 4: Telephone follow-up
Step 5: Reminder postcard
Step 6: Second mailing of survey packet
Each step is described below.
Step 1: Phone call establishing initial contact with mine representatives. Within two weeks after sample selection, and prior to mailing any materials, the survey contractor will make telephone contact with each sampled mining operation (Appendix G). The purpose of this call will be to verify the correct mailing information for the mine, establish the appropriate representative to receive and complete the questionnaire, record alternative contact methods (e.g., e-mail address) for the respondent, and to encourage completion of the survey. The survey team will strive for phone contact with representatives from 100% of the sampled mines prior to the mailing of the pre-notification letter and the survey packet. Calls will be made early in the day (local time) and, that failing, at varying times during the day in order to reach the operator.
Step 2: Pre-notification letter. Prior to the first mailing of the questionnaire, each mine representative will be sent a pre-notification letter on behalf of NIOSH announcing the pending arrival of the survey and requesting prompt completion upon its arrival (Appendix K). The letter will notify each mine representative that they have been selected to participate in the study, describe the purpose of the study, and inform them that they will receive the short questionnaire within the next two weeks.
Step 3: Survey packet mail out. Survey packets — including a cover letter on NIOSH letterhead (Appendix D), a copy of the paper questionnaire (Appendix B), directions to access the Web questionnaire (Appendix C), and a postage-paid return envelope — will be mailed to the entire study sample approximately one week after the pre-notification letters are mailed. The cover letter will restate the purpose of the survey, encourage respondents to complete their questionnaire and mail it back in timely fashion, and thank them for their participation. The letter also will reference the Web-based version of the survey and invite respondents to complete the questionnaire using that method.
Additionally, the letter will provide contact information for those who may have questions about the questionnaire or the overall survey effort. The survey team will establish and operate a toll-free survey assistance line, which will allow respondents to call with questions or concerns about the survey. Staff will be available to answer telephone calls during regular business hours (8:30 a.m. to 6 p.m. ET). A voice messaging system will be available to receive messages after regular business hours. A log of all questions and contacts will be maintained, and respondent questions will be answered within 24 hours.
All survey materials will be inserted into a mailing envelope, and mailing labels will be prepared for each mine representative and affixed to the mailing envelopes. Sufficient numbers of survey packets will be prepared to support the initial mailing, plus a full second mailing for three-fourths of the initial sample. To improve mine responsiveness, the survey contractor also will seek to obtain the endorsements of the survey from the National Mining Association (NMA), the Bituminous Coal Operators Association (BCOA) (Appendix E) and from regional mining associations. The reason for recruiting regional mining associations is that they include mine operators that are not members of the NMA and are more likely to know potential respondents personally. When mailings are sent out, the letter of endorsement from the NMA would go to all operators, a letter from the BCOA would go to all of its members and affiliates, and letters of endorsement of regional mining associations would go to operators in the relevant regions.
Step 4: Telephone follow-up. In order to enhance response rates, starting one week after the initial mailing of survey packets, the survey team will begin telephone prompting of those mine representatives who have not yet returned a completed questionnaire. (Appendix H).
Step 5: Reminder postcards. Research and the team’s past experience conducting surveys indicate that the use of reminder postcards (Appendix I) as a follow-up tends to increase response rates by as many as five to eight percentage points. The use of both reminder postcards and a second survey mailing can almost double the response rate. Approximately two weeks after the survey packets have been sent, labels will be printed with the names and addresses of mine representatives from whom a completed questionnaire has not been received. The labels will be affixed to pre-printed postcards and mailed. The postcard will remind mine representatives that they were sent a questionnaire, encourage their participation, remind them of the web-based option, and thank them if they have already completed and returned the questionnaire.
Step 6: Second mailing of survey packet. Approximately two weeks after the reminder postcard has been mailed, a second packet of survey materials will be mailed to all respondents from whom a completed questionnaire has not been received either by mail or through the Web.
Survey Pre-Test
As noted in Section A, ICF conducted a pre-test of the instrument with nine mines in April, 2007. Of the mines selected, eight surveys were completed and returned. The pre-test instrument contained 81 questions, including five questions specifically measuring respondents’ impressions of the clarity, burden level, and relevance of the survey itself. All respondents indicated the survey took 30 minutes or less to complete. Respondents also indicated the survey was not usually burdensome. On a 5-point Likert scale, most respondents rated the survey “very easy” to complete.
Maintaining a Toll-Free Survey Hotline
During the period that the surveys are in the field, the survey team will provide and maintain a toll-free telephone line to answer any questions respondents may have about the survey (e.g., how to interpret questions and response items, the purpose of the survey, how to get another survey if their copy has been lost/damaged). Project staff will be available to answer telephone calls during regular business hours (8:30 a.m.- 6 p.m. ET). A voice messaging system will be available to receive messages after regular business hours so after-hours calls can be responded to within 24 hours.
Excluding Questions of a “Sensitive” Nature
None of the questions included in the survey are sensitive, or private in nature, which will encourage compliance.
The initial draft of the survey was pre-tested with nine mine operators during April and May, 2007. The pretest instrument contained 81 questions, including five questions specifically measuring respondents’ impressions of the clarity, burden level, and relevance of the survey itself. ATL staff with knowledge of the mining community selected the candidate mines for the pre-test. Both individual mine operators and the directors of safety at the headquarters level were informed by mail of the nature and purpose of the survey pre-test. Eight completed surveys were returned. All respondents indicated the survey took 30 minutes or less to complete. Respondents also indicated the survey was not usually burdensome. On a 5-point Likert scale, most respondents rated the survey “very easy” to complete.
Additionally, as part of the pre-test activities conducted for a separate survey — the National Survey of the Mining Population — a total of 16 pre-test respondents participated in a debriefing interview. There were questions asking whether the mine had access to the Internet, and how convenient this would be for the completion of the questionnaire. More than 50% of the respondents indicated that their mining operation had a high speed internet connection and that they would have preferred to have access to: (1) a Web version of the questionnaire; or (2) both a paper questionnaire and a Web questionnaire. Though part of an unrelated information collection, this information, in part, led to the decision to administer the Survey of Coal Mine Safety Interventions through both Web and paper-and-pencil methods.
B5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
The following individual is the project staff from NIOSH.
Linda J. McWilliams, M.S. – Project Officer
Supervisory Statistician
Team Leader - Surveillance & Statistics Team
NIOSH, Pittsburgh Research Laboratory
P.O. Box 18070
Pittsburgh, PA 15236-0070
412-386-6116
The following individuals are the project staff from the survey contractors.
ICF International: ATL International:
Bradford Booth, Ph.D. Kathryn Makeig
Project Manager Sr. Program Manager
ICF International ATL International
9300 Lee Hwy. 20010 Century Blvd.
Fairfax, VA. 22031 Germantown, MD. 20874
703-934-3164 301-515-6799
[email protected] [email protected]
Y. Michael Yang, Ph.D. James Weeks, Ph.D.
Chief Statistician Mining Health and Safety Expert
ICF International ATL International
9300 Lee Highway 20010 Century Blvd.
Fairfax, VA 22031 Germantown, MD. 20874
703-934-3320 240-364-6009
[email protected] [email protected]
References
Butani S.J. and Bartholomew, A.M. [1988]. Characterization of the 1986 Coal Mining Workforce. U.S. Department of the Interior, U.S. Bureau of Mines, Information Circular #9192.
Antao VC, Petsonk EL, Sokolow LZ, Wolfe AL, Pinheiro GA, Hale JM, Attfield MD. 2005. Rapidly progressive coal workers' pneumoconiosis in the United States: geographic clustering and other factors. Occup Environ Med 62:670-674.
National Research Council and Institute of Medicine. Mining Safety and Health Research at NIOSH; Committee to Review the NIOSH Mining Safety and Health Research Program. Rpt. No 2, 1-264. 2007. Washington, DC, The National Academies Press. Reviews of the Research Program of the National Institute for Occupational Safety and Health. 2007.
Stayner L, Dankovic D, Smith R, Steenland K. 1998. Predicted lung cancer risk among miners exposed to diesel exhaust particles. Am J Ind Med 34:207-219.
Weeks JL. 1993. From Explosions to Black Lung: A History of Efforts to Control Respirable Coal Mine Dust. State of the Art Reviews: Occupational Medicine, The Mining Industry, 8:1-18.
1 Start-up mines not yet in production are less likely to be confronted on a daily basis by the safety and health issues associated with ongoing production and will, therefore, only have limited experience with the issues covered by the survey.
2 The 3-category variable used to define this stratum is based on the Department of Energy’s regional classification of coal mines: “Interior”, “Appalachian” and “Western.” See: Energy Information Administration. (2006). Annual Coal Report: 2005. (DOE/EIA-0584) Washington, D.C: Department of Energy.
| File Type | application/msword |
| File Title | SURVEY OF COAL MINE SAFETY INTERVENTIONS |
| Author | tqs7 |
| Last Modified By | ziy6 |
| File Modified | 2010-08-11 |
| File Created | 2010-08-11 |