NIOSH Sampling Plan

Identification of Hazards and Risk Factors for Demolition Robot Operators

Sampling Plan for Human Subject Recruitment

The Protective Technology Branch (PTB) of the Division of Safety Research (DSR) of NIOSH is conducting a project titled “Identification of Hazards and Risk Factors for Demolition Robot Operators.” This project aims to understand the factors (human-related, environment-related, and control-pad-interface-related) that may lead the operators of demolition robots to position themselves within the hazard zone of the robot, i.e., within a distance to the robot where they could get hurt (pinched, hit, or crushed) by the robot or by the structure being demolished. Data will be collected in NIOSH’s Virtual Reality Lab located in Morgantown, WV. The purpose of this Sampling Plan is to ensure, to the extent possible, that the participants will reflect the demographic composition of the construction workforce across the United States.

1. Respondent Universe and Sampling Methods

Potential Respondent Universe

This project includes data collection with operators of demolition equipment, including robots, who work in the construction sector. The project aims to recruit a study population that is representative to the current composition of the construction workforce in the United States. This composition, according to data from the Bureau of Labor Statistics (2018), is 9.9% women, 88.4% White, 6.2% Black or African American, 2.0% Asian, and 30.7% Hispanic or Latino. Table 1 shows the potential respondent universe in the recruiting area.

Table 1. Potential Respondent Universe in Recruiting Area

Number of Construction Companies in Recruiting Area (Dun&Bradstreet, 2024)	Number of Construction Employees in Recruiting Area (BLS, 2023)
There are approximately 3,637 construction companies in the Pittsburgh Metropolitan Area.	46,850 individuals employed in Construction and Extraction Occupations.
There are approximately 352 Construction companies in the Morgantown Metropolitan Area	2,780 individuals employed in Construction and Extraction Occupations
Potential establishment respondent: 3,989	Potential workforce respondent population: 49,630

2. Sampling Methods

This study utilizes both a convenience and purposive sampling strategy (Yin, 2011). A convenience method is used based on organizations or individuals who respond to recruitment materials (Attachment A). From experience gained while conducting similar projects in the Virtual Reality Laboratory of NIOSH in Morgantown, West Virginia, one of the main challenges that researchers expect to encounter during the recruitment process is low response numbers. The reasons being that 1) the study population has specific age (18 years of older) and experience (as construction workers) requirements, i.e., Purposive Sampling, and 2) the relatively small population of the Morgantown metropolitan area. To circumvent these challenges, the recruitment area was extended to include the Pittsburgh metropolitan area, as discussed in the previous section.

Initially, researchers will include all interested human subjects that meet the inclusion criteria; this sampling process will reflect a random selection process of participants from a convenience sample. Then, as additional screening and eligibility processes occur, researchers will analyze the demographic composition of the human subjects that are volunteering to participate and will compare this composition against the targeted demographic composition aimed by the study, until the quota for every demographic composition is filled as much as reasonably practical, thereby composing our purposive sample.

3. Respondent Selection Methods and Anticipated Sample

According to the Bureau of Labor Statistics, as shown in Table 1, the demographic composition of the construction workforce in the United States in 2018 was 9.9% Women, 88.4% White, 6.2% Black or African American, 2.0% Asian, and 30.7% Hispanic or Latino. According to the CensusReporter website, the metropolitan area of Morgantown, WV is shown in Figure 1a, and has a total population of 141,041 inhabitants, of which 47.9% are Women, 86.2% are White, 3.7% are Black or African American, 2.6% are Asian, and 2.8% are Hispanic or Latino. To increase the pool of potential participants, project researchers will expand the recruitment area to the Pittsburgh metropolitan area, which includes Pittsburgh, New Castle, Weirton PA-OH-WV combined statistical areas. The Pittsburgh metropolitan area, as shown in Table1, and Figure 1b, has a demographic composition of 50.8% Women, 82.7% White, 7.3% Black or African American, 2.7% Asian, and 2.1% Hispanic or Latino.

Morgantown metropolitan area.	Pittsburgh metropolitan area.
Figure 1. Potential metropolitan areas for recruitment of human subjects.

Figure 2 shows the quotas for Study 1 and Study 2. From this figure it can be seen that for Study 1, researchers will try to recruit 68 participants who are White; 6 that are Black or African American, and 2 that are Asian. Furthermore, among the 68 White participants, 20 should be Hispanic or Latino, and 7 should be women. Similarly, of the 6 Black or African American participants, 3 should be Hispanic or Latino and 1 should be a Woman. Similarly, for Study 2, the goal of researchers is to recruit 85 participants who are White, 7 participants who are Black or African American, and 4 participants who are Asian. Furthermore, among the 85 White participants, 24 should be Hispanic or Latino, and 9 should be Women; on the same token, among the 7 Black or African American participants, 5 should be Hispanic or Latino, and 1 should be a Woman.

(a)	(b)
Figure 2. Quota for participants for (a) Study 1 and (b) Study 2.

4. Sampling and Recruitment Procedures

The instrument that will be used to recruit potential human subjects is a Recruitment Flyer, shown in Attachment A. Researchers will contact construction companies and labor organizations and ask them to post the flyers in common areas of their workplace. Interested parties who call the telephone number listed on the recruitment flyer will be provided information about the project, its purpose, duration, and compensation; they will be asked two questions to determine whether they are susceptible to motion sickness, and if they meet the inclusion criteria of age, experience as construction workers, and US citizenship (to access government facilities), they will be mailed (via regular or electronic mail) their scheduled date and time, along with a copy of the informed Consent form (Attachment C). The recruitment flyer will also be posted in social networks like Facebook, and LinkedIn, to reach as many potential participants as possible.

Up to approximately 40 construction companies of varying experience with demolition activities are anticipated to receive materials about this project. Size of companies range from approximately 5 to 1,000 employees. Among the construction/demolition companies that the flyers will be sent are:

Morgantown Area Construction Companies:

March-Westin Company, Inc.

Barclay Construction, LLC

Lanta Contractors, Inc.

Rising Sun Construction

Parrotta Paving Co inc.

Orange Construction Corporation

Anderson Excavating LLC

Carline Hauling and Excavation, LLC

Laurita, Inc.

Weaver Pro Service Excavating

Reclaim Company

Morgantown Area Construction Unions:

West Virginia Laborers’ District Council 

Pittsburgh Area Construction Companies that use Demolition Robots:

Concrete Cutting Systems inc.

Matcon Diamond

Alloy Group Services

Pittsburgh Area Construction Companies:

Mascaro Construction Company Lp

Foresight Construction Services

RJ Steel Construction

Ryan Homes

Heartland Homes

Charter Homes and Neighborhoods

DRB Homes

J Francis Company, LLC

Flynn Construction Management

Pittsburgh Area Demolition Companies:

Nate Brown Demolition

Noralco Corporation

Watts Contracting Demolition

Allen Demolition

ORM Contracting Corporation

Schaaf Excavating Contractors

Quality Construction Group

Pittsburgh Area Construction Unions:

Pittsburgh Regional Building Trades Council

Pennsylvania Building and Construction Trades Council

Builders Guild of Western Pennsylvania

Builders Association of Metropolitan Pittsburgh

5. Statistical Method for Sample Selection

This project includes construction companies of all types and size. With varying organizational type, size and locations, statistical differences among those participating is probable. Studies 1 and 2 will preferably and if possible, use the same group of participants. The adequate sample sizes are estimated to be 76 for Study 1 and 96 for Study 2 using fixed-effects ANOVA with effect size of 0.40 for a perceived safety rating (on a 6-level scale) between two study conditions at a power of 0.80 and a 0.05 significance level. Nevertheless, considering that there will be cases when a human subject may only participate in one study, a maximum of 200 human subjects will be recruited to conduct both studies. Furthermore, recruitment of up to 200 human subjects will allow for dropouts without compromising study power to detect effects.

6. Data Collection and Organization

Participants will arrive to the NIOSH facilities in Morgantown, WV and upon completing the security screening, will be escorted to the Virtual Reality Lab, where a brief description of the lab will be provided. The Virtual Reality Lab is a controlled laboratory setting, where only one participant at a time will undergo the VR experience. This lab uses a CAVE-type surround screen virtual reality system, which consists of four large projected screens: front, left, right, and floor; and provides a semi-immersive virtual reality experience. Participants will wear the Volfoni VR 3D glasses, waist and hand markers. A hand-held Logitech wireless controller will be used for operating the virtual demolition robot, while the participants conduct the assigned demolition tasks. The markers on the VR 3D glasses, as well as the markers on the participants waist and hand, and the markers on the Logitech remote control will serve to collect the participant’s position and motion data. During the data collection, and while the participant is conducting his/her assigned tasks, the participant will be in a standing position, and may require moving around a short distance, i.e., two to three steps in each direction.

The project will consist of two studies: Study 1 and Study 2. A set of up to 200 human subjects will be recruited for both studies; preferable, one human subject will participate in both studies and if possible, both studies will be conducted during the same day, i.e., during the same visit to the NIOSH’s facilities. Both studies will be conducted in the NIOSH Morgantown Virtual Reality Lab.

6.1 Study 1

It is expected that each test subject will need approximately 2.5 hours to complete Study 1. This estimated time includes introduction to the Virtual Reality Lab; time to conduct the Virtual Reality Sickness Questionnaire, read and sign the informed consent form, and conduct the Construction/Demolition Robot Experience Questionnaire; these questionnaires will be administered by an authorized researcher in paper format and will be filled by each participant.

Then, it will take approximately 10 minutes for marker setup via the Logitech wireless controller and virtual reality 3D glasses; 10 minutes for training, where the participants will be explained the various functions of the remote control and then will be allowed to freely operate the robot to familiarize with the various buttons and joysticks of the remote control. All subjects will complete tasks in four scenarios, which will be presented in the same order to each participant. The markers that will be used to track human subject motions consist of small plastic balls of 0.5-inch diameter; they will be on the head (mounted on the VR glasses), the right and left hand (mounted on adjustable straps), and the waist (mounted on a provided leather belt). An authorized researcher will explain to each subject how to mount the right hand, left hand and waist markers, which will be worn on top of the participants regular clothing, and then will make a visual inspection to make sure they are properly mounted.

Study 1 will take about 70 minutes for participation in the virtual reality simulations and data collection (including two breaks of 5 minutes each). Upon study completion, the markers will be removed by the participant (an authorized researcher will assist the participant if needed); then, the participant will complete the Post-Test Questionnaire by the authorized researcher in paper format.

If the subject is participating in both studies, then the subject will be administered the VRSQ before being released for an unpaid break; otherwise, if the subject is only participating in Study 1, the payment will be processed, and then the VRSQ will be administered by the authorized researcher in paper format, and filled by the participant, before the participant is safely released from the lab.

Both Study 1 and Study 2 will preferably be completed by one participant the same day, on a single visit to the NIOSH’s Morgantown facilities. Since both studies are lengthy, there will be an unpaid break of at least 10 minutes between them to allow the human subjects some rest.

2. Study 2

If the subject is only participating in Study 2, then it is estimated they will spend approximately 2 hours and 50 minutes to complete Study 2. This estimate includes an introduction to the Virtual Reality Lab; time for the participant to complete the Virtual Reality Sickness Questionnaire, read and sign the informed consent form, and conduct the Construction/Demolition Robot Experience Questionnaire; these questionnaires will be administered by an authorized researcher in paper format and will be filled by each participant. Then, during marker placement followed by 10 minutes for training, participants will be explained the various functions of the remote control and then will be allowed to freely operate the robot to familiarize with the various buttons and joysticks of the remote control.

Data collection is estimated to be completed in approximately 70 minutes, including the administration of three NASA Task load Index Questionnaires, and Robot Trust Questionnaire, and two 5-minte breaks. Therefore, the total immersion time in the VR environment for Study 2 will be approximately 40 minutes. Once Data collection is complete, markers will be removed, and the participant will be administered the Post-Test Questionnaire, the VRSQ, and his/her payment will be processed.

Data collection for Study 2 will consist of three blocks: 1) Participant will conduct demolition tasks and no robot unexpected motions will occur; 2) participants will conduct demolition tasks, and robot/structure unexpected tasks will occur in order, and 3) participants will conduct demolition tasks and robot/structure unexpected motions will occur randomly. After each of these blocks, the NASA Task Load Index, and the Robot Trust Questionnaire will be administered by an authorized researcher in paper format and will be filled by the participant.

2. Independent Variables

The following independent variables will be controlled in Study 1 and Study 2.

1. Simulated Lighting Conditions

   1. Training session: Good lighting conditions will be used for this session.

   2. Study 1: There will be three levels: Good, medium, and poor lighting conditions which correspond to 10 foot-candle, 5 foot-candle, and 3 foot-candle, respectively, as per OSHA recommendations (29 CFR 1926.56 Illumination).

   3. Study 2: Good lighting conditions will be used for this study.

2. Virtual/simulated Visibility

   1. Training session: Unblocked (good) visibility will be used for this session.

   2. Study 1: There will be two levels: Blocked and unblocked line-of-sight.

   3. Study 2: Unblocked (good) visibility will be used for this study.

3. Simulated Control Direction/Orientation

   1. Training session: During this session, the remote control will be aligned with the robot, i.e., both the robot and the operator will be facing in the same direction.

   2. Study 1: Two conditions will be tested: 1) Remote control functions aligned with the robot, i.e., both the robot and the operator will be facing in the same direction, and 2) operator is facing the robot. In this later condition, the remote-control functions are inverted with respect to the right/left directions of motion; that is, when the operator pushes the joystick to the left, the robot’s arm will actually rotate to the right.

   3. Study 2: No Control Direction/Orientation will be used in this study.

4. Simulated Unexpected Motions (Outrigger motion, arm swing, structure fail, and hydraulic hose fail)

   1. Training session: No unexpected motions will be present during the training session.

   2. Study 1: No unexpected motions will be present during Study 1.

   3. Study 2: The following unexpected motions will take place during this study: Outrigger motion, arm swing, structure fail, and hydraulic hose fail. These unexpected motions will occur while the participant is conducting the demolition task that he/she was assigned.

6.4 Dependent Variables

The measures for this study are selected among the widely used metrics that measure human-robot interaction [Steinfeld et al. 2006]. Study 1 and Study 2 will include three measures:

1. Human motion behaviors:

   1. Speed is measured by mean walking speed in both Study 1 and Study 2, using motion capture technology.

   2. Separation distance will be measured by minimal distance from the closest body part to the axis of rotation of the robot’s turret.

2. Task performance is measured by the completion time (i.e., the time for the participatnt to complete the given task).

3. Self-reported, perceived safety measures: Only in Study 2. Will be measured by the ratings on “surprise” and “fear”, on a 6-level scale using 1 being “never” and 6 being “very much” [Bartneck et al. 2008].

   1. Mental workload. Only in Study 2. A higher level of mental workload will be associated with a higher level of distraction and a higher level of unawareness of where the operator locates him/herself (within or outside the hazard zone of the robot). Mental work will be measured by the response to the NASA Task Load Index [Hart and Staveland 1988].

   2. Human operator’s trust in the robot. Only in Study 2. A higher level of trust in the robot is associated with smaller separation distances between the operator and the robot, and lower human walking speed around the robot. This self-reported measure will be assessed by the responses to the 14 item robot trust scale [Schaefer 2013].

6.5 Data analysis

Repeated measures analysis of variance ANOVA will be used to examine effects of different risk factors for demolition robot operators on human behaviors, task performance, and self-reported safety measures. All data will be recorded and saved in Excel files for analysis. Statistical software, SAS or equivalent, will be used for data analysis and graphic presentations. Tables and figures will be generated to illustrate the differences between the conditions and any interaction effects. It is anticipated that no further analysis of the data will be undertaken.

7. Methods to Maximize Response Rates and Deal with Nonresponse

To maximize participation, we will offer the following incentives: 1) Compensation for the participants time at a rate of $30/hour, with partial payments at a rate of $10 per 20 minutes, with periods less than 20 minutes rounded up; 2) flexible hours that include early hours in the mornings and/or evening hours; 3) flexibility regarding the participation in one or both studies, in the same day or in different days.

In addition, to boost the recruitment rate, we will ask all human subjects that participated in one or both studies to pass the information to their coworkers, to encourage them to participate.

References

[1] Bureau of Labor Statistics (BLS) [2018]. Labor Force Statistics from the Current Population Survey. Household Data, Annual Averages. Employed persons by detailed industry, sex, race, and Hispanic or Latino ethnicity, 2018. [https://www.bls.gov/cps/cpsaat18.htm]. Date accessed: January 2020.

[2] Pittsburgh-New Castle-Weirton, PA-OH-WV CSA - Profile data - Census Reporter

[3] Morgantown-Fairmont, WV CSA - Profile data - Census Reporter

[4] Bureau of Labor Statistics (BLS) [2023]. May 2023 Metropolitan and Nonmetropolitan Area Occupational Employment and Wage Estimates [https://www.bls.gov/oes/current/oes_38300.htm#47-0000]