Section B FatigueEMSExperimentalStudy 09252018

Information Collection Request Supporting Statement: Section B

The EMS Sleep Health and Fatigue Education Study

NHTSA is seeking approval to conduct a field experiment. The experiment seeks to gather objective data regarding the impact of a sleep health and fatigue education and training program on diverse indicators of sleep and fatigue among Emergency Medical Services (EMS) personnel that operate ambulances on the roadway. The overarching goals of this project are to 1) enhance our understanding of the relationships between shift work, sleep, and fatigue in EMS operations, and 2) determine if providing education and training to EMS personnel on the importance of sleep health and dangers of fatigue impact diverse indicators of sleep, fatigue, and safety.

The study team will accomplish these goals by using a cluster-randomized trial study design with a wait-list control group. The study team will use nationwide convenience sampling to recruit 30 EMS agencies and 40 EMS personnel per agency for a total sample of 1,200 EMS personnel (see Appendix B for more information on methodology). As described in A.12, the proposed data collection will include a total of 10,141 burden hours to reach the study objectives, which includes measuring changes in sleep quality with the Pittsburgh Sleep Quality Index (PSQI) and fatigue with the Chalder Fatigue Questionnaire (CFQ).

Individual study participants (EMS personnel) will use a secure, password-protected web-based data collection tool to answer baseline and follow-up surveys, and to watch intervention-related materials. Participants will also be asked to respond to text-message queries sent to their cellular, mobile, or smartphones that ask about sleep and fatigue.

According to the project schedule, recruitment and enrollment of EMS organizations and personnel is planned to begin May of 2019. The final report is due August of 2020, with estimated publication in 2021.

B.1. Describe the potential respondent universe and any sampling or other respondent selection to be used.

The researchers will recruit EMS nationwide from EMS agencies to participate in this study. Moderately sized organizations with between 50 and 300 employees will be eligible to participate.

Organization type determinants of eligibility include:

• Eligible types:

  • EMS operations that deploy EMS crews using ground-based ambulance services;

  • Dual ground-based and air-medical organizations; and

  • Dual fire and EMS organizations.

• Exclude organizations that are exclusively air-medical.

The research team will use nationwide convenience sampling. This approach will increase the likelihood of meeting enrollment goals by maximizing visibility of the study to as many EMS organizations as possible. Nationwide recruitment will help achieve participation from EMS organizations located in all U.S. Census regions (i.e., Midwest, Northeast, South, and West). The research team will coordinate with EMS leaders, national EMS organizations, professional societies, state EMS agency directors, and professional membership groups to distribute study flyers and links to a study-designated website to advertise the opportunity to participate in this research project (Appendix C).

The study team will also use scheduled webinars and conference calls to review the study and enhance recruitment. The EMS organizations that reach out to the study team to learn more about the study will be contacted later by telephone and webinar to determine whether they qualify for study participation. The EMS organizations that are eligible, as determined by Form 1460, will be invited to participate in the study.

We have 80% power to detect 0.4 standard deviation difference in Pittsburgh Sleep Quality Index (PSQI) score with n=10 agencies in each group (intervention and control) and n=30 EMS personnel per agency. We have 83% power to detect a 20% reduction in fatigue with n=20 agencies and n=30 personnel per agency assuming the control group has 50% reported prevalence of fatigue. We conservatively assumed a 50% prevalence of fatigue for the control group because this results in the maximum variance for the test of proportions. We also have preliminary data to suggest this is a reasonable assumption.¹

Timeline & Key Steps	CONSORT DIAGRAM / FLOW CHART		Screened OUT Attrition
Agency-level Screening (Estimated n=200 agencies express interest in participating)	Nationwide Recruitment of EMS Agencies (Goal enrollment n=30 agencies)		85% of agencies will be screened out or decide not to participate
	Agencies RANDOMIZED to Intervention or Wait-List Control
	INTERVENTION GROUP (n=15 agencies)	WAIT-LIST CONTROL GROUP (n=15 agencies)
Individual-level Screening (Estimated n=100 individuals per agency are screened, totaling ~n=3,000 screened)	Estimated 50% of individuals at the agency agree to participate	Estimated 50% of individuals at the agency agree to participate	~n=3,000 total individuals screened.
			50% lost
Individual-level video-based consent procedure	~n=750	~n=750	~n=1,500
			20% lost
Baseline Survey	~n=600	~n=600	~n=1,200
			0% lost
12-week follow-up surveys	~n=600**	~n=600**	~n=1,200**
			0% lost
24-week follow-up surveys	~n=600**	~n=600**	~n=1,200**
Study End	Estimated Number of Individuals that Complete the Study as Designed (~n=30 per agency retained) (~n=450 total complete study)	Estimated Number of Individuals that Complete the Study as Designed (~n=30 per agency retained) (~n=450 total complete study)	Of the ~n=3,000 individuals screened at the start of the study, we anticipate that (~n=900 total individuals complete the study)

NOTES: The ~ symbol = estimated/approximately. **Signifies a conservative estimate of attrition for purposes of overestimating the burden on participants.

B.2. Describe the procedures for the collection of information.

Our research team will communicate first with administrators, managers, and officials at each EMS agency that express interest in participation. A telephone call will be scheduled between the EMS agency administrator and a member of the study team. Form 1460 will be completed to confirm agency level eligibility. We anticipate that 85% of EMS agencies screened will be screened out or decide not to participate. Eligible agencies will be enrolled and then instructed to circulate a study flyer directly to their EMS personnel (Appendix D). The flyer will describe the study and emphasize voluntary participation and confidentiality of participation. The flyer will instruct the EMS personnel that wish to participate to contact the research team via email or telephone. The research staff will screen the person using Form 1461. Persons eligible to participate will then be instructed to visit the study website and follow instructions for watching the informed consent video (Form 1462) and creating their own unique login to the study website. We anticipate that 50% of individuals screened will be eligible to participate. Participants will use the study website to complete baseline and follow-up surveys and enter in their shift schedules during the study period. These entries will inform the texting platform and timing of when text messages are sent to individual participants.

At the completion of agency level screening (Form 1460), participating agencies will be randomized to one of two groups: [1] the intervention group, or [2] the wait-list control group. Individual participants affiliated with an agency randomized to the intervention group that are interested in participating will be screened with Form 1461.

Eligibility criteria include:

• 18 years of age or older;

• Live/reside in the United States (Hawaii & Alaska included);

• A licensed/certified EMS professional (e.g., EMT-Basic, Firefighter, Paramedic, Flight Nurse);

• Currently work shifts (e.g., 12-hour shifts);

• Works at least one shift per week;

• Currently owns and uses a cellular phone or smartphone that can both send and receive text messages;

• Willing to answer online surveys and respond to text-message queries seven days in a row every third week of the month for a total of 24 weeks;

• Willing to answer a follow-up survey at the end of the study period; and

• Currently works as a full-time or part-time EMS clinician at an EMS agency that has agreed to participate in this research study.

If eligible, the intervention group will:

• Complete baseline survey (Form 1463);

• Granted access to the intervention materials;

• Complete training intervention through study website using secure login to access 10 unique educational/training modules that last approximately 10 minutes each within a 10-day period;

• Receive text-message queries for 7 straight days every third week for the duration of the 24-week study period (Form 1464); and

• Complete the follow-up survey (Form 1465) made available to participants during weeks 23 and 24 of the study period.

Individual participants affiliated with an agency and randomized to the wait-list control group will be screened based upon the same eligibility criteria (Form 1461).

If eligible, the wait-list control group will:

• Complete the baseline survey (Form 1463);

• Receive text-message queries for 7 straight days every third week for the duration of the 24-week study period (Form 1464);

• Crossover to intervention group at 12 weeks into the study;

• Complete training intervention of 10 unique educational/training modules that last approximately 10 minutes each within a 10-day period; and

• Complete the follow-up survey (Form 1465) made available to participants during weeks 23 and 24 of the study period.

Participants at one EMS agency proximal to the University of Pittsburgh will:

• Voluntarily wear an actigraph wristwatch seven days straight, every third week of the study, coinciding with the text-message query component of data collection protocol;

• Complete a paper-based sleep diary (Form 1466); and

• Voluntarily complete psychomotor vigilance testing (PVT) at the start and end of at least one shift during the period when wearing the actigraph and answering text-message queries (Form 1467).

Table 1: Study Design for Cluster Randomized Controlled Trial with Wait-List Control Group
Inter Group	EDU
	BSL										FUP												FUP
			TX			TX			TX			TX			TX			TX			TX		TX
Wait-list Cont Group												CRO
												EDU
	BSL										FUP												FUP
			TX			TX			TX			TX			TX			TX			TX		TX
WEEK	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24
Table 1 Legend: Inter=Intervention group. Cont=Wait-list control group. CRO=Crossover from control group status to intervention group. BSL=Baseline assessment. FUP=Follow-up assessment. TX=Texting assessment and intervention messages. EDU=Education intervention delivered. Grey colored cells added to further indicate intervention group status.

B.3. Describe methods to maximize response rates.

Participation in this study is voluntary. To maximize interest in the study, participation, and response to study surveys and queries, the research team will:

• Include letters and statements of support for this study in the study flyers sent to potential EMS agencies and potential participants located within EMS agencies that agree to participate (Appendices B and C). Prominent leaders of national EMS organizations and other prominent professional societies and organizations will author these letters and/or statements of support. These leaders and the organizations that they represent understand that this research is important and they support the mission to improve the health and safety of EMS personnel and the patients they serve;

• Outline assurances of confidentiality, such that no individual will be identified in reports of the study’s findings, nor will any individual’s data be shared with any licensing regulatory authority; and

• Provide participants with remuneration of $20 (see A.9).

B.4. Describe any tests of procedures or methods to be undertaken.

All analyses for treatment group comparisons will use an intention-to-treat approach and results will be reported used the CONSORT extension to cluster randomized trials.² Baseline characteristics for agencies and workers will be compared between the intervention and wait-list control groups to assess effectiveness of the randomization, and statistical differences will be adjusted for in secondary analyses. Two sample t-tests for continuous measures and chi-square tests for categorical measures will be used to compare agency characteristics. Mixed models will be used to compare the EMS worker baseline characteristics between groups.

The primary outcome for this agency-randomized trial is sleep quality, measured with the Pittsburgh Sleep Quality Index (PSQI) at baseline at 12 weeks and 24 weeks. We will use a linear mixed model (assuming normality of these numeric outcomes) controlling for baseline sleep quality and EMS agency size. We will also report the intraclass correlation coefficients for fatigue and sleep quality to assess our assumptions for sample size analyses.

A secondary outcome of interest for this agency-randomized trial is EMS worker fatigue related to shift work, measured by the and fatigue with the Chalder Fatigue Questionnaire (CFQ) at baseline, 12 weeks, and 24 weeks after randomization. To test the effect of the intervention on worker fatigue, we will use a generalized linear mixed model with logit link (due to the dichotomous nature of outcome: fatigued versus not-fatigued). We will use a random agency effect to account for the clustering of EMS workers within agencies and a random worker effect to account for the correlation between repeated measures within worker. The main effects of time, intervention, and intervention*time interaction will be fit as fixed effects with primary focus on the contrast between the intervention and APC groups at 12 weeks and 24 weeks post start of intervention period. The mixed model using both time points allows us to borrow information from the correlation between the time points for those workers with missing data on either time point. We will control for the size of EMS agency.

Secondary analyses will adjust for EMS worker characteristics that are potentially imbalanced at baseline and have been shown to be associated with fatigue and sleep quality. Factors include, but are not limited to, years of experience, employment status, shifts worked per month, the number of patients seen during shift, shift length, health status, and number of medical conditions.

We are interested in worker and agency level characteristics associated with reduction of EMS worker fatigue. The characteristics of interest are:

• Age, sex, and years of experience;

• Health status, medical conditions, and body mass index;

• Sleep quality;

• Shift work (e.g., shift length, shift rotation, time off between shifts);

• Number of jobs;

• Volume/intensity of work while on duty (i.e., number of patients seen), and

• Recovery between shifts.

We will first test the interaction between characteristics and the intervention on the primary dichotomous outcome of fatigue using generalized linear mixed models with a logit link. We will estimate and report the intervention effects by subgroups to assess significance within groups and the consistency across groups. We will limit these exploratory analyses to EMS worker factors (stratified into two categories / dichotomous) with a distribution no worse than 40%-60% between the stratums (e.g., status of working multiple jobs stratified into Yes vs. No). In addition, we will explore if these baseline characteristics are associated with fatigue and sleep quality and improvement in these measures. We will use generalized linear mixed models to adjust for clustering and repeated measures over time with random provider effects and random agency effects. In addition, we will use the ecological momentary assessment data collected with the mobile phone texting platform to determine if measures of fatigue, sleepiness, and difficulty with concentration change with changes in sleep behavior (e.g., amount of sleep between shifts) and use of strategies promoted by the intervention. This analysis will use linear mixed models with random subject effects to incorporate time varying covariates (between and within subject effects) and model the between subject and within subject variance.

In a sub-set of participants (estimated n=30), we will collect psychomotor vigilance testing (PVT) measures of reaction time, lapses, and false starts. We anticipate that the number of PVT lapses would follow a Poisson (or negative binomial) distribution. We will use a generalized estimating equation (GEE) to estimate mean lapses and false starts at start and end of a shift for each participant that completes a PVT measurement at the start and end of a shift. We will control for shift duration. We will use mixed effects linear models to compare PVT measures of mean reaction time by shift duration, and we will use a random effects linear model to account for multiple measures per participant and test for the association between shift duration and sleep hours (pre-shift and post-shift).

In a sub-set of participants (the same sub-set that participates in PVT measures), we will use the actigraph wrist devices to validate self-reported sleep. Analyses of these data requires a special software (e.g., Actilife Version 6 by the Actigraph Corporation). We will use the software to examine actigraph measures of sleep and triangulate these data with a paper-based sleep diary (Form 1466) and sleep reported via the text message query platform (Form 1467).

B.5. Provide the names and telephone numbers of individuals consulted on statistical aspects of the design.

Charity G. Moore, PhD, MSPH

Professor

Director, Physical Therapy Data Center

Department of Physical Therapy

School of Health and Rehabilitation Sciences

University of Pittsburgh

[email protected]

412-383-4812

1 Patterson PD, Weaver MD, Hostler D. EMS Provider Wellness. In: Cone DC, Brice JH, Delbridge TR, Myers JB, editors. Emergency Medical Services: Clinical Practice and Systems Oversight. Second Edition. Chichester, West Sussex; Hoboken: John Wiley & Sons Inc., 2015 pp. 211-216. ISBN: 978-1-118-86530-9. 

2 Campbell MK, Elbourne DR, Altman DG, CONSORT-group. CONSORT statement: extension to cluster randomised trials. BMJ. 2004;328(7441):702-708.

7