0703-cope_ssb_3.8.2024

SUPPORTING STATEMENT – PART B

B.  COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1.  Description of the Activity

The purpose of the present study is to identify work-related stressors and individual and organizational factors that are associated with mental and behavioral health issues over time as certain Naval vessels transition throughout the various phases of the ship’s life cycle (e.g., in prolonged maintenance phases, during sea trials, throughout homeport shift). All Sailors attached to the participating commands are eligible to participate; up to 1,000 Sailors per vessel per data collection event will be recruited. For example, when fully manned, aircraft carriers have a crew of 5,000 (including air wing), which would result in a 20% response rate. At certain stages of the aircraft carrier life cycle such as during the Refueling and Complex Overhaul (RCOH) period, manning drops to approximately 2,000, which would represent a 50% response rate. Unfortunately, a tabular depiction of the potential respondent universe is not available as current manning for operational ships is not publicly available and is subject to change with changing Navy policy in response to reports of poor mental health among Sailors aboard the USS George Washington during the RCOH period.

2.  Procedures for the Collection of Information

All Sailors attached to the participating command will be eligible to participate. The research team will engage in specific actions to maximize response rate and ensure it is as representative as possible as described below. Statistical methodologies will not be used to determine sample selection. During data analysis, data will be stratified in several ways including: no stratification, stratified by rank (junior enlisted [E1-E6] vs. senior enlisted [E7-E9] vs. and officers [W1-O6]), first tour status (first tour vs. not first tour), gender (male vs. female), department (for departments with n > 30), and survey repetition status (where those who participate in only one wave of the data collection are analyzed separately from those included at multiple time points).

All validated measures (e.g., PHQ-9 for depression) will be coded in accordance with published instructions. When available, comparisons will be made to other sources of data. For example, while the proposed surveys will not be linked with medical records, a separate report of the prevalence of mental health issues at that command will be requested from the Navy Marine Corps Public Health Center. Additionally, measures of command climate will be compared with the command’s Defense Organizational Climate Survey (DEOCS) results. While comparison with these other tools is valuable, many of the measures assessed in the proposed survey are inherently subjective and thus a comparison to an objective benchmark is not possible. However, ensuring that participants are aware of the anonymity of their voluntary responses will promote participant honest and accurate self-reporting. These methods are expected to be sufficient to ensure that the proposed research will be able to achieve its purpose as described previously.

Because the proposed research will continue to follow specific commands over several years, it is expected that the sample of participants will change over time as Sailors transition in and out of working aboard that vessel. As a result, missing data is to be expected for longitudinal analyses. To address these concerns, data will be analyzed both cross-sectionally and longitudinally and separate inclusion and exclusion criteria for each analysis have been identified. When necessary, statistical methods for addressing missing data will be implemented (e.g., multiple imputation). Because missing data may be higher than most scientific studies, the study team is in the process of hiring a data analyst with extensive experience in longitudinal data analysis with a high prevalence of missing data. In the interim, the research team has consulted with biostatisticians in the Military Population Health Directorate of the Naval Health Research Center to ensure that the data analysis plan is appropriate to the study design. While missing data may thus pose a surmountable challenge for the proposed research, the combination of cross-sectional as well as longitudinal analyses will allow the study to achieve its intended purpose. Furthermore, this data analytic approach will ensure maximization of sample size and thus statistical power to observe relevant effects. It will also ensure that generalizability of the study findings is maximized as the transition of personnel in and out of a specific command is a facet of military workplace environments.

In order to reduce respondent burden, data will be collected approximately every 6 months. The exact timing of data collection will be coordinated with the ship to ensure that the crew is assessed during each phase of the ship’s lifecycle. A minimum of at least 3 months between data collection periods will be required. These requirements will minimize response burden as much as possible without compromising the study’s ability to meet its stated objective.

3.  Maximization of Response Rates, Non-response, and Reliability

The research team takes several actions to ensure maximation of response rates. First, the research team maintains a very positive relationship with participating Commands and works to ensure that information about the study is adequately disseminated prior to the research team’s arrival. A designated Command point of contact with whom the research team can work through logistical considerations is identified. The research team works closely with this Command point of contact to suggest various communication methods. For example, the research team provides verbiage for texts of emails that can be disseminated in meetings (e.g., Executive Steering Committee meetings, department head meetings), via email, and via flyers in common areas. Upon initiation of the data collection event, the research team again works with the Command point of contact to ensure that relevant leaders (e.g., department heads, chiefs) are aware of the research team’s presence and the opportunity to participate. When possible, the Command POC facilitates announcements made on the 1MC (the ship’s loudspeaker) to announce the study.

Second, the research team works with the command to identify a high foot traffic area in which to recruit participants. For the present study, the aft mess deck is typically utilized as it is a central point on the ship, is located near where junior enlisted personnel return their plates after meals, has ample seating, and minimizes barriers to the ship’s operations. The team also visits both the Wardroom and the Chief’s Mess to provide Officers and Chiefs an opportunity to participate in the survey during lunch. Third, the research team provides a modest $15 gift card incentive to encourage participation.

Fourth, the research team uses a thorough informed consent process that ensures sailors understand the purpose of the research and the potential impact that it will have, that their responses are fully anonymous, and that their participation is voluntary. Because concerns about anonymity have been identified as reasons why some Sailors opt out of participating in surveys, the research team clearly conveys how anonymity is protected. For example, the team will explain that responses will not be reported for groups of fewer than 30 people to protect sailors’ anonymity. To ensure that Sailors understand, the team says, “For example, in order for us to report information on the experiences of female African American Chiefs, we would need at least 30 female African American Chiefs; if we have fewer than 30 people, we cannot report information for that group of participants.” Additionally, the research team consents Sailors separately from their leadership to ensure that their participation is truly voluntary, and Sailors are not being coerced or mandated to participate by leadership.

When possible, approximately halfway through the data collection week, the research team calculates the number of respondents from each department, rank, and gender to identify departments that may be under-represented. The study coordinator then asks the Command POC whether there may be avenues to recruit participants from these departments by emailing their leadership to remind them of the opportunity to participate, determining if it is possible to visit Sailors in the department’s spaces, and explore whether providing additional data collection times (e.g., in the evening after a drill) may promote participation. These methods have been used successfully in prior research in a variety of operational environments to maximize response rates, minimize non-response bias, and ensure reliability of data.

4.  Tests of Procedures

The Psychological Health & Readiness Department Research Team has extensive experience conducting shipboard-specific research studies and has developed a routine method to test procedures and methods prior to data collection efforts. These include but are not limited to utilizing validated scales wherever possible, obtaining stakeholder input on survey materials through an official or unofficial advisory board, cross-checking the programmed electronic survey and the corresponding data file it creates, pilot testing of the survey by the research team, and rehearsal of the informed consent process. Together, these procedures ensure that the study team minimizes respondent burden to the greatest extent possible.

When possible, the research team utilizes rigorously validated measures to assess critical constructs of interest (e.g., mental health screeners). After designing the survey, the study team informally consults with relevant subject matter experts (e.g., active duty or retired Sailors who have experience in the given environment in which the team is working) using community based participatory research best practices; SMEs are asked to provide input on whether the survey makes sense (e.g., are any questions worded in a confusing manner?), assesses critical constructs of interest (e.g., are there topic areas that are missing?), etc.

Upon obtaining relevant regulatory approvals, the study team then programs the survey in an online survey platform (e.g., Qualtrics). Once programmed, several members of the study team (n < 9) pilot test the survey by going through it as if they were a participant representing different characteristics (e.g., a female senior chief); special attention is paid to skip logic questions when appropriate. Once the team pilots the survey, the data analyst accesses the data file created by the online survey platform and ensures that all variables are included and coded appropriately. After the survey programming has been validated but prior to commencement of data collection, all test responses are deleted.

Prior to data collection, the principal investigator and study coordinator train all members of the team on informed consent procedures and each member of the data collection team participates in a role play scenario. These scenarios also include opportunities to practice answering common questions that participants may ask during the consent process or while completing the survey. Additionally, all members of the team review the final survey materials together.

5.  Statistical Consultation and Information Analysis

Jennifer N. Belding, PhD

Principal Investigator & Research Psychologist

Psychological Health & Readiness Department

Naval Health Research Center

[email protected]

Office: (619) 553-6889

Cell (preferred): (678) 270-0191

Psychological Health & Readiness Department Staff

Naval Health Research Center

Please direct inquiries to the Principal Investigator named above.

2