Evaluation of Healthcare Worker Mental Health Campaign
Request for Office of Management and Budget Review and Approval
for Federally Sponsored Data Collection
Supporting Statement A
February 16, 2023
Contact Information:
Amia Downes
CDC/NIOSH/OD
(404) 498-2544
Table of Contents
Justification
Circumstances Making the Collection of Information Necessary
Purpose and Use of the Information Collection
Use of Improved Information Technology and Burden Reduction
Efforts to Identify Duplication and Use of Similar Information
Impact on Small Businesses or Other Small Entities
Consequences of Collecting the Information Less Frequently
Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency
Explanation of Any Payment or Gift to Respondents
Protection of the Privacy and Confidentiality of Information Provided to Respondents
Institutional Review Board (IRB) and Justification for Sensitive Questions
Estimates of Annualized Burden Hours and Costs
Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers
Annualized Cost to the Federal Government
Explanation for Program Changes or Adjustments
Plans for Tabulation and Publication and Project Time Schedule
Reason(s) Display of OMB Expiration Date is Inappropriate
Exceptions to Certification for Paperwork Reduction Act Submissions
Attachments
Att 1a. — COVID-19 American Rescue Plan of 2021
Att 1b. — Dr. Lorna Breen Health Care Provider Protection Act
Att 2a. — 60 day FRN
Att 2b. — 60 day FRN comments
Att 03. — NIOSH IRB Determination
Att 04. — Healthcare Worker Follow-up Survey
Att 05. — Hospital Leader Follow-up Survey
Att 06. — Invitation and Reminder Emails
A. JUSTIFICATION
1. Circumstances Making the Collection of Information Necessary
The National Institute for Occupational Safety and Health (NIOSH), within the Centers for Disease Control and Prevention, is taking an active stance to address mental health concerns among the more than 20 million workers in the nation’s healthcare sector. This activity is in response to the congressional mandate within the COVID-19 American Rescue Plan of 2021 (Attachment 1a), and on the heels of the passage of the Dr. Lorna Breen Health Care Provider Protection Act (Attachment 1b). For many years now, healthcare workers have reported feeling undervalued, overworked, and overwhelmed. A 2012 study found nearly half of physicians had symptoms of burnout.1 The COVID-19 pandemic has only exacerbated the strain and pressure facing healthcare workers as they endure unprecedented challenges that make working in this field exponentially harder on their own health and wellbeing. Depression, anxiety, and PTSD are prevalent among healthcare workers across the United States.2,3
While many Americans experienced some respite from COVID-19 since the start of the pandemic, healthcare workers remained on the front lines, in communities where infections and deaths remained highest and in settings where their charge was to care for the most immunocompromised and sickest Americans. Add to this staffing shortages, a lack of resources and beds across health centers of all sizes, public mistrust in medical professionals in certain areas, and hesitancy of healthcare workers to access support due to licensure and credentialing issues, it is no wonder that our nation’s healthcare workers need support, especially from the systems that employ them.
NIOSH, the federal agency tasked with conducting research that contributes to reductions in occupational illnesses, injuries, and hazards, and its contractor, JPA Health, plan to develop, implement, and evaluate a social marketing campaign for hospital leaders and healthcare workers who provide direct patient care in hospitals. The goal is to awareness of mental health risks, promote help seeking and treatment among healthcare workers experiencing burnout and job-related distress, and establish organizational policies and practices that support worker mental health.
While many interventions focus on health workers’ behaviors, few address the organizational causes of health worker burnout. For this reason, NIOSH proposes to evaluate the campaign through a follow-up survey of hospital leaders and healthcare workers. This activity supports NIOSH’s goal to improve work design and well-being among healthcare workers in the current NIOSH Strategic Plan as well as NIOSH’s efforts to build evaluation capacity.
This is a new information collection request for one year of data collection.
The purpose of this ICR is to collect data to document outcomes of a social marketing campaign to improve the mental health and well-being of healthcare workers (e.g., mental health help seeking, modifications to working conditions, and healthcare worker well-being). The campaign will be developed by NIOSH’s contractor, JPA Health (JPA), with input from NIOSH. Education Development Center (EDC), a subcontractor to JPA and in partnership with JPA’s evaluation team, will oversee the collection, cleaning, storing, protection, and analysis of data collected through this project within NIOSH’s Edge Computing Platform (NECP).
The social marketing campaign, and therefore the evaluation, focuses on two populations:
Hospital leaders, defined as executives or senior managers who have decision making authority within hospital systems.
Healthcare workers, defined as workers who provide direct patient care in hospitals. This includes, but is not limited to, physicians, nurses, therapists (e.g., respiratory, physical), and patient care technicians.
Follow-up surveys will examine whether hospital leader and healthcare worker exposure to and engagement with campaign activities and messages contribute to changes in their knowledge, beliefs, and practices thought to promote healthcare worker mental health and well-being. A post-only design was chosen to reduce burden and maximize response rates, compared to a baseline and follow-up design. Data collection includes 10-minute online surveys for healthcare workers and hospital leaders.
This study will include a representative sample of healthcare workers and hospital leaders that hail from relevant partner network organizations of the All In: WellBeing First for Healthcare network (e.g., American Medical Association, American Hospital Association, American Nurses Association). The hospital leader and healthcare worker surveys will be conducted at two points in time:
December 2023, 10 months after campaign launch, and
February 2023, 12 months after campaign launch
Half of the sample will be collected during each data collection period; each person will only be surveyed once.
The table below presents the types of information that will be disseminated to whom and how. Results will be presented in aggregate. Results do not allow us to generalize to the entire U.S. healthcare system.
Products |
Information |
Audience |
Peer-reviewed publications |
Aggregate results |
Public health and healthcare researchers; policymakers |
Presentations at professional conferences |
Aggregate results |
Healthcare and public health practitioners; healthcare executives |
Internal and external evaluation reports |
Aggregate results |
CDC/NIOSH and other DHHS agency officials; U.S. legislators |
Short articles in professional group, trade, and labor union press |
Aggregate results |
Healthcare workers and hospital leadership |
To supplement the survey data, JPA/EDC will also track campaign reach, engagement, and reception. This includes tracking social media mentions, conversations, and connections about the campaign using JPA’s Gretel® software. JPA will also monitoring dissemination channel and document which messages or campaign products are shared by whom and when. User feedback on the campaign website will be obtained through a voluntary web survey (OMB Control 0920-0953).
Survey information is important to collect because, as noted above, very few interventions address the organizational causes of healthcare worker burnout. In turn, few data exist to inform how the field can and should address this major public health and occupational safety issue. Without a seeking direct feedback from a representative sample of the target audiences of the campaign, NIOSH would not understand the impact of this investment or be able to make informed decisions about future investments in this area.
Survey data will be collected via REDCap to reduce participant burden and ease data collection, entry, cleaning, analysis, and security. NIOSH and EDC/JPA are cognizant of participant burden and both surveys will take approximately 10 minutes to complete. The surveys also do not contain open-ended questions, which are more time consuming to answer. Survey questions have been carefully aligned to elements on the campaign logic model, to ensure that there are no extraneous questions. Skip patterns will be programmed in REDCap to reduce the number of questions asked of respondents that do not apply. Participants may use desktop and mobile devices to complete surveys. Compared to paper surveys, online surveys reduce burden, require less time, and are less costly to employ.
Existing epidemiological data sources – including SAMHSA’s National Survey on Drug Use and Health and CDC’s Behavior Risk Factor Surveillance system – collect information on various aspects of health and well-being among adults in the U.S. However, existing epidemiological sources like these do not collect the detailed information needed to evaluate outcomes specific to this population (i.e., hospital leaders and healthcare workers). Further, existing data sources do not collect information on specific outcomes of interest to this project (e.g., perceptions of workplace changes and beliefs and practices related to help seeking for mental health).
NIOSH conducted an environmental scan to assess the extent to which other public or private partners are engaged in similar projects. They found that while many individual-level interventions specific to healthcare and healthcare workers exist, very few interventions address the organizational-level causes of healthcare worker burnout. JPA Health conducted an environmental scan of existing health communication campaigns focused on addressing the mental health and well-being of healthcare workers. They found no true social marketing campaigns, and none from other federal health agencies. This campaign, and its associated evaluation surveys, takes a unique approach that focuses on employers and organizational change.
5. Impact on Small Businesses or Other Small Entities
This data
collection will not involve small businesses.
This a one-time, retrospective data collection. Although the data will be collected at two points in time, each wave uses a different sample of hospital leaders and healthcare workers. In other words, there are two sample groups that will be surveyed once. This post-only design was chosen to reduce burden and maximize response rates, compared to a baseline and follow-up design.
This request
fully complies with the regulation 5 CFR 1320.5.
A 60-day Federal Register Notice was published in the Federal Register on November 16, 2022, vol. 87, No. 1779, pp. 56953-55 (see Att 2a). CDC received three comments; two expressed support for the campaign and one was non-substantive. Each received a standard CDC response (Att 2b). NIOSH received one request for additional information via the CDC Information Request Collection Office and responded over email (Att 2b).
The version of the information collection available during this public comment period included a quasi-experimental study with 12 hospitals (6 intervention and 6 comparison) and pre-post surveys and interviews. Due to logistical challenges and time constraints, the quasi-experimental study has been discontinued.
NIOSH and JPA Health have made several efforts to consult outside the agency. On June 30, JPA Health led a briefing session for the All In: WellBeing First for Healthcare steering committee. Participants reflected on their experiences around mental health in healthcare and provided individual input on campaign audiences and objectives. Many of the participants expressed a desire for organizational change. NIOSH led a second briefing session for its partners on July 19, 2022, which included representatives from healthcare systems, labor unions, academics who run NIOSH-funded Centers of Excellence for Total Worker Health®, and other relevant organizations. Participants provided individual input on potential actions that the campaign could promote to employers. Like the first briefing session, participants expressed strong support for organizational change rather than individual approaches like resilience training. This input shaped both the campaign and its evaluation.
JPA/EDC provided drafts of the survey instruments to 3 members of the All In: WellBeing First for Healthcare steering committee for comments in August 2022. As a result of this input, the surveys were shortened from 15-20 minutes to 10 minutes and the language was simplified wherever possible. For example, surveys originally included the long, technical ICD-11 definition of burnout and has since been replaced with a shorter version that uses everyday words. Reviewers expressed a strong preference for the term “healthcare worker” over “health worker”, and for “staff” rather than “clinical staff” so adjustments were made accordingly. Additionally, feedback indicated that the information on the first page of the surveys was too long, technical, and made the survey questions seem more invasive than they really were. The language has since been revised to focus on the most important points using plain language.
NIOSH sought input from three peer reviewers in October 2022. Peer reviewers are academics or practicing professionals whose identity is held secret from the study investigators. Two reviewers felt the anticipated response rate of 44% was overly optimistic for an online survey. The response rate was adjusted downward to 25% and the sample increased to maintain the goal of 3,000 healthcare workers and 500 hospital leaders. One peer reviewer also recommended a greater focus on health equity, which was incorporated into the evaluation questions and analysis plan.
NIOSH will not be providing financial incentives to participants.
10. Protection of the Privacy and Confidentiality of Information Provided by Respondents
ISSO determined in conjunction with the CDC Privacy Office that Privacy Act is not applicable. The collection contains PII with demographic information in the survey (i.e., Age, Race, Ethnicity and Gender).
Research Electronic Data Capture (REDcap) and NIOSH Edge Computing Platform (NCEP) include the in-place technical, physical, or administrative controls (safeguards).
Research Electronic Data Capture (REDcap) and NIOSH Edge Computing Platform (NCEP) System Security Plan (SSP) defines the process for handling security incidents. The system’s team and the Cybersecurity Program Office (CSPO) share the responsibilities for event monitoring and incident response. Direct reports of suspicious security or adverse privacy related events to the component’s Information Systems Security Officer (ISSO), CDC helpdesk, or to the CDC Security Incident Response Team (CSIRT). The CDC CSPO reports to the HHS Computer Security Incident Response Center (CSIRC), which reports incidents to US-CERT as appropriate.
Throughout the project, best practices in participant protections will be applied to ensure the security, privacy, and confidentiality of respondent information.
JPA and EDC researchers will manage the quantitative data collection using the REDCap survey system overseen by CDC’s National Center for Emerging Zoonotic and Infectious Diseases, and all data will be stored on the NIOSH Edge Computing Platform. All survey data files will be saved and archived in their raw format. Additional data files containing processed, reformatted, or summarized data will be created, edited, or reformatted as necessary to meet the requirements for conducting various statistical analyses in SPSS. These files will be regularly inspected by team members for quality (i.e., accuracy, omissions, and errors).
Participants will be informed that the survey is voluntary and that they are free to answer or decline to answer any particular question for any reason. Participants will be informed of the purpose of the study, the benefits of participation (i.e., to further the field), and the risks to participation (i.e., minimal, with breach of confidentiality being one potential risk). Participants will be instructed that if they continue with the survey, it will be considered consent to participate. Participants will be asked to indicate their occupation (e.g., physician, nurse, social worker), but will not be asked further identifying information (e.g., name, email address, or IP address). Participants will be informed of storage procedures and protections. JPA/EDC will report all results in aggregate. Demographic data will also be reported in aggregate, to protect participant privacy.
Because the purpose of the proposed project is for evaluation, the activity does not meet either the definition of research as specified under 45 CFR 46.102 (l) or the definition of clinical investigation as specified in 21 CFR 56.102 (l) (See attachment 3).
Questions about burnout and well-being may be considered private. Further, as noted by OMB, demographic data, including race and ethnicity data, may be considered sensitive. Section 10 (above) describes how sensitive information will be recorded and protected. As noted above in Section 10, participation in the online survey is voluntary and participants may choose to stop at any time or decline to answer any individual question. At the conclusion of the surveys, participants will be provided with links to mental health resources. The collection of this information is essential to answering the project’s main evaluation questions, which focus on the contribution of campaign efforts to healthcare worker mental health and well-being.
Acknowledging that organizations view information collections pertaining to organizational policies, performance data, or practices as sensitive, disclosure of some of the information collected through this project could create liability or competitive disadvantage. All efforts will be made to protect the identity of participating organizations.
(a) Estimated Annualized Costs
The goal is a representative sample of 3,000 healthcare workers and 500 hospital leaders that hail from relevant partner network organizations (e.g., American Medical Association, American Hospital Association, American Nurses Association) of the All In: WellBeing First for Healthcare network. Assuming a 25% response rate, JPA/EDC must include 12,000 healthcare workers and 2,000 hospital leaders. Both surveys take no more than 10 minutes to complete.
If the response rate is 100%, the burden is 2333 hours. If the response rate is 25% as expected, the burden is 583 hours.
Type of Respondents |
Attachment Name |
No. of Respondents |
No. of Responses per Respondent |
Average Burden per Response (in hours) |
Total Burden Hours |
Healthcare Worker |
Follow-up Survey (Att 4) |
12,000 |
1 |
10/60 |
2000 |
Hospital Leader |
Follow-up Survey (Att 5) |
2000 |
1 |
10/60 |
333 |
TOTAL |
|
|
|
|
2333 |
(b) Annualized Cost to Respondents
As noted above, JPA/EDC will work with partner organizations to invite hospital leaders and healthcare workers to participate. The healthcare worker audience include patient-facing staff in specific occupation groups. To calculate an hourly wage for hospital healthcare workers JPA/EDC used a weighted average based on the following assumptions (e.g., when considering these 9 occupation groups only, registered nurses comprise 60% of the hospital workforce):
Occupation |
Hourly Rate |
Weight |
Total |
Registered Nurse |
37.31 |
0.600 |
22.39 |
LPNs |
23.11 |
0.025 |
0.58 |
Nurse Practitioner |
59.51 |
0.020 |
1.19 |
Physician |
100.00 |
0.030 |
3.00 |
Phlebotomist |
17.97 |
0.010 |
0.18 |
Pharmacists |
61.81 |
0.030 |
1.85 |
Patient Care Tech |
29.80 |
0.190 |
5.67 |
Therapist |
45.97 |
0.085 |
3.91 |
Physician Assistant |
58.43 |
0.010 |
0.58 |
TOTAL |
|
|
39.35 |
Hourly rates included in the table above are based on median hourly wages provided by the May 2021 National, State, Metropolitan, and Nonmetropolitan Area Occupational Employment and Wage Estimates, Bureau of Labor Statistics (BLS) for General Medical and Surgical Hospitals. Median hourly wages for patient care technicians were based on those for radiologic technician; and wages for therapists were based on those for physical therapist. Median wage calculations for hospital leaders in the table below are also derived from the BLS and are based on those for chief executives. The table below provides an overview of total respondent costs based on type of respondent, survey to be completed, total burden in hours to complete the survey, and median hourly wage.
If the response rate is 100%, the burden is $112,000. If the response rate is 25% as expected, the burden is $28,000.
Type of Respondents |
Attachment Name |
Total Burden Hours |
Hourly Wage Rate |
Total Respondent Costs |
Healthcare Worker |
Follow-up Survey (Att 3) |
2000 |
$39.35 |
$78,700 |
Hospital Leader |
Follow-up Survey (Att 4) |
333 |
$100.00 |
$33,300 |
TOTAL |
|
2333 |
|
$112,000 |
Total capital and start-up cost (annualized over its expected useful life).
JPA/EDC anticipate that 11 partner member organizations will incur some costs associated with the following tasks: Developing a sampling frame based on eligible respondents (8 hours); and helping draw a stratified (as appropriate), random sample from their member rosters (8 hours). These costs per partner are estimated to be $780 (based on a median hourly rate of $48.72 * 16 hours). This hourly rate is based on the median salary for medical and health services managers (derived from the May 2021 National, State, Metropolitan, and Nonmetropolitan Area Occupational Employment and Wage Estimates, Bureau of Labor Statistics). Each partner member will also incur $585 associated with emailing survey invitations and reminders to members selected to participate ($48.72 * 12 hours). The total cost for each partner organization to participate in the survey is $1365. The total across all 11 partner organizations is $15,015, for an annual cost of $7,508.
Expense Type |
Expense Explanation |
Annual Costs (dollars) |
Personnel Salary |
Partner network survey sampling strategy and survey distribution |
$7508 |
|
TOTAL START UP COSTS |
$7508 |
Total operation and maintenance and purchase of services component.
NIOSH and JPA/EDC do not expect respondents to purchase any equipment in order to participate in these evaluation studies.
The total cost of this data collection is $473,878 over six months
Expense Type |
Expense Explanation |
Annual Costs (dollars) |
Personnel Salary |
Project Manager (technical oversight) – DrPH Program Evaluator (GS-14 685) at 5% FTE |
$3139 |
Personnel Salary |
Project Manager (technical oversight) – MPH Program Evaluator (GS-13 601) at 5% FTE |
$2739 |
Contract |
Evaluation task in contract National Education and Awareness Social Marketing Campaign Employer Efforts to Support the Mental Health of Healthcare workers |
$388,000 |
Administrative |
Background checks and network access for four JPA/EDC researchers |
$80,000 |
|
TOTAL COST TO THE GOVERNMENT |
$473,878 |
15. Explanation for Program Changes or Adjustments
This is a new data/information collection.
Projected Time Schedule
Activity |
Timeline |
Follow-up survey - wave 1 |
Dec 2023 |
Follow-up survey – wave 2 |
Feb 2024 |
Cleaning and analysis of post-campaign data (Quantitative) |
Feb-Mar 2024 |
Publications, presentations, reports |
Apr-May 2024 |
Analyses of retrospective data will examine reported perceptions of well-being and changes in outcomes such as increases in awareness and knowledge as well as, changes or intent to change working conditions and help seeking behaviors. These analyses will not only be conducted across all participants, but data will be stratified by race, ethnicity, gender, and occupational groups to determine whether associations exist between outcomes within and across groups to better understand whether the campaign benefits some groups more than others.
The display of the OMB expiration date is not inappropriate.
There are no exceptions to the certification.
1 Shanafelt, Tait D., et al. “Burnout and Satisfaction with Work-Life Balance among Us Physicians Relative to the General US Population.” Archives of Internal Medicine, vol. 172, no. 18, 2012, p. 1377., https://doi.org/10.1001/archinternmed.2012.3199.
2 Galanis, Petros et al. “Nurses' burnout and associated risk factors during the COVID-19 pandemic: A systematic review and meta-analysis.” Journal of advanced nursing, vol. 77, no. 8 (2021): 3286-3302. doi:10.1111/jan.14839
3 “The Mental Health of Healthcare Workers in COVID-19.” Mental Health America, Mental Health America, Inc., https://mhanational.org/mental-health-healthcare-workers-covid-19.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Dahm, Matthew M. (CDC/NIOSH/DFSE/FRB) |
| File Modified | 0000-00-00 |
| File Created | 2023-08-18 |