Supporting Statement B
Health Center Workforce Well-being Survey Evaluation and Technical Assistance
OMB Control No. 0915-XXXX
1. Respondent Universe and Sampling Methods
The respondent universe includes all full-time and part-time staff across all HRSA-funded health centers nationally. It is expected that the total number of eligible staff will be approximately 450,000 individual staff, making up approximately 280,000 full-time equivalents (FTE). The targeted completion rate for the voluntary Health Center Workforce Survey is an ambitious 85%-or more of the total health center full and part-time staff. Surveying the HRSA-funded health center workforce is necessary to create a baseline understanding of workforce wellbeing including burnout, job satisfaction, and intention to turnover for all staff. The intention is also to provide aggregate responses at the health center level for feedback and quality improvement purposes. Therefore inviting all staff to participate would maximize the reliability and validity of this information. Analysis of the survey data will provide national comparisons across health center occupation groups and will provide insight for quality improvement initiatives including future targeted technical assistance to promote workforce recruitment and retention, as well as improve patient quality of care.
2. Procedures for the Collection of Information
HRSA and the HRSA contractor will employ various methods to maximize participation and quality of data collection.
Health Center Recruitment Procedures
To maximize health center engagement, HSRA and HRSA’s contractor will make direct contact with staff at all of the approximately 1,400 national HRSA-funded health centers. The approach is to designate approximately 40 experienced professionals as health center Liaisons, each of whom will work with approximately 35 health centers to obtain the health center’s commitment to participate in the Health Center Workforce Well-being Survey effort. The entire recruitment, engagement, and readiness to launch process will take place over a five-month period. Activities conducted by the health center Liaisons to promote survey participation by health center directors include:
Sending out initial email invitations to their assigned health centers. The initial outreach will include a survey flier which describes survey development, topics, confidentiality of responses, and the importance of participation.
Tracking who sent back confirmation engagement emails (expected 20% received).
Two weeks later, sending follow-up emails to those not yet responding with FAQs included.
Tracking who responded based on the follow-up provided confirmation of engagement (expected 30% more agreeing to participate).
Making phone calls to health center directors (or designee) to explain the request, answer questions, and obtain commitment. This process may involve additional emails that provide answers to specific issues raised by the health centers as well as calls back and forth as needed.
Extending these efforts over 3 months to result in another 25% engaged (75% total of health centers after 4 months).
In the final 2 months, working to obtain engagement commitment from the remaining 25% (estimated at perhaps 9 health centers for each liaison) invoking information and encouragement from national, regional, and state partners; HRSA; and other local area health centers.
Pre-Launch Activities
After obtaining commitment from the health center to participate, the health center Liaisons will work with the health center director or their designee to ensure their understanding and to undertake a limited number of pre-launch procedures provided in steps a.-h. below. This will be a rolling process spread over approximately 5 months and will begin as soon as health centers confirm their engagement in the Health Center Workforce Survey effort.
Communication will be primarily via email with requested confirmation by email. For those having questions, follow-up phone calls will be made. The pre-launch activities and commitments are as follows:
The health center understands that about two months before launch they will enter the email addresses of all staff (full- and part-time) into an invitation interface that the HRSA contractor will use to generate individual survey invitation emails. Liaisons will request through email or phone contacts that the health center director or designee provide the HRSA contractor with email addresses of all staff two months before the launch using an electronic form provided by the HRSA contractor. It is expected that the total number of eligible staff will be approximately 400,000 individual staff, making up approximately 250,000 full-time equivalents.
The health center understands and will implement their IT department whitelisting of emails from the HRSA contractor so that the emails reach the employees and do not get intercepted as spam.
The Liaisons will discuss and encourage health center directors to consider a variety of local incentives to encourage staff participation. Although health center directors will not have access to information about which individual staff members participated in the survey, Liaisons will discuss ways in which local incentives might be promulgated.
The health center understands and will be prepared to send out their own emails (draft content to be supplied by the HRSA contractor) to all staff at about two weeks and again one day before launch, emphasizing that the health center is participating in the HRSA Health Center Workforce Well-being Survey.
Survey and that leadership strongly encourages everyone to participate.
The health center understands and will be prepared to send emails from the health center director’s office (draft content to be supplied by the HRSA contractor) one day before launch and 10 days after launch encouraging all to participate.
Liaisons will inform the health center director that the HRSA contractor will send a pre-notification email about 3–4 weeks before launch to all employees alerting them to the up-coming survey invitation. This pre-notification email will alert the HRSA contractor to any problematic emails and Liaisons can follow-up with their health centers to correct these problems (although, if this is the same list health center directors used to notify employees of the survey, there should be relatively few email problems).
BPHC will be working in collaboration with HRSA partners including, Primary Care Associations, and National Training and Technical Assistance Centers, to promote the survey and to ensure health center leadership are aware of the survey and how this effort supports the U.S. Department of Health and Human Services’ Health Workforce Strategic Plan.
The survey will be launched as an online survey, using SurveyAnalytics, a selected federally approved and 508 compliant survey software. There will be a live link to the survey embedded in the email (in the invitation and subsequent reminders). The link will be unique to each staff member and will also indicate each staff member’s health center. Staff will be able to complete the survey on a variety of devices including desktops, laptops, tablets and smartphones. It is anticipated that approximately 400,000 full and part-time staff will be invited to participate.
Health Center Director Efforts. After launch, Liaisons will be the channel for communication with their assigned health center directors or designees on progress, as well as any issues that may arise.
Liaisons will send data to the health center director on a weekly basis that shows how the health center is performing on response rates.
At 10 days post-launch, Liaisons will request that health center directors send out their own emails (content provided by the HRSA contractor but can be modified as needed) to their staff encouraging a final push to reach at least an 85% response rate.
Data Collection Activities.
The HRSA contractor
will promote a variety of actions to facilitate staff participation
in the survey including: The HRSA contractor will send emails to
staff individually. These emails will have a unique link to the
survey embedded in them, which will associate the responses with a
particular individual and the health center to which they belong. The
individual link will allow the staff member to complete the survey in
multiple sittings and from multiple types of devices if they are
unable to complete the survey in one sitting. These unique email
links will allow the HRSA contractor to know which staff have
completed their surveys and who to send reminders to. These links can
only be used once for submitting a completed survey. The HRSA
contractor will have earlier, during pre-launch activities, sent out
a pre-notification email to make sure the emails work. If any email
address bounces back, Liaisons will check with their health center to
resolve the problem.
The contractor will send out
invitations in multiple cohorts. The cohorts will be established in
advance of the survey launch to allow for a predictable and balanced
load, and earlier cohorts may be smaller to assure that processes
function smoothly and lessons learned can be assimilated into the
process of future cohorts. The timing of survey cohort invitations
will be recorded such that follow up can be done in a consistent and
timely manner, and that the response tracking dashboards will be able
to show any response lags accurately. The timing of each survey’s
invitation date/time will be available in the tracking data and used
to allow monitoring of the ‘aging’ of survey invitations
and response progress based on the duration since the invitation and
timing of reminders.
Reminders will be sent out by the HRSA contractor to those who have not submitted a completed survey at intervals of 3-5 days for up to 3 reminders.
As soon as a staff member submits their survey, reminders to that person will cease. However, if in the data cleaning process, issues are identified, the HRSA contractor may follow-up to clarify the responses with that person.
3. Methods to Maximize Response Rates and Deal with Nonresponse
The HRSA contractor will employ 3 main methods in conjunction with the pre-and-post-launch data collection activities listed previously in order to maximize response rates.
Technical Assistance During the Data Collection Period
The HRSA contractor will develop a data collection process supported by a multi-layered support team capable of addressing the full range of issues that may arise from the pool of respondents. In the invitation emails and in the survey itself, an email link for technical assistance will be provided, should respondents have difficulty completing the survey. In addition to on-line references and FAQ materials that will be made available to respondents, the HRSA contractor will make live support trained staff available by phone, email, and, potentially, web chat. Technical assistance will respond to technical issues related to the survey software, as well as questions regarding the survey process, confidentiality, and the use of the data.
The goal of the technical support function will be to provide live support within a short wait time. As noted, staggering the release of the survey invitations will avoid bottlenecks to the degree possible. If any backlog builds up, the HRSA contractor will also offer callers the opportunity to receive a call back instead of waiting. Email questions will be fielded as they come in. Zoom meeting support will be available if the nature of the problem requires someone to examine computer settings or other issues requiring sharing screens or control.
Procedures to minimize sample non-response and item non-response
There are several different ways that non-response can be minimized within the online survey software.
Staff who have not started the survey and those that have started but not completed/submitted the survey will be targeted with email reminders encouraging them to start the survey or finish the survey if already started.
Item non-response is a second issue that the survey system will attempt to minimize. (1) The informed consent and instructions at the beginning of the survey will encourage staff to answer all questions and specifically emphasize anonymity of individual answers. (2) There are certain questions that require a response (i.e., occupational group). Notes at these locations will explain the requirement to answer and provide additional assurances of confidentiality. (3) A note will be placed at the beginning of the demographic section to emphasize the importance of these answers for the national purposes of the survey analysis and give reassurances that individual answers will not be identified. (4) The survey software can be programmed at the end of each survey “page” to provide a warning before proceeding to the next screen that identifies unanswered questions and encourages a respondent to go back and complete their answers before moving on.
A third issue is staff who submit a survey with a lot of missing answers. It is anticipated that with the reminders mentioned in “a” and “b” above, this will be a relatively rare occurrence. However, the HRSA contractor will have staff email addresses at this point and can email them a special request to go back to the survey (with the link embedded in the email) and complete the survey.
Protecting Individual Staff Identities
The data collection process will emphasize the protection of individual identities and explain how their answers will be kept confidential in the public data sets. The ability to assure health center staff that their data will be kept confidential at the individual level, as it will not be possible to link individual answers with a person’s identity, will be of the utmost importance. Responses will be kept confidential in the analytic data set. Methods of assuring protection include disassociating responses from the respondent’s email address; removing specific job title from the public data file; and providing only aggregate data back to health centers with minimum record count suppression to remove the possibility of respondent’s being identified through job titles or demographic characteristics. Furthermore, using an unidentifiable code in place of the health center grant numbers in the final data set provided to HRSA so the possibility of identifying specific health centers is removed. Internally the HRSA contractor will retain the ability to track responses at the individual and organizational level while data collection is underway. After data collection and cleaning is completed the contractor will take steps to sever the connection to individually identifying attributes permanently as there is no intention or reason to retrieve any information by individual respondents and any identifying information will not be part of the public database. Steps will also be taken to remove the linkage to individual health centers from any public facing version of the data, and the ability to view data at the organization level, or link it to other data such as the UDS, will be protected using cryptic codes kept separately in a crosswalk file that requires its own controlled permissions. This process will remove personal identifiers from the survey results to avoid triggering the Privacy Act.
In addition, HRSA has granted the project a Certificate of Confidentiality which adds an additional layer of protection from legal action to force disclosure of any individual answers.
4. Tests of Procedures or Methods to be Undertaken
Prior to the rollout of the national Health Center Workforce Well-being Survey, the HRSA contractor will have conducted two pilot surveys. A pilot test was conducted (OMB Control No. 0915-0379) in the development phase of the Health Center Workforce Survey. The pilot test allowed the opportunity to (1) calculate reliability coefficients for intended scales and (2) to identify low reliability scales by identifying questions that weakened the reliability of scales in this population or indicated the need for additional questions to more reliably measure the concept. A second pilot survey to be conducted prior to the national roll-out will be used to test the online survey tool systems functionality internally including the link to survey tool, skip patterns, selection of Spanish language version, reminder notifications, data collection and downloading of data files and correcting them before launch.
Before launching this pilot test, HRSA’s OIT office will confirm 508 compliance. OMB approval for the Health Center Workforce Survey Evaluation and Technical Assistance pilot survey will be under the generic clearance OMB Control No. 0915-0379 used for the Health Center Workforce Well-Being/Satisfaction pilot survey and will be obtained before launch.
5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
Meresa Stacy
Project COR
Health Resources and Services Administration
Jayne Berube, MS, RD
Team Lead, Care Integration and Workforce Team
Bureau of Primary Health Care
Health Resources and Services Administration
jberube@hrsa.gov
Lara Chausow, PhD
Data Statistician, Quality Division
Health Resources and Services Administration
Larry
Horlamus, MS
Deputy
Director, Quality Division
Office
of Quality Improvement
Bureau
of Primary Health Care
HRSA
Thomas Mangione
Principal Investigator, Senior Research Scientist
JSI Research & Training Institute, Inc.
Eric Turer
Health Systems Consultant
John Snow, Inc.
Tabeth Jiri
Senior Epidemiologist/Statistician
JSI Research & Training Institute, Inc.
Wendy Chow
Senior Data Analyst
JSI Research & Training Institute, Inc.
Mihaly Imre
Data Analyst
JSI Research & Training Institute, Inc.
Natalie Truesdell
Senior Consultant
JSI Research & Training Institute, Inc.
Rebecca Millock
Data Collection Specialist
JSI Research & Training Institute, Inc.
Leanna Pham
Project Associate
JSI Research & Training Institute, Inc.
Ann Keehn
Project Director, Senior Consultant
John Snow, Inc.
Laura Steere
Project Manager
JSI Research & Training Institute, Inc
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Jodi.Duckhorn |
| File Modified | 0000-00-00 |
| File Created | 2022-09-14 |