EFFECT OF WELLNESS GRANT ON WORKER HEALTH AND SAFETY
Request for Office of Management and Budget (OMB) Review and Approval
for a Federally Sponsored Data Collection
Supporting Statement B
Alysha R. Meyers, Ph.D.
Epidemiologist
Project Officer
National Institute for Occupational Safety and Health
Division of Surveillance, Hazard Evaluations, and Field Studies
1090 Tusculum Ave., Mail Stop R15
Cincinnati, Ohio 45226
513-841-4208 (phone)
513-841-4486 (fax)
January 1, 2016
Table of Contents
B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS 3
B1. Respondent Universe and Sampling Methods 3
B2. Procedures for the Collection of Information 4
B3. Methods to Maximize Response Rates and Deal with Nonresponse 5
B4. Tests of Procedures or Methods to be Undertaken 5
B5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 6
This information collection does not employ statistical methods to select respondents.
Statistical sampling methods are not being used to select employers to be interviewed for either the economic interviews or the annual case study verification interviews.
For the economic interviews, the respondent universe includes employers who were awarded their first wellness grant at least one year prior to the interview and had been insured by OHBWC for at least two years prior to receiving their first wellness grant. As of January 31, 2015 no more than 247 employers would be eligible and as of August 31, 2015 this could include no more than 280 employers. WWGP enrollment is ongoing so the number of eligible employers may increase depending on when we receive OMB approval to proceed with data collection. In addition, employers that could be interviewed for the annual case study verification interviews only would also have had to submit at least one annual case study. The annual case study is a mandatory requirement of the program so we anticipate that the response rate will be very high eventually, although not all employers submit the case studies in a timely manner. OHBWC bills employers who do not submit an annual case study and the employer is expected to pay back any grant funding. Regardless, it is impractical for the selection of which employers participate in the economic interviews to be completely random.
For the economic interviews we have used a random number generator to rank each grantee and we will continue to call then in consecutive order until we have recruited the maximum of 25 participating employers. However, depending on the time of year and funding levels, we may add geographic constrictions to avoid having to stay overnight or drive long distances in inclement weather. It is unlikely that modifying our selection criteria based on location alone would bias our sample in a way that would introduce a differential error on the information we receive regarding time and money spent on their wellness programs. Based on the response rate to the six initial interviews we do not expect a response rate greater than 50%, which is fairly typical for this type of field research.
For the annual case study verification interviews we are not trying to obtain a random or representative sample. Instead these interviews are being used to verify responses, improve data quality and validate some survey items. Therefore we will target some employers when they skipped questions, gave unusual responses, or intriguing responses to open-ended questions on the survey, or opted to drop out of the three-year program prematurely (1-2 years).
For this study, the target population includes any employer insured by OHBWC across all industry sectors in Ohio, including public and privately owned establishments. OHBWC insures all Ohio public employers and all private employers with <500 employees. The sampling frame (segment of the target population) includes an OHBWC-insured employer who received grant funding as part of the OHBWC WWGP and have completed at least one year of the program. The study sample (employers chosen from the sampling frame) includes OHBWC-insured employers who agree to participate in this OHBWC-NIOSH collaboration research project.
For each interview we have chosen to interview the minimal number of employers that should provide meaningful insights into the economic experiences of employers in the WWGP, is feasible for our research team, and help to understand employer motivation for starting wellness programs. We do not have to resources to conduct any more interviews than the modest number that we have planned. By conducting six interviews it’s clear from the variety of responses that we need to conduct more interviews. We expect to add meaningful data to inform our analyses after conducting 25 economic interviews and 50 annual case study verification interviews. The two data collections described in this request are being conducted using limited funds. In an ideal world it would be better to conduct more interviews but we are only committing to the least amount that are meaningful, feasible and cost effective for our team and budget.
For the economic interviews NIOSH and OHBWC are conducting these in-depth, structured interviews with at least nine but no more than 25 participating employers. These semi-structured interviews (Attachment J) will be administered once to up to 50 key informants from 25 randomly selected employers that received grant funding from the WWGP.
The annual case study verification telephone interviews will be will be administered 1-2 times per employer, so up to 100 key informants may participate from randomly selected employers that have received grant funding from the WWGP and are not participating in the semi-structured economic interviews. The topics of theses telephone interviews will be limited to asking the employer to verifying responses to selected questions in Section II what their motivation was for starting a wellness program and applying for the grant and then clarifying an employer’s answers to selected questions about the employer and their
The purpose of the data collection we are proposing is to provide important data to help address the need to 1) assess the effectiveness and cost-benefit of an intervention that funds workplace wellness programs among employers in Ohio insured by the OHBWC and 2) understand the impact of integrating of wellness with traditional occupational safety and health (OSH) programs.
The proposed data collection will be used to address the following specific aims from the overall study:
Determine the total costs, savings, and savings (benefits) to cost ratios associated with grant-supported wellness programs from the perspective of OHBWC and the participating employers.
Determine the relationship between WC claim rates and changes in yearly pre- and post-intervention measures for OSH-wellness program integration measures
Final interview recruitment and survey administration:
To collect the data for the economic interviews, a research assistant assists with recruiting and scheduling the interviews. Then 1-2 investigators conduct the interview, typically with one key informant. All investigators are PhD-level researchers from different areas of expertise relevant to this study (e.g. economics, epidemiology, occupational and environmental health). All investigators are given the opportunity to observe at least one interview conducted by the economist or another trained interviewer prior to conducting an interview on their own.
Interviewees are sent an information sheet (Attachment M) in advance that describes the general content of the questions so that can prepare some data in advance, if possible. For all the pilot interviews we have had two investigators present. One conducts the interview and takes notes while talking while the second person also takes notes and makes sure the first person doesn’t skip any questions. If the employer agrees we make a digital recording of the conversation. Upon return to the office a research assistant helps the investigator with an initial transcription no the interview and then the investigator checks the transcript for accuracy and organizes the notes by topic. Later, we will develop a database of responses, to provide some standardization across interviews, when possible. The format itself is being used because it can be flexible and the interview can probe further on some questions, as needed.
For the telephone interviews, the investigators conducting the interviews may also be recruiting participants so that interviews can be conducted immediately, if time allows, or scheduled for later time. Prior to making these phone calls OHBWC will send an email to all potential participants to give them advance notice that a NIOSH researcher may contact that person and invite them to participate in the verification survey by phone. The initial contact information (firm phone number and/or email address, and name of key informant) will be provided by OHBWC for the targeted sample
As stated in section B1 we do expect a response rate of no more than 50%. Response rates of about 50% are fairly typical for field research like this study. The response to the initial economic interview recruitment efforts are as follows: attempted to contact 27 employers, nine were passive non-responders after several attempts, 18 were contacted, of which two said “maybe,” and seven agreed to participate.
To improve our response rate for both planned data collections, recruiters will make ten attempts to make contact the given participant, after which the employer will be classified a “passive non-responder.” Previously we were less persistent. The recruiters will cycle through contacting the targeted sample of firms until an interview is schedule, the firm has refused to participate, or the firm has not responded to ten requests (any combination of voice/ email contacts). After making the initial phone call an information sheet about the study will be emailed to each contact (Attachment M). There are particular employers with a stronger relationship with OHBWC that would be more likely to participate but those employers are likely to be systematically different than other employers who we invite to participate, which is why would rather use a more arbitrary calling order for recruitment purposes for the economic interviews. To maximize response rates we are encouraging the employers to contact OHBWC to if they want to verify the legitimacy of our study and we will be copying OHBWC staff on email correspondence with the employer after initial contact. We have developed a recruitment script in collaboration with our colleagues at OHBWC that was developed to encourage participation. Following the script should help keep our response rates consistent across employers. No employer specific incentives are planned at this time other than providing the employers who participate with a copy of their own cost-benefit analysis results in the future.
Estimates of time burden and usability for semi-structured interview pilot-testing are based on recent pilot testing conducted at NIOSH. So far the semi-structured economic interviews have been piloted with six OHBWC-insured employers using the person responsible for their workplace wellness program as the key informant for the employer. On average, each interview lasted 90 minutes (no more than two hours) plus an average of 30 minutes to prepare so we determined the time burden based on these observations. The pilot tests went well and very few changes were made to the initial interview format so all data collected from the ‘pilot’ tests will be used in our final analyses.
The annual case study verification interviews have not been formally pilot tested independent of the semi-structured economics interviews. The need for these verification surveys independent of the economic interviews became apparent based on responses we received during interview when we verified their responses to some questions in Part II of the annual survey or asked them what their motivation was for applying for the grant and starting a workplace wellness program. To ensure clarity and understandability of questions, and to minimize the inconvenience of the procedures undertaken, we will pilot the format of the verification interviews separately on fewer than 10 OHBWC, NIOSH or employer volunteers soon.
This protocol was reviewed by Dr. Elizabeth Whelan, the Industrywide Studies Branch Chief of NIOSH’s Division of Surveillance, Hazard Evaluations, and Field Studies (DSHEFS), as well as Dr. Douglas Trout, Associate Director for Science of NIOSH’s DSHEFS. As part of the NIOSH peer review process, in 2012, the WWGP evaluation study was peer-reviewed and rated based on project approach, potential impact, innovation, and significance by the NIOSH Safety and Occupational Health Study Section as part of the NIOSH National Occupational Research Agenda (NORA) competitive process for intramural research. The project received favorable scores and was chosen for funding by NIOSH from Fiscal Year 2014 through Fiscal Year 2017. One criticism of the proposal was that nine semi-structured economic interviews was not an adequate sample to estimate what employers were spending on their wellness programs and it was strongly recommended that we conduct more interviews so that the sample can be more representative of the entire population of grant recipients included in the study. We agreed with the review and decided that 25 interviews would be an adequate and feasible goal for our research team if we obtain OMB approval. The review panel for the NORA Fiscal Year 2014 process is listed in Section A8 of this request.
NIOSH personnel, OHBWC and NIOSH contractors co-designed data collection and data analysis plans. NIOSH investigators will perform the data collection, and analyze the data. Below is a summary of individual roles on this project.
NIOSH personnel
Name |
Job Title |
Division |
Contact Information |
Roles on Project |
Alysha Meyers, Ph.D. |
Epidemiologist |
Division of Surveillance Hazard Evaluation and Field Studies (DSHEFS) |
513.841.4208 |
Project Officer:
Co-designed data collection and data analysis plans
|
Tim Bushnell, PhD, MPA |
Economist |
Economic Research Support Office (ERSO) |
Economist and lead on designing, conducting and analyzing data from the economic interviews. |
|
Steve Wurzelbacher, Ph.D. |
Research Industrial Hygienist |
Division of Surveillance Hazard Evaluation and Field Studies (DSHEFS) |
513.841.4322 |
Co-designed data collection and data analysis plans
|
Steve Bertke, Ph.D. |
Statistician |
Division of Surveillance Hazard Evaluation and Field Studies (DSHEFS) |
513.841.4493 |
Co-designed data collection and data analysis plans
|
Chia Wei, PhD |
Epidemiologist |
Division of Surveillance Hazard Evaluation and Field Studies (DSHEFS) |
513.841.4562 |
Data collection |
Kaori Fujishiro |
Epidemiologist |
Division of Surveillance Hazard Evaluation and Field Studies (DSHEFS) |
513.841.4120 |
Co-designed data collection and data analysis plans, data analysis
|
The OHBWC also helped design the data collection. Below is a summary of individual OHBWC staff roles on this project
Name |
Job Title |
Division |
Contact Information |
Roles on Project |
Carol Morrison |
Manager, DSH Outreach Programs and Services |
Division of Safety and Health |
Co-designed data collection |
|
Michael L. Reinerth, CIE, CHSP |
Ergonomics Technical Advisor |
Division of Safety and Health |
216.538.9724
|
Co-designed data collection
|
Abe Tarawneh, Ph.D. |
Superintendent |
Division of Safety and Health |
614.466.0384
|
Supervising OHBWC role on overall project |
Dayona Turner |
Grants Program Coordinator |
Division of Safety and Health |
614-728-3008 |
Data management, quality control |
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Meyers, Alysha R. (CDC/NIOSH/DSHEFS) |
| File Modified | 0000-00-00 |
| File Created | 2021-01-24 |