Appx E_ExtRevCmets

Appendix E: External Review Comments

Appendix O. NORA Review Comments

SUMMARY STATEMENT

(Privileged Communication)

Review Manager: Release Date: July 25, 2011

K. Ann Berry, Ph.D., M.B.A.

404-498-2503

[email protected]

Project Officer: Jean Cox-Ganser

Project ID#: NORA_FY12_DRD_Cox-Ganser_Flavorings

Solicitation: NORA FY12 Intramural Project Funding Opportunity: Advancing Sector Priorities

Review Group: SRA International, Inc External Peer Review

Overall Score: 32

Range: 10 (Exceptional) to 90 (Poor)

Meeting Date: 06/06/2011–06/07/2011

Project Title: Spectrum of Flavoring Chemical–Related Lung Disease

Total Years Funding Requested for:	Total NORA Funds Requested (All Years):
4	$239,873

Rating Criteria

Criterion 1—Significance: Does the project address an important problem or a critical barrier to progress in the field? If the aims of the project are achieved, how will scientific knowledge, technical capability, and/or the safety and health of workers improve? How will successful completion of the aims change the concepts, methods, technologies, services, or preventative interventions that drive this field?

Criterion 2—Project Officers and Key Personnel: Are the project leader(s) well suited to conduct this work? For project leaders who are in the early stages of their work in this area, do they have appropriate experience and training and/or will they receive appropriate mentoring and supervision? Have established project leaders demonstrated an ongoing record of accomplishments that have advanced their field(s) of work? If the project is collaborative, do the key personnel have complementary and integrated expertise and is the project’s leadership approach, governance and organizational structure appropriate?

Criterion 3—Innovation: Is a refinement, improvement, or new application of concepts or approaches proposed? Does the proposal challenge and seek to shift current research or workplace practice paradigms by utilizing novel concepts or approaches? Does the project address a recently recognized critical barrier to progress in the field or take unique or significant advantage of the results from a recent project?

Criterion 4—Approach: Are the overall strategy, methodology, and analyses well-reasoned and appropriate to accomplish the specific aims of the project? Are potential problems, alternative strategies, and benchmarks for success presented? If the project is in the early stages of development, will the strategy establish feasibility and will particularly risky aspects be managed? If an intervention project, does the proposal have an appropriate evaluation plan? Does the Project Officer include appropriate partners throughout the project and are their roles well-defined?

Criterion 5—Impact: Does the Project Officer describe how the expected activities or findings, e.g., knowledge, interventions, or technologies, will plausibly lead to a safer, healthier, workforce in either the near- or long-term? Does the Project Officer describe who will benefit from the project activities? If needed, does the Project Officer include appropriate partners and/or stakeholders in the project activities to help ensure successful transfer of the findings to the end users/intended audience?

Criterion 6—Environment: Will the environment in which the work will be done contribute to the probability of success? Are the institutional support, equipment and other physical resources available to the investigators adequate for the project proposed? Will the project benefit from unique features of the scientific environment, public health environment, or subject populations?

Score for Individual Criterion: 1 (Exceptional) to 9 (Poor)

Panel Discussion:

The investigators of this proposal intend to establish the spectrum of lung diseases associated with flavoring chemicals. It has been previously established that irreversible obstructive lung disease is related to diacetyl exposure in microwave popcorn plants, but since the focus was on the rare disease, bronchiolitis obliterans, the possibility of excess restrictive disease in flavoring-exposed populations has not been explored. As a result, possible work-related lung disease is not being diagnosed and is not motivating preventive interventions in workforces. The investigators seek to remedy the current practices in flavoring and food production industries by establishing a scientific basis for whether spirometric restriction reflects lung disease, whether restrictive lung disease satisfies attributes of work-related causality such as demonstration of a temporal relationship between workplace exposures and the deterioration of lung function, and the presence of exposure-response relationships. This proposal has three specific aims: To determine whether excess spirometric restriction exists among flavoring-exposed workers as compared to the United States population, to investigate exposure-response relationships that would support flavoring exposure as an etiologic factor for restrictive disease, and to characterize the nature of restrictive lung disease occurring in flavoring workers If successful, this proposal could assist in the diagnosis of possible work-related lung disease and motivate preventive interventions in these workforces.

The panel members considered this to be an excellent proposal that is very strong with only some minor weaknesses. A strength of this study is that it addresses the true extent of adverse respiratory disease associated with diacetyl exposure, providing useful information as to what medical conditions need to be monitored and protected against. This proposal could have a major impact as the study addresses a potentially important and clearly emerging occupational health issue. The environment at the NIOSH respiratory testing center is impressive, and the personnel involved in this project are highly experienced and have particular expertise in pulmonary function testing, assessment, and quality control. The project is quite innovative in that it proposes to investigate the occurrence of restrictive changes in what is considered an obstructive disease and proposes to use existing data from multiple institutions and to conduct followup activity from previous cohorts.

The proposal does contain some minor weaknesses. There was concern that Specific Aim 3 may not be able to be carried out on a sufficient number of individuals to confirm that the restrictive changes seen on spirometry truly reflect restriction and have an association with measures of exposure to flavoring agents. There were also concerns about the reliance on historical data from multiple sites, particularly for the exposure data.

In summary, the panel members concluded that this proposal is excellent and could have an important impact on knowledge, care, and management of workers with exposure to flavoring agents. The strengths of the proposal are that it utilizes an innovative approach to address a potentially important and emerging occupational health issue, the highly experienced research team, and the impressive research environment. Minor weaknesses include concerns about the approach and the reliance on historical exposure data. The panel members reviewed the proposal on the basis of the published evaluation criteria and rated it as described below.

Critique 1:

Significance Score: 1

The proposed research addresses the important issue of the true extent of adverse respiratory disease associated with diacetyl exposure. It will provide useful information to healthcare providers and to safety managers in the flavoring industry as to what medical conditions need to be monitored and protected against. This study will address whether diacetyl causes restrictive as well as obstructive disease.

Project Officers and Key Personnel Score: 1

The personnel involved in this project are highly experienced and have particular expertise in pulmonary function testing, assessment, and quality control. The management and coordination of the project is excellent.

Innovation Score: 2

The project is innovative in that it proposes to investigate the occurrence of restrictive changes in what is considered an obstructive disease and proposes to use existing data from multiple institutions and conduct followup activity from previous health hazard evaluation cohorts.

Approach Score: 5

As indicated in the proposal, the investigators very likely will be able to carry out Specific Aims 1 and 2. Sample size is adequate, data are available, and the analysis plan is described. Improvements in the approach for Specific Aims 1 and 2 could be made. More detail of the problems and how they will address problems of combining data from 29 cohorts from the 3 institutions involved needs to be provided. Also more details need to be provided on which ATS quality control parameters they will use to exclude spirometric results and assurance that the spirometric results of the reference group (National Health and Nutrition Examination Survey [NHANES]) are comparable in terms of the quality control measures used. The part of the proposal that potentially will provide the most important information, Specific Aim 3, has the least detail. It does not include power/sample size analysis, and there are no letters of support from partners in Missouri or the union or other partners in Indiana and no discussion of how the investigators will use measures that confirm restriction such as total lung capacity in the analysis versus the forced vital capacity screening results. There is a brief, inadequate analysis section that does not mention an approach to include an exposure estimate for the flavoring manufacturing workers.

Impact Score: 3

How the investigators will effect impact is well described, including involvement of partners.

Environment Score: 2

The environment is excellent. The equipment to be used is excellent. Institutional support is excellent. Letters of support for Specific Aims 1 and 2 are included although not for Specific Aim 3.The project will benefit from access to previously collected pulmonary function testing on a large number of exposed workers. These cohorts are not available to other investigators.

Overall Evaluation

The strengths of this proposal are the importance of expanding knowledge regarding the adverse respiratory health effect of exposure to diacetyl and working in the flavoring manufacturing industry, the excellent research team, and the use and combination of existing data from multiple previous investigations from three different institutions. The major weakness involves whether Specific Aim 3 can be carried out on a sufficient number of individuals to confirm that the restrictive changes seen on spirometry truly reflect restriction and have an association with measures of exposure to flavoring agents. If successfully carried out, the proposal will have an important impact on knowledge, care, and management of workers with exposure to flavoring agents.

Resubmitted Applications

Not applicable.

Other Considerations

Subjects are those workers found in the 29 plants, and inclusion of women and minorities and children are appropriate.

Critique 2:

Significance Score: 3

Strengths: The main significance of the proposed project is identification and characterization of a novel etiology for restrictive lung diseases that could lead to regulations to protect workers exposed to these types of agents. In addition, there are strengths in the broadening of the subject population from diacetyl (and related compounds) to other flavoring sources. The preliminary data strongly suggest that this may be a previously unrecognized issue. In addition, there is a laudable goal to protect the subjects from further deterioration of their lung function although this does not come through in the specific aims.

Weaknesses: Stand-alone epidemiological studies are always susceptible to confounding factors, and since the proposed questionnaire is not provided, it is unclear whether adequate attention will be paid to these types of issues; e.g., cigarette smoking (although it is listed as something to be included in that questionnaire). However, it is not obvious whether other factors (socioeconomic index, smoking history more extensive than “ever” or “never,” prior work history) will be available and, if so, if that information will be included. If these factors (including smoking) contribute to or interact with the observed effects, it is likely that the power calculations overestimate the power.

Project Officers and Key Personnel Score: 2

Strengths: The Project Officer (PO) has a Ph.D. in animal science with a subsequent M.S. in statistics. She has participated in several environmental studies although mostly in the field of exposures to biological agents. She was, however, first author on an abstract related to the topic of this proposal. The rest of the team is quite strong, including Kriess, who has been involved in the diacetyl problem for years; Bailey, in designing the medical surveys; and Piacitelli, an industrial hygienist with experience in diacetyl. Two epidemiologists, Park and White, will assist with risk assessment and statistical analyses. Park’s experience is primarily with cancer end points and metals/silica, and no biosketch is provided for White. Edwards (no biosketch provided) will also assist in the data management and analysis. Tift (no biosketch provided) will contribute to the generation of the questionnaire and reduction of the data from this and the physiological tests. Two pulmonary function technologists, Freeland and Spainhour, (no biosketches provided) will assist with the data collection phases. Fedan has prior experience with statistical analyses in the context of prior diacetyl studies. In addition, several consultants who have data of importance to the study (Lockey and McKay from U. Cincinnati) and Materna (California Department of Public Health) will participate. In addition, a pulmonologist with expertise in the diacetyl and other flavoring-induced lung dysfunction (Enright), and a radiologist with expertise in high resolution computed tomography (HRCT) interpretation (Tallaksen) are keys to this study. Finally, Hobbs, an experienced statistician, will provide support.

Weaknesses: The percent effort for many of the personnel listed is not provided, so it is difficult to determine whether their contributions are sufficient (or excessive). In addition, no biosketches are provided for several personnel listed in the budget justification, although the roles appear to be relatively minor.

Innovation Score: 4

Strengths: The proposal includes the unusual approach of retesting cohorts previously analyzed, which permits a considerably longer timeframe than is usually possible under conventional funding mechanisms. The protocol also includes some (relatively conventional) lung function tests that have apparently not previously been applied to NIOSH field studies. While somewhat innovative, the integration of data from several previous historical data sets presents some challenges. If the applicant is successful in performing this, it could serve as a model for future retrospective studies of combined datasets.

Weaknesses: The integration of the previous datasets presents some significant challenges and, although the applicant mentions some of these challenges, the discussion of approaches for dealing with them is rather limited.

Approach Score: 5

Strengths: An important strength of the proposed study is the availability of existing longitudinal datasets from the collaborators. A second strength is the comprehensive lung function testing planned in Specific Aim 3.

Weaknesses: The first strength stated above is also a weakness in that there is little control over the quality of the data. The applicants do indicate that the data will be screened for acceptability, but it is not clear how similar the questionnaires are and how difficult it will be to integrate data collected with different questionnaires and possibly different spirometry methods. It does not appear from the descriptions of the data for Specific Aim 2 that there is really sufficient exposure data to perform more than the most basic exposure response relationships: Workers that were likely to have been exposed based on job description versus those unlikely to have been exposed. Moreover, for the “flavorings” workers, there does not seem to be any attempt to segregate or subdivide the worker population for which flavoring or flavorings they might have been exposed to. Attempts to contact and recruit workers previously tested between 2000 and 2003 may be quite challenging, even in the modern age. The applicants indicate that contact information includes information for friends and relatives, and that the Missouri population is generally stable. However, there does not seem to be a sufficient plan for inclusion of people who might have moved substantial distances from the worksite.

Impact Score: 3

Strengths: The study addresses a potentially important and clearly emerging occupational health issue. Support for the hypothesis that restrictive lung diseases with rather rapid progression are common among workers in this industry could well lead to regulations that would protect these workers.

Weaknesses: It is unclear how large the population of exposed workers might be.

Environment Score: 3

Strengths: The capabilities and equipment for the NIOSH respiratory testing center are impressive.

Weaknesses: Most of the actual testing (except the HRCTs) will be done at hospitals (Jasper County Health Department) or at “union facilities, a local health department, or in rented hotel conference space.” It is not clear whether the mobile testing laboratory would be used for these analyses, which would ensure a more homogeneous dataset. It is not clear whether the travel funds requested (including a truck driver) imply that this laboratory will indeed be used or if only the administration of questionnaires is planned for these “medical survey” trips, in which case the budget to include a truck driver is puzzling. If the mobile laboratory will be used, there is some concern that the time allotted for these trips may be insufficient for the number of subjects to be tested.

Overall Evaluation

This proposal describes proposed studies to further investigate the effects of exposure to flavoring vapors (e.g., diacetyl and others) in the context of lung function testing for restrictive lung disease.

Strengths: The major strength of this proposal is that it addresses a somewhat broader range of flavoring-induced diseases and restrictive rather than obstructive diseases. The key personnel have, collectively, the breadth and depth of experience to carry out these studies.

Weaknesses: The most critical weakness is the reliance on historical data from multiple sites. There are concerns regarding the integration of the data from these disparate sources, particularly for the exposure data.

Resubmitted Applications

N/A

Other Considerations

There is inadequate documentation of the procedures to protect the subject confidentiality. No children will be included, which is appropriate since this is a study of occupational exposures. Women and minorities are appropriately included.

Critique 3:

Significance Score: 2

Strengths:

• This project is addressing an important problem in food flavoring workplaces. Exposure to flavorings has been related to obstructive lung disease, making it a serious and irreversible workplace health issue. The PO states that it is of high public interest to understand if exposure to food flavoring leads to restrictive lung disease. Findings from this proposed study would add valuable knowledge to the understanding of lung diseases associated with exposure to food flavoring.

• This study is significant because restrictive lung disease occurs much earlier than obstructive lung disease and, in some instances, only 3 years after working in such an industry. Preventive measures or changes in work placement could be done early to avoid further development of obstructive lung disease. By doing so, emerging lung disease hazard in flavoring-exposed workers can be corrected. The addition of the followup fieldwork to conduct more specific spirometry lung disease tests for scarring and inflammatory lung disease is another significance of this study.

• The need for this study is supported by the demand from physicians, workers, employers, and regulators to have a comprehensive scope of all lung diseases related to flavoring exposure. Past research has found that workers in microwave popcorn and flavoring manufacturing plants had a high risk of developing obstructive lung disease and that these health effects develop in an exposure-dependent way. In early work by NIOSH, flavoring-exposed workers showed manifestations of both restrictive and obstructive lung disease but were classified as having obstruction. Further investigations being done in other plants uncovered spirometric restriction and the development of restrictive lung disease at the followups among flavoring-exposed workers. In addition, physicians have brought forth several case reports of workers with restrictive lung disease. Based on these findings, researchers have reason to believe that flavoring exposure could mediate the development of restrictive lung disease.

• A NIOSH Criteria Document for Recommended Standard for Diacetyl and 2,3-Pentanedione is currently undergoing review and has the potential to influence OSHA regulations for reducing obstructive lung disease. If the aims of this proposal are met, then similar regulations could come about to also ensure the reduction of restrictive lung diseases.

Weaknesses: Obstructive lung disease occurs in other conditions such as asthma, chronic bronchitis, and emphysema. Further medical testing may be required to distinguish between these obstructive lung diseases.

Project Officers and Key Personnel Score: 2

This is a multidisciplinary project in structure, with internal and external collaborators from NIOSH, California Department of Public Health, and the University of Cincinnati. The proposal requires combining the data on microwave popcorn and flavor manufacturing workers from NIOSH, California Department of Public Health, and the University of Cincinnati to investigate the occurrence of restrictive lung disease. The PO is the Branch Research Team Supervisor and has extensive experience with multidisciplinary and collaborative research. Collectively, the assembled research team demonstrates the necessary qualifications for performing and achieving the intended aims for the purposed study. Inclusion of Dr. Paul Enright, a pulmonologist, adds strength to this research team. His expertise in spirometry is important for ensuring the quality of the spirometry measures, which are susceptible to controversies with respect to utility, methods, interpretation, sensitivity, and specificity.

Weaknesses: No weaknesses were identified.

Innovation Score: 3

Currently, research on flavoring-related lung disease has focused on obstructive lung disease, specifically those with bronchiolitis obliterations. Evidence is mounting that suggests that the spectrum of lung disease in food production and flavoring manufacturing is broad and also includes restrictive lung diseases. This proposal will use diagnostic methods for further assessing restrictive lung disease in a field setting, which is unique to this study. By establishing an association between restrictive lung disease and flavoring exposure, regulations can be made to prevent these health indices in flavor manufacturing workers. Findings from this project would help understand the best preventive efforts for mitigating the development of restrictive lung diseases, such as lowering flavoring exposure, work practices, and personal protective equipment. This research will apply diagnostic methods for restrictive lung disease in field settings that have rarely been used, including the gas transport assessment, lung volume through nitrogen washout, and low dose HRCT. Nitrogen washout and HRCT have never been used by NIOSH’s field investigations. These tests will distinguish between lung diseases due to fibrotic or inflammatory nature and those from spirometric restrictions. In addition, this study will establish the presence of an emerging restrictive lung disease in flavoring production industries in which obstructive lung disease is already an identified risk.

Strengths: This is the first proposal to add additional testing of participants in hazard health evaluation after completion of such investigations by NIOSH.

Weaknesses: There may be difficulty in obtaining approval from the Office of Management and Budget to trace participants at offsite workplaces for the proposed field testing. Some partnership has been already established with sites such as Jasper County Health Department for testing former workers.

Approach Score: 3

To meet Specific Aims 1 and 2, data from NIOSH, the California Department of Public Health, and the University of Cincinnati will be analyzed. To address Specific Aim 3, investigators at NIOSH will collect additional medical test data from a subset of participants from previous studies at the Missouri popcorn factory and Indiana flavorings plant. Medical test data will be collected offsite; however, the PO notes that, in the past, the Indiana flavoring company was adverse to a NIOSH followup visit after a litigated health hazard evaluation. This could deter employees from partaking in this proposed project and potentially harm participation rates. Although personal identifying information and names will not be disclosed, employees may still be leery about participating and concerned that this medical information may get out and affect their employment. The PO provides a brief description of all the partners’ roles throughout the project.

Strengths:

• In this proposed work, the investigators seek to characterize the presence of restrictive lung disease in a food production and a flavoring manufacturing plant where they have already established a high prevalence of obstructive lung disease by using additional diagnostic testing not currently being used.

• Participants will be reimbursed for their time and travel expenses with a $75 gift card.

• Based on the power analysis provided for each specific aim, it appears that the proposed sample could provide enough power for significance to be identified.

Weaknesses:

• To meet Specific Aim 1 and 2, databases from NIOSH studies, the California Department of Public Health surveillance study on flavoring manufacturing workers, and the University of Cincinnati studies on microwave popcorn workers will be combined and will need to be harmonized and analyzed across the studies. This is not specifically a weakness, but a challenging adventure.

• Job exposure matrices have been established for four of the six NIOSH-investigated microwave popcorn plants. However, there are limited data on exposure assessment for the flavoring manufacturing companies, and they do not have the ability to create cumulative diacetyl exposures for individuals or quantitative matrices of exposure for other types of flavoring chemicals. It may thus be difficult to combine data for the microwave popcorn plants and flavoring manufacturing companies or examine across these workplaces.

• The quality of the spirometry can vary across individual databases and among the three different databases due to lack of tester-tester reliability.

• This proposal relies on recruitment of participants from a previously investigated Indiana flavoring plant and a Missouri microwave popcorn plant. From the 112 Indiana flavoring plant workers, only 30 tested had restriction, 1 had mixed abnormalities, and 3 had obstruction as their last test. There is also the possibility that all of these participants will not choose to partake in this study, which would affect the sample size.

• The proposal acknowledges that spirometry is subject to limitations. An accurate spirometric result involves completion of exhalation before ending the test and thus, could falsely indicate restriction if the participant does not complete a full exhalation.

• This proposal will pioneer the additional testing of participants in health hazard evaluations after an initial public health investigation; however, it requires tracing participants to obtain contact information. The PO implies that there may be challenges in obtaining and tracing current residence and contact information to invite participants to an off-worksite location for the additional testing.

Impact Score: 3

An immediate impact for participants of this study is that participants will receive followup results on their medical test and proper advisement for necessary diagnostic followup. Findings from this proposal could support the development of new preventive measures for curtailing workers flavoring exposure. New knowledge gained from this study would help physicians make more accurate recommendations for secondary prevention measures for affected workers. Findings from this study could also improve regulatory processes to better protect workers and limit flavoring exposure and would require employers to take all necessary precautions to protect the health of their employees. The stakeholders are physicians, workers, employers, and regulators.

Strengths: Achieving the proposal's three aims will lay the foundation for development of preventive measures in occupational health practices and in flavoring regulations to reduce the risk of restrictive abnormalities among flavoring-exposed workers.

Weaknesses: No weaknesses were identified.

Environment Score: 1

Strengths: The research environment proposed for this study is well equipped for the successful completion of the proposed work.

Weaknesses: No weaknesses were identified.

Overall Evaluation

This is a multidisciplinary proposal requiring collaboration from NIOSH, the California Department of Public Health, and the University of Cincinnati. This proposal has three specific aims: Specific Aim 1: Determine whether excess spirometric restriction exists among flavoring-exposed workers compared to the U.S. population. Specific Aim 2: Investigate the exposure-response relationship that would support flavoring exposure as an etiologic factor for restrictive lung disease. Specific Aim 3: Characterize the nature of restrictive lung disease. The investigators seek to provide new measures for establishing a cause and effect relationship between food flavoring exposure and restrictive lung disease. Furthermore, this proposed work will demonstrate a temporal relationship between workplace exposures and the deterioration of lung function and the presence of an exposure-response relationship. The study will provide beneficial information to many stakeholders—including physicians who manage work-related lung disease—for handling the level of workers’ exposure, reassignment, and workers’ compensation if warranted. Other stakeholders could be companies to set up data-driven priorities for work-related preventive strategies and workers for making decisions regarding either continuation of their jobs or changing their jobs. The investigators have access to large datasets including questionnaire and spirometry data previously collected by NIOSH investigators in six microwave popcorn plants and three flavoring plants in both cross-sectional and longitudinal studies (almost 3 years of data). In addition, previous participants from one of the flavoring manufacturing plants and one of the microwave popcorn plants will also be asked to partake in further lung function testing and CT lung scans.

Strengths:

• Specific Aims 1 and 2 are both exploratory in nature, and these aims will develop background information regarding spirometric restriction among flavoring-exposed workers in comparison to the U.S. population and the exposure-response relationships that would support flavoring exposure as an etiology for restrictive lung disease.

• If successful, the project will support regulations that address health and safety hazards among such populations and would also have applications beyond these workplaces being investigated.

Weaknesses: One concern is that this is a very large and comprehensive multidisciplinary project between different groups and geographical locations. Managing such projects usually requires time, management skills, and efficient use of resources. The investigators propose a plan as to how to manage the project and appear to have experience doing these types of activities.

Resubmitted Applications

Not applicable.

Other Considerations

None

r2p Review

Overall Rating: Accepted as is; no revisions required

Reviewer Comments

No changes necessary. Great use of all the r2p elements in the proposal.