ADSOManual

ADSOManual.pdf

Deepwater Horizon Response Worker Rostering Survey and Worker Health Survey

ADSOManual

OMB: 0920-0857

Document [pdf]

Download: pdf | pdf

Policy and Procedures for
ADSO* Review of NIOSH Publications
August 2009

*Office of the Associate Director for Science

Department of Health and Human Services
Centers for Disease Control and Prevention
National Institute for Occupational Safety and Health

Disclaimer
This document is for use by the National Institute for Occupational Safety and Health (NIOSH), including
all of its Divisions, Laboratories, and Offices. This document is not intended to and does not create any
right or benefit, substantive or procedural, enforceable at law or in equity, against the United States, its
agencies or other entities, its officers, employees, or any other person.

Acronyms and Abbreviations
ADS
ADSO
CDC
DART
DLO
EID
FACE
FRN
GGP
HHE
HHS
HISA
ISI
MMWR
NIOSH
NORA
OD
OHC
OMB
PEERA
REL
TCM

Associate Director for Science
Office of the Associate Director for Science
Centers for Disease Control and Prevention
Division of Applied Research and Technology
Division, Laboratory, and Office
Education and Information Division
Fatality Assessment and Control Evaluation
Federal Register Notice
Good Guidance Practices
Health Hazard Evaluation
Department of Health and Human Services
Highly Influential Scientific Assessment
Influential Scientific Information
Morbidity and Mortality Weekly Report
National Institute for Occupational Safety and Health
National Occupational Research Agenda
Office of the Director
Office of Health Communications
Office of Management and Budget
Peer Review Agenda
Recommended Exposure Limit
Topic Concept Memo

Table of Contents
Disclaimer

Acronyms and Abbreviations

Table of Contents

iii

I. Introduction

Purpose
Audience
Role of the Office of the Associate Director for
Science
Content Overview

1
1
1

II. Publications Reviewed by the Office
of the Director

Publications Exempted From OD Review and
Clearance
Publications Requiring OD Review
No Change in Policy for Publications Reviewed
by DLOs

III. Planning for Peer, Stakeholder, and
Public Review

Initial Review Plan
Internal Peer Reviewers
External Peer and Stakeholder Reviewers
External Peer Review Process
Stakeholder Peer Review Process

4
4
4
4
5

IV. Internal and External Review

NIOSH OD Review Requirements At-A-Glance

A. Clearance for External Review Not Required
by NIOSH OD

B. External Review Required of Scientific
Information

C. External Review Required of Influential
Scientific Information

D. External Review With Public Comment
Before Peer Review – Option 1

E. External Review With Public Comment After
Peer Review – Option 2

List of Appendices

Appendix A. Director’s Memorandum on
NIOSH Scientific Information Quality
Review

2
2

iii

Appendix B. Classification of NIOSH
Scientific Information Products by
Applicability of Peer Review Bulletin and
Good Guidance Practices Bulletin

Appendix C. Summary of Review Policy
for Publication of Influential Scientific
Information and Highly Influential Scientific
Assessments

Appendix D. Summary of Review Policy
for Significant Guidance Publications

Appendix E. Advantages and
Disadvantages of the Relative Timing of
External Review and Public Comment

Appendix F. ADSO Review Worksheet

Appendix G. Considerations for Writing a
NIOSH Publication

Appendix H. Decision Logic for
Determining the Applicability of Peer
Review or Good Guidance Practices
Bulletin and Use of Disclaimers

Appendix I. Charge to Reviewers

Appendix J. Summary Flowchart for
Public Review of Draft Significant
Guidance Publications

Appendix K. Process Flowchart for
Advance Notification of HHS for Significant
Guidance Publications

Appendix L. HHS Advance Notification
Template and Example for Significant
Guidance Publications

Appendix M. Summary Flowchart for
Electronic Docket and FRN

Appendix N. FRN Example for Significant
Guidance Publications

I. Introduction
reviews, an instrumental part of achieving a
larger NIOSH goal of scientific quality.

Purpose
1

This document provides guidance on policies
and procedures for scientific review of NIOSH
publications by the Office of the Associate
Director for Science (ADSO), Office of the
Director (OD), NIOSH.
The terms NIOSH
document, NIOSH publication, and NIOSH
scientific information product are used
interchangeably in this document to refer to any
scientific information product, regardless of
format, that is officially distributed by NIOSH and
is not exempt from OD review.

Content
This document includes information on the types
of publications the OD reviews and the criteria
and process used for the ADSO review during
document development. The content of this
document is organized into the following
sections:

Audience

Section II: Publications Reviewed by the
Office of the Director. This section provides
information on which publications require
and which are exempted from OD review.

This document is intended to be a resource
primarily for authors and others in the NIOSH
Divisions, Laboratories, and Offices (DLOs) who
are developing publications that require review
by the ADSO. It also may be useful as a best
practices
resource
for
NIOSH
authors
developing other scientific information products.

Section III: Planning for Peer, Stakeholder,
and Public Review. This section provides
information
to
assist
authors
with
anticipating
review
issues
affecting
publication development, such as required
levels of peer, stakeholder, and public
review.

Role of the Office of the Associate Director
2
for Science

Section IV: Internal and External Review of
NIOSH Publications. This section explains
what to expect during internal and external
review of NIOSH publications, documenting
responses to reviewer comments, and
making revisions to address reviewer
comments.

Procedures to review the quality, objectivity,
utility, and integrity of scientific information
disseminated by NIOSH are the responsibility of
the Director, NIOSH, and have been delegated
to the ADSO.
When reviewing NIOSH publications, the
Associate Director for Science (ADS) is
responsible for ensuring they meet scientific
information quality standards as required by
the Office of Management and Budget (OMB),
the Department of Health and Human Services
(HHS), the Centers for Disease Control and
Prevention (CDC), and NIOSH.
Senior
Scientists in the ADSO are charged with
conducting these scientific information quality

Supersedes and merges Policy and Procedures for
Document Review by the Office of the Director and NIOSH
Good
Guidance
Practices:
Interim
Supplemental
Procedures for Document Review by the Office of the
Director.
2
Director’s Memorandum on NIOSH Scientific Information
Quality Review, March 18, 2005 (see Appendix A).

II. Publications Reviewed by the Office of the Director
Publications Exempted From OD Review and
Clearance

All other scientific information products must be
submitted for NIOSH OD review and clearance.
All publications sent to the OD for review and
clearance should be submitted through
Documentum
(see
Scientific
Information
Products).

The OD reviews all scientific information
products by NIOSH authors, except those on the
following list:

Publications Requiring OD Review

1. Fire fighter fatality investigation reports.
2. Fatality Assessment and Control Evaluation
(FACE) investigation reports.

Examples of NIOSH publications that require
review and clearance by the OD include:

3. Individual worker notification products for
epidemiologic studies.

Any product that recommends a new
Recommended Exposure Limit (REL)
Criteria Documents
Alerts
Current Intelligence Bulletins
Manual of Analytical Methods
Pocket Guide to Chemical Hazards
Bibliographies, Anthologies, and Research
Compendia
Conference Proceedings (entire volume)
Fact Sheets, Brochures, Pamphlets, and
Newsletters
Mining Information Circulars
Morbidity and Mortality Weekly Report
(MMWR) articles
National Occupational Research Agenda
(NORA) Reports
Mining Reports of Investigation
Statistical Reports (e.g. surveillance)
Technical Reports
Training Materials
Web Pages
Workplace Solutions

4. Health Hazard Evaluation (HHE) reports (42
CFR Part 85) and site visit reports (42 CFR
Part 85A), unless they contain policy
positions or raise national media interest.
5. Division
of
Applied
Research
and
Technology (DART) engineering survey
reports, unless they contain policy positions
or raise national media interest.
6. Journal articles, book chapters, articles in
conference proceedings, unless they contain
policy positions, raise national media
interest, are authored by a DLO Director, or
raise concerns of dual use research (note
that journal articles are subject to DLO
3,4
approval processes) .
7. Individual presentations, (e.g. speeches,
PowerPoint, or talking points) and abstracts
unless they involve terrorism related topics,
contain policy positions, raise national media
interest, or raise concerns of dual use
research (note that presentations and
abstracts are subject to DLO approval
3,4
processes) .

No Change in Policy for Publications
Reviewed by DLOs
A large number of publications, such as most
journal articles, book chapters, articles in
conference proceedings, presentations, and
abstracts, currently are reviewed and cleared at
the DLO level.
DLOs have the primary
responsibility to conduct scientific review of
publications exempted from OD review,
according to the list above, to ensure scientific
quality. DLOs continue to be responsible for
determining whether publications in these
categories require interagency cross-clearance
or cross-clearance from other Coordinating

Disseminating scientific information that is not an official
agency position, sharing draft scientific information
confidentially with colleagues, or disseminating scientific
information for the purposes of peer review may require the
use of a disclaimer (see NIOSH Policy on the Use of
Disclaimers for Scientific Information Products).
4
The ADSO coordinates the review of dual use research for
NIOSH, in order to meet CDC requirements (see Oversight
and Clearance of Dual Use Research of Concern).

Centers, Coordinating Offices, or National
Centers within CDC. There also is no change in
DLO responsibility to determine whether
publications require OD review and clearance
because they contain policy positions or
sensitive issues such as those that generate
Congressional and media interest or raise
concerns about dual use research.

III. Planning for Peer, Stakeholder, and Public Review
The terms peer review, stakeholder review, and
public review refer to distinct processes that
serve different purposes.
Peer reviewers assess the scientific and
technical quality of
publications, make
recommendations for improvement, or identify
areas where scientific uncertainties should be
addressed more fully. Choose peer reviewers
for their technical and scientific expertise,
independence, and freedom from conflicts of
interest.
Stakeholder reviewers provide crucial feedback
on how the document will be accepted by the
parties impacted by its publication. Choose
stakeholder reviewers for their vested interest in
the publication as NIOSH customers, including
employers, employees, and others such as
product manufacturers or impacted government
agencies. Stakeholder reviewers may or may
not be technical experts.
Peer review frequently is conducted in parallel
with stakeholder review. In some cases, it may
be desirable to conduct peer review prior to
stakeholder review so that stakeholders receive
the most accurate technical information possible.
Public review allows an opportunity for the
general public to provide comments and
suggestions on NIOSH publications of interest to
them. This process often captures additional
stakeholder
viewpoints
and
ensures
transparency in the development of NIOSH
publications.
Initial Review Plan
Prior to initiating publication development, a
Topic Concept Memo (TCM) must be written
and approved by the NIOSH Leadership Team
(see Policy and Guidance for Developing the
Topic Concept Memo). The TCM provides an
opportunity for authors to develop an initial plan
for peer, stakeholder, and public review of their
publication. The TCM requires a brief description
of the anticipated internal and external review
process and the names and affiliations of
proposed peer and stakeholder reviewers. If
specific individuals cannot be named, list the
organizations from which peer and stakeholder

reviewers will be drawn (e.g. academia,
employer groups, or labor organizations). The
plan may be updated when the publication is
submitted for OD review prior to external review.
Internal Peer Reviewers
Identify NIOSH staff knowledgeable about the
subject to review the draft publication prior to
first submission to the OD for ADSO review.
Internal reviewers assess technical accuracy,
content format, comprehensibility, completeness
of references, and scientific interpretation of
data. Policies for internal review are established
by each DLO. The ADSO relies on comments
by internal reviewers, and the responses to
those comments by the authors, to certify the
scientific and technical quality of the publication.
External Peer and Stakeholder Reviewers
Choose external peer and stakeholder reviewers
according to the level of sensitivity and
complexity of the publication. Anyone outside of
NIOSH is considered an external reviewer,
including staff from another CDC center or office.
External peer review and stakeholder review are
required for most publications reviewed by the
NIOSH OD. The ADSO relies on comments by
external reviewers and the authors’ responses to
those comments to certify the scientific and
technical quality of the publication, as well as the
soundness of any scientific policy statements.
An exception to external peer review may be
made if the scientific information and
conclusions have been peer reviewed previously
(for an example, see Clearance of Workplace
Solutions Documents).
In this case the
publication only requires stakeholder review to
ensure that the material is appropriate for its
intended audience.
External Peer Review Process
Determine the appropriate level of peer review
based on the complexity and sensitivity of the
science. Publications undergo more extensive
review if they are complex, include novel or
controversial methods or interpretations, or are
likely to have a significant impact on
stakeholders. The OMB Peer Review Bulletin

and Good Guidance Practices Bulletin contain
specific requirements for external peer review.

are NIOSH contractors or recipients of
funding through cooperative agreement
mechanisms. NIOSH grantees, who receive
funding through a competitive extramural
process, are not precluded from being
reviewers and often are excellent choices.
Conflict of Interest: Avoid reviewers who
have financial or other conflicts that might
affect their judgment. Avoid both real and
perceived conflict of interest.

All NIOSH publications classified as Influential
Scientific Information (ISI) must meet OMB
requirements described in the Final Information
Quality Bulletin for Peer Review or Peer Review
Bulletin (see page 11 and Appendices B and C).
There are additional public participation
requirements for documents classified as Highly
Influential Scientific Assessments (HISAs) (see
pages 13-18 and Appendix B and C). During
review, the ADSO will assess whether NIOSH
publications classified as ISI or HISA meet the
requirements of the Peer Review Bulletin.

Stakeholder Review Process
Determine the appropriate level of stakeholder
review by the likely influence of the publication
on workers and industry.
A minimum
requirement is one stakeholder representing
workers and one representing industry. For
HISAs or other publications that are especially
important or controversial, include more
stakeholder reviewers and plan to receive public
comments in an open meeting and through
posting of a draft publication on the NIOSH
Internet website.

Peer Reviewer Criteria
Select peer reviewers for their scientific
expertise but give additional consideration to
conflicts of interest and impartiality.
At a
minimum, two peer reviewers are used.
Additional peer reviewers are selected for HISAs
and other complex or high-profile publications.
The National Academies criteria for selection of
committee members provide a useful guide for
selecting external peer reviewers
(www.nationalacademies.org/coi/index.html).

Public participation is part of the external review
procedures. It also satisfies the public comment
requirement for publications that are revised in
response to external review and contain
significant guidance as defined by the Final
Bulletin for Agency Good Guidance Practices, or
Good Guidance Practices Bulletin (see pages
13-18 and Appendices B and D). During review,
the ADSO will assess whether NIOSH
publications classified as containing significant
guidance meet the requirements of the Good
Guidance Practices Bulletin.

Expertise: Choose peer reviewers to ensure
that the full range of scientific disciplines
related to the publication is represented.
For publications containing potentially
controversial scientific approaches, multiple
reviewers are selected with expertise in the
controversial
areas.
For
publications
containing detailed statistical analyses, at
least one reviewer is selected with expertise
in statistical methodology.
Balance: Select reviewers to represent a
balanced range of scientific opinions. This is
especially
important
for
publications
containing potentially controversial scientific
conclusions.
Independence: Select reviewers with no
prior involvement with the publication being
reviewed who are outside of CDC and HHS,
if possible. Avoid reviewers who currently

Stakeholder Reviewer Criteria
Choose stakeholder reviewers to represent the
range of individuals or organizations affected by
the publication. Stakeholder reviewers are
always external to NIOSH and, if possible,
include representatives of affected industries,
impacted workers, interested government
agencies, and other end-users.

IV. Internal and External Review

NIOSH publications, along with supporting
materials, are reviewed by the ADSO
subsequent to submission to the Documentum
electronic content management system. One to
three OD submissions for ADSO review may be
required to meet all scientific, policy, and
administrative review requirements, and to
provide opportunities for public comment.
MMWR articles, journal articles needing policy
review, journal articles needing cross-clearance,
and miscellaneous reports, pamphlets, and fact
sheets may require only one submission to the
OD for ADSO review. Criteria Documents,
Current Intelligence Bulletins, Alerts, and other
major NIOSH publications require multiple
submissions.

The ADSO also coordinates review of materials
classified as Dual Use Research of Concern.
Depending on the publication type, materials
containing dual use research may require
multiple levels of review to resolve issues of
concern.
The ADSO review sequence usually is the same
for each submission:
1. The ADS or Deputy ADS accepts the
submission and assigns the document to a
Senior Scientist.
2. The Senior Scientist performs a review for
science quality, policy, and communication
according to the ADSO Review Worksheet
(see Appendices F and G). The Senior
Scientist then recommends approval, with or
without
changes,
or
revision
and
resubmission.
3. The Senior Scientist may work directly with
the author or lead DLO on revisions.
4. Following Senior Scientist review, the ADS
or Deputy ADS provides a similar review,
highlighting points raised by the Senior
Scientist, and makes the final ADSO
recommendation.
5. The
ADS
may
provide
additional
recommendations for NIOSH publications
that contain complex or controversial
scientific policy statements or highly
influential scientific information.

Review requirements and review process for
typical publications are outlined in Table 1. As
shown in Columns D and E of Table 1, there are
two options for timing the solicitation of public
comment for documents requiring it, either
before or after external peer and stakeholder
review. See Appendix E for advantages and
disadvantages of each option. Subsections A
through E of this Section IV (pages 8-18)
describe the review sequence in detail for each
document type.
The ADSO coordinates cross-clearance within
CDC (see Clearance of Information Products
Disseminated Outside CDC for Public Use) and
cross-clearance with other federal agencies.

Table 1: NIOSH OD Review Requirements At-A-Glance
1

Document Examples

Comments

Clearance for External
1
Review Not Required
by NIOSH OD
A

Clearance for External Review Required by NIOSH OD
No Public Comment

DLOs may require external
review even when the OD does
not assess it.

Announcements
Fact sheets
Journal articles needing
policy review and/or cross
clearance
Manuscripts
MMWR
NORA Council Reports
Novelty Items
Pocket Guide Updates
Pamphlets
Posters
Postcards
Slide presentations
Testimony
Web pages

C
Covered by OMB
Peer Review Bulletin

Revisions to
NMAM methods
Statistical
Reports
Technical
Reports
Workplace
Solutions

Public Comment

New NMAM
methods
Others on case by
case basis

Covered by OMB Peer
Review Bulletin & Good
Guidance Practices
Bulletin
Public Comment Before
Peer / Stakeholder
Review (Option 1)2

Covered by OMB Peer
Review Bulletin & Good
Guidance Practices
Bulletin
Public Comment After
Peer / Stakeholder
Review (Option 2)2

Alerts
Criteria Documents
Current Intelligence
Bulletins
Publications with a
new REL

Alerts
Criteria Documents
Current Intelligence
Bulletins
Publications with a new
REL

Others on case by case
basis

Others on case by
case basis

Others on case by case basis
Initial Submission

Initial Submission

ADSO Review

Cross-Clearance, if required

Cross-Clearance, if
required

OD Clearance

Peer / Stakeholder
Review

Public Comment / Public
Meeting

Peer / Stakeholder
Review

ADSO Review

Peer / Stakeholder
Review

ADSO Review

OD Clearance

Review Process

OD Clearance
External Review conducted
outside of NIOSH, if required
Publish

OD Clearance
ADSO Review

Publish

Public Comment
OD Clearance
ADSO Review
Publish
OD Clearance
Publish

See Subsection A
See Subsection B
See Subsection C
See Subsection D
See Subsection E
1. External review may consist of any combination of peer review, stakeholder review, and/or public comment
2. There are two options for timing the solicitation of public comment, either before or after external peer and stakeholder review.
See Appendix G for advantages and disadvantages of each option.

A. Clearance for External Review is Not Required by NIOSH OD
This category of NIOSH publications generally
includes:
MMWR articles
Journal articles needing policy review
Journal articles needing cross-clearance
NORA Council Reports
Pamphlets and fact sheets
Other publications on a case-by-case
basis

Submit publication to the OD using
Documentum Transmittal Form A

Note that DLOs may require external review for
certain documents even when the OD does not
require it.

Completion of ADSO review

Submission Procedure

Publication proceeds through
NIOSH OD final clearance

Submit Publication to the OD Using
Documentum Transmittal Form A. Include:
Cover memo from the lead author’s DLO
Director describing the purpose and type of
review (e.g. policy, cross-clearance) and
highlighting potential areas of concern.
Documentation of internal reviewer comments
and how they were addressed.
Other items listed in Transmittal Form A.

2. Completion of ADSO Review
After review of the materials and resolution of
all science-related issues, the ADSO review is
complete and the document proceeds through
the OD for final clearance for publication.

B. External Review Required of Scientific Information
This category of NIOSH publications
generally includes:

Submit publication to the OD
using Documentum Transmittal
Form B, Checklist 1

Scientific Information, defined by the Peer
Review Bulletin as any communication or
representation of scientific knowledge
containing factual inputs, data, models,
analyses, technical information, or scientific
assessments based on physical, biological,
behavioral, or social sciences.

Completion of ADSO review for
external peer/stakeholder review

This category does not include:
Influential Scientific Information (ISI). As
defined by the Peer Review Bulletin, ISI is
scientific information that NIOSH determines
will have, or does have, a clear and
substantial impact on important public
policies or private sector decisions (see
Appendix C).
Highly Influential Scientific Assessments
(HISAs). As defined by the Peer Review
Bulletin, a ”Scientific Assessment” is an
evaluation of a body of scientific or technical
knowledge which typically synthesizes
multiple factual inputs, data, models,
assumptions,
and/or
applies
best
professional
judgment
to
bridge
uncertainties in the available information.
HISA applies to scientific assessments that
NIOSH determines: (i) could have a
potential impact of more than $500 million in
any year, or (ii) is novel, controversial, or
precedent-setting
or
has
significant
interagency interest (see Appendix C).
Significant Guidance. NIOSH publications
contain significant guidance if they
recommend new guidelines or include new
or revised Recommended Exposure Limits
(see Appendix D and the Good Guidance
Practices Bulletin for details).

Publication proceeds through NIOSH
OD clearance for external peer
review

Conduct external
peer/stakeholder review

Revise publication after
external peer/stakeholder review

Submit revised publication after
external peer/stakeholder review
using Documentum Form B,
Checklist 2

Refer to Appendix H for a decision flowchart on
the applicability of the OMB Bulletins on Peer
Review and Good Guidance Practices.

Publication proceeds through the
NIOSH OD for final clearance

Submission Procedure
1. Submit Document to the OD Prior to
External Review Using Documentum
Transmittal Form B, Checklist 1, and
Include:

Completion of ADSO review
Publication proceeds through NIOSH
OD final clearance

Draft publication.
Cover memo from the lead author’s DLO
Director stating:
o
Purpose and type of review (e.g. policy,
cross-clearance).
o
Certification that the publication is not
ISI, HISA, or significant guidance
o
Description of how the external review
will be conducted (e.g. by mail or public
meeting).
o
Statement whether any additional
background
information
will
be
provided to reviewers.
o
Description of how reviewer comments
will be handled (e.g. whether NIOSH
will publicly disseminate a response to
reviewer comments).
o
Potential areas of concern.
Documentation
of
internal
reviewer
comments and how they were addressed.
Charge to reviewers (see Appendix I).
Other items listed in Transmittal Form B,
Checklist 1.

Revised publication.
Cover memo from the lead author’s DLO
Director
certifying
the
review
and
accommodation
of
comments
and
highlighting potential areas of concern.
For internal use, a comprehensive listing of
all external comments and how they were
addressed in a point-by-point format.
Reviewer
names
and
organizational
affiliations are associated with specific
comments.
For release to the public upon request (i.e.,
typically not posted with unlimited access on
the NIOSH Internet website):
o A detailed response to reviewer
comments (similar to the internal
response), or a synopsis of reviewer
comments with highlights of revisions
made to the document
o Peer reviewer names and affiliations
which are not associated with specific
comments
o Stakeholder names and affiliations
associated with specific comments
Other items listed in Transmittal Form B,
Checklist 2.

2. Obtain OD Clearance for External Review
3. Conduct External Review

6. Completion of ADSO Review
Forward the publication and supporting
materials to external reviewers following OD
clearance for external review.
For
relatively
straightforward
and
uncontroversial publications, review by
regular or electronic mail usually is sufficient.

After review of the materials and resolution
of all science-related issues, the ADSO
review is complete and the document
proceeds through the OD for final clearance
for publication.

4. Revise Publication after External Review
Incorporate
relevant
comments
and
suggestions from external reviewers into the
document.
o Make a good faith effort to incorporate
comments whenever reasonable and
feasible.
o The manner in which the comments are
incorporated is at the discretion of
NIOSH.
o All comments are addressed either
through incorporation or by documenting
reasons
why
they
cannot
be
accommodated.
5.

Submit Revised Publication after
External Review Using Documentum
Transmittal Form B, Checklist 2, and
Include:

C. External Review Required of Influential Scientific Information
This category of
generally includes:

NIOSH

publications

Submit publication to the OD using
Documentum Transmittal Form C,
Checklist 1

Completion of ADSO review for
external peer/stakeholder review

This category does not include:
Publication proceeds through NIOSH
OD clearance for external peer review

Highly Influential Scientific Assessments
(HISAs). As defined by the Peer Review
Bulletin, a ”Scientific Assessment” is an
evaluation of a body of scientific or technical
knowledge which typically synthesizes
multiple factual inputs, data, models,
assumptions,
and/or
applies
best
professional
judgment
to
bridge
uncertainties in the available information.
HISA applies to scientific assessments that
NIOSH determines: (i) could have a
potential impact of more than $500 million in
any year, or (ii) is novel, controversial, or
precedent-setting
or
has
significant
interagency interest (see Appendix C).
Significant Guidance. NIOSH publications
contain significant guidance if they
recommend new guidelines or include new
or revised Recommended Exposure Limits
(see Appendix D and the Good Guidance
Practices Bulletin for details).

Conduct external peer/stakeholder
review

Revise publication after
external peer/stakeholder review

Submit revised publication after
external peer/stakeholder review
using Documentum Transmittal
Form C, Checklist 2

Refer to Appendix H for a decision flowchart on
the applicability of the OMB Bulletins on Peer
Review and Good Guidance Practices.

Publication proceeds through the
NIOSH OD for final clearance

Submission Procedure
1. Submit Publication to the OD Prior to
External Review using the Documentum
Transmittal Form C, Checklist 1 and
include:

Completion of ADSO review
Publication proceeds through NIOSH
OD final clearance

Draft publication
Cover memo from the lead author’s DLO
Director stating:
o
Purpose and type of review (e.g. policy,
cross-clearance).

Certification that the publication is ISI,
and is not HISA or significant guidance.
o
Description of how the external review
will be conducted (e.g., by mail or
public meeting).
o
Statement whether any additional
background
information
will
be
provided to reviewers.
o
Statement that NIOSH will publicly
disseminate a response to reviewer
comments.
o
Potential areas of concern.
Documentation
of
internal
reviewer
comments and how they were addressed.
Charge to reviewers (see Appendix I).
Other items listed in Documentum
Transmittal Form C, Checklist 1.

5. Submit Revised Publication after External
Review using Documentum Transmittal
Form C, Checklist 2 and Include:
Revised publication.
Cover memo from the lead author’s DLO
Director
certifying
the
review
and
accommodation
of
comments
and
highlighting potential areas of concern.
For internal use, a comprehensive listing of
all external comments and how they were
addressed in a point-by-point format.
Reviewer
names
and
organizational
affiliations are associated with specific
comments.
For release to the public upon request (i.e.,
typically not posted with unlimited access on
the NIOSH Internet website):
o A detailed response to reviewer
comments (similar to the internal
response), or a synopsis of reviewer
comments with highlights of revisions
made to the document.
o Peer reviewer names and affiliations
which are not associated with specific
comments.
o Stakeholder names and affiliations
associated with specific comments.
Other items listed in Documentum
Transmittal Form C, Checklist 2.

2. Obtain OD Clearance for External Review
3. Conduct External Review
Forward the publication and supporting
materials to external reviewers following
OD clearance for external review.
For
relatively
straightforward
and
uncontroversial publications, review by
regular or electronic mail usually is
sufficient.
4. Revise Publication after External Review
Incorporate
relevant
comments
and
suggestions from external reviewers into the
publication.
o
o

All comments are addressed either
through incorporation or by documenting
reasons
why
they
cannot
be
accommodated.

6. Completion of ADSO Review

Make a good faith effort to incorporate
comments whenever reasonable and
feasible.
The manner in which the comments are
incorporated is at the discretion of
NIOSH.

After review of the materials and resolution
of all science-related issues, the ADSO
review is complete and the document
proceeds through the OD for final clearance
for publication.

D. External Review With Public Comment Before Peer Review: Option 1
Option to Solicit Public Comment
Prior to External Peer and
Stakeholder Review

Submit publication to the OD using
Documentum Transmittal
Form D, Checklist 1

There are two options for timing solicitation of
public comment, either before (this subsection
D) or after (see subsection E) external peer and
stakeholder review. See Appendix E for
advantages and disadvantages of each option.
This category
includes:

NIOSH

Completion of ADSO review prior
to public comment and external
peer/stakeholder review

Publications

Influential Scientific Information (ISI). As
defined by the Peer Review Bulletin, ISI is
scientific information that NIOSH determines
will have, or does have, a clear and
substantial impact on important public
policies or private sector decisions (see
Appendix C).
Highly Influential Scientific Assessments
(HISAs). As defined by the Peer Review
Bulletin, a ”Scientific Assessment” is an
evaluation of a body of scientific or technical
knowledge which typically synthesizes
multiple factual inputs, data, models,
assumptions,
and/or
applies
best
professional
judgment
to
bridge
uncertainties in the available information.
HISA applies to scientific assessments that
NIOSH determines: (i) could have a
potential impact of more than $500 million in
any year, or (ii) is novel, controversial, or
precedent-setting
or
has
significant
interagency interest (see Appendix C).
Significant Guidance. NIOSH publications
contain significant guidance if they
recommend new guidelines or include new
or revised Recommended Exposure Limits
(see Appendix D and the Good Guidance
Practices Bulletin for details).

Publication proceeds through NIOSH
OD clearance for external peer review

Solicit public comment

Conduct external peer/stakeholder
review

Revise publication after
external peer/stakeholder review

Submit revised publication after
external peer/stakeholder review
using Documentum
Transmittal Form D, Checklist 2
Publication proceeds through the
NIOSH OD for
clearance
Completion
offinal
ADSO
review

Refer to Appendix H for a decision flowchart on
the applicability of the OMB Bulletins on Peer
Review and Good Guidance Practices.

Publication proceeds through NIOSH
OD final clearance

Submission Procedures
1. Submit Publication to the OD Prior to
External Review
using
Documentum
Transmittal Form D, Checklist 1 and Include:

complex recommendations, such as
establishing a new REL

Draft publication
Cover memo from the lead author’s DLO
Director stating:
o
Purpose and type of review (e.g. policy,
cross-clearance).
o
Statement whether the publication is ISI
or HISA.
o
Statement whether the publication
contains significant guidance.
o
The public comment and external
review sequence.
o
Description whether a public meeting
will be held.
o
Statement whether any additional
background
information
will
be
provided to reviewers.
o
Statement that NIOSH will publicly
disseminate a response to reviewer
comments.
o
Potential areas of concern.
Documentation
of
internal
reviewer
comments and how they were addressed
Charge to reviewers (see Appendix I).
Advance HHS notification of the intent to
request public comments on the draft
publication (see Appendices J and K).
Draft Federal Register notice (FRN) inviting
the public to comment on the draft
publication (see Appendices M and N).
Other items listed on Documentum
Transmittal Form D, Checklist 1.

4. Summarize comments provided by the public
For brevity, organize comments according to
common themes, avoiding an extensive pointby-point format.
5. Forward the publication, summary of public
comments, and other supporting materials to
external peer and stakeholder reviewers
6. Revise Document After External Peer and
Stakeholder Review
Incorporate
relevant
comments
and
suggestions from external peer and
stakeholder reviewers and from the public.
o Make a good faith effort to incorporate
comments whenever reasonable and
feasible.
o The manner in which the comments are
incorporated is at the discretion of
NIOSH.
o All comments are addressed either
through incorporation or by documenting
reasons
why
they
cannot
be
accommodated.
7.

2. Obtain OD Clearance for External Review

Submit Revised Publication After External
Review Using Documentum Transmittal
Form D, Checklist 2 and Include:
Revised publication.
Cover memo from the lead author’s DLO
Director
certifying
the
review
and
accommodation
of
comments
and
highlighting potential areas of concern.
For internal use, a comprehensive listing of
all external comments and how they were
addressed in a point-by-point format.
Reviewer
names
and
organizational
affiliations are associated with specific
comments.
For release to the public:
o A detailed response to reviewer
comments (similar to the internal
response), or a synopsis of reviewer
comments with highlights of revisions
made to the document.
o Peer reviewer names and affiliations
which are not associated with specific
comments.
o Stakeholder names and affiliations
associated with specific comments.

The ADSO submits advance notification to
HHS at least 5 days before release of the
publication for public comment
3. Invite Public comment
Establish an electronic docket where public
comments are received and eventually
made available to other members of the
public (see Appendix M).
Publish an FRN inviting the public to
comment on the draft publication.
Provide opportunities for public comment
through:
o Posting the publication on the NIOSH
Internet website (always required)
o Mailing the publication to the public
(upon request)
o Organizing a public review meeting (on
a case-by-case basis). A public meeting
may be useful for open discussion of

HHS Advance Notification Form stating
intent to publish (see Appendix L).
FRN prepared announcing availability of the
final publication.
Other items listed in Documentum
Transmittal Form D, Checklist 2.
8. Completion of ADSO Review
The ADSO:
o Reviews the final HHS Advance
Notification Form (see Appendix L)
prepared by lead author’s DLO.
o Provides HHS with advance notification
of the intent to issue a final publication.
o Reviews the final FRN prepared by lead
author’s
DLO
announcing
final
availability of the publication (see
Appendices M and N).
After review of the materials and resolution
of all science-related issues, the ADSO
review is complete and the document
proceeds through the OD for final clearance
for publication.
9. Publish the Document
Publish the FRN announcing availability of
the final publication.
Post the publication on the NIOSH Internet
website.
Provide a link from the final publication to
the peer, stakeholder, and public comments
report and the NIOSH response those
comments.

E. External Review With Public Comment After Peer Review: Option 2
Option to Conduct External Peer and
Stakeholder Review Prior to
Soliciting Public Comment

Submit publication to the OD using
Documentum Transmittal Form
Form E, Checklist 1

There are two options for timing solicitation of
public comment, either before (previous
subsection D) or after (this subsection E)
external peer and stakeholder review. See
Appendix E for advantages and disadvantages
of each option.
This category
includes:

NIOSH

Completion of ADSO review for
external peer/stakeholder review
Publication proceeds through NIOSH
OD clearance for external
peer/stakeholder review

Publications

Conduct peer/stakeholder review
Influential Scientific Information (ISI). As
defined by the Peer Review Bulletin, ISI is
scientific information that NIOSH determines
will have, or does have, a clear and
substantial impact on important public
policies or private sector decisions (see
Appendix C).
Highly Influential Scientific Assessments
(HISAs). As defined by the Peer Review
Bulletin, a ”Scientific Assessment” is an
evaluation of a body of scientific or technical
knowledge which typically synthesizes
multiple factual inputs, data, models,
assumptions,
and/or
applies
best
professional
judgment
to
bridge
uncertainties in the available information.
HISA applies to scientific assessments that
NIOSH determines: (i) could have a
potential impact of more than $500 million in
any year, or (ii) is novel, controversial, or
precedent-setting
or
has
significant
interagency interest (see Appendix C).
Significant Guidance. NIOSH publications
contain significant guidance if they
recommend new guidelines or include new
or revised Recommended Exposure Limits
(see Appendix D and the Good Guidance
Practices Bulletin for details).

Revise publication following
external peer/stakeholder review

Submit publication to the OD using
Documentum Transmittal Form E,
Checklist 2
Completion of ADSO review for
public comment
Publication proceeds through NIOSH
OD for clearance for public comment

Solicit public comment
Revise publication following
public comment

Submit publication to the OD using
Documentum Transmittal Form E,
Checklist 3

Refer to Appendix H for a decision flowchart on
the applicability of the OMB Bulletins on Peer
Review and Good Guidance Practices.

Completion of final ADSO review
Publication proceeds through NIOSH
OD final clearance

Submission Procedures
1.

Submit Publication to the OD Prior to
External Review Using the Documentum,
Transmittal Form E, Checklist 1 and Include:

Draft publication
Cover memo from the lead author’s DLO
Director stating:
o Purpose and type of review (e.g. policy,
cross-clearance).
o Statement whether the publication is ISI
or HISA.
o Statement whether the publication
contains significant guidance.
o Potential areas of concern.
o Description of the public comment and
review sequence.
o Statement whether a public meeting will
be held.
o Statement whether any additional
background information will be provided
to reviewers.
o Statement that NIOSH will publicly
disseminate a response to reviewer
comments.
Documentation
of
internal
reviewer
comments and how they were addressed.
Charge to reviewers (see Appendix I).
Other items listed in the Documentum
Transmittal Form E, Checklist 1.

Revised publication.
Cover memo from the lead author’s DLO
Director
certifying
the
review
and
accommodation
of
comments
and
highlighting potential areas of concern.
For internal use, a comprehensive listing of
all external comments and how they were
addressed in a point-by-point format.
Reviewer
names
and
organizational
affiliations are associated with specific
comments..
For release to the public:
o A detailed response to reviewer
comments (similar to the internal
response), or a synopsis of reviewer
comments with highlights of revisions
made to the document.
o Peer reviewer names and affiliations
which are not associated with specific
comments.
o Stakeholder names and affiliations
associated with specific comments.
Advance HHS notification of the intent to
request public comments on the revised
publication (see Appendices K and L).
Draft Federal Register Notice (FRN) inviting
the public to comment on the draft
publication (see Appendices M and N).
Other items listed in the Documentum
Transmittal Form E, Checklist 2.

2. Obtain OD Clearance for External Review
3. Conduct External Peer and Stakeholder
Review
Forward the publication and supporting
materials to external reviewers following OD
clearance for external review.

Obtain Clearance
Comment

Post

for

Public

After review and resolution of issues in the
revised document, the ADSO recommends
posting for public comment.
The ADSO submits advance notification to
HHS no less than 5 days before release of
the document for public review.

4. Revise Publication After External Review
Incorporate
relevant
comments
and
suggestions from external peer and
stakeholder reviewers.
o Make a good faith effort to incorporate
comments whenever reasonable and
feasible.
o The manner in which the comments are
incorporated is at the discretion of
NIOSH.
o All comments are addressed either
through incorporation or by documenting
reasons
why
they
cannot
be
accommodated.

7. Invite Public Comment
Establish an electronic docket where public
comments are received and eventually
made available to other members of the
public (see Appendix M).
Publish an FRN inviting the public to
comment on the draft publication.
Provide opportunities for public comment
through:
o Posting the publication on the NIOSH
Internet website (always required).
o Mailing the document to the public
(upon request).

5. Submit Revised Publication After External
Review Using Documentum Transmittal
Form E, Checklist 2 and Include:

Organizing a public review meeting (on
a case-by-case basis). A public meeting
may be useful for open discussion of
complex recommendations, such as
establishing a new REL.

a point-by-point format; names and
organizational affiliations are associated with
specific comments.
10. Completion of ADSO Review

8. Revise Document Following Public Comment

The ADSO:
o Reviews the final HHS Advance
Notification Form (see Appendix L)
prepared by lead author’s DLO.
o Provides HHS with advance notification
of the intent to issue a final publication.
o Reviews the final FRN prepared by lead
author’s
DLO
announcing
final
publication (see Appendices M and N).
After review of the materials and resolution
of all science-related issues, the ADSO
review is complete and the document
proceeds through the OD for final clearance
for publication.

Incorporate
relevant
comments
and
suggestions from the public.
o Make a good faith effort to incorporate
comments whenever reasonable and
feasible.
o The manner in which the comments are
incorporated is at the discretion of
NIOSH.
o All comments are addressed either
through incorporation or by documenting
reasons
why
they
cannot
be
accommodated.
9. Submit Revised Document After Public
Comment
Using
the
Documentum
Transmittal Form E, Checklist 3 and Include:

11. Publish the Document
·
·

Revised publication.
Cover memo from the lead author’s DLO
Director certifying the revised publication
and accommodation of public comments
and highlighting potential areas of concern.
Comprehensive listing of all public
comments and how they were addressed in

Publish an FRN announcing availability of
the final publication.
Post the publication on the NIOSH Internet
website.
Provide a link from the final publication to
the peer, stakeholder, and public comments
reports and the NIOSH response to those
comments.

List of Appendices
Appendix A. Director’s Memorandum on
NIOSH Scientific Information Quality Review

Appendix J. Summary Flowchart for Public
Review
of
Draft
Significant
Guidance
Publications

Appendix B. Classification of NIOSH Scientific
Information Products by Applicability of Peer
Review Bulletin and Good Guidance Practices
Bulletin

Appendix K. Process Flowchart for Advance
Notification of HHS for Significant Guidance
Publications

Appendix C. Summary of Review Policy for
Influential Scientific Information and Highly
Influential Scientific Assessment Publications

Appendix L. HHS Advance Notification
Template and Example for Significant Guidance
Publications

Appendix D. Summary of Review Policy for
Significant Guidance Publications

Appendix M. Summary Flowchart for Electronic
Docket and FRN

Appendix E. Advantages and Disadvantages of
the Relative Timing of External Review and
Public Comment

Appendix N. Federal Register Notice Example
for Significant Guidance Publications

Appendix F. ADSO Review Worksheet
Appendix G. Considerations for Writing a
NIOSH Publication
Appendix H. Decision Logic for Determining the
Applicability of Peer Review or Good Guidance
Practices Bulletin and Use of Disclaimers
Appendix I. Charge to Reviewers

Appendix A. Director’s Memorandum on NIOSH Scientific Information
Quality Review

Appendix B. Classification of NIOSH Scientific Information
Products by Applicability of Peer Review Bulletin and Good
Guidance Practices Bulletin1

Scientific Information Product
Any products that recommend a new REL
Criteria Documents
Alerts
Current Intelligence Bulletins
Manual of Analytical Methods
(new methods or significant updates only)

Peer Review
Bulletin

GGP
Bulletin

HISA
HISA
ISI
ISI
ISI

SG
SG
SG
SG

Pocket Guide to Chemical Hazards
Bibliographies, Anthologies, and Research
Compendia
Conference Proceedings
Fact Sheets, Brochures, Pamphlets, and
Newsletters
Mining Information Circulars
MMWR articles
NORA Reports
Reports of Investigation
Statistical Reports (e.g. surveillance)
Technical Reports
Training Materials
Web pages
Workplace Solutions
1

Note that while publications are classified on a case-by-case basis, the table demonstrates how
publications usually are classified. The term publication refers to print, electronic, or video format.

HISA – Highly influential
scientific assessment
ISI – Influential scientific
information

OMB Peer Review Bulletin
http://www.whitehouse.gov/omb/fedreg/2005/011405_peer.pdf

SG – Significant guidance

OMB Good Guidance Practices Bulletin
http://www.whitehouse.gov/omb/memoranda/fy2007/m07-07.pdf

Appendix C. Summary of Review Policy for Publication of
Influential Scientific Information and Highly Influential Scientific
Assessments
OMB has defined two special categories of
influential information, ISI and HISAs. According
to the Peer Review Bulletin, ISI is information
that the agency expects to have a clear and
substantial impact on important public policies or
private sector decisions. A HISA is defined as an
evaluation of a body of scientific or technical
knowledge which typically synthesizes multiple
factual inputs, data, models, and assumptions,
and then applies best professional judgment to
bridge uncertainties in the available information.
A HISA meets either one of the following criteria:

It has a potential impact of more than $500
million in any one year on either the public
or private sector; or
It is novel, controversial, or precedentsetting, or has significant interagency
interest.
In general, NIOSH publications are considered
ISI if they identify new hazards or recommend
new guidelines. Publications are HISAs if they
include new or revised RELs. Appendix B
contains a table that shows the suggested
classification of NIOSH publications according to
Peer Review Bulletin requirements.

Appendix D: Summary of Review Policy for Significant Guidance
Publications
The Good Guidance Practices Bulletin
establishes technical definitions for guidance
publications, significant guidance publications,
and
economically
significant
guidance
publications and outlines procedures for their
development and revision. The term “guidance
publication” means an agency statement of
general applicability and future effect, other than
a regulatory action that sets forth a policy on a
statutory, regulatory, or technical issue or an
interpretation of a statutory or regulatory issue.
The definition of a guidance publication
encompasses all guidance materials, regardless
of format or title.

Raise novel legal or policy issues arising out
of legal mandates, the President’s priorities.
Economically significant guidance publications
are a subset of significant guidance publications;
they may reasonably be anticipated to lead to an
annual effect on the economy of $100 million or
more or adversely affect in a material way the
economy or a sector of the economy.
Economically significant guidance publications
do not include guidance publications on federal
expenditures and receipts.
Although OMB only requires public notice and
an opportunity for public comment with agency
feedback for economically significant guidance
publications, this distinction is inconsistent with
NIOSH values for scientific quality. Given the
national and international impact of NIOSH
activities, it is expected that the majority of
NIOSH significant guidance publications will also
be determined to be economically significant.
Consequently, all NIOSH guidance publications
that meet the OMB definition for a significant
guidance publication will comply with all of the
requirements for economically significant
guidance publications as outlined in the Good
Guidance Practices Bulletin (see NIOSH Policy
on the Implementation of the OMB Final Bulletin
for Agency Good Guidance Practices).

The Good Guidance Practices Bulletin also
distinguishes
between
significant
and
economically significant guidance publications.
The term “significant guidance document”
means a guidance document disseminated to
regulated entities or the general public that may
reasonably be anticipated to:
Lead to an annual effect on the economy of
$100 million or more or adversely affect in a
material way the economy, a sector of the
economy, productivity, competition, jobs, the
environment, public health or safety, or
state, local, or tribal governments or
communities;
Create a serious inconsistency or otherwise
interfere with an action taken or planned by
another agency;
Materially alter the budgetary impact of
entitlements, grants, user fees, or loan
programs or the rights and obligations of
recipients thereof; or

In general, NIOSH publications are considered
to contain significant guidance if they
recommend new guidelines or include new or
revised RELs. Appendix B contains a table that
shows the general classification of NIOSH
publications by applicability of the Good
Guidance
Practices
Bulletin.

Appendix E. Advantages and Disadvantages of the Relative Timing
of External Review and Public Comment

E. Option to Conduct External Peer and
Stakeholder Review Prior to Soliciting
Public Comment

D. Option to Solicit Public Comment Prior
to Conducting External Peer and
Stakeholder Review

Advantages

Disadvantages

Public comments can be shared with
peer reviewers for consideration during
their review. This is likely to improve the
quality of the peer review.
There is no NIOSH OD clearance checkpoint after public comment is solicited
and before external peer and stakeholder
review is conducted. This might result in
a shorter overall review and clearance
process.

Publications revised in response to peer
review are closer to their final version and
therefore, the public is presented with a
higher quality publication. This likely
improves the public comment process.
The NIOSH OD clearance check-point
after external peer and stakeholder
review is conducted and before public
comment is solicited provides an
opportunity to identify important and
unaddressed issues early. Addressing
these issues will likely not require a lot of
effort.

Publications may change significantly in
response to external peer review. This
may draw criticism from the public. In
order to avoid this, the NIOSH OD may
recommend the publication receive a
second round of public comment (on a
case-by-case basis).
Because there is no NIOSH OD
clearance check-point after public
comment is solicited and before external
peer and stakeholder review is
conducted, important but unaddressed
issues may first be discovered during the
final NIOSH OD review and clearance.
Addressing these issues may require
significant effort and cause delays.

The NIOSH OD clearance check-point
after external peer and stakeholder
review is conducted and before public
comment is solicited might increase the
overall time required for the review and
clearance process.
Peer reviewers do not have access to
public comments during their review
process. As a result, some public
comments may need further scientific
clarification in order to be adequately
addressed. Then, the NIOSH OD may
recommend that the publication receive a
second round of peer review (on a caseby-case basis).

Appendix F. ADSO Review Worksheet
I.

Publication Thumbnail Sketch
Review Process Summary
A. Thumbnail Sketch Elements
 Purpose
 Audience
 Need for information product
B.

 Does the final publication reflect
adequate consideration of internal and
external peer review comments?
 Were sources of uncertainty or alternate
interpretations for the data addressed,
and were caveats acknowledged?
 Were associations among variables
distinguished from causal relationships?
 Were intermediate steps in the flow of
arguments presented?
 Are
the
conclusions
and
recommendations
appropriately
generalized and supported?

and

Publication Development and Review
Process Summary Elements
 Summary of previous ADSO review
issues (if applicable)
 Publication developmental stage (e.g.
draft publication for public review)
 Requested review outcome (e.g. NIOSH
OD clearance for final publication)

B. NIOSH Scientific Policy
 Is the publication’s topic sensitive, which
may affect the level of scrutiny it will
receive after publication?
 Does this scientific product contain
findings or policy related to:
 Departmental, Presidential, OMB or
Congressional priorities
 Media interests
 High profile health and safety topics
(e.g. emergency preparedness or
terrorism,
nanotechnology,
pandemic influenza)
 Is the publication expected to impact
regulatory activities?
 Is there an urgent need for the
publication’s dissemination?
 Has the DLO or document author
identified,
acknowledged
and/or
characterized any policy-related issues
or controversies?
 Has the ADSO reviewer identified,
acknowledged and/or characterized any
policy-related issues or controversies?
 Did prior participants in the review and
clearance
process
(e.g.
peer,
stakeholder, or public reviewers) identify
and/or characterize any controversial
issues with policy statements?
 Did the authors provide clear and
concise responses to policy-related
comments
received
during
peer,
stakeholder, and public review that were
of sufficient quality to represent the
Institute’s viewpoint in a manner that is
publishable on the Institute’s Internet
website?

II. Overall
Assessment
and
Recommendations
 This publication has been
 Approved
 Approved with changes (additional
review not required)
 Approved with changes (additional
review is required)
 Not approved at this time
 Issues requiring further resolution (if
applicable)
 Suggestions for changes (if applicable)
 Guidance
for
future
document
development (if applicable)
III. Review Elements
A. Science Quality
 Did the lead DLO solicit collaboration
from other DLOs to ensure that others
with relevant expertise within NIOSH are
aware of the publication and concur with
its
content,
conclusions,
and
recommendations?
 Are the objectives of the work and the
need that prompted its development
clearly stated?
 Does
the
publication
contain
preliminary/unpublished data, and if so,
are conclusions drawn from this data
justified?
 Are scientific data accurately displayed?
 Are factual statements adequately
supported by scientific citation?

 Are science policy statements new,
consistent with prior expressions of
NIOSH policy, or revisions to existing
NIOSH policy?
If science policy
statements are revisions, are departures
from previous NIOSH policy clearly
explained?
 Does the product contain or create any
inconsistencies between NIOSH policy
and another agency (e.g. Department of
Labor)?
 Does the product contain critical
language that
 May be critical of other agencies or
departments?
 May create a perception of bias,
limiting the impact of the underlying
message?
 May be critical of companies,
products, institutions, organizations,
or societies?
 Has care been taken to specifically
avoid identifying, criticizing and/or
endorsing
companies,
products,
institutions, organizations, or societies?
 Does the publication provide critique or
recommendations
that
have
the
potential for unintended consequences
or adverse effect?
 Is
mandatory
language
used
appropriately (e.g. shall, must, is
required)?
C. Administrative Standards
 Does this product contain influential
scientific information or a highly
influential scientific assessment as
defined by the OMB Peer Review
Bulletin? If yes,
 Is there a plan to update the Peer
Review Agenda Internet website (i.e.
identification of stakeholders and
peer reviewers, charge to peer
reviewers, comments, etc.)?
 Does the submission include the
Peer Review Agenda Transmittal
form?
 Are
stakeholders
and
peer
reviewers appropriately identified?
 Are
statements
of
independence/conflict of interest for
peer reviewers available (required
for highly influential scientific
assessments)?
 Are appropriate disclaimers used?
For example, if the publication is
being submitted for clearance for

external review, does the publication
bear the following disclaimer: “This
information is distributed solely for
the purpose of pre dissemination
peer review under applicable
information quality guidelines. It has
not been formally disseminated by
the
National
Institute
for
Occupational Safety and Health. It
does not represent and should not
be construed to represent any
agency determination or policy,”
(see NIOSH Policy on the Use of
Disclaimers for Scientific information
Products)?
 Are there any recommendations
regarding notification of authors to
make document reviewers or
collaborators aware that they may
be included on the Peer Review
Agenda Internet website?
 Are there any recommendations
regarding the plan for a public
meeting
(required
for
highly
influential scientific assessments)?
 Does this product contain significant
guidance as defined by the OMB Good
Guidance Practices Bulletin? If yes, are
there any recommendations regarding:
 A plan to provide HHS with 5 day
advance notification?
 The HHS Advance Notification Form
(e.g. is it clear and concise?)?
 Writing a Federal Register Notice (if
applicable)?
 Establishment of a public docket for
comments?
 Notification of authors to provide
document reviewers or collaborators
with a courtesy notification that the
document will be made available for
public comment?
 Response to comments document
and its posting to the NIOSH
Internet website?
 Updating the Peer Review Agenda
Internet website (if applicable)?
 Coordination of the Federal Register
Notice
with
posting
of
the
publication to the NIOSH Internet
website,
and
the
anticipated
clearance by HHS?
 Does this product require crossclearance by other CDC Centers or
Offices or other federal agencies (see
Clearance of Information Products

 Does this product conform to an
established publication format (see the
NIOSH Identity Guide)?
 Is the publication written with a
consistent style throughout and does it
read as a whole?
 Is the publication clearly organized?
 Is the intended audience clearly
defined?
 Are references complete and do they
include the sources for statistical tables
and graphs?
 Are recommendations clearly written for
the intended audience?
 Is this publication likely to generate
Congressional, media, or political
interest?
 Are
critical
audiences
effectively
anticipated (e.g. if it is already known
that the audience will be skeptical of the
publication’s conclusions, a more
lengthy and detailed rationale may need
to be presented to establish credibility
with the audience)?
 Are there specific communication issues
that need to be referred to OHC?

Disseminated Outside CDC for Public
Use)?
 Does the research raise dual use
concerns (see Oversight and Clearance
of Dual Use Research of Concern)?
Dual use research of concern is
research that, based on current
understanding, can be reasonably
anticipated to provide knowledge,
products, or technologies that could be
directly misapplied by others to pose a
threat to public health, agriculture,
plants, animals, the environment, or
materiel. Such research requires
scrutiny and review for dual use
potential prior to initiation of research as
well as during the scientific review and
publication process. Scientists must be
mindful of dual use issues while
research is being conducted as dual use
concerns may develop during the
execution of a research plan.
D. Communication Issues
See Appendix N for details.
 Is the publication free of grammatical or
typographical errors?

Appendix G. Considerations for Writing a NIOSH Publication
State Goals

Characteristics of NIOSH publications include
provision of sufficient background information,
balanced presentation of issues, presentation of
new information, and respectful treatment of
audiences, including audiences that may
disagree with NIOSH conclusions. Authors of
NIOSH publications are responsible for
producing high-quality science and evidencebased guidelines and recommendations that are
effectively communicated. Recommendations
are most relevant and have the most impact
when they are based on analyses that use
sound and transparent methodology and highquality data. Effectiveness of the presentation is
improved when the message is relevant,
substantiated, clear, and important.

Provide NIOSH stakeholders with useful
scientific information or guidance on existing
or emerging issues.
Link goals for the publication to the need
that prompted its development.
Anticipate Critical Audiences
Anticipate the need for more background or
detailed information for some audiences.
If it is known that an audience will be
skeptical, additional rationale may be
required to establish credibility.
Use High-Quality Data

Consider the following elements during the
writing phase of document development. These
are the same elements on which the ADSO
review focuses.

Use data derived from sound methodology.
Avoid over-generalization of preliminary or
unpublished data.
Display data accurately.

Attend to the Basics

Support Factual and Background Information

Eliminate grammatical and typographical
errors.
Conform to an established format (see the
NIOSH Identity Guide).
Write in a consistent style throughout the
document.
Logically organize the content.
Include complete references and sources for
tables and figures.

Reinforce
factual
and
statements with references.

background

Substantiate Conclusions
·
·

Add Value

·
·

Be sure the publication’s contribution is clear
and relevant.
Base conclusions and recommendations
either on original NIOSH-supported research
or on existing research that is synthesized in
a new and useful manner.
Avoid
duplicating
conclusions
and
recommendations
that
are
available
elsewhere.

Ensure that conclusions are appropriately
generalized and supported.
Address areas of uncertainty and data
limitations.
Acknowledge alternate interpretations.
Distinguish associations among variables
from putative causal relationships.
Present intermediate steps in the flow of
arguments in a logical format.
Derive Recommendations from
Scientific Conclusions
Base policies and recommendations clearly
and directly on conclusions derived from
high-quality science.
Formulate recommendations with the
understanding that NIOSH publications
represent official positions of the Institute,
are broadly disseminated, may be used to

address urgent public health needs, and
may influence regulatory activities.

Language that is explicitly or harshly critical
of other agencies, departments, or
organizations
detracts
from
NIOSH
credibility and impartiality.
Inappropriate language can create a
perception of bias, limiting the impact of the
underlying message.

Synthesize Information
Draw upon other peer-reviewed studies and
relevant information when NIOSH research
does not address all policy issues.

Use Mandatory Language Appropriately
Use the Hierarchy of Controls
Clearly distinguish between guidance and
statutory or regulatory requirements.
Avoid mandates such as “shall,” “must,”
“required,” or “requirement,” unless the
terms are used to describe statutes or
regulations.
Laws of nature, scientific principles, and
technical requirements may be described in
mandatory terms as long as it is clear that
NIOSH guidance does not inappropriately
impose legally enforceable rights or
obligations.

Arrange recommendations in a sequence
that follows the hierarchy of controls (see
http://www.cdc.gov/niosh/topics/engcontrols/
) as closely as possible.
Cite Previous Policy
Explicitly state whether policy statements
are consistent with prior expressions of
NIOSH policy or are departures from it.
Anticipate Sensitivities, Complexities,
and Priorities

Relate Recommendations to the Audience

The following types of publications may
receive higher than usual attention after
publication:
o Topics that are sensitive, technically
complex, original, or unique
o Publications related to high-profile
health and safety topics, e.g. emergency
preparedness,
terrorism,
nanotechnology,
and
pandemic
influenza
o Subject matter that is included in
Presidential, OMB, Congressional, or
HHS priorities
Complexity will increase the difficulty of
clearly communicating a topic and
appropriately disentangling the policy issues.

Compose recommendations in language
appropriate for the intended audience. Avoid
highly technical terms in recommendations
aimed at non-experts.

Acknowledge Controversies
Identify controversies explicitly and present
alternative viewpoints in a balanced manner.
Avoid Overly Critical Language or
Unintended Endorsements
Ensure
that
companies,
products,
institutions, or organizations are not
inappropriately identified, criticized, or
endorsed.

Appendix H. Decision Logic for Determining the Applicability of
Peer Review or Good Guidance Practices Bulletin and Use of
Disclaimers
Determine the applicability of Peer Review Bulletin and use of disclaimers
No
Scientific information? See Appendix B
Yes

Not covered by Peer Review Bulletin
No

Disseminated to public?
Yes

The official position of NIOSH?

Not covered by Peer Review Bulletin
Use disclaimer: See NIOSH Policy on
the Use of Disclaimers for Scientific
Information Products

Yes
Influential Scientific information?
See Appendices B and C

No
Not covered by Peer Review Bulletin

Yes
Highly Influential Scientific
Assessment? See Appendices B and C

Yes
Designate as HISA
Planning for review: See Policy and
Guidance for Developing the Topic
Concept Memo
During review: See Section IV
“External Review With Public
Comment Required (D or E)”

Designate as ISI
Planning for review: See Policy and
Guidance for Developing the Topic
Concept Memo
During review: See Section IV
“External Review of ISI (C)” or
“External Review With Public
Comment Required (D or E)”

Determine the applicability of Good Guidance Practices Bulletin
No
Significant Guidance?
See Appendices B and D

Not covered by
Good Guidance Practices Bulletin

Yes
Designate as Significant Guidance
Planning for review: See Policy and
Guidance for Developing the Topic
Concept Memo
During review: See Section IV
“External Review with Public
Comment Required (D or E)”

Appendix I: Charge to Reviewers
Include a charge to reviewers in Documentum
for publications requiring external peer and
stakeholder review. Based upon technical or
end-user issues, it may be necessary to draft
different charges for peer and stakeholder
reviewers. The charge communicates the scope
and expectations for the review with the
following information:

Specific questions that the reviewers should
address
How reviewer comments will be used and
distributed, including what will be made
public following the review
Whether reviewer names will be kept
anonymous and whether their comments will
be distributed with attribution or as part of an
overall summary of the review
Other directions or instructions to reviewers
that may be necessary for completing the
review

Purpose of the publication and any other
background information that may not be selfevident from the publication itself
Purpose of the review
Scope of the review

Appendix J. Summary Flowchart for Public Review of Draft
Significant Guidance Publications

Obtaining public review

Significant Guidance?
See Appendices B and D
Yes

Not covered by
Good Guidance Practices Bulletin
No public comment required

NIOSH OD clearance for public review?

Obtain OD clearance for public review

Yes
Formatted and ready for publishing
to NIOSH Internet Website?

Yes
Provide advance notification to HHS
See Appendices K, L

Publish FRN, following standard
NIOSH procedures
See Appendices M, N

Establish docket to receive public
comments, following standard NIOSH
procedures See Appendix M

Post to NIOSH Internet Website

Work with EID and OHC
to prepare for publishing

Appendix K. Process Flowchart for Advance Notification of HHS for
Draft Significant Guidance Publications

Providing advance notification to HHS

DLO completes HHS Advance
Notification Template (Appendix L)
and submits to NIOSH OD through
Documentum

ADSO submits template to HHS

HHS requests to review draft
publication?

Return to summary flowchart
(Appendix J) to publish FRN

Yes
Provide HHS draft publication

No
HHS approves draft publication?
Yes
Return to summary flowchart
(Appendix J) to publish FRN

Work with HHS to have draft
publication approved

Appendix L. HHS Advance Notification Template and Example for
Significant Guidance Publications
NIOSH provides HHS with advance notification:
No less than 5 days before dissemination of a draft significant guidance NIOSH publication
for public review
No less than 5 days before NIOSH issues a final significant guidance publication

Template
Date
Title

File Type	application/pdf
Author	Regina Pana-Cryan
File Modified	2009-12-31
File Created	2009-12-31