NSF Biological Sciences Directorate Surveys of Applicants and Reviewers to Assess the Preliminary Proposal Process

Reviewer Survey
May 2016 v3
Panel reviewers: All questions in the survey refer to your experience as a panel reviewer
_____ for the _____ [NSF program name] in 2015. 1
Ad hoc reviewers: All questions in the survey refer to your experience as a reviewer of __
[proposal info] ___ for the _____ [NSF program name] in 2015. 2

RESPONDENT CHARACTERISTICS
Panel reviewers for preliminary and full proposals, ad hoc reviewers
Which of the following best describes your current academic rank?
□ Postdoctoral fellow
□ Research scientist
□ Adjunct professor
□ Assistant professor
□ Associate professor
□ Full professor
□ Other ___________
What is your tenure status?
□ Tenure-tracked or equivalent, but not yet tenured
□ Tenured or equivalent
□ Not tenure-tracked
Have you ever been a PI or Co-PI on a preliminary proposal submitted to core programs in the
Division of Environmental Biology (DEB) or the Division of Integrative Organismal Systems
(IOS)?
□ Yes  A
□ No
□ Unsure
A. Have you ever been PI or Co-PI on an award from the core programs in DEB or IOS
stemming from a preliminary proposal?
□ Yes
□ No
□ Unsure

1
2

This information will be preloaded and visible to respondent
This information will be preloaded and visible to respondent

1

WORKLOAD
Panel reviewers for preliminary and full proposals
Please complete the following information about this panel experience. Do not include travel and
panel meeting time in your estimates.
Review format: Was
your participation in
the panel primarily inperson or through
virtual meeting
technologies?

Approximate
number of
reviewers on the
panel, including
yourself

Number of
proposals
assigned to you
for review (N)

Approximate
number of hours
you spent per
review prior to
the panel
meeting (H)

□
□

____

____

____

Virtual
In-person

Total number of
hours you spent
on all reviews
prior to panel
meeting
(T=N*H)

Panel reviewers for preliminary proposals
How would you compare the time and effort to review each preliminary proposal to a full
proposal? On average, a preliminary proposal took approximately…
□ ¼ of the time or less to review
□ ½ of the time to review
□ The same time to review
□ 25% more time to review
□ 50% more time to review
□ 100% more time to review
□ More than twice the amount of time to review
Ad hoc reviewers for full proposals
Approximately how many hours did it take you to review this proposal [pre-load proposal info]?
__________

2

SATISFACTION WITH THE PROPOSALS
Panel reviewers for preliminary and full proposals
How satisfied were you with the proposals reviewed by your panel in terms of…
Not at all
satisfied

Somewhat
satisfied

Neutral

Satisfied

Very satisfied

Do not
recall/do not
know

Intellectual merit
Broader impact
Potential to advance
your field
Potential to transform
your field

Approximately what fraction of the proposals reviewed by your panel had the following
characteristics?
10% or
less

Onefourth

One-third

One-half

Twothirds

> Twothirds

Do not
recall

Addressed a significant
question/problem/opportunity
Clearly conveyed the idea and
approach
Presented an approach that
was feasible and appropriate
Contained convincing
preliminary data
Could be characterized as
high-risk/high-reward
Demonstrated that
investigators had the requisite
expertise
Were collaborative and/or
interdisciplinary

3

What percentage of the proposals reviewed by your panel was clearly not worth funding?
_____ percent
□ Do not recall/do not know
Ad hoc reviewers for full proposals
Did this proposal have the following characteristics?
Addressed a significant research
question/problem/opportunity

□

Clearly conveyed the idea and approach

□

Presented an approach that was feasible and appropriate

□

Contained convincing preliminary data

□

Could be characterized as high-risk/high-reward

□

Demonstrated that investigators had the requisite expertise

□

Was collaborative and/or interdisciplinary

□

Was clearly worth funding

□

Panel reviewers for preliminary proposals
Was the 4-page Project Description narrative sufficient for you to evaluate the proposed idea and
approach?
□ Yes
□ No
How many pages would you recommend as optimal for the Project Description section in a
preliminary proposal?
_____ pages
Panel and ad hoc reviewers for full proposals
Was the 15-page Project Description narrative sufficient for you to evaluate the proposed idea
and approach?
□ Yes
□ No

4

How many pages would you recommend as optimal for the Project Description section in a full
proposal?
_____ pages

SATISFACTION WITH THE REVIEW
Panel reviewers for preliminary and full proposals

Very satisfied

Satisfied

Neutral

Unsatisfied

Very unsatisfied

What is your level of satisfaction with the following….

Number of proposals you were assigned to review
Quality of written reviews by other panelists/ad hoc
reviewers
Rigor of proposal discussion in panel
Balance in the reviewers’ consideration of proposal
strengths and weaknesses
Willingness of other reviewers to support highrisk/high-reward projects
Fairness of final panel ratings
Level of consensus among the reviewers
Quality of panel summaries
Review process overall

Panel reviewers for preliminary and full proposals, ad hoc reviewers for full proposals
Please describe one change you would make to the merit review process.
___________________________

□ I would not make any changes

5