NSF/BIO DEB and IOS Web Survey for Soliciting Feedback on Program Changes

NSF/BIO DEB and IOS web survey for soliciting feedback on program changes

proposed February 2013

Introduction:

In 2011 the Division of Environmental Biology (DEB) and Division of Integrative Organismal Systems (IOS) in the Biological Sciences Directorate (BIO) at the National Science Foundation (NSF) made significant changes to the core program review process for the 2012 funding year. These changes (including the new preliminary proposal requirement, a switch to an annual deadline, and a cap on the number of proposals Principal Investigators (PIs) can submit per cycle) were made to protect the integrity of the merit review system, which was increasingly compromised by steep increases in proposal submissions, diminished participation of the community in the proposal review process, and flat budgets. Sinking success rates and the constant churn of excellent proposals were symptoms of a complex, system-wide problem that motivated a 5-year investigation into ways to help alleviate the problems associated with flat budgets.

We have received considerable feedback from the community on those changes -- including discussions between Program Officers and panel members at NSF, discussions at professional meetings with many PIs and community members, and via letters, phone calls and emails. We are also compiling multiple forms of data on the outcomes after the first year of implementation of these changes.

This survey is another mechanism that we are using to obtain feedback about the new review process. Our target audience is investigators who have had funding from DEB or IOS or who are likely to submit a proposal to DEB and/or IOS in the next 1-2 years. We will use the results of the survey, which will be anonymous, to assess the level of satisfaction with various aspects of the new review process. Your answers will be confidential, and will have no bearing on your future interactions with the NSF. The survey should require 10 minutes to complete. Thank you for taking the survey and providing NSF with feedback.

Section A. The new process. (If you did not submit a preliminary proposal to either DEB or IOS in 2012 or 2013 please SKIP sections A & B and proceed to C1.)

A1. Four pages of text in the preliminary proposal is ________________ space for me to describe my ideas and explain why they are important.

Sufficient 1

Not Sufficient 2

Don’t Know 3

A2. When you submitted proposals to DEB in the 5 years before the new procedure (i.e., prior to January 2012), on average, how many unique proposals per year did you routinely submit as PI or coPI to the core programs in DEB? (Do not count submissions to other solicitations like Doctoral Dissertation Improvement Grants, Dimensions of Biodiversity, CAREER, Macrosystems Biology, Ecology and Evolution of Infectious Diseases, etc.).

• 0, I didn’t submit any proposals to NSF prior to 2012.

• 0, I submitted a proposal(s) to NSF but not to the core programs in DEB.

• 1

• 2

• 3

• ≥4

A3. When you submitted proposals to IOS in the 5 years before the new procedure (i.e., prior to January 2012), on average, how many unique proposals per year did you routinely submit as PI or coPI to the core programs in IOS? (Do not count submissions to other solicitations like Doctoral Dissertation Improvement Grants, Plant Genome Research Program, CAREER, Macrosystems Biology, Ecology and Evolution of Infectious Diseases, etc.).

• 0, I didn’t submit any proposals to NSF prior to 2012.

• 0, I submitted a proposal(s) to NSF but not to the core programs in IOS.

• 1

• 2

• 3

• ≥4

A4a. Before the new procedure (prior to January 2012) when a proposal was declined, did you typically revise and resubmit the proposal at the next proposal deadline (i.e., roughly 6 months after the previous submission)?

• Yes.

• No.

A4b. If you answered “no” in A4a please provide a reason for this. Check all that apply.

• I typically chose to take more time to revise and resubmit.

• I did not receive comments in time to revise and resubmit by the next deadline.

• I would submit a different, unique proposal at the next deadline.

• I normally did not submit more than one proposal per year.

• Other.__________________________

A5. To what extent has the cap on the number of submissions to 2 per PI/coPI/subaward lead per Division influenced your ability to submit proposals with your established collaborators?

Strongly Increased 1

Increased 2

No Change 3

Decreased 4

Strongly Decreased 5

Not Applicable

A6. To what extent has the cap on the number of submissions influenced your ability to submit proposals with new collaborations?

Strongly Increased 1

Increased 2

No Change 3

Decreased 4

Strongly Decreased 5

Not Applicable

A7. To what extent has the cap on the number of submissions influenced aspects of your research productivity other than grant writing (e.g., writing manuscripts; conducting research; etc.)?

Strongly Increased 1

Increased 2

No Change 3

Decreased 4

Strongly Decreased 5

Not Applicable

A8. Before January 2012, researchers could submit full proposals twice per year (winter and/or summer). Please assess the timing of the new proposal submission process.

a. Preliminary proposal submission in mid-January?

Very Convenient 1

Somewhat Convenient 2

Neutral 3

Somewhat Inconvenient 4

Very Inconvenient 5

Decline to Answer

b. Full proposal submission in early August?

Very Convenient 1

Somewhat Convenient 2

Neutral 3

Somewhat Inconvenient 4

Very Inconvenient 5

Decline to Answer

B. Level of satisfaction with the 3 main components of DEB and IOS’s new submission and review system (preliminary proposal requirement, capping number of submissions, and limit to single submission date per year). When answering the questions, keep in mind that there have been relatively flat budgets in the past several years, and while the new system does not change that, it was put into place to help offset some of the negative consequences of low funding rates.

Please try to answer the questions below with respect to the new system and not the limited budget, which is something that NSF does not control.

B1. What is your level of satisfaction with the following?

Requirement of a preliminary proposal screen before invitation to submit a full proposal.

Very Satisfied 1

Satisfied 2

Neutral 3

Dissatisfied 4

Very Dissatisfied 5

Capping the number of submissions per investigator per year to 2 per Division.

Very Satisfied 1

Satisfied 2

Neutral 3

Dissatisfied 4

Very Dissatisfied 5

Limit to single submission date per year for preliminary proposals.

Very Satisfied 1

Satisfied 2

Neutral 3

Dissatisfied 4

Very Dissatisfied 5

B2. Rank the three procedural changes that were made in DEB and IOS according to your level of satisfaction. (1=most satisfied or least dissatisfied; 3=least satisfied or most dissatisfied)

__ Requirement of a preliminary proposal screen before invitation to submit a full proposal.

__ Capping the # of submissions per investigator per year to 2 per Division.

__ Limit to single submission date per year for preliminary proposals.

B3. When broadly considering the scientific fields generally supported by DEB and IOS, in your opinion, how have the following aspects of these sciences been impacted by the DEB and IOS’s new system?

The ability of new investigators and untenured faculty to advance their careers.

Very Positively 1

Positively 2

Neutral 3

Negatively 4

Very Negatively 5

Don’t Know 6

The ability to work collaboratively.

Very Positively 1

Positively 2

Neutral 3

Negatively 4

Very Negatively 5

Don’t Know 6

The ability to submit new ideas on pace with advancements in the science.

Very Positively 1

Positively 2

Neutral 3

Negatively 4

Very Negatively 5

Don’t Know 6

B4. In your opinion, are there other aspects of science that have been negatively affected by the new system?

B5. In your opinion are there other aspects of science that have been positively affected by the new system?

C. Communication

C1. How do you receive information about new solicitations or other news from DEB and IOS? (Check all that apply.)

• I receive emails from NSF.

• I visit the NSF website.

• I hear it from other colleagues.

• I attend NSF sessions at professional meetings.

• Other (fill in)

C2. How would you like to receive information from DEB and IOS? (Check all that apply.)

• Emails from NSF

• Visit the NSF website

• Social media updates

• Webinars

• Blog

• Sessions at professional meetings

• Other (fill in)

C3. If you have any other comments related to the new procedure in DEB and IOS, please provide them here or at the DEB blog (www.XXXXX.org). Keep in mind that returning to the old system is not tenable.