CSR Attachment 1 - Applicant Survey Instrument

OMB No: 0925-0474 Expiration Date: 9/30/2011

Enhancing Peer Review Initiative

Survey of Applicants

Sponsored by:

National Institutes of Health

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number of this information collection is 0925-0474. The time required to complete this information collection is estimated to average 15 minutes per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have any comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: NIH, Project Clearance Branch, 6705 Rockledge Drive, MSC 7974, Bethesda, MD 20892-7974 Attn: PRA (0925-0474). If you have comments or concerns regarding the status of your individual submission of this form, write directly to: RTI International, 3040 Cornwallis Road, PO Box 12194, Research Triangle Park, NC 27709-2194. 1-800-334-8561 Attn: RTI Project 0212255)

Introduction

This survey of NIH grant applicants is to help examine NIH’s Enhancing Peer Review Initiative (http://enhancing-peer-review.nih.gov/). The objectives of the initiative are to engage the best reviewers, improve the quality and transparency of peer review, and ensure balanced and fair reviews.  This is the first annual “point in time” survey to gather applicants’ opinions about the peer review process. This information will be useful in assessing the changes introduced by the Enhancing Peer Review Initiative and may be used to further improve the peer review process. 

You have been randomly selected to participate in this survey from a pool of individuals who submitted at least one NIH research grant application from January 2008 through May 2009. We are interested in your opinions, regardless of whether or not the application(s) you submitted during this time period was funded. Even if you have limited experience submitting grant applications, your opinions are very important to us.

The survey should take approximately 15 minutes to complete. You can stop at any point and continue at another time. There are no right or wrong answers, so please give the answer that best describes your opinion. While we would like you to answer all the questions in this survey, you may skip any questions that you do not wish to answer.

Your participation is entirely voluntary. If you choose to complete the survey, your responses will remain anonymous. Your responses will not be linked to your name and will not be made known to NIH staff or peer reviewers. They will not be used to assess the performance of individual NIH Institutes, Centers, or Scientific Review Groups. Aggregate responses will be used to guide NIH management in refining enhancements to the peer review process.

Your participation is greatly appreciated.

Screening Questions

Please think of the most recent application you submitted to NIH (for either a single-PI or multiple-PI grant) that was:

• reviewed and

• for which a funding decision (either not funded or funded) was reached.

Please do not include applications for administrative supplements, as these undergo administrative review instead of peer review.

S1. When did you submit this research grant application?

O January 2008 – December 2008

O January 2009 – May 2009

IF JAN08-DEC08 THEN DISPLAY ALTERNATE VERSION OF SECTIONS A AND B WITH NO PEER REVIEW CHANGES, ELSE CONTINUE (ALTERNATE VERSION OF SECTIONS A AND B ARE PROVIDED AT THE END OF THIS QUESTIONNAIRE)

S2. Was this application submitted in response to any of the NIH funding opportunity announcements related to the American Recovery and Reinvestment Act (ARRA) of 2009, such as the Challenge grants program and Grand Opportunities “GO” grants program?

O Yes

O No

SECTION A: Your Most Recent Experience As an Applicant

Please think of the most recent application you submitted to NIH (for either a single-PI or multiple-PI grant) that was:

• reviewed and

• for which a funding decision (either not funded or funded) was reached

Please do not include applications for administrative supplements, as these undergo administrative review instead of peer review.

A1. What was the activity code for this NIH research grant application?

An activity code refers to the 3-character code used by NIH to identify a specific category of extramural research activity, applied to various funding mechanisms (e.g., R01, P01).

Enter activity code here 

PROGRAMMING NOTE: THIS WILL BE A DROP-DOWN LIST. WE WILL NEED A COMPLETE LIST OF 3-LETTER CODES TO INCLUDE IN DROP-DOWN. RESPONDENT WOULD TYPE IN FIRST LETTER AND THEN THE LIST WILL SHOW UP FOR THAT LETTER GROUP.

A2. Was this the first NIH research grant application for which you were the Principal Investigator (PI)?

THE QUESTION WORDING “PRINCIPAL INVESTIGATOR” WILL INCLUDE A HYPERLINK AND IF CLICKED THE FOLLOWING DEFINITION WILL APPEAR ON SCREEN:

NIH Definition of a Principal Investigator:  The individual(s) judged by the applicant organization to have the appropriate level of authority and responsibility to direct the project or program supported by the grant.  The applicant organization may designate multiple individuals as PDs/PIs [Program Directors/Principal Investigators] who share the authority and responsibility for leading and directing the project, intellectually and logistically.  Each PD/PI is responsible and accountable to the applicant organization, or, as appropriate, to a collaborating organization, for the proper conduct of the project or program including the submission of all required reports.  The presence of more than one identified PD/PI on an application or award diminishes neither the responsibility nor the accountability of any individual PD/PI.

O Yes

O No

IF S2=2 CONTINUE ELSE GO TO A6

A3. Did your eRA Commons account indicate that you were identified as a New or Early Stage Investigator on this application?

Note: NIH defines a New Investigator as an applicant who has not yet competed successfully for a significant NIH independent research award, such as an R01. An Early Stage Investigator is defined as a New Investigator who is within 10 years of completing his/her terminal research degree or is within 10 years of completing medical residency (or the equivalent).

O Yes

O No

A4. Was this an application to conduct clinical research, defined by NIH as research involving human subjects?

O Yes

O No

A5. Was this a resubmission (amended) application (A1 or A2)?

O No, original, unamended application

O Yes, A1 (First re-submission)

O Yes, A2 (Second re-submission)

A6. Approximately how many days did you spend on the following aspects of the application process?

IF A5=”NO” DO NOT DISPLAY COLUMNS A1 OR A2

IF A5=“YES- A1” DO NOT DISPLAY COLUMNS ORIGINAL APPLICATION OR A2

IF A5=”YES- A2” DO NOT DISPLAY COLUMNS A1 OR ORIGINAL APPLICATION

Task	Number of Days
	Original Application	A1	A2
A6a. Writing and editing the application (including locating, downloading, and completing the various required forms)
A6b. Compiling and preparing supporting documents for an Appendix
A6c. IF A5=YES A1 OR YES A2: Responding to reviewer critiques (including conducting additional laboratory experiments or analyses)?

A7. Was your application assigned a numerical overall impact/ priority score?

O Yes

O No

IF A7=YES CONTINUE ELSE GO TO A10

A8. Have you received a Notice of Award letter indicating that your application has been funded?

O Yes

O No

A9. How helpful was each of the following in understanding the overall impact/priority score assigned to your application?

	Very helpful 	Moderately helpful 	Somewhat helpful 	Not at all helpful 	Not applicable 
a. Resume and Summary of Discussion	O	O	O	O	O
b. Reviewer written critiques	O	O	O	O	O
c. Individual criterion scores	O	O	O	O	O
d. Discussion with my NIH Program Officer	O	O	O	O	O
e. General information on NIH’s Web sites	O	O	O	O	O

A10a. Please indicate if you agree or disagree with the following statement. If a statement does not apply to your application, please select “Not applicable.”

The Summary Statement demonstrated that the reviewers understood my proposed approach (research plan).

O Strongly agree

O Agree

O Neither agree nor disagree

O Disagree

O Strongly disagree

O Not applicable

A10b. Please indicate if you agree or disagree with the following statement. If a statement does not apply to your application, please select “Not applicable.”

Based on the written critiques in the Summary Statement, my application was evaluated by reviewers with the appropriate expertise.

O Strongly agree

O Agree

O Neither agree nor disagree

O Disagree

O Strongly disagree

O Not applicable

IF A7=YES CONTINUE ELSE GO TO A10e

A10c. Please indicate if you agree or disagree with the following statement. If a statement does not apply to your application, please select “Not applicable.”

The Summary Statement was helpful in understanding the overall impact/priority score.

O Strongly agree

O Agree

O Neither agree nor disagree

O Disagree

O Strongly disagree

O Not applicable

A10d. Please indicate if you agree or disagree with the following statement. If a statement does not apply to your application, please select “Not applicable.”

The overall impact/priority score assigned to my application appeared consistent with the comments in the Resume and Summary Discussion section.

O Strongly agree

O Agree

O Neither agree nor disagree

O Disagree

O Strongly disagree

O Not applicable

A10e. Please indicate if you agree or disagree with the following statement. If a statement does not apply to your application, please select “Not applicable.”

The bulleted comments for the individual review criteria aligned with the corresponding criterion scores.

O Strongly agree

O Agree

O Neither agree nor disagree

O Disagree

O Strongly disagree

O Not applicable

A10f. Please indicate if you agree or disagree with the following statement. If a statement does not apply to your application, please select “Not applicable.”

The individual criterion scores were helpful in understanding the strengths and weaknesses of the application.

O Strongly agree

O Agree

O Neither agree nor disagree

O Disagree

O Strongly disagree

O Not applicable

IF A7=YES CONTINUE ELSE GO TO A10h

A10g. Please indicate if you agree or disagree with the following statement. If a statement does not apply to your application, please select “Not applicable.”

The criterion scores helped me understand the overall impact/priority score.

O Strongly agree

O Agree

O Neither agree nor disagree

O Disagree

O Strongly disagree

O Not applicable

IF A7=NO CONTINUE ELSE GO TO A10i1

A10h. Please indicate if you agree or disagree with the following statement. If a statement does not apply to your application, please select “Not applicable.”

The Summary Statement helped me understand why the scientific review group did not discuss my application.

O Strongly agree

O Agree

O Neither agree nor disagree

O Disagree

O Strongly disagree

O Not applicable

A10i1. Please indicate if you agree or disagree with the following statement. If a statement does not apply to your application, please select “Not applicable.”

The Summary Statement helped me know whether or not to resubmit the application.

O Strongly agree

O Agree

O Neither agree nor disagree

O Disagree

O Strongly disagree

O Not applicable

A10i2. Please indicate if you agree or disagree with the following statement. If a statement does not apply to your application, please select “Not applicable.”

The Summary Statement helped me focus on problem areas in the application that could be corrected.

O Strongly agree

O Agree

O Neither agree nor disagree

O Disagree

O Strongly disagree

O Not applicable

A10j1. Please indicate if you agree or disagree with the following statement. If a statement does not apply to your application, please select “Not applicable.”

The individual criterion scores helped me know whether or not to resubmit the application.

O Strongly agree

O Agree

O Neither agree nor disagree

O Disagree

O Strongly disagree

O Not applicable

A10j2. Please indicate if you agree or disagree with the following statement. If a statement does not apply to your application, please select “Not applicable.”

The individual criterion scores helped me focus on problem areas in the application that could be corrected.

O Strongly agree

O Agree

O Neither agree nor disagree

O Disagree

O Strongly disagree

O Not applicable

SECTION B: Your Opinions about the NIH Peer Review Process Since 2009

When answering the questions in this section, please think of the peer review process at NIH after enhancements were made in 2009, the one under which your most recent NIH grant application was reviewed.

B1. Overall, which peer review system do you prefer – the new system (with enhancements) or the old system (without enhancements)?

O New system

O Old system

O No preference between the new or the old systems

O Not applicable – no experience with the new system

B2. How fair is the peer review process at NIH after enhancements were made in 2009?

O Very fair

O Somewhat fair

O Neither fair nor unfair

O Somewhat unfair

O Very unfair

B3. How satisfied are you with the peer review process at NIH after enhancements were made in 2009?

O Very satisfied

O Somewhat satisfied

O Neither satisfied nor dissatisfied

O Somewhat dissatisfied

O Very dissatisfied

SECTION C: Background

As a reminder, the information provided in this survey will remain anonymous. No individual respondents will be identified, and all responses will be summarized and reported in aggregate form.

C1. What type of organization do you work for?

Select all that apply

University

Research Foundation

Private Sector/For-profit Organization

Hospital/Medical Center

Federal, State, or Local Government Agency

Other Non-profit Organization

Other (specify): _____________________

C2. What is your job title or position?

O Professor

O Associate Professor

O Assistant Professor

O Adjunct Professor

O Senior Scientist

O Other (specify): _____________________

C3. Have you ever served as a peer reviewer for NIH?

O Yes

O No

IF C3=YES CONTINUE ELSE GO TO C5

C4. Have you reviewed any grant applications related to the American Recovery and Reinvestment Act (ARRA) of 2009, such as the Challenge grants program and Grand Opportunities “GO” grants program?

O Yes

O No

C5. When did you submit your first research grant application to NIH as a Principal Investigator (PI) for a single-PI or multiple-PI grant?

O 2008-2009 O 1996-1998

O 2005-2007 O 1993-1995

O 2002-2004 O 1990-1992

O 1999-2001 O Prior to 1990

THE QUESTION WORDING “PRINCIPAL INVESTIGATOR” WILL INCLUDE A HYPERLINK AND IF CLICKED THE NIH DEFINITION OF A PRINCIPAL INVESTIGATOR WILL APPEAR ON SCREEN

C6. As a PI, have you ever received a R01, R03, or R21 grant from NIH? (Please include single-PI grants and multiple-PI grants.)

 O  Yes

 O  No

IF C6=Yes CONTINUE ELSE GO TO C8

C7.  In total, how many years of NIH funding have you received as a PI on R01, R03, and R21 grants? (Please include single-PI grants and multiple-PI grants.)

Enter total number of years here  __________

C8. In which of the following fiscal years did you receive any type of NIH funding as a PI?  (Please include single-PI grants and multiple-PI grants.) 

Examples of NIH funding include research grants (R series), program project/center grants (P series), career development awards (K series), research training and fellowships (T and F series), and SBIR/STTR grants/contracts.

 

Select all that apply

 O FY 2009 (October 2008 – September 2009)

 O  FY 2008 (October 2007 – September 2008)

 O FY 2007 (October 2006 – September 2007)

 

 O  Did not receive NIH funding for the fiscal years listed

C9. Please indicate the degree(s) you have.

Select all that apply

Ph.D. or other research doctorate

M.D.

D.D.S.

D.V.M. or V.M.D.

Other (specify): _______________

C10. What is your age?

O Under 35 O 46-50 O 61-65

O 35-40 O 51-55 O 66-70

O 41-45 O 56-60 O Over 70

C11. What is your gender?

O Female

O Male

C12. What is your ethnicity?

O  Hispanic or Latino

O  Not Hispanic or Latino

C13. What is your race?

Select all that apply

O  American Indian or Alaska Native

O  Asian

O  Black or African American

O  Native Hawaiian or Other Pacific Islander

O  White

Thank you very much for completing the survey!

For more information about the peer review changes that have been implemented at NIH, please visit http://enhancing-peer-review.nih.gov/faqs.html.

If you have any ideas for improving the peer review process at NIH, please enter your suggestions at [insert URL].

ALTERNATE VERSION OF SECTIONS A AND B FOR RESONDENTS WHO HAVE NOT EXPERIENCED ENHANCED PEER REVIEW

SECTION A: Your Application Experience

Please think of the most recent application you submitted to NIH (for either a single-PI or multiple-PI grant) that was:

• reviewed and

• for which a funding decision (either not funded or funded) was reached.

Please do not include applications for administrative supplements, as these undergo administrative review instead of peer review.

A1. What was the activity code for this NIH research grant application?

An activity code refers to the 3-character code used by NIH to identify a specific category of extramural research activity, applied to various funding mechanisms (e.g., R01, P01).

Enter activity code here 

PROGRAMMING NOTE: THIS WILL BE A DROP-DOWN LIST. WE WILL NEED A COMPLETE LIST OF 3-LETTER CODES TO INCLUDE IN DROP-DOWN. RESPONDENT WOULD TYPE IN FRIST LETTER AND A LIST WILL COME UP FOR THAT LETTER SECTION.

A2. Was this the first NIH research grant application for which you were the Principal Investigator (PI)?

O Yes

O No

A3. Was this a resubmission (amended) application (A1 or A2)?

O No, original, unamended application

O Yes, A1 (First re-submission)

O Yes, A2 (Second re-submission)

A4. Approximately how many days did you spend on the following aspects of the application process?

IF A3=”NO” DO NOT DISPLAY COLUMNS A1 OR A2

IF A3=“YES, A1” DO NOT DISPLAY COLUMNS ORIGINAL APPLICATION OR A2

IF A3=”YES, A2” DO NOT DISPLAY COLUMNS A1 OR ORIGINAL APPLICATION

Task	Number of Days
	Original Application	A1	A2
A4a. Writing and editing the application (including locating, downloading, and completing the various required forms)
A4b. Compiling and preparing supporting documents for an Appendix
A4c. IF A3=YES- A1 OR YES-A2: Responding to reviewer critiques (including conducting additional laboratory experiments or analyses)?

A5. Was your application assigned a numerical priority score?

O Yes

O No

IF A5=No GO TO A8 ELSE CONTINUE

A6. Have you received a Notice of Award letter indicating that your application has been funded?

O Yes

O No

A7. How helpful was each of the following in understanding the priority score assigned to your application?

	Very helpful 	Moderately helpful 	Somewhat helpful 	Not at all helpful 	Not applicable 
a. Resume and Summary of Discussion	O	O	O	O	O
b. Reviewer written critiques	O	O	O	O	O
c. Discussion with my NIH Program Officer	O	O	O	O	O
d. General information on NIH’s Web sites	O	O	O	O	O

A8a. Please indicate if you agree or disagree with the following statement. If a statement does not apply to your application, please select “Not applicable.”

The Summary Statement demonstrated that the reviewers understood my proposed approach (research plan).

O Strongly agree

O Agree

O Neither agree nor disagree

O Disagree

O Strongly disagree

O Not applicable

A8b. Please indicate if you agree or disagree with the following statement. If a statement does not apply to your application, please select “Not applicable.”

Based on the written critiques in the Summary Statement, my application was evaluated by reviewers with the appropriate expertise.

O Strongly agree

O Agree

O Neither agree nor disagree

O Disagree

O Strongly disagree

O Not applicable

IF A5=Yes CONTINUE ELSE GO TO A8e

A8c. Please indicate if you agree or disagree with the following statement. If a statement does not apply to your application, please check the box for “Not applicable.”

The Summary Statement was helpful in understanding the priority score.

O Strongly agree

O Agree

O Neither agree nor disagree

O Disagree

O Strongly disagree

O Not applicable

A8d. Please indicate if you agree or disagree with the following statement. If a statement does not apply to your application, please select “Not applicable.”

The priority score assigned to my application appeared consistent with comments in the Resume and Summary of Discussion section.

O Strongly agree

O Agree

O Neither agree nor disagree

O Disagree

O Strongly disagree

O Not applicable

A8e1. Please indicate if you agree or disagree with the following statement. If a statement does not apply to your application, please select “Not applicable.”

The Summary Statement helped me know whether or not to resubmit the application.

O Strongly agree

O Agree

O Neither agree nor disagree

O Disagree

O Strongly disagree

O Not applicable

A8e2. Please indicate if you agree or disagree with the following statement. If a statement does not apply to your application, please select “Not applicable.”

The Summary Statement helped me focus on problem areas in the application that could be corrected.

O Strongly agree

O Agree

O Neither agree nor disagree

O Disagree

O Strongly disagree

O Not applicable

SECTION B: Your Opinions about the NIH Peer Review Process Prior to 2009

When answering the questions in this section, please think back to the peer review process at NIH prior to 2009, the one under which your most recent NIH grant application was reviewed.

B1. Overall, which peer review system do you prefer – the new system (with enhancements) or the old system (without enhancements)?

O New system

O Old system

O No preference between the new or the old systems

O Not applicable – no experience with the new system

B2. How fair was the peer review process at NIH prior to 2009?

O Very fair

O Somewhat fair

O Neither fair nor unfair

O Somewhat unfair

O Very unfair

B3. How satisfied were you with the peer review process at NIH prior to 2009?

O Very satisfied

O Somewhat satisfied

O Neither satisfied nor dissatisfied

O Somewhat dissatisfied

O Very dissatisfied

17

APPLICANT SURVEY (Version 8-18-09)