Satisfaction of Applicants (Principal Investigators) and Reviewers with NSF's Merit Review Process

Appendix A. 2019 Merit Review Survey
A. Survey Introduction
Welcome
Intro: Thank you for your participation. This survey is designed to help NSF understand the
factors that affect researchers as they submit proposals to or review proposals for NSF, and the
impact of various approaches to proposal review. Your responses will help NSF to improve its
service to the community of proposers and reviewers.
The results will be reported in such a way that no single individual can be identified. Your
response is voluntary. Your decision to participate or not to participate in this survey will not
adversely affect consideration of your pending or future proposals.
This survey should take about 20 minutes to complete.
If you have any difficulty taking this survey, please contact
[email protected] for assistance.
Please scroll down and click the blue "Forward" arrow to proceed with the survey.
Paperwork Burden Statement
OMB: This information is collected under the authority of the National Science Foundation Act
of 1950, as amended. According to the Paperwork Reduction Act of 1995, no persons are
required to respond to a collection of information unless it displays a valid OMB control number.
The valid OMB control number for this information collection is 3145-0215. The time required
to complete this voluntary information collection is estimated to average 20 minutes, including
the time to review instructions, and complete and review responses. If you have any comments
or concerns about the contents or the status of your individual submission of this questionnaire,
e-mail [email protected], or write directly to: Merit Review Survey, Insight
Policy Research, 1901 N. Moore Street, Suite 1100, Arlington, VA 22209.

5

About This Survey
Main_Intro: This survey consists of three sections. The first asks about your experiences as
someone who has reviewed proposals for NSF (if applicable), the second asks about your
experiences as someone who has submitted proposals to NSF (if applicable), and the third
contains questions that will help NSF understand how experiences may vary between subgroups
and career stages. Someone who submits a proposal to NSF, a proposer, is also called a
Principal Investigator (PI).
Q1_Intro. For the purpose of this survey, please do not count post-doctoral fellowship
applications or student fellowship applications as proposals.
Q1A. * Since October 1, 2015, have you reviewed a proposal for NSF, other than a post-doctoral
or student fellowship application?
 Yes
 No
Q1B. * Since October 1, 2015, have you submitted a proposal to NSF, other than a post-doctoral
or student fellowship application? (Do not include your experience as a co-investigator.)
 Yes
 No
[If answers to both Q1A and Q1B are ‘No’, thank them for their participation, and exit.]
Q2A-C. Since October 1, 2015, with which NSF Directorate(s) and Division(s) have your
scholarly activities been most closely affiliated? (Note: If your work aligns with more
than one, select up to three Directorate/Division combinations in the drop-down menus
below.)
Q2A1.
Q2A2.
Q2B1.
Q2B2.
Q2C1.
Q2C2.

Directorate 1
Division 1
Directorate 2
Division 2
Directorate 3
Division 3

Drop-down list of NSF Directorates:
1, BIO = Biological Sciences
2, CISE = Computer & Information Science & Engineering
3, EHR = Education & Human Resources
4, ENG = Engineering
5, GEO = Geosciences
6, MPS = Mathematical & Physical Sciences
7, SBE = Social, Behavioral & Economic Sciences
-8 = Skip
-9 = Missing
Drop-down list of NSF Divisions:
1, DBI = Biological Infrastructure
2, DEB = Environmental Biology
3, IOS = Integrative Organismal Systems

6

4, MCB = Molecular & Cellular Biosciences
5, OAC = Office of Advanced Cyberinfrastructure (formerly, Division of Advanced
Cyberinfrastructure)
6, CNS = Computer & Networking Systems
7, CCF = Computing & Communication Foundations
8, IIS = Information & Intelligent Systems
9, DGE = Graduate Education
10, HRD = Human Resource Development
11, DRL = Research on Learning in Formal & Informal Settings
12, DUE = Undergraduate Education
13, CBET = Chemical, Bioengineering, Environmental, and Transport Systems
14, CMMI = Civil, Mechanical & Manufacturing Innovation
15, ECCS = Electrical, Communications & Cyber Systems
16, EEC = Engineering Education & Centers
17, IIP = Industrial Innovation & Partnerships
18, AGS = Atmospheric & Geospace Sciences
19, EAR = Earth Sciences
20, OCE = Ocean Sciences
21, PLR = Office of Polar Programs (formerly, Division of Polar Programs)
22, AST = Astronomical Sciences
23, CHE = Chemistry
24, DMR = Materials Research
25, DMS = Mathematical Sciences
26, PHY = Physics
27, BCS = Behavioral & Cognitive Sciences
28, SES = Social & Economic Sciences
29, Other organization unit; list here _________________
-8 = Skip
-9 = Missing
[If “No” selected for Q1A, and “Yes” for Q1B, skip to PI_Intro [i.e. jump to questions for
investigators.]
[If “Yes” selected for Q1A, continue to Reviewer_Intro.]

7

[Visible only if answered “Yes” to question 1A; if “No” is selected for 1A, skip to PI_Intro]

B. Experiences as a Reviewer
Reviewer_Intro: The following questions ask about your experiences reviewing NSF proposals.
For these questions, please use the definitions below.
There are two types of reviewers:
 An ad hoc reviewer is someone who submits a written review of a proposal but does not
participate in a discussion of the proposal with other reviewers.
 A panelist, or panel reviewer, is someone who participates in a discussion of a proposal
(usually more than one proposal) with other reviewers. A panelist may or may not
prepare a written review but has access to the reviews written by others. Panelists may
meet face to face or remotely.
Q3.

Since October 2015, I have served as ____ for NSF:
 An ad hoc reviewer only
 A panelist/panel reviewer only
 Both an ad hoc reviewer and a panelist/panel reviewer

Q4.

Approximately how many reviews of individual proposals have you written for NSF
since October 1, 2015, regardless of whether as an ad hoc reviewer or a panelist? (Your
best estimate is fine; no decimals, please.) [text box]

Q5.

Approximately how many reviews of individual proposals or applications have you
written for other funding agencies since October 1, 2015? (Your best estimate is fine; no
decimals, please.) [text box]

Q6. * During the past 12 months, have you declined to…
(Note: If you have not been asked to perform one or more of these functions, please
answer “no”.)
Yes

No

Q6A. Serve as an ad hoc reviewer for NSF?
Q6B. Serve as a face-to-face panelist on an NSF review panel?
Q6C. Serve as a remote panelist on an NSF review panel?

[Visible if “yes” to any option in Q6]
Q7.

Thinking about the most recent time you declined to participate in a review, to what
extent did the following factors influence your decision?

8

To a Great Extent

To a Moderate
Extent

To a Small Extent

Not at all

Q7A.
Q7B.
Q7C.
Q7D.
Q7E.

Proposal or program was not related to my professional interests
Lack of time
Conflict of interest
Too many NSF review requests
Competing professional pressures (including teaching, organizational
administration service, etc.)
Q7F. Dissatisfaction with the proposal review process
Q7G. Increasing commitments as a reviewer to other funding agencies
Q7H. [Visible only if Q6B is “Yes”] Unable to travel to a face-to-face panel
Q7I. [Visible only if Q6C is “Yes”] Dislike participating in discussions over phone,
video-conference, or web-based meeting technology
Q7J. Some other factor (Specify):
Q8.

Thinking about the most recent time you wrote a review of an NSF proposal, please
estimate the amount of time (rounded to the nearest hour) that it took you to read the
proposal, write, and submit that single written review. Please do not count time spent
travelling to or sitting in panels.
(Please enter a whole number in the box below). [text box]

Q9.

When do you typically read proposals and write reviews of NSF proposals?
 Mainly during your normal work-day
 Mainly outside of your normal working hours
 Both during the work-day and outside your normal working hours

Q10. How does your employer view your participation as a reviewer (for NSF or other funding
agencies)?
 My employer considers my participation as a reviewer to fall within the scope of my
normal work duties.
 My employer considers my participation as a reviewer to fall outside the scope of my
normal work duties.
 I am unsure how my employer views my participation.

OverallPropQual_Intro: The following questions will ask you about your perceptions about the
quality of the proposals you have reviewed.

9

Q11.

Based on your experience reviewing proposals for NSF, to what extent do you agree or
disagree with the following statements?

Strongly Disagree

Disagree

Agree

Strongly Agree

Q11A. Overall, the majority of proposals I have reviewed in recent years have been of
high quality.
Q11B. Individuals submitting proposals are treated fairly.
RO_Intro: The following questions ask about your experience preparing to review proposals for
NSF.
Q12.

To what extent do you use the following strategies when completing proposal reviews?

To a Great Extent
Q12A.
Q12B.
Q12C.
Q12D.
Q12E.
Q12F
Q12G.
Q12H.
Q12I.
Q12J.

To a Moderate Extent

To a Small Extent

Not at All

Read the merit review criteria before you read the proposal(s)
Take notes when reading the proposal
Focus on strengths and weaknesses with respect to the review criteria
Include specific and concrete examples
Critically read your review
Actively reflect on your own thought processes
Think of alternative views
Play a devil’s advocate to your own assessment
Take time with your decision
Use some other strategy (specify): _________________

Q13. To what extent are you familiar with the following unconscious cognitive biases that can
affect reviews?
To a Great Extent

To a Moderate Extent

To a Small Extent

Not at All

Q13A. Anchoring bias: Relying too heavily on one piece of information or an initial
impression (the anchor) and neglecting subsequent information.
Q13B. Confirmation bias: Unconsciously attending to evidence that confirms our
existing beliefs or expectations.
Q13C. Halo effect: When an overall positive impression of a person’s past achievements
influences judgements of the specific merits of a proposal.
Q13D. Language bias: Tendency to judge ideas or statements from non-native speakers
more critically.
Q13E. Social stereotype bias: Unconscious and automatic thoughts and feelings about
other people influenced by social categories (e.g., age, ethnicity, race, nationality,
gender, occupation).

10

Q14. * [REVIEWER ORIENTATION FILTER] NSF recently began offering reviewer orientation
information in a 20-minute video with tips about how to prepare a high-quality review.
Have you seen this video?
 Yes
 No [skip to Q19]
 Unsure [skip to Q19]
Q15.

At what stage of the review process did you watch the reviewer orientation video? If you
watched it at multiple stages, please select all that apply.
Q15A.
Q15B.
Q15C.
Q15D.

Prior to reviewing the proposal(s)
After reading the proposal(s) but before writing my review(s)
After writing my review(s) but before participating in the panel discussion
Not sure/can’t remember [Exclusive answer]

Video_Intro: The reviewer orientation video includes three segments:
1. tips for how to prepare an analytical review,
2. a description of the merit review criteria with guidance on the broader impact criterion,
and
3. information about strategies to mitigate the effects of unconscious cognitive biases.
Q16. Please indicate the degree to which you found the information in these segments to be
helpful:
Very Helpful

Moderately
Helpful

Slightly Helpful

Not Helpful

Do Not Recall

Q16A. Tips on how to prepare an analytical review
Q16B. Guidance to reviewers on the broader impact criterion
Q16C. Information about strategies to mitigate the effects of unconscious cognitive
biases
Q17. Please indicate the degree to which you found the reviewer orientation video helpful
when you prepared your reviews:
Very Helpful

Q18.

Moderately
Helpful

Slightly Helpful

Not Helpful

Do Not Recall

Do you now recall any of the tips provided in the reviewer orientation video?
 Yes
 No

Q19. To what extent has participating as an NSF reviewer…

11

To a Great Extent
Q19A.
Q19B.
Q19C.
Q19D.

To a Moderate Extent

To a Small Extent

Not at All

Improved your understanding of the proposal process?
Provided useful information for improving your next proposal?
Influenced you to submit to another funding agency?
Discouraged you from submitting your proposals to NSF?

[Visible only if answered “Yes” to question 1B; if “No” is selected for 1B, skip to Q29]
C. Experiences as an Investigator
PI_Intro: NSF is interested in your experience seeking funding from NSF and other sources.
Please answer the following questions based on your experience as a principal investigator (PI),
not on any experience that you may have had as a co-investigator. Please think only of the
proposals you have submitted to NSF since October 1, 2015.
Q20. * Since October 1, 2015, how many proposals have you submitted to NSF? (Note: Please
enter a whole number in the box below.) [textbox]
Q21. * Since October 1, 2015, have you applied for funding from a federal agency other than the
National Science Foundation?
 Yes
 No  Skip to Q23
[Visible if Q21= “Yes”] Q22. Compared to other federal agencies' proposal submission systems,
how much effort, on the part of a researcher preparing a proposal, does it take to write
and complete a proposal in the required format and submit it to NSF?
 More Effort
 Nearly the Same Effort
 Less Effort
Q23.

Thinking about the most recent proposal you submitted to NSF, how much time did you
spend preparing (writing, formatting and submitting) the proposal?







Less than 40 hours
41 - 80 hours
81 - 120 hours
121 - 160 hours
161 - 200 hours
More than 200 hours

Q24. * Since October 1, 2015, have you received a funding decision for any proposals you
submitted to NSF?
 Yes, I have received a decision for at least 1 proposal submitted to NSF since October
1, 2015.
 No.
Q25. * Have you ever submitted a proposal to NSF that was declined?

12

 Yes
 No  Skip to PI_Sat _Intro
[Visible only if Q25 = “Yes”] Q26. To what extent did the written reviews that accompanied the
declination of one of your NSF proposals…
To a Great Extent
Q26A.
Q26B.
Q26C.
Q26D.

To a Moderate Extent

To a Small Extent

Not at All

Improve your understanding of the proposal process?
Provide useful information for revising and improving your next proposal?
Influence you to submit to another funding agency?
Discourage you from revising and submitting your proposals to NSF?

[Visible if Q24 = “Yes”] PI_Sat_Intro: For the following questions, please refer to the most
recent proposal that you submitted to NSF for which you have received an award or
decline decision.
Q27.

How satisfied or dissatisfied were you with...

Very Satisfied

Somewhat
Satisfied

Neither
Dissatisfied
nor Satisfied

Somewhat
Dissatisfied

Very
Not Applicable
Dissatisfied

Q27A. The quality of the information NSF provided during the proposal submission
process (i.e., FastLane, FAQs, web site content)
Q27B. The timeliness of the decision to award or decline funding
Q27C. Your interaction with NSF staff
Q27D. The overall quality of NSF’s merit review process
Q28.

Based on your experience submitting proposals to NSF, to what extent do you agree or
disagree with the following statements?

Strongly Disagree

Disagree

Agree

Q28A.
Q28B.
Q28C.
Q28D.
Q28E.

Strongly Agree

Not Applicable

Written reviews are thorough
Written reviews are technically sound
Overall, written reviews are of high quality
The panel summary or summaries are of high quality
The information provided regarding the outcomes of the competition is of high
quality
Q28F. The PO Comments I viewed in FastLane helped me understand the decision to
decline or award my proposal
Q28G. The conversations (email, phone, face-to-face) I had with my program officer
provided me with helpful feedback about my proposal
Q28H. Individuals submitting proposals are treated fairly

13

D. All Respondents

Q29.

Think about your collective experiences with the NSF merit review process. Based on
your experience, to what extent do the following factors influence the ratings given by
reviewers?

To a Great
Extent

To a Moderate
To a Small Extent
Extent

Not at All

Don’t Know

Q29A. PI career stage
[Show if Q29A = To a Great Extent] Please describe
_________________________
Q29B. PI geographic location
[Show if Q29B = To a Great Extent] Please describe
_________________________
Q29C. PI gender
[Show if Q29C = To a Great Extent] Please describe
_________________________
Q29D. PI institution
[Show if Q29D = To a Great Extent] Please describe
_________________________
Q29E. PI race/ethnicity
[Show if Q29E = To a Great Extent] Please describe
_________________________
Q29F. PI reputation/experience
[Show if Q29F = To a Great Extent] Please describe
_________________________
Q29G. Level of risk of proposed research
[Show if Q29G = To a Great Extent] Please describe
_________________________
Q29H. Reviewer interest in proposed research topic
[Show if Q29H = To a Great Extent] Please describe
_________________________
Q29I. Other (Please describe) _________________________

Q30.

Please indicate whether you agree or disagree with the following statements:
Strongly
Disagree

Disagree

Neither Agree
nor Disagree

Agree

Q30A. Overall, I am satisfied with NSF’s merit review process

14

Strongly Agree

Q30B. Overall, I think NSF’s merit review process is fair
Q30C. Overall, I think NSF’s merit review process is effective
Q30D. [Show if answered “Yes” to question 1A-Reviewers] Overall, I intend to continue
to review proposals for NSF in the future
Q30E. [Show if answered “Yes” to question 1B-Investigators] Overall, I intend to
continue to submit proposals to NSF in the future
Q31.

This survey has asked about your experiences with NSF’s merit review process. In your
opinion, improving which one of the following factors will have the most significant
effect on improving the merit review process? Please select one.







Q32.

Timeliness of decisions about, and responsiveness to, proposals by NSF staff
Quality of feedback to PIs in the form of comments in written reviews
Quality of feedback to PIs in the form of comments in panel summaries
Quality of PI conversations with, and written comments from, program officers
Quality of information available during proposal submission
Quality of the review process from the perspective of a reviewer

Please enter any additional comments you may have about NSF’s merit review process in
the space below: ____ [Essay text box]

Demo_Intro: The following questions will prompt you to provide basic information on your
early career experience with NSF, and your institution/organization as well as some demographic
information. If you work for multiple organizations, please pick the one you consider to be your
primary employer and answer in terms of that organization. (Data will be reported at an
aggregated level and are requested to help us understand the experiences of different groups.)
Q33. In what year did you receive your highest degree? (Please do not count honorary
degrees.) Please select a year from the drop-down menu. [drop down menu including years 1950
through 2017 + ‘before 1950’]
Q34. Did you receive any financial support (e.g. research assistantship, fellowship, traineeship,
scholarship, other grants) from NSF as an undergraduate or graduate student?
 Yes
 No Skip to Q37
 Don’t know Skip to Q37

Q35. [Visible If Q34= “Yes”] What type of financial support did you receive from NSF while
you were an undergraduate or graduate student?
Yes
No
Q35A.
Q35B.
Q35C.
Q35D.

REU (research experience for undergraduates) support
Research assistantship
Fellowship support
Traineeship support

15

Q35E. Scholarship
Q35F. Travel grant
Q35G. Other – please specify [… Text box with character limit …]
Q36.

[Visible If Q34 = “Yes”] If appropriate, please provide the name(s) of the NSF
program(s) you received support from as an undergraduate or graduate student.
[… Text box with character limit …]

Q37.

Are you a “soft-money” researcher (your appointment requires that 75% or more of the
annual salary for the research position you hold is funded by grant monies, rather than
your employer)?
 Yes
 No
 Not sure

Q38. * Do you work for an institution of higher education?
 Yes
 No  Skip to Q44
Q39.

Please select the basic classification that best describes your institution.
 Doctoral university. Includes institutions that award at least 20 research/scholarship
doctoral degrees/year. Excludes Special Focus Institutions and Tribal Colleges.
 Master's College or University. Generally includes institutions that award at least 50
master's degrees and fewer than 20 doctoral degrees/year. Excludes Special Focus
Institutions and Tribal Colleges.
 Baccalaureate College. Includes institutions where baccalaureate or higher degrees
represent at least 50 percent of all degrees but where fewer than 50 master's degrees or
20 doctoral degrees are awarded/year. Excludes Special Focus Institutions and Tribal
Colleges.
 Baccalaureate/Associate's College. Includes four-year colleges that confer more than
50 percent of degrees at the associate's level/year. Excludes Special Focus Institutions
and Tribal Colleges, and institutions that have sufficient master’s or doctoral degrees
to fall into those categories.
 Associate's College. Institutions at which the highest level degree awarded is an
associate's degree. Excludes Special Focus Institutions and Tribal Colleges.
 Special Focus Institution. Institutions where a high concentration of degrees is in a
single field or set of related fields. Excludes Tribal Colleges.
 Tribal College. Colleges and universities that are members of the American Indian
Higher Education Consortium.  Skip to Q41

Q40.

Is it a minority-serving institution (i.e., a Tribal College or University, an Historically
Black College or University, or an Hispanic-Serving Institution)? (These are Department
of Education Title IV designations for institutions of higher education that enroll
populations with significant percentages of undergraduate minority students.)
 Yes
 No

16

Q41.

Do you have tenure?
 Yes
 No

Q42.

Does your contract of employment provide fewer than 9 months of salary?
 Yes
 No

Q43.

What is your position?









Post-doctoral fellow
Assistant Professor
Associate Professor
Full Professor
Adjunct Professor
Emeritus/Emerita Professor
Retired
Other (please specify): [… Text box with character limit …]

Q44. * Which of the following best describes your organization?
 An institution of primary and/or secondary education
 Federally Funded Research and Development Center (FFRDC)  Skip to
Demographics_Info
 Other non-profit research organization  Skip to Demographics_Info
 For-profit research organization  Skip to Demographics_Info
 Industrial or commercial company  Skip to Demographics_Info
 Federal government  Skip to Demographics_Info
 State government  Skip to Demographics_Info
 Local government  Skip to Demographics_Info
 Professional society  Skip to Demographics_Info
 Other, please specify: [… Text box with character limit …]  Skip to
Demographics_Info
Q45.

Which of the following best describes your position?





Teacher or Instructor
Curriculum design specialist
Administrator
Other

Demographics_Info: The next three questions request demographic information in standard
categories that are defined government-wide. You may have wondered why you see such
requests when interacting with NSF. We use the demographic information to generate statistics
that help us gauge whether our programs and other opportunities in science and technology reach
and benefit everyone regardless of demographic category. The most helpful way to complete
these questions is to pick the category or categories that you feel best describe yourself. You are
also provided the option to not specify a category for each question.

17

Q46.

Gender:
 Female
 Male
 Prefer not to specify

Q47.

Ethnicity:
 Hispanic or Latino
 Not Hispanic or Latino
 Prefer not to specify

Q48.

Race (select one or more):
Q48A.
Q48B.
Q48C.
Q48D.
Q48E.
Q48F.

American Indian or Alaska Native
Asian
Black or African American
Native Hawaiian or Other Pacific Islander
White
Prefer not to specify [exclusive answer]

18

Appendix B: Survey Recruitment and Reminder Materials
Initial Prenotification Email (from NSF):
Dear [NAME]:
I am writing to request your participation in a survey funded by the National Science Foundation
(NSF). The purpose of this survey, entitled “Satisfaction of Applicants (Principal Investigators)
and Reviewers with NSF’s Merit Review Process,” is to provide updated data with respect to
participants’ satisfaction with various aspects of NSF’s merit review process, including reviewer
workload and other factors influencing review quality.
Your participation in the survey will help NSF ensure that the merit review process continues to
serve the needs of the Nation, the Foundation, and the proposal-writing and reviewer
communities. Your responses will help NSF to maintain the quality of the review process while
minimizing the burden on proposers and reviewers and exploring potential technological
enhancements.
Our contractor, Insight Policy Research (Insight), an independent research organization, has been
tasked with surveying all proposers and reviewers that participated in the merit review process in
fiscal year 2016 to the present. In the next few days, you will receive an email from Insight
inviting you to participate in this survey. The survey should take no longer than 20 minutes to
complete and will be open until Month Day, 2019.
I would like to thank you in advance for your important contribution to this research. Be assured
that survey responses will only be shared in the aggregate, and specific findings will not be
attributed to particular individuals. Your thoughts and feedback on the merit review process are
very important and will help the NSF continue to improve both the process and the experiences
of proposers and reviewers.
If you have questions regarding the survey, please contact the Insight Helpdesk at
[email protected].
Sincerely,
Suzanne Iacono
Office Head
Office of Integrative Activities
National Science Foundation
OMB Control Number: 3145-0215
Expiration Date: [insert date]

19

Initial Invitation Email (from Insight):
Dear NSF Reviewer or Principal Investigator:
This email is a follow-up to an email you received from Suzanne Iacono, Office Head of NSF’s
Office of Integrative Activities. Insight Policy Research (Insight) is conducting a survey on
behalf of NSF to collect data about applicants (principal investigators) and reviewers’
experiences with the merit review process.
Please respond to this survey by Month, Date 2019.
This survey will take approximately 20 minutes of your time and will ask you about your
experiences as a proposer and/or reviewer for NSF. These data will be used by NSF to improve
the merit review process.
As a reminder, survey results will only be shared in the aggregate, and specific findings will not
be attributed to particular individuals. In order to enhance the security of the information you
provide, you will be required to enter a password to access the survey. If you exit the survey for
any reason, you may click on the survey URL and enter your password again to resume your
progress.
To complete the survey, please click on the unique link provided below.
Unique survey link: [URL]

We at Insight and NSF know how busy you are, and we appreciate you sharing your time to
assist NSF in this way. If you have any questions about the survey or its administration, please
contact the Insight Helpdesk at [email protected].

Sincerely,
Meg Trucano, Ph.D.
Data Collection Lead

20

First Reminder Email (from Insight):
Dear [NAME]:
This email is a follow-up to remind you that Insight Policy Research (Insight) is conducting a
survey on behalf of NSF to collect data about applicants (principal investigators) and reviewers’
experiences with the merit review process.
Please respond to this survey by Month, Date, 2019.
This survey will take approximately 20 minutes of your time and will ask you about your
experiences as a proposer and/or reviewer for NSF. These data will be used to improve the merit
review process.
As a reminder, survey results will only be shared in the aggregate, and specific findings will not
be attributed to particular individuals. In order to enhance the security of the information you
provide, you will be required to enter a password to access the survey. If you exit the survey for
any reason, you may click on the survey URL and enter your password again to resume your
progress.
To complete the survey, please click on the unique link provided below.
Unique survey link: [URL]
We at Insight and NSF know how busy your schedules are, and we appreciate you sharing your
time to assist NSF in this way. If you have any questions about the survey or its administration,
please contact the Insight Helpdesk at [email protected].
Sincerely,
Meg Trucano, Ph.D.
Data Collection Lead
Insight Policy Research

21

Second Reminder Email (from NSF):
Dear Colleague,
I am writing again to request your participation in an important survey about the NSF merit
review process.
As someone who participated in the merit review process between fiscal years 2016 and 2018,
you should have received an email from Insight Policy Research (Insight), including a link to the
survey. If you have already completed the survey, I thank you for your time and effort. If you
have not yet completed the survey, I highly encourage you to do so as soon as possible. Your
feedback will help the NSF continue to improve both the process and the experiences of
applicants (principal investigators) and reviewers.
If you have any questions regarding the survey, please contact the Insight Helpdesk at
[email protected].

Sincerely,
Suzanne Iacono
Office Head
Office of Integrative Activities
National Science Foundation
OMB Control Number: 3145-0215
Expiration Date: [insert date]

22

Final Reminder Email (from Insight):
Dear [NAME]:
This email is to a final reminder that Insight Policy Research (Insight) is conducting a survey on
behalf of NSF to collect data about proposers’ and reviewers’ experiences with the merit review
process.
We hope that you can respond to this important survey by Month, Date, 2019.
This survey will take approximately 20 minutes of your time and will ask you about your
experiences as a proposer and/or reviewer for NSF. These data will be used to improve the merit
review process.
As a reminder, survey results will only be shared in the aggregate, and specific findings will not
be attributed to particular individuals. In order to enhance the security of the information you
provide, you will be required to enter a password to access the survey. If you exit the survey for
any reason, you may click on the survey URL and enter your password again to resume your
progress.
To complete the survey, please click on the unique link provided below.
Unique survey link: [URL]

We at Insight and NSF know how busy your schedules are, and we appreciate you sharing your
time to assist NSF in this way. If you have any questions about the survey or its administration,
please contact the Insight Helpdesk at [email protected].
Sincerely,
Meg Trucano, Ph.D.
Data Collection Lead
Insight Policy Research

23

Insight Thank You Email:
Dear [NAME]:
We would like to thank you for your time in completing NSF’s survey about your experiences
with the merit review process. Your responses are very important to NSF.
If you have any questions about the survey or its administration, please contact the Insight
Helpdesk at [email protected].
Thank you,
Meg Trucano, Ph.D.
Data Collection Lead
Insight Policy Research

24