Supporting Statement for Satisfaction of Applicants (Principal Investigators) and Reviewers with NSF’s Merit Review Process

Request for Approval under the “Generic Clearance for the Collection of Routine Customer
Feedback” (OMB Control Number: 3145-0215)
TITLE OF INFORMATION COLLECTION:
Satisfaction of Applicants (Principal Investigators) and Reviewers with NSF’s Merit Review
Process
PURPOSE:
The National Science Foundation (NSF) receives close to 50,000 proposals for funding each
year, each of which undergoes a rigorous merit review process that is designed to ensure all
proposals are fairly and thoroughly reviewed. The merit review process comprises three phases:
1. NSF announces funding opportunities on the NSF website and Grants.gov. Principal
Investigators (PIs) prepare proposals in response to these opportunities and submit their
proposals via FastLane (NSF’s web-based system for proposal submission and review) or
Grants.gov.
2. Proposals are assigned to the appropriate program(s) for review. Each proposal is
assigned a Program Officer (PO) who selects external reviewers to evaluate the proposal
according to the two NSF merit review criteria, Intellectual Merit and Broader Impacts.
The Intellectual Merit criterion encompasses the potential to advance knowledge. The
Broader Impacts criterion encompasses the potential to benefit society and contribute to
the achievement of specific, desired societal outcomes. Programs may have additional
review criteria particular to the goals and objectives of the program. The NSF guidelines
for the selection of reviewers are designed to ensure selection of experts who can give
program officers the proper information needed to make a recommendation in accordance
with the merit review criteria. POs utilize the proposal’s reference list, the investigator’s
suggested reviewers, and personal knowledge of individual reviewers to identify a pool
of diverse experts with respect to type of organization represented, demographics,
experience, and geographic balance, selecting appropriate reviewers with no apparent
potential conflicts. Most proposals are reviewed by 3–10 content expert reviewers who
provide written feedback on the proposal through FastLane. POs synthesize reviewer
comments and issue a recommendation to either decline or award funding based on
reviewer feedback, panel discussions, the amount of available funding, and portfolio
balances (i.e., the diversity of a portfolio, including factors such as award type, career
stage, demographic characteristics, geographic location, institution type, research topic,
laboratory funding status, and intellectual risk). The proposal and PO recommendation is
then forwarded to the appropriate Division Director or other NSF official for additional
review and action to either decline or award.
3. Each proposal recommended for award undergoes an administrative review conducted by
NSF’s Office of Budget, Finance, and Award Management. If it passes this review, the
proposal is awarded.
Through this review process, NSF aims to identify the highest quality proposals to receive
funding. The success of this process hinges on the assumptions that PIs will continue to submit
to NSF their ideas for cutting-edge research and that experts in their respective fields will
continue to provide high-quality reviews of those proposals.

1

The goal of this data collection is to assess the experiences of PIs and reviewers and their
satisfaction with the National Science Foundation’s (NSF’s) merit review process. The customer
satisfaction information collection for which this OMB approval is requested includes a Webbased survey that will be administered to all recent PIs and reviewers. The specific research
objectives are to—
1. Examine PI and reviewer perceptions of, and satisfaction with, the merit review process.
2. Document the time burden the proposal submission and merit review process places on
PIs and reviewers.
3. Examine PI and reviewer perceptions of the quality of reviews and of proposals.
4. Describe the extent to which NSF’s reviewer orientation pilot has been successful to date.
5. Describe the outcomes associated with participation in NSF early career programs.
Data from the survey will be used to improve customer service provided by NSF for the merit
review process.
DESCRIPTION OF RESPONDENTS:
The survey will be administered to PIs and reviewers who have participated in NSF’s merit
review process between FY 2016 and FY 2018.
TYPE OF COLLECTION: (Check one)
[ ] Customer Comment Card/Complaint Form
[ ] Usability Testing (e.g., Website or Software
[ ] Focus Group

[X] Customer Satisfaction Survey
[ ] Small Discussion Group
[ ] Other:

CERTIFICATION:
I certify the following to be true:
1. The collection is voluntary.
2. The collection is low-burden for respondents and low-cost for the Federal Government.
3. The collection is non-controversial and does not raise issues of concern to other federal
agencies.
4. The results are not intended to be disseminated to the public.
5. Information gathered will not be used for the purpose of substantially informing influential
policy decisions.
6. The collection is targeted to the solicitation of opinions from respondents who have
experience with the program or may have experience with the program in the future.
Name:__Suzanne H. Plimpton, NSF Reports Clearance Officer:
To assist review, please provide answers to the following question:
Personally Identifiable Information:
1. Is personally identifiable information (PII) collected? [ ] Yes [X] No
2. If Yes, is the information that will be collected included in records that are subject to the
Privacy Act of 1974? [ ] Yes [ ] No
3. If Applicable, has a System or Records Notice been published? [ ] Yes [X] No

2

Gifts or Payments: Is an incentive (e.g., money or reimbursement of expenses, token of
appreciation) provided to participants? [ ] Yes [X] No
BURDEN HOURS
Category of Respondent
NSF PIs and reviewers
Totals

No. of
Respondents
36,000
36,000

Participation
Time
20 minutes
20 minutes

Burden
12,000 hours
12,000 hours

FEDERAL COST: The estimated annual cost to the Federal government is $0
If you are conducting a focus group, survey, or plan to employ statistical methods, please
provide answers to the following questions:
The selection of your targeted respondents
1. Do you have a customer list or something similar that defines the universe of potential
respondents and do you have a sampling plan for selecting from this universe?
[X] Yes
[ ] No
If the answer is yes, please provide a description of both below (or attach the sampling plan)? If
the answer is no, please provide a description of how you plan to identify your potential group of
respondents and how you will select them?
NSF will provide to its contractor, Insight Policy Research (Insight), a universe file containing
the names and email addresses of all PIs and reviewers who have participated in the merit review
process from FY 2016-FY 2018. There is no sampling plan; Insight will conduct a census of all
PIs and reviewers included in the universe file provided by NSF.
Administration of the Instrument
1. How will you collect the information? (Check all that apply)
[X] Web-based or other forms of Social Media
[ ] Telephone
[ ] In-person
[ ] Mail
[ ] Other, Explain
2. Will interviewers or facilitators be used? [ ] Yes [X] No
Please make sure that all instruments, instructions, and scripts are submitted with the
request. See appendix A for the survey instrument and appendix B for survey recruitment and
reminder materials.

Instructions for completing Request for Approval under the “Generic Clearance for the
Collection of Routine Customer Feedback”

TITLE OF INFORMATION COLLECTION: Provide the name of the collection that is the
subject of the request. (e.g. Comment card for soliciting feedback on xxxx)

3

PURPOSE: Provide a brief description of the purpose of this collection and how it will be used.
If this is part of a larger study or effort, please include this in your explanation.
DESCRIPTION OF RESPONDENTS: Provide a brief description of the targeted group or
groups for this collection of information. These groups must have experience with the program.
TYPE OF COLLECTION: Check one box. If you are requesting approval of other
instruments under the generic, you must complete a form for each instrument.
CERTIFICATION: Please read the certification carefully. If you incorrectly certify, the
collection will be returned as improperly submitted or it will be disapproved.
Personally Identifiable Information: Provide answers to the questions.
Gifts or Payments: If you answer yes to the question, please describe the incentive and provide
a justification for the amount.
BURDEN HOURS:
Category of Respondents: Identify who you expect the respondents to be in terms of the
following categories: (1) Individuals or Households;(2) Private Sector; (3) State, local, or tribal
governments; or (4) Federal Government. Only one type of respondent can be selected.
No. of Respondents: Provide an estimate of the Number of respondents.
Participation Time: Provide an estimate of the amount of time required for a respondent to
participate (e.g. fill out a survey or participate in a focus group)
Burden: Provide the Annual burden hours: Multiply the Number of responses and the
participation time and divide by 60.
FEDERAL COST: Provide an estimate of the annual cost to the Federal government.
If you are conducting a focus group, survey, or plan to employ statistical methods, please
provide answers to the following questions:
The selection of your targeted respondents. Please provide a description of how you plan to
identify your potential group of respondents and how you will select them. If the answer is yes,
to the first question, you may provide the sampling plan in an attachment.
Administration of the Instrument: Identify how the information will be collected. More than
one box may be checked. Indicate whether there will be interviewers (e.g. for surveys) or
facilitators (e.g., for focus groups) used.
Please make sure that all instruments, instructions, and scripts are submitted with the
request.

4