Supporting Statement for Retrospective Survey of Participants in the National Science Foundation (NSF) International Research Ex

Request for Approval under the “Generic Clearance for the Collection of
Routine Customer Feedback” (OMB Control Number: 3145-0215)
TITLE OF INFORMATION COLLECTION:
Retrospective Survey of Participants in the National Science Foundation (NSF) International
Research Experiences for Students (IRES) Program
PURPOSE:
The purpose of the retrospective survey is to learn about former IRES participants’ characteristics,
experiences and satisfaction with program, and educational and employment outcomes. The survey
will be designed, developed, and administered by Mathematica. It will be administered once with one
cohort of former program participants and results will be used to inform program improvement and
report to OMB. Mathematica will analyze the survey data and share results with NSF in a report.
DESCRIPTION OF RESPONDENTS:
Respondents are young adults who participated in the IRES program as undergraduates or graduate
students through awards made by NSF in 2013.
TYPE OF COLLECTION: (Check one)
[ ] Customer Comment Card/Complaint Form
[ ] Usability Testing (e.g., Website or Software
[ ] Focus Group

[X] Customer Satisfaction Survey
[ ] Small Discussion Group
[ ] Other: ______

CERTIFICATION:
I certify the following to be true:
1. The collection is voluntary.
2. The collection is low-burden for respondents and low-cost for the Federal Government.
3. The collection is non-controversial and does not raise issues of concern to other federal
agencies.
4. The results are not intended to be disseminated to the public.
5. Information gathered will not be used for the purpose of substantially informing influential
policy decisions.
6. The collection is targeted to the solicitation of opinions from respondents who have
experience with the program or may have experience with the program in the future.
Name:__Suzanne H. Plimpton, NSF Reports Clearance Officer_______________________
To assist review, please provide answers to the following question:
Personally Identifiable Information:
1. Is personally identifiable information (PII) collected? [X] Yes [ ] No
2. If Yes, is the information that will be collected included in records that are subject to the
Privacy Act of 1974? [X] Yes [ ] No
3. If Applicable, has a System or Records Notice been published? [ ] Yes [ ] No
Gifts or Payments:
Is an incentive (e.g., money or reimbursement of expenses, token of appreciation) provided to
participants? [X] Yes [ ] No

1

We expect difficulties in obtaining an adequate response rate among IRES participants about 5.5
years after participation. A review of similar surveys—focused on participants in programs offering
research experiences to undergraduate and/or graduate students—suggests that response rates range
from 40 to nearly 80 percent, with only those that administered the survey immediately after or
within a year of participation reaching response rates above 70 percent (Lopatto 2004, 2007; Russell
et al. 2006). Instead, surveys administered several years after participation, and not offering
incentives, reach much lower response rates. For example, Martinez et al. (2015) surveyed
participants in the NSF Partnerships for International Research and Education (PIRE) program up
to 10 years of participation and had a response rate of 40 percent among those who participated as
undergraduates and 54 percent among graduate-level participants. Similarly, Fitzsimmons (1990)
surveyed participants in the NSF Research Experiences for Undergraduates (REU) program after 3
years of participation and obtained a response rate of 53 percent.
To encourage participation, IRES participants who complete the survey will be offered a $20 gift
card as a token of appreciation. The proposed incentive is modeled on previous incentives used in
this study. In developing a survey for NSF participants in IRES’s sister program—REU—students
participating in roughly one-hour sessions of usability testing received a $40 gift card. Expecting the
IRES survey to take less than half of that time, we propose to offer $20 as a thank you.
BURDEN HOURS
Category of Respondent
Participants
Principal Investigators
Totals

No. of Respondents
243
14
257

Participation Time
0.3 hours
1.5 hours

Burden
73 hours
21 hours
94 hours

Explanation:
Participants: Respondents are former IRES participants in the 2013 cohort of awards—defined as
the 243 students supported through 14 awards made by IRES in 2013. (Two additional awards were
made by IRES in 2013, but the projects supported through these awards did not report supporting
students.). The survey will contain a total of 46 items (including screening questions), most of which
are multi-choice or dropdown menu questions. Based on preliminary testing, we expect that the
survey will take 15 to 20 minutes to complete. We will confirm through further testing once the
survey is programmed and adjust as needed to stay within the target time limit.
Principal Investigators: Respondents are principal investigators (PIs) who led the IRES award for
the 2013 IRES cohort of participants. We will be contacting PIs to provide the most recent email
address or other contact information they might have for former participants. Because contact
information reported in the NSF annual reports is likely outdated, engaging PIs in the location effort
will be critical. We assume PIs will spend about 5 minutes per student searching their own emails or
social media accounts to find more updated information. There are 14 awards in the 2013 cohort
and, on average, each supported 17.4 student participants. Therefore, we expect PIs to spend a total
of 1.5 hours on this task.
FEDERAL COST:
The estimated annual cost to the Federal government is $127,509

2

Explanation:
This estimate includes the cost to develop the survey instrument; program the survey in Confirmit—
a secured online survey platform; locate participants, and administer the survey (including follow-up
calls, incentive payments, and technical help-desk support).
If you are conducting a focus group, survey, or plan to employ statistical methods, please
provide answers to the following questions:
The selection of your targeted respondents
1. Do you have a customer list or something similar that defines the universe of potential
respondents and do you have a sampling plan for selecting from this universe?
[X] Yes
[] No
If the answer is yes, please provide a description of both below (or attach the sampling plan)? If
the answer is no, please provide a description of how you plan to identify your potential group of
respondents and how you will select them?
Explanation:
We obtained information on all former participants in the 2013 cohort from NSF (through the annual
reports submitted by PIs). Given the small cohort size (N=243), we will survey the universe of
participants to ensure we obtain responses from students of different background characteristics and
can produce valid estimates.
We selected the 2013 cohort of awards for two reasons. The first is that program guidance changes as
of 2012 resulted in better reporting of information about student participants in annual and final
reports. This enhances our ability to identify participants to survey. The second reason is that selecting
this year (2013) ensures that enough time elapsed since program participation (5.5 years) for key
outcomes to occur, such as graduation from undergraduate or graduate school and entry into further
education or employment.
Administration of the Instrument
1. How will you collect the information? (Check all that apply)
[X] Web-based or other forms of Social Media
[ ] Telephone
[ ] In-person
[ ] Mail
[ ] Other, Explain
2. Will interviewers or facilitators be used? [ ] Yes [X] No

3