Common Application and Registratin Pilots - Reference Writers - Common Application (Questions 26-37)

NATIONAL SCIENCE FOUNDATION

SUPPORTING STATEMENT FOR PAPERWORK REDUCTION ACT SUBMISSION

NATIONAL SCIENCE FOUNDATION

RESEARCH EXPERIENCES FOR UNDERGRADUATES PROGRAM

A. Justification

1. Necessity for the Data Collection

This study is being conducted by Mathematica Policy Research on behalf of the National Science Foundation (NSF) and in support of the Research Experiences for Undergraduate (REU) program. The REU program was created in 1987 to strengthen the science, technology, engineering, and mathematics (STEM) workforce. Building on research experiences as “one of the most effective avenues for attracting students to and retaining them in science and engineering, and for preparing them for careers in these fields,” the program is designed to foster student research and promote diversity (NSF 13-542). The program provides awards to principal investigators (PIs) at institutions—mostly universities and research centers—throughout the nation. Called REU Sites, these institutions engage groups of about 10 to 12 students in ongoing research programs or in research projects designed for the REU program, and operate mostly in the summer.

The main goal of the current study is to pilot test alternative approaches to collecting data required by Congress in the America COMPETES Reauthorization Act of 2010, which states that students in the REU program must “be tracked, for employment and continued matriculation in STEM fields, through receipt of the undergraduate degree and for at least three years thereafter” (Section 514[a][6] of Public Law 111-358). The legislation also mentions specific demographic characteristics of participants that need to be collected, such as gender, ethnicity, and enrollment in a two-year college. In addition to needing these data to report to Congress, NSF program officers (POs) and leadership need a more robust data system to enhance their efforts to monitor participation in the program and eventually to assess its effectiveness.

Data will be collected mostly through a newly designed REU Data System. The design of this system is based on (1) a careful review of the legislation (to ensure testing of a collection system that responds to the Congressional requirement) and the feasibility study conducted by the Science and Technology Policy Institute (Zuckerman et al. 2016); (2) existing REU Site proposals, reports, and current applications (to design a system that is tailored to the REU program); (3) the scholarly literature (to inform the general approach to data collection and enhance the NSF’s ability to use it in the future for evaluations); and (4) the Graduate Research Fellowship Program common application (GRFP; for benchmarking).

The present study includes designing the system and pilot testing it with a sample of REU Sites that volunteer to participate. By participating in this study, these Sites will have the opportunity to experience the data collections first hand and provide feedback that will be used to determine which approach will be the most effective, most efficient, and least burdensome for possible future implementation across all REU Sites.

Description of the Pilot

The pilot includes:

1. Testing two web-based approaches to obtain basic student background and participation information (see the Appendix A for specifications):

• Registration. The registration will be designed to collect the basic demographic and contact information needed for analysis and tracking purposes. Students wishing to participate in the REU program will be asked to register at a website through which they will submit information and obtain a unique ID. With this unique ID, they will then apply directly to the REU Sites using the existing Site application processes. Staff at REU Sites will later use the IDs provided by students to record application decisions and participation status of admitted applicants. Site staff will also be able to use the system to obtain data submitted by students who applied to their Sites.

• Common Application (including the Registration). The common application will be modeled after existing REU Site applications and will replace those applications for the upcoming 2018 application cycle among participating Sites (for applications to participate in the program in 2019). The common application will enable students to apply to multiple Sites through one application.

Students will first complete the REU Registration described above, and then proceed to the common application through which they will submit additional information commonly required by Sites as part of their applications, such as transcripts. The system will also allow some level of customization for Sites to include additional requirements. Staff at REU Sites will use the system to provide information needed by potential applicants, retrieve applicant information, record application decisions and participation status among admitted applicants, and produce reports and run queries of data submitted by applicants to their Sites.

2. Obtaining and integrating educational and employment information. The study will follow the subset of rising seniors who participate in the REU program in 2019 in the disciplines included in the pilot (rising seniors are the large majority of participants) to:

• Obtain educational outcomes information from the National Student Clearinghouse (NSC)

• Administer a survey to obtain information on employment outcomes (among those not pursuing studies at the time of the survey)

3. Conducting visits at a few REU Sites participating in the pilot to interview principal investigators/coordinators and mentors and conduct focus groups with REU students. Conducted when most programs are operating in the summer (in this case, summer 2019), the site visits will be used to obtain in-depth feedback on the registration, common application, and data analytics tools available through the REU Data System for PIs.

The draft employment survey and site visit protocols (student focus groups, and PI/coordinator and mentor interviews) are included in Appendix B and C, respectively.

Description of Respondents

• College/university students interested in applying to the REU program at the pilot Sites. Students at two- and four-year colleges and universities are the target population of the REU program. Sites recruit incoming freshmen and rising sophomores, juniors and seniors. Through the registration or common application, these respondents will be asked to submit basic demographic and educational background information.

• Reference letter writers. Applicants in the common application will be required to provide contact information of two reference letter writers. The REU Data System will send reference writers an email requesting a reference letter in support of their application to the program.

• Principal investigators (PIs) or staff designated by the PIs of participating Sites. PIs or their designees will submit (1) information useful to prospective applicants (such as the opening and closing dates of applications) and (2) decisions/participation information (they will indicate which students were admitted and which later participated in the program at their Site). This information will be submitted through a module designed for PIs.

• Fomer REU Site participants. After students participate in the REU program, we will follow them to see if they can be found in the NSC and, if so, obtain educational participation and outcomes such as graduation from an undergraduate program and enrollment in graduate school. We will then seek to gather data on employment outcomes among those not enrolled in education programs or not found in the NSC.

2. Use of Information

The information collected through the pilot will enable NSF and other program stakeholders to assess these alternative approaches to collecting data needed to respond to the congressional requirement and meet the information needs of REU program officers and NSF leadership. These are:

• Registration + educational outcomes from the NSC + employment outcomes through a survey

• Common application + educational outcomes from the NSC + employment outcomes through a survey

These approaches will be analyzed based on several factors—including data quality, user burden, and user feedback—and compared against the current approach used by NSF to collect data.

3. Improved Information Technology to Reduce Burden

The pilot will create one online application to test both the registration and the common application. This online application will:

• Reduce application burden for students applying to the REU Program, as the same application can be submitted to multiple Sites (at present, most Sites require that applicants follow each Site’s application procedures)

• Reduce burden on Site PIs and administrators who will no longer need to administer their own applications

• Provide analyses of applicant and participant data that is not readily available, is helpful to PIs, and is needed by NSF to monitor the program and comply with the America COMPETES Reauthorization Act; obtaining these data would otherwise generate additional burden for PIs and NSF staff

In addition, the pilot will include obtaining educational outcomes in a way that creates no burden for respondents (by relying on existing information available through the NSC).

4. Efforts to Identify Duplication

To minimize duplication, the registration and common application are designed to complement each other, including mutually-exclusive data items that are necessary for the pilot. In addition, PIs participating in the pilot will be asked to use the system to obtain the data they need, rather than requesting that applicants also submit the same information directly to the Sites. To this end:

• Registration. REU PIs participating in the registration pilot will get to preview the system before it is launched. They will also be informed ahead of time of the information to be collected through the registration to facilitate revisions to their applications and minimize the chances of collecting duplicate data.

• Common application. To ensure that the system captures the information they need and avoid concurrent application efforts led by program Sites participating in the pilot, PIs will be encouraged to preview the online application and customize it (1) to include additional questions they may wish to ask of applicants or (2) to request additional documentation not included in the common application already.

5. Efforts to minimize burden on small business

Small businesses are not affected by this information collection.

6. Consequences of Less Frequent Data Collection

This collection is needed to test alternative ways of collecting data needed to meet the congressional reporting requirements and monitor the REU Program. NSF commissioned a feasibility study to assess whether its current data collection approach would be adequate to meet the congressional requirements and consider alternative ways of collecting needed information. This feasibility study was conducted by the Science and Technology Policy Institute (STPI) and concluded that

“new data collection will be required, as the status quo of [REU] participants providing demographic information to NSF’s Research Performance Report System, coupled with voluntary tracking of participants’ career choices by the REU [principal investigators], is clearly insufficient to meet the [congressional] mandate” (Zuckerman et al 2016)

Based on an analysis of more recent cohorts of REU participants (other than those included in the STPI study) who responded to NSF’s new data collection through Research.gov (STPI analyzed data from earlier data collections through FastLane), Mathematica arrived at the same conclusion as STPI.

7. Special Circumstances

Not applicable.

8. Federal register announcement and consultation outside the agency

a. Federal Register announcement

A 60-day notice to solicit public comments was published in the Federal Register,

_PLACEHOLDER for NSF to update once the notice is posted_.

Comments received will be included in this section.

b. Consultation outside the agency

NSF commissioned a feasibility study to assess its ability to respond to the congressional mandate with its existing data collection efforts. As mentioned earlier, this study was conducted by STPI. NSF officers and Mathematica staff involved in this study reviewed the STPI report and attended their presentation at NSF—during which they were able to ask questions regarding issues raised and suggestions made in the report. An expert advisory board will also be convened to obtain feedback prior to launching the pilot.

9. Payment or gifts to respondents

Ten college students and five REU PIs will test the application before it is launched. The students will be offered a gift of $40 for their participation. This amount is roughly equivalent to four hours of time at minimum wage—including about 2 hours to test the system, 1 hour debrief to share experiences using the system, and 1 hour for logistics (coordination and review of background information).¹ PIs will not receive a gift for testing the system as they will be current beneficiaries of the REU program.

Other than payments to the ten college student application testers, no incentives or payments will be given to respondents.

10. Assurance of confidentiality

The collection of this information is a part of all regular applications to Sites participating in the REU Program. Applicants’ information will be maintained in accordance with the requirements of the Privacy Act of 1974. No personal information will be released to the public. The system includes two notices:

The first one is based on the notice displayed in the NSF GRFP collection (OMB control number 3145-0023) and appears when users first access the system to obtain a user ID and a password. The notice reads as follows:

Notice

An agency may not conduct or sponsor, and a person is not required to respond to, an information collection unless it displays a valid Office of Management and Budget (OMB) control number. The OMB control number for this collection is XXXX-XXXX. Public reporting burden for this collection of information is estimated to average XX hours for registering and YY hours for registering and submitting an application, including the time for reviewing instructions. Send comments regarding the burden estimate and any other aspect of this collection of information, including suggestions for reducing this burden, to:

Suzanne Plimpton

Reports Clearance Officer

Division of Administrative Services                                                             

National Science Foundation

Arlington, VA 22230

Please note that information provided through the REU Data System will be used for admissions decisions, audits, and research and evaluation purposes. All applicants’ information will be maintained in accordance with the requirements of the Privacy Act of 1974. No personal information will be released to the public.

The second notice appears when users certify and submit their registration or application. This notice reads as follows:

By clicking on the SUBMIT button below, I am certifying that the information provided is true and complete to the best of my knowledge.  I understand that I am consenting to the confidential use of the information I provided for admissions decisions, audits, and research and evaluation purposes. 

11. Justification for sensitive questions

Information regarding applicants’ characteristics (such as gender, ethnicity and educational achievement) will be collected, as this information is needed by REU Sites to make admissions decisions and by NSF to report to Congress and monitor the program. Note that REU Sites normally collect this information as part of their own application processes.

Date of birth will also be collected to ensure proper identification of applicants and request educational information (enrollment and graduation information) from the NSC, which is important to comply with the legislative mandate while not increasing burden on participants.

Note that these data are also collected as part of the NSF GRFP common application (OMB control number 3145-0023).

12. Estimate of respondent burden

Table A1. Estimates of respondent burden by respondent type

Category of Respondent		No. of Respondents	Participation Time	Annual Burden (hours)
Common Application and Registration Pilots
REU Applicants – Registration		8,436	.5 hours	4,218
REU Applicants – Common Application		6,865	12 hours	82,380
Reference Writers– Common Application		13,730	.5 hours	6,865
REU PIs (or their designated staff) who will complete and use the PI module		263	8.9 hours	2,341
Site Visits**
PI/coordinator interview		6	1 hour	6
Mentors small group interview		12	.5 hour	6
Student focus group		36	1 hour	36
Educational Outcomes
This information will be obtained from the NSC, creating no burden for former REU participants	1,578 (excluded from the total count)		0	0
Employment Survey**
REU Participant Employment Survey (estimate includes college students included in the field test)		1,113	under 15 minutes	278
Totals		30,461		96,130 hours

Supporting information for Table A1

1. REU Applicants participating in the Registration

Number of respondents: Based on responses from NSF POs and REU PIs, we estimate that 145 REU Sites will participate in the registration pilot—121 Sites in Biology and 24 Sites in Earth Sciences. Based on past experience and existing NSF data, the NSF REU Program Coordinator estimates that, on average, students apply to 3.3 Sites and Sites in these disciplines receive 192 applications. We therefore estimate 8,436 unique system users in the registration pilot (192 applications per Site multiplied by 145 Sites and divided by 3.3 to identify unique applicants).

Participation time (number of burden hours per respondent): We estimate that the subset of 26 mostly multi-choice questions included in the registration would take a respondent about .5 hours to complete, assuming about 1 minute per question and 4 to 5 minutes to answer a handful questions needed to obtain a username and create a password.

2. REU Applicants participating in the Common Application

Number of respondents: Based on responses from NSF POs and REU PIs, we estimate that 118 REU Sites will participate in the common application pilot—98 Sites in Engineering and 20 Sites in Mathematics. Students applying to these Sites will complete the questions included in the registration before proceeding to additional items needed to submit their full applications through the common application. Based on past experience and existing data, the NSF REU program coordinator estimates that, on average, students apply to 3.3 Sites and Sites in these disciplines receive 192 applications. We therefore estimate 6,865 unique system users in the common application pilot (192 applications per Site multiplied by 118 Sites divided by 3.3 to identify unique applicants).

Participation time (number of burden hours per respondent): Based on the NSF GRFP common application, we estimate that it will take 12 hours to complete the full application.

3. Reference Writers– Common Application

Number of respondents: Students using the common application are required to submit contact information for two reference writers. The REU Data System contacts reference letter writers by email. Reference writers will be asked to complete only one form for each applicant that names them as a reference, regardless of the number of Sites to which students apply. We estimate 6,865 applicants in the common application and, therefore, at most 13,730 reference writers.

Participation time (number of burden hours per respondent): We estimate that it will take 0.5 hours for reference writers to draft a short paragraph about the applicant and rate the applicant on 11 characteristics (by selecting check-boxes).

4. REU Principal Investigators (PIs) in Registration and Common Application

Number of respondents: We estimate that a total of 263 PIs or their designees—one for each Site—will provide information through the REU Data System that will house the registration and common application. This includes 121 Sites in Biology, 24 in Earth Sciences, 98 in Engineering, and 20 in Mathematics.

Participation time (number of burden hours per respondent): We also estimate that the PI burden is 8.9 hours for each Site. This number comes from the time it takes PIs to:

1. Provide information about their Site and submit additional application requirements (24 items at a rate of 1.25 minutes per item = 0.5 hours per Site)

2. Record applicant IDs and admission and acceptance decisions for Sites in the registration (3 items per each of the 192 applicants at a rate of 1 minute per item = 9.6 hours per Site)

3. Record admission and acceptance decisions for Sites in the common application (2 items per each of the 192 applicants at a rate of 1 minute per item = 6.4 hours per Site)

4. Record participation information (1 item for 12 participants at the rate of 1 minute per item = 0.2 hours)

Weighing the estimates in b) and c) by the number of Sites piloting each approach to data collection (145 Sites for the registration and 118 for the common application)

5-7. Site Visits

Number of respondents: We estimate a total of 6 PIs and 12 mentors will be interviewed (two per Site) and 36 students will participate in focus groups (6 per Site).

Participation time (burden hours per respondent): We will conduct one-hour interviews with PIs/coordinators and one-hour focus groups with students. Interviews with mentors will last 30 minutes (if individual) to an hour (if small group).

8. Educational Outcomes

Number of respondents: We will obtain educational outcomes data from the NSC. We will request data for individuals who (1) participated in the common application or registration pilots, (2) were selected and participated in the REU program, and (3) were rising seniors at the time of their participation. We assume that approximately 10 students participate in each of the 263 Sites in the pilots and nearly 60% of them are rising seniors (based on STPI’s analysis), resulting in a total of 1,578 students (10*263*.6).

Participation time (burden hours per respondent): Not applicable, as the data are available through the NSC.

8. Employment Survey

Number of respondents: The employment survey will be administered to students who (1) participated in the common application or registration pilots, (2) were selected and participated in the REU program, (3) were rising seniors at the time of their participation, and (4) are not enrolled in further education the year after obtaining a bachelor’s degree. Given that the program seeks to serve 10 to 12 per Site, and some Sites enroll 8 to 10 students, we assume that approximately 10 students participate in each of the 263 Sites in the pilot, nearly 60% of them are rising seniors (based on STPI’s analysis), and about 70% are not enrolled in school.² In addition, 8 students will field test the survey (2 in each participating discipline). This results in a total of 1,113 survey respondents ((10*263*.6*.7) +8).

Participation time (burden hours per respondent): The survey will take no more than 15 minutes to complete. This time should be sufficient to answer the few questions needed to comply with the congressional requirement, plus a few questions needed to meet the information needs of NSF leadership and program officers. We will seek to keep the survey short to avoid unnecessary burden and improve response rates and data quality. Note that the literature does not indicate a strong correlation between survey length and response rates. However, research does suggest that “stated” survey length of time is associated with lower response rates, and survey length influences data quality of questions positioned later in the survey (Galesic and Bosnjak 2009).

13. Cost Burden to Respondents

There are no direct costs to respondents. The cost to respondents involves solely the time spent completing their REU applications (which they would incur in the absence of this pilot) and any follow-up surveys.

14. Cost Burden to the Federal Government

The estimated cost of this data collection is $980,562, which includes the design, development, and implementation of the REU Data System for the 2019 REU cycle.

15. Reason for Change in Burden

Not applicable. This is a new information collection.

16. Schedule for information collection and publication

Task	Date
REU Registration and Common Application Pilot	August 2018
Site Visits	Summer 2019
Interim Report	Winter 2019-2020
National Student Clearinghouse	Fall 2020
Employment Follow-up Survey	Spring-Summer 2021
Final report	Winter 2021

17. Display of OMB expiration date

The expiration date for Office of Management and Budget (OMB) approval will be displayed as shown in section 10 of this document.

18. Exception to the certification statement

There are no exceptions to the certification statement.

B. Collections of information employing statistical methods

1. Respondent universe and sampling methods

• REU Data System

Universe. The respondent universe for the REU Data System are:

• College and university students interested in applying to the REU program (registration pilot) or to the Sites participating in the pilot (common application). These individuals will submit background information described in Part A as part of their application to the REU program.

• Principal investigators (PIs) of the Sites participating in the pilot or their designees (such as co-PIs or Site administrators). These individuals will be asked to provide background information on their Sites (information useful to prospective applicants or NSF), admission decisions (who was admitted to their Site), and participation information (who actually participated in the REU program at their Site).

Sampling methods. We will not use sampling as Sites need information from every applicant. Consequently, the system will collect basic information from the universe of about 8,436 students who wish to submit an application to an REU program Site in biology and earth sciences in summer 2019, and additional information from the subset of 6,865 students who wish to submit an application to an engineering or mathematics Site that is participating in the pilot (restricted to applications for summer 2019). A justification for the estimated numbers of students is provided in section 12 of Part A of this request. Note that collecting this information from the universe of applicants does not add cost or burden to the government or respondents. Indeed, it should reduce cost and burden on Sites and burden on respondents by providing an online, centralized alternative to the applications currently offered by individual Sites (see section 3 of Part A of this request for more details).

• National student clearinghouse (NSC)

Universe. 1,578 rising seniors participating in the registration and common application pilots for the 2019 program cycle. These students are expected to graduate in 2020. In 2020, we will obtain information on educational outcomes (enrollment and graduation) from the NSC.

Sampling methods. We will not use sampling given that the information we seek is available in a centralized location (the NSC) at zero burden for the participants and a low marginal cost for additional records ($.60 per student) after the initial 1,000 records are purchased at $1 per student. We also want to ensure adequate representation of groups that are traditionally underrepresented in STEM (minorities, students with disabilities, and others).

• Employment Survey

Universe. 1,113 rising seniors participating in the registration and common application pilots for the 2019 program cycle who, based on the NSC data, are not enrolled in undergraduate or graduate programs (or this outcome is unknown).

Sampling methods. We will not use sampling because (1) this is a relatively small population and (2) we seek to obtain an accurate measure of the expect response rate for the future. Note that the logic of this pilot is to test a system that can be easily implemented by NSF in the future. The information collected—including a non-response bias analysis comparing survey data to NSC and registration/common application data—will enable the study team to make a recommendation to NSF regarding whether and how to sample in future data collection rounds, if this approach to data collection is adopted.

2. Procedures for the collection of information

a. Statistical methodology for stratification and sample selection

Sites participating in the pilot were selected in two stages. In the first stage, we identified disciplines that may be good candidates for the pilot (note that the REU program is organized into 11 mostly disciplinary groups, with Sites supported in each discipline ranging from 6 to 121). To do so, we analyzed NSF awards data and consulted with NSF POs who are knowledgeable about the Sites in their disciplines. We assessed disciplines to ensure (1) the discipline had a large enough number of Sites to offer informative findings that would be useful for other disciplines, (2) the disciplines were mostly self-contained (this was important to reduce potential confusion among applicants to disciplines included and excluded from the pilot, such as astronomy that has content overlaps with physics and Sites in both disciplines may receive applications from the same individuals), and (3) the discipline did not already have a common application that would be a confounding factor for the pilot. These conditions resulted in the exclusion of the following disciplines: computer sciences and advanced cyberinfrastructure; education and human resources; and social, behavioral, and economics sciences.

In the second stage, we offered NSF POs the possibility of contacting PIs across the remaining 8 disciplines to gauge interest in participating in the pilot (participation is voluntary). Five program officers invited PIs in their disciplines to take a brief survey and indicate their interest in participating in the pilot selected by the program officer (as indicated in Table 1; OMB control number 3145-0215). This yielded 4 disciplines selected for inclusion in the pilot based on two criteria: (1) high interest among PIs (37 to 72 percent of Site PIs responded to the survey and, on average, 92 percent of respondents expressed interest in participating) and (2) large enough numbers of Sites to run a robust pilot and receive feedback from representatives of key program constituencies. The disciplines selected for the pilot are: Biology and Geosciences (Earth Sciences³) for the registration, and engineering and mathematics for the common application.

Table 1. Principal Investigator Survey Results (selected disciplines in bold)

Discipline	Pilot	Total Active Sites a	Surveys Completed	Volunteer to Participate
Biology	Registration	121	76	72
Engineering	Common Application	98	71	25
Mathematics	Common Application	58	29	25
Geosciences	Common Application	71	26	18
Chemistry	Registration	72	20	18
Astronomy	Registration	21	0
Physics	Registration	50	0
Materials	Registration	64	0

^a Total active awards were obtained from FastLane and represent sites active as of May 31, 2016.

b. Data collection

• REU Data System. This is a web-based system to which users will have access for different purposes and at different points in time:

Student applicants will be instructed to access the electronic system to complete their registrations or applications. Information will be disseminated widely through the NSF website, the Site websites, emails sent to all REU PIs (who will, in turn, inform prospective applicants), and announcements made at PIs meetings (PIs in some disciplines, such as biology, hold discipline-wide meetings virtually, in person near NSF, or at national/disciplinary conferences widely attended by academics).

The system will be launched in the Fall of 2018 to coincide with the earliest opening of applications by Sites that start accepting applications in September and October. Applications to individual Sites will be accepted within the dates specified by PIs (to respect the current practice of allowing Site PIs to determine the application period).

• NSC. Mathematica will prepare and submit to the NSC a request for data on REU participant enrollment and graduations a year after participation in the pilot.

• Employment Survey. In alignment with current best practices in data collection, we will use a multi-mode contact protocol to guide respondents to a web-based employment survey. We will attempt to reach participants via email, text (if they consent) and, if needed, postal mail to take them to a survey instrument that will be dynamically optimized for a range of screen sizes, including computers and mobile devices (Dillman 2017; Finamore and Dillman 2013).

3. Methods to maximize response rates and deal with nonresponse

The pilot is designed to determine what response rate is possible using only low-cost outreach strategies while maximizing data quality.

• REU Data System. Non-response is not a concern for the common application, as students will be asked by PIs to submit their applications through the system. Non-response is a concern, however, among registration users as it requires enforcement by PIs, who will be asking students to complete their Sites’ application processes in addition to the registration. Participating PIs will be asked to (1) adapt their applications to avoid redundancies, as they will have access to student information submitted online by students who later apply to their Sites, and (2) require that all applicants complete the registration and provide them with their user ID. PIs will need the students’ IDs to obtain information through the system and record admissions and participation decisions. To encourage adherence with these guidelines, we will host webinars with participating PIs to introduce them to the system, share detailed guidelines regarding participation in the pilot, and answer questions. We will also share guidelines in writing among all participating PIs. The pilot will test the success of this approach and elicit feedback to consider improving the system in the future if NSF choses to adopt it.

• NSC. Non-response is not a concern, but non-matches is. The purpose of the pilot is, in part, to test the feasibility of obtaining educational outcomes information on REU participants through the NSC.

• Employment Survey. We will ask NSF to send (or ask PIs to send) an email announcing the survey to former REU participants and encouraging participation. Mathematica will then send an initial welcome email to all sample students that includes information about the study and a link to access the survey. Students will be given 8 weeks to complete the survey and nonresponders will receive several reminders. If participants consent, we would also like to send text message invitations. In addition, in weeks 4 and 8 of data collection, we will forward to the REU program a list of nonrespondents, requesting its support in encouraging the students to complete the survey. We will also consider sending a mailed paper invitation with a link and PIN in a later contact. Research suggest that, although this strategy does not increase responses rates significantly, it may encourage responses from different types of respondents who are not well represented among the initial Internet responses (Dillman 2017). A postal service mail contact attempt may also have a better chance of being opened and read by respondents who receive a large volume of email.

4. Tests of Procedures or Methods to be Undertaken

• REU Data System. We will conduct two types of pre-tests—alpha testing to ensure the system is working properly, and beta- or usability-testing to ensure the content works as intended, that is, items are eliciting the responses we seek. Specifically:

Alpha-testing. The study team will conduct testing of the web-based system to ensure that 1) the site functions properly, 2) the user interface is friendly and clear, and 3) the business logic aligns with specifications. Site functionality testing will involve testing the flow of the site between pages, verifying the ability to enter and retrieve appropriate information based on user role, and ensuring that the data collection capabilities of the site work properly and securely with website’s attached database. Site user interface testing ensures that the site’s buttons, tabs, scroll bars, text boxes and other inputs are laid out in an intuitive way and function properly. Business logic verification ensures that all text content and status reports are aligned with the business needs for the instrument and the specifications which document those business needs. The goal of these tests is to ensure that the products—the registration and the common application—work properly before we move to the stage of beta-testing.

Beta- or usability-testing. We will pre-test the registration and common application with intended users. Both were designed based on existing REU applications and modeled after the NSF GRFP common application (OMB control number 3145-0023). We will request fast-track clearance using NSF’s existing active generic approval for information collection (3145-0215). We plan to reach out to 10 college students (5 will test the registration and 5 will test the common application) and 5 PIs (at least one in each of the disciplines included in the pilot).We will also share a link to the system with the 29 NSF POs associated with this work or the REU program (this includes our contracting officer representative and NSF REU POs) for them to review the system if they wish to do so (all are government employees). Before launching the collection, we will make revisions to the system based on feedback received from students, PIs, and NSF POs.

• NSC. Not applicable as, for many years, the NSC has been collecting and providing the data we will be requesting.

• Employment survey. Pretesting the web-based employment survey is vital to the integrity of data collection. We will conduct alpha and beta testing similar to that described earlier for the REU Data System. The goal of the alpha testing will be to ensure that the survey behaves as intended. The goal of the beta-testing will be to assess how respondents understand the terms and questions presented in the survey, assess the accuracy and relevance of the questions, determine if any important questions are missing, and determine the length of time the survey takes to complete. We will test the survey with 8 former REU participants, 2 in each of the 4 participating disciplines.

5. Individuals consulted on statistical aspects of the design

Dr. Michael Sinclair, senior fellow at Mathematica, is a statistician who was formerly with the U.S. Department of Justice and Census Bureau and has extensive expertise in areas such as sampling methodology, survey weighting and imputation, and data linkage.

Andrew Weiss, vice president at Mathematica, is an expert in data collection systems, research operations, and survey methodology.

REFERENCES

Baum, S., and P. Steele. “Who Goes to Graduate School and Who Succeeds?” 2017. Available at https://www.urban.org/sites/default/files/publication/86981/who_goes_to_graduate_school_and_who_succeeds_1.pdf. Accessed February 1, 2018.

Dillman, D. “The Promise and Challenge of Pushing Respondents to the Web in Mixed-Mode Surveys.” Survey Methodology, vol. 43, no. 1, 2017, pp. 3–30.

Finamore, J., and D.A. Dillman. “How Mode Sequence Affects Responses by Internet, Mail, and Telephone in the National Survey of College Graduates.” Presentation to the European Survey Research Association, Ljubljana, Slovenia, July 18, 2013.

Galesic, M., and M. Bosnjak. “Effects of Questionnaire Length on Participation and Indicators of Response Quality in a Web Survey.” Public Opinion Quarterly, vol. 73, no. 2, 2009, pp. 349–360.

National Science Foundation. “REU Program Solicitation NSF 13-542.” 2013. Available at https://www.nsf.gov/pubs/2013/nsf13542/nsf13542.pdf. Accessed February 1, 2018.

U.S. Department of Education. “Employment and Enrollment Status of Baccalaureate Degree Recipients 1 Year After Graduation: 1994, 2001, and 2009.” 2016. Available at https://nces.ed.gov/pubs2017/2017407.pdf. Accessed February 1, 2018.

Zuckerman, B., J. Doyle, A. Mudd, T. Jones, and G. Davis. “Assessment of the Feasibility of Tracking Participants from the National Science Foundation’s Research Experiences for Undergraduates (REU) Sites Program.” Final report. Washington, DC: STPI, 2016.

APPENDIX A

DRAFT

REU Electronic Data System Overview

This document describes the REU electronic data system—which includes a students’ log-in page (to obtain the REU ID), a registration (to obtain basic descriptive information on program applicants), and a common application (to obtain additional information needed by principal investigators (PIs) to make admissions decisions). The system also includes a module for PIs to enter and access data.

The system design was informed by several sources: (a) the REU feasibility study conducted by Science and Technology Policy Institute (STPI); (b) the NSF Graduate Research Fellowship Program (GRFP) common application system; (c) information included in the Research Experience for Undergraduate (REU) sites’ websites; (d) REU program documents; (e) interviews with NSF program directors; and (f) the literature on monitoring and evaluation of research experiences for undergraduates.

This appendix is organized into 4 sections:

• Section 1 provides an overview of the key sections of the system under the Applicant and the PI modules.

• Section 2 presents the full list of data elements proposed for the Applicant module (Table 1) and the PI module (Table 2).

• Sections 3 and 4 provide the draft items for the Applicant and PI module, respectively.

Note that data elements:

• Will be revised based on feedback from NSF program officers, REU PIs, and students who test the system.

• Are intentionally drawn from national surveys (for future benchmarking) or the NSF GRFP collection (as NSF seeks to harmonize collections across programs to facilitate aggregated analyses)

1. Overview of Modules in the REU Data System

APPLICANT MODULE

Create an account/Log-in

This section requires that applicants provide a valid email address and enables them to create an account and obtain information needed to log into the REU data system. It includes instructions to obtain an REU ID, reset a password, or retrieve their REU ID. More details are provided in Table 1.

This module also enables verification of students’ email addresses before they enter the other sections in the system.

Registration

The main goal of this module is to collect the basic information that NSF needs for basic monitoring and to comply with the Congressional requirement.

This module requests information from applicants that can be used to identify unique individuals, match them to National Student Clearinghouse records (to obtain educational outcomes information), monitor program eligibility, and track applicants over time. It includes demographic information (such as gender and citizenship) and current college enrollment (such as college name and years completed).

The last question in the registration module asks applicants to identify the discipline to which they are applying. This question will be used as a filter to sort students into the registration or common application pilots as follows:

• Students who select the discipline that is participating in the registration pilot will be asked to “certify and submit” the information provided and instructed to apply to REU sites using their REU ID.

• Students who indicate that they are applying to Sites in the discipline participating in the common application will experience a seamless transition into the common application module.

Common Application

The purpose of this module is to allow students to apply to sites in the discipline participating in the pilot. To this end, this module includes:

• Questions about previous REU participation and prior academic achievement

• Additional information (Sites to which students wish to apply; a personal statement, a resume and college transcript, and the names of two references); note that transcripts will be unofficial and reference letters will be obtained electronically through an email sent to referees through the system with instructions to provide a reference.

• Additional space or functions (space for text responses and instructions to upload documents) for applicants to answer questions or provide additional materials requested by individual Sites

Certify and Submit

The purpose of this module is to ensure applicants certify that the information provided is true and complete and consent to the confidential use of their information. The “certify and submit” section applies to applicants in both pilots and reads as follows:

By clicking on the SUBMIT button below, I am certifying that the information provided is true and complete to the best of my knowledge. I understand that I am consenting to the confidential use of the information I provided for admissions decisions, audits, and research and evaluation purposes.

PI MODULE

Through the PI module, PIs and their designated program staff administrators will be able to:

• Customize the application to meet their needs by inserting or uploading additional requirements (beyond those included in the common application)

• Provide information that may be useful to applicants (such as expected start and end dates of the research experience at the given Site) or that signal important Site requirements (such as application start and end dates)

• Obtain information (for example, view and download applicant data and generate data reports)

PIs in disciplines participating in the registration will be asked to provide information on students who applied, were admitted, and participated in research at their Sites (the system will allow them to enter this information manually or to upload a spreadsheet with applicant ID numbers).

PIs in the Sites using the common application will be asked to provide information on admissions and participation.

2. REU Data System Data Elements

The tables below present the list of data elements proposed for the system:

• Table 1 presents the tentative list of elements to be collected from applicants.

• Table 2 presents the tentative list of elements to be collected from PIs.

Table 1. Data Elements Included in the REU Data System— Applicant Module

Sections	Data Element
Create an account/Log-in	Personal Information Name Date of birth Email address	Password and security questions Password Security questions 1-10
Registration (Pilot 1)	Demographic Information (Questions 1 – 13) Name (pre-populated from account creation) Date of birth (pre-populated from account creation) Citizenship Email address (pre-populated from account creation) Cell phone number Address and contact information Race and ethnicity Gender Veteran status Disability status	Current enrollment (Questions 14 – 20) Name of college or university Enrollment status Degree pursued Primary field of study Completion date or expected completion date Years of college completed Additional Information (Questions 21 – 25) Financing undergraduate degree Education and occupation of applicant’s parents or guardians Awareness of REU program REU discipline to which they intend to apply
Common application (Pilot 2)	Application (Questions 26 – 37) Prior REU program participation Prior research experience (outside of REU) College GPA REU sites applicant is applying	Personal statement Transcript CV or resume References’ contact information Additional materials requested by Sites
Certify & Submit	Certify & Submit (For both pilots)

Table 2. Data Elements Included in the REU Data System—Principal Investigator Module

Sections	Data Element
Create an account/Log-in	Personal Information Name Email address	Password and security questions Password Security questions 1-10
REU Site/PI/Users information	REU Award Information (Questions 1-6) REU Award Number REU Division (populated from Fastlane) NSF Directorate (populated from Fastlane) REU Site Institution Name (populated from Fastlane) REU Title (populated from Fastlane) REU Abstract (populated from Fastlane) PI information (Questions 7- 10) PI Name PI Institution NSF ID ORCID Membership and ID	REU Site Information (Questions 11-17) REU Site Name REU Site Location REU Site Website Number of Years REU Site in Operation REU Site 2019 Cycle Schedule Site Open to Foreign Applicants REU Site Priorities Additional users Name, email, position (up to 5 additional users)
Application information (Common application pilot only)	Program application requirements (Common application pilot only) Application open and close date Admissions decisions timeline, notification strategy Additional application requirements (optional)
Applicants and Participants	Applicants REU_ID (Pre-populated with student applications in common application pilot; entered by PI in registration pilot) Admission decision Applicant accepted offer	Participants (Pre-populated with accepted offer from applicant table) Participation decision
Data Download, Reports, and Feedback	Data Download and Reports Download all application data or subset of the data Data reports available end of summer 2019	Feedback Open text box for feedback

3. Applicant Module

Personal Information

1. Name

   1. First name

   2. Middle name

   3. Last name

   4. Suffix

   5. Previous last name

2. Date of birth

3. Citizenship

 Citizen or national of the United States

 Permanent resident of the United States (green card holder)

 Neither a citizen/national nor a permanent resident of the United States (green card holder)

4. Email address

   1. Primary email address

   2. Alternate email address

5. Cell phone number

6. Current mailing address

Country

Street address 1

Street address 2

City

State

ZIP code

7. Permanent mailing address

   1. Mother’s Contact Information

Country

Street address 1

Street address 2

City

State

ZIP code

2. Father’s Contact Information (if different from mother’s)

Country

Street address 1

Street address 2

City

State

ZIP code

8. Other contact information (optional)

   1. Facebook

   2. Linkedin

   3. Twitter

Demographic Information

9. Race (select one or more)

□ American Indian or Alaskan Native

□ Asian

□ Black or African American

□ Native Hawaiian or Other Pacific Islander

□ White

□ I do not wish to provide this information

10. Ethnicity

 Hispanic or Latino

 Not Hispanic or Latino

 Unknown

 I do not wish to provide this information

11. Gender

 Male

 Female

 I do not wish to provide this information

12. Veteran Status

 Yes

 No

 I do not wish to provide this information

13. Disability

What is the usual degree of difficulty you have with…	None	Slight	Moderate	Severe	Unable to do	I do not wish to provide this information
a. Seeing words or letters in ordinary newsprint (with glasses/contact lenses, if you usually wear them)?	1 	2 	3 	4 	5 	6 
b. Hearing what is normally said in conversation with another person (with hearing aid, if you usually wear one)?	1 	2 	3 	4 	5 	6 
c. Walking or using stairs without human or mechanical assistance?	1 	2 	3 	4 	5 	6 
d. Lifting or carrying something as heavy as 10 pounds, such as a bag of groceries?	1 	2 	3 	4 	5 	6 
e. Concentrating, remembering, or making decisions because of a physical, mental, or emotional condition?	1 	2 	3 	4 	5 	6 

Current Enrollment

14. College or University [Drop-down list populated with all IPEDS institutions]

15. Other institution (if not listed in drop-down)

Name of your college or university

City

State or Province

Country

16. Enrollment status at this college/university

 Full-time

 Part-time

 I am not currently enrolled

17. Degree program at this college/university

 Associate’s degree

 Bachelor’s degree

 Other degree (please specify)

 None of the above, I am just taking courses

18. Primary field of study at this college/university

[Drop-down with National Survey of College Graduates (NSCG) fields of study]

Other Primary Field (please specify, if not listed in drop-down)

19. Expected date of bachelor’s degree completion

If you are currently enrolled in a four year institution, or are currently enrolled in community college and plan to transfer to a 4-year institution, please enter the date in which you expect to graduate from a 4-year institution.

20. Years of college you will have completed by July 2019

 0 years of college (incoming freshman)

 1 year of college (freshman)

 2 years of college (sophomore)

 3 years of college (junior)

 4 years of college (senior)

Additional Information

21. How are you financing your undergraduate degree? (Mark one response per row.)

 Not applicable, I am not enrolled in an undergraduate program or studies
 I do not wish to provide this information
	YES	NO
Financial assistance from parents, spouse, other relatives, not to be repaid		
Financial assistance from your employer		
Financial assistance from the Veterans Educational Assistance Act (i.e., the G.I. Bill)		
Loans from parents or other relatives, to be repaid		
Loans from the school you attended, banks, federal or state government		
Tuition waivers, fellowships, grants, scholarships		
Assistantships or work study		
Earnings from employment		
Personal savings		
Other (Specify) _______ (text field)		

22. What is the highest level of education completed by your parents or guardians? (Mark one response for each parent.)

	Mother or female guardian	Father or male guardian
Less than high school completed		
High school diploma or equivalent		
Some college, vocational, or trade school (including 2-year degrees)		
Bachelor’s degree (e.g., BS, BA, AB)		
Master’s degree (e.g., MS, MA, MBA)		
Professional degree (e.g., JD, LLB, MD, DDS, DVM)		
Doctorate (e.g., PhD, DSc, EdD)		
Not applicable		
I do not know or I do not wish to provide this information		

23. Which of the following best describes your parents’ or guardians’ occupation?

	Mother or female guardian	Father or male guardian
Biological/Life Scientists		
Clerical/Administrative Support Occupations		
Clergy/Other Religious Workers		
Computer Occupations		
Consultants		
Counselors		
Engineers/Architects		
Engineering Technologists/Technicians/Surveyors		
Farmers/Foresters/Fisherman		
Health Occupations		
Lawyers/Judges		
Librarians/Archivists/Curators		
Managers and Supervisors, First-Line		
Managers, Top-level Executives/Administrators		
Managers, Other		
Management-Related Occupations		
Mathematical Scientists		
Physical Scientists		
Research Associates/Assistants		
Sales/Marketing Occupations		
Service Occupations, Except Health		
Social Scientists		
Social Workers		
Teachers – Precollege		
Teachers/Professors – Postsecondary		
Teachers – Other		
Writers/Editors/Public Relations Specialists/Artists/Entertainers/Broadcasters		
Other		
Not applicable		
I do not know or I do not wish to provide this information		

24. How did you hear about the REU Program? (Select one or more responses.)

□ Professor

□ Friend

□ Internet

□ Career fair

□ Other. Please specify: _________

□ I do not wish to provide this information

25. What is the discipline of the REU Program you are applying to? (Select all that apply.)

□ Biology

□ Engineering

□ Mathematics

□ Earth Sciences

Common Application Additional Information

26. Have you participated in the REU program before?

 Yes

 No

 I do not wish to provide this information

27. [If Q26=Yes]

Please provide information about the REU Sites in which you had a research experience:

REU Site institution name:

Year of participation:

Professor first name:

Professor last name:

REU Site institution name:

Year of participation:

Professor first name:

Professor last name:

28. Have you had any prior research experiences outside of the NSF REU Program?

 Yes

 No

29. [If Q28 = Yes]

Please describe your prior research experiences.

30. What is your college’s GPA scale?

 4.0

 5.0

31. What is your current overall college GPA?

Common Application Materials

32. Please identify the REU Sites to which you are applying.

33. Personal Statement (text box—500 words maximum)

34. Transcript (upload)

35. CV or resume (upload)

36. References

Reference 1:

First Name

Last Name

Position

Institution

Relationship to Applicant

Email

Phone

Reference 2:

First Name

Last Name

Position

Institution

Relationship to Applicant

Email

Phone

37. Additional Materials and Questions
    Site 1

    1. Additional Materials (Link to PDF provided by PI)

    2. Additional Questions (Displays questions added by PI)

Site 2

1. Additional Materials (Link to PDF provided by PI)

2. Additional Questions (Displays questions added by PI)

Site 3

1. Additional Materials (Link to PDF provided by PI)

2. Additional Questions (Displays questions added by PI)

Site 4

1. Additional Materials (Link to PDF provided by PI)

2. Additional Questions (Displays questions added by PI)

Site 5

1. Additional Materials (Link to PDF provided by PI)

2. Additional Questions (Displays questions added by PI)

Site 6

1. Additional Materials (Link to PDF provided by PI)

2. Additional Questions (Displays questions added by PI)

Site 7

1. Additional Materials (Link to PDF provided by PI)

2. Additional Questions (Displays questions added by PI)

If you need to submit additional information to your sites, you have two options:
Option 1: Use the space below to upload any additional documentation as a pdf file.
Option 2: Use the space provided below to provide any additional information requested by your site(s).

1. Site 1: Option 1 or Option 2

2. Site 2: Option 1 or Option 2

3. Site 3: Option 1 or Option 2

4. Site 4: Option 1 or Option 2

5. Site 5: Option 1 or Option 2

6. Site 6: Option 1 or Option 2

7. Site 7: Option 1 or Option 2

Common Application Reference Form (Completed by References)

Please tell us about yourself:

Name ___________________

Title/Position ___________________

Institution ___________________

Department ___________________

Please tell us about [FIRST_NAME] [LAST_NAME] (pre-populated from common application)

How long and in what capacity have you known the applicant?

Months:

Capacity:

How would you rate the applicant in overall ability and potential in comparison to others at the same academic level with respect to pursuing undergraduate research? (double click the box to check)

 Top 5%

 Top 10%

 Top 25%

 Top 50%

 Below 50%

Please rate the applicant on the following abilities and skills:

	Outstanding	Good	Fair	Poor	Unable to judge
Intellectual ability					
Integrity					
Work habits					
General motivation					
Leadership					
Creativity					
Ability to work with others					
Maturity					
Writing skills					
Verbal communication					

Please provide your opinion on the applicant’s academic performance, research aptitude, and other factors that you consider relevant for the applicant to be successful in this program and his/her future career. Provide examples where appropriate.

4. Principal Investigator Module

Site and PI Information

1. REU Award Number
   *If you have more than one active REU Site award, please report on the award that you will use to support REU participants in the 2019 cycle.

2. REU Division (pre-populated Fastlane after Award Number is entered)

3. NSF Directorate (pre-populated Fastlane after Award Number is entered)

4. REU Site Institution name (pre-populated Fastlane after Award Number is entered)

5. REU Title (pre-populated Fastlane after Award Number is entered)

6. REU Abstract (pre-populated Fastlane after Award Number is entered)

7. REU Principal Investigator Name (pre-populated Fastlane after Award Number is entered)

8. Institution of Principal Investigator

   1. Country [semi/pre-populated]

   2. Street Address 1 (pre-populated Fastlane after Award Number is entered)

   3. City (pre-populated Fastlane after Award Number is entered)

   4. State (pre-populated Fastlane after Award Number is entered)

   5. Zip Code (pre-populated Fastlane after Award Number is entered)

9. NSF ID

10. ORCID Membership (optional)
    Please indicate if you are a member of ORCID.
    
     Yes

 No

 I do not wish to provide this information

ORCID ID Number: _____________

REU Site Information

11. REU Site Name
    Please write the name of your site as you would like it displayed on the list of sites to which students may apply.

12. *REU Site Location
    □ Same as “Institution of Principal Investigator” [if box is checked, address (a-e) will be pre-populated]

    1. Country

    2. Street Address 1

    3. City

    4. State

    5. Zip Code

13. REU Site website (if available)

Please copy and paste the address of your Site’s website.

14. Number of years your REU Site has been operating (including the current year)
    Please enter all years this site has operated under your leadership or that of other PIs or Co-PIs.

15. REU Site Schedule for 2019 cycle

    1. Program start date: [mm/dd/yyyy] □ Expected [Checkbox]

    2. Program end date: [mm/dd/yyyy] □ Expected [Checkbox]

□ Program dates to be determined

16. Site welcomes applications from foreign students

To be eligible for the NSF REU program, students must be citizens or permanent residents of the United Sates by the time the program starts. Mark “Yes” if your Site has other sources of funding and is willing to consider applications from students ineligible to receive REU funds.

 Yes

 No

17. REU Site priorities

Identify student groups that your site prioritizes in recruiting REU participants.

Years of college completed by July 2019 (mark only one)

 0 years of college (incoming freshman)

 1 year of college (freshman)

 2 years of college (sophomore)

 3 years of college (junior)

 4 years of college (senior)

 We do not prioritize based on years of college completed

Students enrolled at: (check all that apply)

 Two-year colleges, including community, technical and junior colleges

 Tribal colleges

 Historically-black colleges and universities

 Hispanic-serving institutions

User Designation

Authorize Additional Users: Please identify below the individuals you authorize to access the REU data system. They will be sent an email inviting them to register to this site. Note that they will have administrative rights to enter, change, and view information associated with your REU Site.

Staff 1 Name

Staff 1 Position [Dropdown to Select Position]
 Co-PI

 Program Manager

 Administrative Staff

Staff 1 Email

Staff 2 Name

Staff 2 Position [Dropdown to Select Position]
 Co-PI

 Program Manager

 Administrative Staff

Staff 2 Email

Staff 3 Name

Staff 3 Position [Dropdown to Select Position]
 Co-PI

 Program Manager

 Administrative Staff

Staff 3 Email

Staff 4 Name

Staff 4 Position [Dropdown to Select Position]
 Co-PI

 Program Manager

 Administrative Staff

Staff 4 Email

Staff 5 Name

Staff 5 Position [Dropdown to Select Position]
 Co-PI

 Program Manager

 Administrative Staff

Staff 5 Email

Application Info – Common Application

1. Application timeline

   1. Application open date (mm/dd/yyyy)

   2. Application close date or deadline (mm/dd/yyyy)

Your program application will automatically open and close on these dates. However, you will be able to close applications at any point after they are opened using the ‘Close Applications’ button on the main page.

2. Application decisions

   1. Rolling admissions
      Please indicate if applications to your Site will be reviewed and decisions made on a rolling basis.
      
      

 Yes

 No

 I do not wish to provide this information

2. If admissions are NOT rolling, please provide the date on or after which admissions decisions may be communicated to applicants: [mm/dd/yy]
   
   

3. Please indicate if decisions will be communicated to:

 All applicants

 All admitted applicants only

3. Additional Application Requirements (Optional)
   
   

Upload a PDF

If you wish to request additional information from applicants beyond that covered in the common application, you may upload a PDF document with instructions and/or add questions below. Applicants will be prompted to view the document uploaded or the questions inserted when they select your Site and will be able to respond to your instructions by (1) uploading additional materials (in PDF format) or (2) entering text of up to 500 words to respond to each question you added.

Add questions
Please enter up to three additional application questions specific to your REU site.

Applicant Page – Registration Pilot

Please enter the REU IDs of all applicants to your summer 2019 REU Site program and indicate whether they were admitted to your program and whether they accepted your offer to participate.

REU IDs are required for all applicants. Students receive these IDs when they register in the system. Please, do not review any application without an REU ID. If you receive an application without an REU ID, remind the applicant to first go to [url for webpage] to obtain his/her REU ID.

You may enter information for a single applicant by clicking the “add new applicant” and then editing information in the table, as needed. To upload information for multiple applicants, please click “add/edit multiple applicants.”

Add New Applicant – Registration Pilot

To add new applicants, the only element that is needed in the file is REU ID. The rest of the information can be provided at a later time by clicking “edit” on a record on the applicants table or through “add/edit multiple applicants”.

Applicant Name: [Pre-populated with REU ID from applicant module data]

Applicant Home Institution: [Pre-populated with REU ID from applicant module data]

Admission Decision:[drop down menu]

 Blank (default)

 Admitted

 Rejected

 Wait Listed

 Not Reviewed

Applicant Accepted Offer? [drop down menu]

 Not Applicable (default)

 Yes

 No

Add/Edit Multiple Applicants – Registration Pilot

Step 1: Download applicant file

Download all applicants displayed on the Applicants tab by selecting a file type and clicking the “download file” button.

Step 2: Update the downloaded file

Any applicant information can be edited in the spreadsheet and uploaded. New applicants can also be added directly to the spreadsheet and uploaded. Once you are finished updating information in the spreadsheet, return to this screen to upload the updated file.

Step 3: Upload the file

Note: Uploading this file will add new applicants to the table but will not delete any REU IDs currently displayed on the applicant table. To add new applicants, the only element that is needed in the file is REU ID.

Uploading this file will replace information (admissions and acceptance decisions) on existing applicants currently displayed on the applicants table. This action cannot be undone, but if necessary you can contact [email protected] to attempt to retrieve prior data.

Applicants Page– Common Application Pilot

REU Site Applications and Admissions

[pre-populated [REU_site] from ‘Site and PI info’]

The table below shows the applicants to your 2019 REU site. Please indicate whether each applicant was admitted to your program and if admitted, indicate whether he/she has accepted your offer to participate.

You may enter information for a single applicant directly in the table below. To upload information for multiple applicants, please click “edit multiple applicants.”

If you have uploaded a letter under the application info tab, you use the “send” feature in the table below to communicate your admission decisions to applicants.

View Filtered Table [Collapse/Expand]

All Data
 All
Demographic Characteristics [COLLAPSE / EXPAND]
Gender  All □ Female □ Male Race  All □ American Indian or Alaskan Native □ Asian □ Black or African American □ Native Hawaiian or Other Pacific Islander □ White □ Not reported Ethnicity  All □ Hispanic or Latino □ Not Hispanic or Latino □ Unknown □ Not reported	Veteran Status  All □ Yes □ No □ Not reported Disability Status  All □ Yes □ No □ Not reported Citizenship Status  US Citizen □ US Permanent Resident □ Neither US citizen nor US resident
Educational information [COLLAPSE / EXPAND]
Years of college completed by July 2019  All □ None (incoming freshman) □ 1 year of college (freshman) □ 2 years of college (sophomore) □ 3 years of college (junior) □ 4 years of college (senior) College or University Level  All □ 4-year (four or more years) □ 2-year (at least 2 but less than 4 years) □ Less than 2 years (below associate) □ Missing	College or University Sector  All □ Public □ Private not-for-profit □ Private for-profit □ Missing
Application Status [COLLAPSE / EXPAND]
Admission Decision  All □ Accepted □ Rejected □ Waitlisted □ Not Reviewed □ Missing

Edit Multiple Applicants Page– Common Application Pilot

Step 1: Download applicant file

Download all applicants displayed on the Applicants tab by selecting a file type and clicking the “download file” button.

Step 2: Update the downloaded file

Any applicant information can be edited in the spreadsheet and uploaded. New applicants can also be added directly to the spreadsheet and uploaded. Once you are finished updating information in the spreadsheet, return to this screen to upload the updated file.

Step 3: Upload the file

Note: Uploading this file will add new applicants to the table but will not delete any REU IDs currently displayed on the applicant table. To add new applicants, the only element that is needed in the file is REU ID.

Uploading this file will replace information (admissions and acceptance decisions) on existing applicants currently displayed on the applicants table. This action cannot be undone, but if necessary you can contact [email protected] to attempt to retrieve prior data.

Participants Page

REU Site Participants

The table below shows the names of the applicants who you previously identified as having accepted your offer to participate in the REU program at your Site. Please confirm that they participated and indicate how they were funded.

Participants of [Self-populated site name]

REU ID	Accepted Applicant Name	Accepted Applicant home institution	Participant Status Participation codes1
(pre-filled based on accepted offer from Applicant table)	(pre-filled based on accepted offer from Applicant table)	(pre-filled based on accepted offer from Applicant table)	 Missing (Default)  Participant fully funded with NSF REU funds  Participant partially funded with NSF REU funds  Participant fully funded with other funds  No Show  Dropout

Participation codes

Participant: The student completed at least 160 hours of the REU experience. Please indicate whether participant is fully, partially, or not funded with funds from the NSF REU program.

No Show: The student did not come to the Site when the program started. Use this code for students who withdrew from the program before the program started or simply did not appear to the Site when the program started.

Dropout: The student attended the program but did not complete at least 160 hours of participation. Use this code for rare instances where a student had to discontinue attendance due to personal or disciplinary reasons.

Data Download Page

Data Download

You may download all available data for applicants and participants to your site. You may also download a subset of the data by using the “Select Variables” and “Select Students” filters below.

Select Variables [COLLAPSE/EXPAND]

Select the variables you would like to download:

All Variables

 All

List of variables available to download

• Registration Columns

• Application / Participation status

Select Students [COLLAPSE/EXPAND]

All Students
 All
Application and Participation Status
Application Status  All □ Accepted □ Rejected □ Waitlisted □ Not Reviewed □ Missing	Participation Status  All □ Participant □ No Show □ Dropout □ Missing
Demographic Characteristics
Gender  All □ Female □ Male Race  All □ American Indian or Alaskan Native □ Asian □ Black or African American □ Native Hawaiian or Other Pacific Islander □ White □ Not reported Ethnicity  All □ Hispanic or Latino □ Not Hispanic or Latino □ Unknown □ Not reported	Veteran Status  All □ Yes □ No □ Not reported Disability Status □ Yes □ No □ Not reported Citizenship Status  US Citizen □ US Permanent Resident □ Neither US citizen nor US resident
Educational information
Years of college completed by July 2019  All □ None (incoming freshman) □ 1 year of college (freshman) □ 2 years of college (sophomore) □ 3 years of college (junior) □ 4 years of college (senior) College or University Level  All □ 4-year (four or more years) □ 2-year (at least 2 but less than 4 years) □ Less than 2 years (below associate) □ Missing	College or University Sector  All □ Public □ Private not-for-profit □ Private for-profit □ Missing

Data Reports Page

Data reports showing descriptive statistics from the pilot’s applicants and participants will become available at the end of the 2019 summer. You will be notified by email when data reports are available. Stay tuned!

Feedback Page

We want to hear from you!

Please share your experience using the REU Data System. We are interested in (1) what worked well, (2) what did not work well, and (3) what was missing (that is, functionality or information you wanted and was not available).

Please use the box below to share your comments or contact us at [email protected] or 1-800-232-8024. We welcome comments at any time!

1 Source of minimum wage information: https://www.dol.gov/whd/minwage/america.htm

2 Available estimates indicate that, among bachelor’s degree recipients, about 30% enroll in further education within a year of completing their graduate degree (U.S. Department of Education 2016) and 39% enroll in a graduate program within four years of graduating from college (Baum and Steele 2017).

3 Earth Sciences, a subdivision of Geosciences, ultimately preferred to be considered for the registration pilot.

16