3145-0121 2006 Part A

Supporting Statement for Paperwork Reduction Act Submissions

Section A. Justification

A1. Circumstances Making Information Collection Necessary

The United States depends on technological leadership to sustain economic growth and national security. It is thus essential to the nation to assure the availability of well-trained scientists and engineers. Critical to providing this assurance is the need to encourage undergraduates to pursue graduate degrees in science, technology, engineering, and mathematics (STEM) and, subsequently, careers in those fields

Chief among the programs intended to increase the graduate degree production in fields covered by the National Science Foundation (NSF) is the Research Experiences for Undergraduates (REU) Program, which in 2007 will have been in existence for 20 years. Since the REU program began, several studies were done specifically about the program in 1989 and the early 1990s. These early studies included surveys of former REU students and principal investigators (PIs). In 2002, NSF’s Division of Research, Evaluation, and Communication (REC) commissioned a nationwide study entitled Undergraduate Research Opportunities (URO). While it contained numerous components, the main part of the study was an examination of all of NSF’s mechanisms that support undergraduate research, including REU Sites and Supplements. The URO study covered a wide range of NSF programs and directorates, but did not obtain detailed information about any of them. Accordingly, NSF’s Directorate of Engineering (ENG) would like to obtain information about its REU programs that is comparable to that provided by the earlier and broader studies. This will be the first study in many years to examine the activities, outcomes, and impacts of REU awards made in a single directorate.

A2. Purposes and Use of Information

ENG has two major funding sources for REUs. REU Site awards, funded by the Division of Engineering Education and Centers (EEC), are independent proposals to conduct projects that engage students in research. They may be focused on a single discipline/academic department or on related interdisciplinary/multi-department research. Site participants, half of whom must attend an institution other than the one hosting the Site, typically participate for 8 to 10 weeks in a full-time hands-on research experience during the summer. REU Supplements may be requested for ongoing NSF-funded research awards or may be included as a component of proposals for new or renewal NSF grants or cooperative agreements. Most REU Supplements support one or two students and take place during the academic year. Although funded as Supplements, Engineering Research Center (ERC) Supplements support an average of about 10 students and take place during the summer. Thus, from a functional perspective, ERC Supplements are more similar to REU Sites than to “typical” Supplements.

The ENG REU program officers want to obtain in-depth information about the activities, outcomes, and impacts of ENG Supplement and Site awards from the perspectives of the former REU students, PIs, co-PIs, and other faculty mentors. It is anticipated that the study results will help NSF better understand the components and characteristics of effective REUs and thus will help provide direction to ENG’s REU program officers in their reviews of REU proposals and in the advice they give to REU PIs.

The study will assess differences between REUs in the various ENG Divisions and, specifically, between EEC REU Sites and ERC Supplements. In addition, the study will assess differences among respondent groups (undergraduates, PIs, other faculty mentors) and, for undergraduates, will assess differences by gender, race/ethnicity, and total duration of undergraduate research experiences.

The study will address the following research topics:

• Students’ satisfaction with various aspects of their REU research experiences

• Students’ gains in research-related understanding, confidence, and awareness

• Importance of research experiences in students’ academic and career decisions and interests

• Changes in students’ interest in engineering-related careers

• Students’ expectations of academic degree attainment

• Mentors’ reports of students’ gains in understanding, confidence, and awareness

• Mentors’ reasons for mentoring

• Mentors’ perceptions of barriers to increasing the number of undergraduates involved in research

A3. Use of Information Technology to Reduce Burden

Web-based questionnaires will be the primary data collection mode. A hard copy of the questionnaire will be sent to survey participants who do not have Internet access. Web surveying provides thorough editing as data are entered for completeness, validity, and consistency. Web-based surveys employ user-friendly features such as automated tabulation, data entry and error messages for easy online correction, standard menus, and, for analysis, predefined charts and graphics. All of these features facilitate the reporting process, provide useful and rapid feedback to the data providers, and reduce the cost of data collection.

A4. Efforts to Identify Duplication; Why Similar Information Cannot Be Used

The study and questionnaire do not duplicate information collected by other NSF efforts from the same respondents. Earlier studies of the REU program, conducted in 1989 and the early 1990s, were focused on participants in the early years of REU awards. The URO study included only a sampling of REU awards from 2002. None of the undergraduate student respondents to these earlier surveys will be included in the proposed survey. Some of the PIs and faculty mentors in the proposed survey may have been surveyed in the earlier studies, but information requested for this study will be specific to the most recent year of participation.

A5. Impact on Substantial Number of Small Businesses or Other Small Entities

No respondents are from small firms.

A6. Consequences of Not Collecting the Information

If the information is not collected, ENG will be unable to report on the results of the REU Program and meet accountability requirements. It will also not be possible to determine what if anything should be modified in the Program’s design and the types of activities, participants, and research setting to enhance program effectiveness.

A7. Special Circumstances that Require Information to be Conducted in a Manner Inconsistent with Guidelines in 5 CFR 1320.6

The data collections will comply with 5 CFR 1320.6.

A8. Consultation with Persons Outside the Agency

A notice of this study was published in the Federal Register on July 6, 2006, volume 71, No. 129. A copy of the notice is included as Appendix A to this submission. One comment from the public was received that had no significant suggestions for altering the study plan.

Potential questionnaire topics have been discussed with several individuals who are active in or familiar with the REU program. They have also reviewed the draft questionnaires. Their comments have contributed to revisions to the instrument. Those consulted include:

Linda E. Parker, PhD

Engineering Program Evaluation Director

Division of Engineering Education and Centers

National Science Foundation

Arlington, VA 22230

(703) 292-5355

Allen Soyster

Director

Division of Engineering Education and Centers

National Science Foundation

Arlington, VA 22230

(703) 292-8380

Joanne Culbertson

Staff Associate for Planning and Evaluation

Office of the Assistant Director for Engineering

National Science Foundation

Arlington, VA 22230

(703) 292-4602

Mary Poats

ERC Program Manager

Division of Engineering Education and Centers

National Science Foundation

Arlington, VA 22230

(703) 292-5357

Esther Bolding

REU Program Manager

Division of Engineering Education and Centers

National Science Foundation

Arlington, VA 22230

(703) 292-5342

A9. Explanation of Payments or Gifts to Respondents

A $20 gift certificate for a popular online retailer will be offered as incentive for students who complete the study questionnaire. A similar incentive was used in the initial and follow-up surveys of 2002 URO student participants. It proved highly effective in that 76% and 80% response rates, respectively, were obtained on those surveys.

A10. Assurances of Confidentiality

Respondents will be advised that any information on specific individuals will be maintained in accordance with the Privacy Act of 1974. Specifically, it has been policy in ENG for studies such as this that only contractor staff will have access to data from individuals. No data that identify an individual will be provided to ENG staff in any form. ENG maintains no Systems of Records with project participant information. Reports from this study will include only aggregate data such that no individual respondent or his/her organization can be identified. In the cover letter for the survey sent to each survey sample member and on the questionnaire’s cover sheet, respondents will see the project’s confidentially statement.

A11. Questions of a Sensitive Nature

No questions of a sensitive nature are included.

A12.1. Number of Respondents, Frequency of Response, and Annual Hour Burden

The study will be conducted through two surveys: an initial survey of faculty and undergraduate participants in ENG REUs and, 2 years later, a follow-up survey of the undergraduate participants. There are three types of respondents who will be surveyed: (1) 413 Principal Investigators (PIs) who received REU awards during 2006, who will provide SRI with names and contact information for faculty and student participants; (2) an estimated 1,349 faculty members (including PIs) who mentored undergraduate students as part of an REU award; and (3) an estimated 1,622 undergraduate students who participated in research as part of these REU awards. It is estimated that, with a 90% response rate, 372 PIs will provide contact information for REU Program participants, and, with a 75% response rate, 1,012 PI/faculty and 1,217 student participants will complete the initial survey questionnaires and 913 students will complete the follow-up survey questionnaire. The PIs and students will respond twice and each faculty mentor will respond only once during the study. The estimate of burden per PI, based on previous similar surveys, is 15 minutes to provide names and contact information. The estimated burden per PI/faculty and student respondent, based on questionnaire pretests, is 30 minutes. The total estimated response burden, calculated by multiplying the number of respondents to each form by the burden per respondent for that form, is 2,400 hours. (See table in Section A12.2, below.)

A12.2. Hour Burden Estimates by Each Form and Aggregate Hour Burdens

There are six data collection forms. One form collects REU Program participant names and contact information; the other five are the survey questionnaires—two for PI/faculty, two for the initial survey of students, and one for the follow-up survey of students. The PI/faculty and initial student questionnaires will each have an academic year and a summer version. The questions will be rephrased appropriate to the time period of participation. The table below shows the number of respondents for each type of form, the respondent burden for each individual and for each group, and the aggregate number of respondents and hour burdens.

Estimated Respondent Hour Burden

Form Type	Number of Respondents	Burden Hours Per Respondent	Aggregate Hour Burden
REU Participant contact information form	413	0.25	103
PI/faculty questionnaires	1,349	0.50	675
Student initial questionnaires	1,622	0.50	811
Student follow-up questionnaire	1,622	0.50	811
TOTAL	5,006	--	2,400

A12.3. Estimates of Respondent Cost Burden

The overall cost to the respondents is estimated to be $32,464. The estimated hourly wage rates for PIs and faculty mentors are based on information found in the Department of Education's National Center for Educational Statistics Integrated Postsecondary Education Data System

(http://nces.ed.gov/programs/digest/d05/tables/dt05_234.asp). The estimated hourly wage rates for students are based on information from the U.S. Department of Labor

(http://www.dol.gov/dol/topic/wages/minimumwage.htm).

Estimated Respondent Cost Burden

Form Type	Number of Respondents	Burden Hours Per Respondent	Estimated Hourly Rate		Estimated Respondent Costs
REU Participant contact information form	413	0.25		$31.00		$3,201
PI/faculty questionnaires	1,349	0.50		$31.00		$20,910
Student initial questionnaires	1,622	0.50		$5.15		$4,177
Student follow-up questionnaire	1,622	0.50		$5.15		$4,177
Total	5,006	--		--		$32,464

A13. Estimate of Total Capital and Startup Costs/Operation and Maintenance Costs to Respondents or Record Keepers

There is no overall annual cost burden to REU PIs or respondents that results from participation in the Evaluation of the Research Experiences for Undergraduates (REU) Program other than the time spent completing the REU Participant contact information form or the questionnaires, which are attached as an appendix to this request.

A14. Estimates of Costs to the Federal Government

The estimated cost to the government of all data collection, analysis, and reporting activities for this study is $294,355 over 3 years (Base Contract Number: GS10F0554N). In addition, an estimated 2 months of NSF staff time will also be expended during the study. Using an average $55 hourly rate covering administrative, program manager, and COTR time, the estimated cost of ENG personnel effort is $17,600.

The estimated costs include:

Contractor Costs		Initial Survey	Follow-up Survey	Total
	Personnel	$138,688	$69,460	$208,148
	Other Direct Costs
	Staff Travel & Per Diem	$ 4,348	$ 2,346	$ 6,694
	Materials and Services	$ 4,100	$ 2,900	$ 7,000
	Incentives	$ 25,824	$ 25,824	$ 51,648
	Indirect Costs
	Research Overhead (@44%)	$ 1,338	$ 1,264	$ 2,602
	G & A and Fee	$ 9,565	$ 8,698	$ 18,263
	Total Contractor Costs	$183,863	$110,492	$294,355
NSF Staff Costs				$ 17,600
Total, All Costs				$311,955

A15. Change in Burden

There is no change in burden.

A16. Plans for Publication, Analysis and Schedule

Time Schedule:

October 2006 to December 2006

• Prepare study design

• Develop and pretest questionnaires

• Submit package to OMB

January 2007 to October 2007

• Obtain OMB clearance

• Conduct initial surveys

• Analyze survey data

• Prepare draft and final reports

January 2009 to September 2009

• Conduct follow-up survey of undergraduate students

• Analyze follow-up survey data

• Prepare draft and final follow-up reports

Like many agencies, NSF is reducing its reliance on paper publication methods in favor of electronic publications that require the same level of formatting and agency clearance process as with paper publications. NSF plans on electronic publication of the initial summary report during the fall of 2007 and the follow-up summary report during the fall of 2009.

There will be no complex analytical techniques used, such as imputation and sampling.

A17. Approval to Not Display Expiration Date

Not applicable

A18. Exceptions to Item 19 of OMB Form 83-1

No exceptions apply.