Section B. Collections of Information Employing Statistical Methods
B1. Survey Respondent Universe
A total of 13 BBSI awards have been made. It is estimated that these 13 awards supported a total of 523 BBSI students from 2003 to 2007 and utilized 167 PIs and faculty to mentor the students. No sampling will be used. A 100% response from the PIs is expected with respect to providing names and contact information for all student and faculty participants. A 75% response to each of the three surveys is expected.
Study of BBSIs in 2003–07: Survey Respondent Universe and Estimated Number of Respondents
Total Years Covered |
Total BBSIs |
Total Students |
Total PIs/Faculty |
Total Declinees |
Total Students, Faculty, & Declinees |
Est’d Response Rate |
Est’d No. of Survey Respondents |
5 (2003–2007) |
13 |
523 |
167 |
49 |
739 |
75% |
554 |
Note: Counts are non-duplicated in this table.
B2. Procedures for the Collection of Information/Limitations of the Study
This study will be conducted primarily through Web surveys. The survey respondent pool is the universe of students and faculty who participated in the BBSI program in 2003–2007 and also the universe of proposers who did not receive a BBSI award in the 2002 and 2006 solicitations.
The PIs of the BBSIs will provide the names and contact information for faculty and student participants and will be interviewed about matters related to running the BBSI program. They also will complete the faculty survey, which asks different questions from the perspective of a faculty participant (instructor, mentor).
There are three survey instruments: one for BBSI faculty (including PIs), one for former BBSI students, and one for declined proposers. Each respondent will provide answers once to the relevant survey instrument.
Names and contact information (e-mail and postal addresses and telephone numbers) for the 2003–2006 BBSI participants have already been provided by the PIs of the 9 original BBSIs. The PIs of all 13 BBSIs will be asked to provide the names and contact information for the 2007 student and faculty participants after the 2007 summer has been completed. The names and contact information for the declined proposers were obtained from NSF records.
Upon approval of the survey instruments, all BBSI student and faculty participants, as well as all declined proposers, will be contacted by e-mail, provided with the URL of the relevant survey questionnaire, and asked to go to that Web site to complete the instrument. If requested, or if an e-mail address is not available for an individual, a hard copy of the questionnaire will be mailed to the individual’s home address, if available.
This study is a correlational design, and as such, will not be represented to yield causal conclusions. Determinations of causality require objective pre- and post-measures as well as random assignment to condition (e.g., participation or non-participation in the program of interest). In the real world, these requirements are impractical, at best, and often impossible. Most program evaluation surveys, including the BBSI surveys, rely on respondents' self-reports of program outcomes and impacts as the most practical alternative. Even though these surveys do not provide strict evidence of causality, self-reports are widely considered to provide valuable information about program outcomes, impacts, areas of strengths and weaknesses, and so on.
B2.1. Statistical methodology for stratification and sample selection
Not applicable. All members of the survey universe are included in the survey.
B2.2. Estimation procedure
The profile of BBSI students’ survey responses will be compared against the universe profile based on award year and BBSI. If there are statistically significant differences in the profiles of respondents, survey responses will be weighted so that the overall respondent profile parallels that of the universe.
B2.3. Degree of accuracy needed for the purpose described in the justification
Not applicable.
B2.4. Unusual problems requiring specialized sampling procedures
Not applicable.
B2.5. Use of periodic (less frequent than annual) data collection cycles
The study is a one-time collection about the BBSI awards and declined proposals in the 2002 and 2006 solicitations.
B3. Methods to Maximize Response and Deal with Issues of Nonresponse
Several study design and timing situations or features are expected to bring about strong response rates:
(1) The BBSI surveys are being conducted via the Internet to minimize the effort and time required of respondents to complete the questionnaire. All of the PIs and faculty have access to the Internet, and based on experience from the 2002 URO study, an estimated 90% of the BBSI student population has access to the Internet;
(2) The PIs are regular NSF awardees who are accustomed to providing information about their project participants, activities, and results for GPRA purposes and individual evaluation studies;
(3) The surveying is timed to take place in winter/early spring when universities and colleges are in session, maximizing the likelihood of reaching the faculty and students well before the end of the academic year;
(4) All BBSI participants will have completed their BBSI experience within four years of the survey date. Much of the contact information provided by the PIs should be accurate because of the recent BBSI experience or because the PIs have kept the contact information of the BBSI students up to date. The recent BBSI experiences should be easy to recall for both students and faculty;
(5) A $20 gift certificate for a popular online retailer will be offered as incentive for students to complete the questionnaire. Use of a similar incentive for the 2002 URO study proved highly effective, resulting in 75% and 80% response rates for the initial and follow-up student surveys.
(6) SRI will send follow-up e-mail reminders to non-respondents approximately once a week over at least a 6-week period following commencement of the surveys. Reminders will be sent on different days of the week and times of the day.
B4. Tests of Procedures or Methods
The questionnaires are similar in format to the instruments used in surveying the 2002 URO participants. However, the questions and categories focus on the special features of the BBSI program. Pretesting of each survey instrument will be done with 6 to 9 people. The only appreciable changes expected to the made to the instruments are the possible additions of a few response categories based on responses of pretest participants.
B5. Names and Telephone Numbers of Individuals Consulted
Agency Project Director:
Linda E. Parker, PhD
Engineering Program Evaluation Director
Division of Engineering Education and Centers
National Science Foundation
Arlington, VA 22230
(703) 292-5355
Esther Bolding,
Program Manager
Human Resource Development
Division of Engineering Education and Centers
National Science Foundation
(703) 292-5342
Agency Program Managers:
Mary Poats
ERC Program Manager
Division of Engineering Education and Centers
National Science Foundation
Arlington, VA 22230
(703) 292-5357
Richard A. Baird
Director, Division of Interdisciplinary Training
National Institute of Biomedical Imaging and Bioengineering
National Institutes of Health
(301) 496-7671
| File Type | application/msword |
| File Title | Section B |
| File Modified | 2007-10-29 |
| File Created | 2007-10-29 |