NIST - SURF Alumni Discussion Guide for Qualitative Interviews - Supporting Statement

OMB Control No. #0693-0033-NIST Generic Clearance for Program Evaluation Data Collections

NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY (NIST)

SUMMER UNDERGRADUATE RESEARCH FELLOWHSIP (SURF)

Alumni Discussion Guide for Qualitative Interviews

FOUR STANDARD SURVEY QUESTIONS

1. Explain who will be survey and why the group is appropriate to survey.

The National Institute of Standards and Technology (NIST) proposes to conduct a telephone survey through the company Camper Consulting, who has been contracted to carry out a two-year comprehensive assessment of the Summer Undergraduate Research Fellowship (SURF) program. A subset of the SURF Alumni who completed the program between 2003 and 2013 will be interviewed to understand their experiences with SURF, its influence in choosing a field of study or work, and whether they have continued in a Science, Technology, Engineering & Mathematics (STEM) field of study or work. It is planned to interview twenty (20) SURF Alumni – thirteen (13) who attended the SURF Program in Gaithersburg, MD and seven (7) who attended in Boulder, CO.

The overarching purpose of the program evaluation is to assess the impact and effectiveness of SURF by measuring the following outcomes:

• Student clarity about the discipline(s) they would like to pursue, STEM-related or not

• Students proceeding to pursue one or more STEM-related graduate degrees

• Students proceeding to STEM-related careers or employment

• Students proceeding to careers with NIST

2. Explain how the survey was developed including consultation with interested parties, pre-testing, and responses to suggestions for improvement.

The discussion guide to be used as the protocol in qualitative interviews with SURF Alumni was developed by Camper Consulting in consultation with NIST-International & Academic Affairs Office (IAAO) and the NIST SURF coordinator. In addition, interviews with SURF Directors, SURF Advisers, and SURF Alumni who are now NIST employees informed the development of the discussion protocol.

3. Explain how the survey will be conducted, how customers will be sampled if fewer than all customers will be surveyed, expected response rate, and actions your agency plans to take to improve the response rate.

The discussion guide is to be used as the protocol in qualitative telephonic interviews with twenty (20) SURF Alumni. This is not a statistical survey. SURF Alumni will be recruited for interviews by Camper Consulting from Alumni lists provided by NIST. Camper Consulting will e-mail and/or phone a random cross-section of SURF Alumni for whom we have, or can obtain, contact information in Gaithersburg and Boulder. They will conduct the random selection through the use of the Excel RAND function, which returns a random number greater than or equal to 0 and less than 1, evenly distributed for each contact. The top 20 randomly generated values will be selected to contact and schedule interviews. Based on invalid contact information, scheduling conflicts, inability to reach some individuals, and unreturned calls, it is anticipated that the process will need to be repeated several times (taking the next 20 randomly generated values, and so on) until 20 interviews with SURF alumni are successfully recruited and scheduled. The information will not be generalized to the entire population of the SURF Alumni; the information gained from these interviews will be utilized to develop the Alumni survey instrument to be used for quantitative data collection at a later date.

For those who do not have contact information in the lists provided by SURF, or for whom contact information is not current, we will do a scan of publicly available information to attempt to obtain contact information (e.g., Google searches, LinkedIn, and Facebook). In order to recruit and schedule twenty (20) interviews, we anticipate contacting more than that number (up to 5-to-1, that is on average up to five for each one interview to be completed) to account for invalid contact information, scheduling conflicts, inability to reach some individuals, and unreturned calls. If initially unresponsive, we will send e-mails and make phone calls to give each randomly selected individual several opportunities to respond.

4. Describe how the results of the survey will be analyzed and used to generalize the results to the entire customer population.

The Alumni Interviews are designed to provide a critical foundation for the overall SURF Assessment, including the development of the Alumni survey instrument to be used for quantitative data collection with SURF Alumni in a subsequent assessment project phase. It is important that we hear from a cross-section of Alumni by campus (Gaithersburg and Boulder) and a mix by SURF participation year to inform the survey development.  Conducting no fewer than twenty (20 interviews) will provide the coverage by campus and a mix of participation year to inform this process.

2