Protocol document

Interviews with R&RA Awardees: Protocol and Questions

Protocol: Introduction for Phone Interviews

Hello, thank you for agreeing to participate in this interview. I am [Insert Name] and I am with Windrose Vision. Our firm has been contracted by NSF to conduct a study on Broadening Participation in science, technology, engineering, and mathematics (STEM). My colleague [Insert Name] will take notes during the interview.

The purpose of the phone interview is to share your experiences with the NSF-funded [Insert Title of Award and Number].

Instructions: Use text below if the interviewee(s) agreed to be tape recorded in the agreement form:

Thank you for agreeing for this interview to be tape recorded. The recording ensures accurate documentation of all responses.

Instructions: Use text below if respondent(s) agreed to participate, but did not email Windrose a signed agreement with respect to the recording:

We would like to record all of our interviews to ensure accurate documentation of all responses. Do we have your permission to record this interview?

Instructions: If yes, thank the respondent(s). If not, say “that’s fine” and record in notes that respondent(s) did not want to be recorded

**************************************************************************

Please note that your participation in this interview is voluntary. You may choose to not answer any question or stop the interview at any time. Our report to NSF will include your name and the name of your institution/organization, but public reports (if any) will exclude personal and institutional identifying information. I want to remind you that the interview questions focus just on [Insert Title of Award] which was funded from [Insert start date] to [Insert end date].

We anticipate that this interview should last no longer than 30 minutes. Do you have any questions before we start?

Let’s start the interview.

Interview Questions

Program

1. Please describe your project.

General

2. Which underrepresented groups in STEM were targeted as part of this award? By “underrepresented groups in STEM,” we mean African Americans, Alaska Natives, Hispanics, Native Americans, Native Hawaiians, Pacific Islanders, women, and individuals with disabilities.

3. Was Broadening Participation the focus of the award?

[“NSF defines broadening participation in terms of individuals from underrepresented groups as well as institutions and geographic areas that do not participate in NSF research programs at rates comparable to others.”]

1. PROBE: [If Yes]: Why did you seek funding for the [insert title of R & RA Award] project from the Research and Related Activities portfolio as opposed to NSF’s Broadening Participation portfolio?

Outreach and Recruitment

4. What activities or strategies were conducted to get underrepresented groups involved in this project?

5. Which strategies were helpful in getting individuals involved in your program?

1. PROBE: How where these strategies helpful?

2. PROBE: What strategies, if any, were not helpful?

   1. PROBE: Why were these strategies not helpful?

Retention

6. What methods or strategies, if any, were used to sustain participation by these underrepresented individuals in your program?

1. PROBE: Would you characterized the retention methods or strategies as novel?

2. PROBE: Were these retention methods or strategies something you have tried in the past?

3. PROBE: Which retention methods or strategies would you characterize as helpful?

4. PROBE: Which retention methods or strategies would you characterize as not helpful?

   1. PROBE: [If examples are not provided for helpful and/or not helpful methods or strategies] Can you provide an example on how the methods or strategies were [Insert: “helpful” or “not helpful]”?

5. PROBE: Which methods or strategies used in this project do you plan on using in the future for other projects?

Output

7. What was the approximate number of individuals who participated in this project?

1. PROBE: Approximately how many of these individuals were underrepresented groups?

8. What was the approximate number of individuals, especially underrepresented groups, who were directly impacted by this project?

[“Impact” can mean subjects’ interest in STEM was promoted, subjects gained insight into STEM, subjects were recruited into STEM fields, etc. It can also mean that subjects or individuals who share some characteristics with the subjects may benefit from the results of the project (e.g., a game promoting social skills for children diagnosed with autism).]

9. Have the findings and or lessons learned from the project been disseminated?

PROBE: [If yes]: How? Examples include: presentations at professional meetings, workshops, webinars, and peer-reviewed articles.

10. Have the products of the project [Insert software/computer games, curricula] been disseminated?

PROBE: [If yes]: How?

Outcomes

11. What has happened with the project since the NSF funding ended?

    1. PROBE: What activities, if any, were used to follow-up with individuals involved in the project?

       1. [If follow-up activities were conducted] What has been the project’s impact on the individuals involved in the project?

12. What changes, if any, have taken place within your institution that are directly or indirectly related to NSF funding for this project?

1. PROBE: Was this project the primary motivation for these institutional changes?

Broader Impact

13. What impact did these NSF-funded activities have at the institutional level?

    1. Exemplar PROBE: The award modernized the XYZ lab at your institution. Is that lab still being used for new projects?

14. What impact did these NSF-funded activities have on the underrepresented groups who worked on the project?

    1. Exemplar PROBE: You were able to involve X number of African American undergraduates to work on your project by collaborating with the XYZ organization on campus. Are you still working with XYZ to recruit underrepresented groups?

       1. [if yes] How is that going?

       2. [if no] Why is that?

15. What impact did these NSF-funded activities have on the individuals who were the subjects or presumed beneficiaries of the project?

    1. Exemplar PROBE: Do you have any information on what the African American students who worked with you are doing now?

       1. [If yes] Did any of them go on to graduate programs or careers in STEM?

          1. [If yes] Tell me a little about that.

16. Beyond the impacts already discussed, did this project have any other impacts related to BP in STEM in the community?

5