Former Scholar/Fellow Focus Group Protocol

Attachment 3a:

Former Scholar/Fellow Focus Group Protocol

Noyce Program Evaluation Focus Group Protocol: Former Scholars/Fellows

OBJECTIVES:

The purpose of this inquiry is to obtain thorough and comprehensive insights from former Noyce scholars and fellows regarding their involvement with Noyce STEM teacher preparation/leadership programs. The questions also address the perspective of former Noyce scholars and fellows on the Noyce Program's perceived impact, effectiveness, and burden as well as interactions with higher education institutions with Noyce STEM teacher preparation/leadership programs.

FACILITATOR SCRIPT:

INTRODUCTION

• Hello, my name is [NAME OF FACILITATOR], and I will be facilitating this confidential focus group today.

• I am an independent evaluation and research consultant. I work for WhitworthKee Consulting. We are partnering with the National Science Foundation to evaluate the Robert Noyce Teacher Scholarship Program and identify areas that are working well and opportunities for growth. NSF has contracted the WKC Evaluation Team as a third-party evaluator so that the Noyce community can express their true perceptions and experiences with Noyce without having personal information tied to responses.

• Joining us today is my colleague [NAME OF CO-FACILITATOR/NOTETAKER] who will function in the background to take notes and observe the session. They will remain in the background without video but may also ask follow-up questions.

• As you may know, the focus groups were organized by your connection to the Noyce Program when possible. This is the former scholar/fellow focus group.

• The purpose of this focus group is to discuss your perceptions of the impact of the Noyce Program, how well the Program has worked, and your experiences in the Program. We will also discuss your perception of the factors that affect the project’s success and perceptions of those invested in the STEM education community. There are no right or wrong answers, and please be straightforward with your responses. You may not have experience with or know the answer to every question. Because Noyce projects and roles can vary, please feel free to share with us if you don’t know the answer to the question or it doesn’t apply to you. This focus group will last no more than 60 minutes.

• You can skip any question that you do not want to answer, and you can stop participating in this focus group at any time. Your participation in this focus group will not impact you or your institution’s relationship with NSF either now or in the future.

CONFIDENTIALITY

Your answers in today’s focus group will remain private. The WKC Evaluation Team will not connect your name or any personal details to your responses. The results will be summarized to reflect the overall group’s perspectives. Before sharing the information with external parties (e.g., NSF, NSF’s website, reports to Congress, Noyce events, other conferences, publication audiences, or other professional communities), the team will make sure that any identifying details are removed.

We would like to record this focus group so that we can ensure that we have time to think deeply about your responses. To maintain data security, the recording will be stored in a password-protected cloud-based drive. It will not be shared with anyone except the researchers working on the Noyce Program evaluation. Please verbally acknowledge your consent to recording by saying, “I agree.” [FACILITATOR NOTE: MAKE SURE TO RECEIVE A VERBAL “I AGREE” FROM ALL FOCUS GROUP PARTICIPANTS. MAKE NOTE OF PARTICIPANT CONSENT (YES/NO) IN THE TRACKER.]

Do you have any questions before we get started?

[IF YES, PLEASE ANSWER ANY QUESTIONS AS BEST YOU CAN. IF YOU ARE UNABLE TO ANSWER A QUESTION, PLEASE REACH OUT TO THE PROJECT LEAD OR CO-LEAD]

[IF NO, BEGIN THE FOCUS GROUP QUESTIONS]

FOCUS GROUP QUESTIONS

[FACILITATOR NOTE: START RECORDING IF PARTICIPANT CONSENTED; IF THEY DID NOT, BOTH THE INTERVIEWER AND NOTETAKER WILL TAKE NOTES TO CAPTURE AS MUCH DATA AS POSSIBLE] I’m going to start the recording now.

Background

[INFORMATION ON NAME, CURRENT ROLE, AND INSTITUTION/ ORGANIZATION WILL BE CAPTURED FROM THE DEMOGRAPHIC QUESTIONNAIRE – SHARE THE RESPONDENTS’ DEMOGRAPHIC QUESTIONNAIRE ANSWERS AND VERIFY THEY ARE CORRECT.

IF ANY INFORMATION IS INCORRECT, PLEASE NOTE THE RESPONDENT’S CORRECTED INFORMATION AND GIVE THE UPDATED INFORMATION TO THE PROJECT LEAD OR CO-LEAD.]

FACILITATOR SCRIPT: In this section, we will discuss your background with the Noyce Program.

1. To start off, please tell us whether you are a Noyce Scholar, Teaching Fellow, or Master Teaching Fellow, and share how you first heard about the Noyce Program. [IF NEEDED: NOYCE SCHOLARS RECEIVE THEIR TEACHING CERTIFICATION VIA AN UNDERGRADUATE OR POSTBAC PATHWAY WITH NO SALARY SUPPLEMENT POST-CERTIFICATION. TEACHING FELLOWS RECEIVED THEIR TEACHING CERTIFICATION VIA A MASTER’S DEGREE PROGRAM PLUS 4 YEARS OF A SALARY SUPPLEMENT. MASTER TEACHING FELLOWS ARE EXPERIENCED CERTIFIED TEACHERS THAT RECEIVE A SALARY SUPPLEMENT FOR 5 YEARS.] [BACKGROUND]

2. How would you describe the Noyce project you were previously a part of? [2A]

STEM Teacher Preparation Field

3. From your perspective, what specific parts of the STEM teacher preparation field do you think should be emphasized and funded? [1A]

   1. Probe: What are some of the challenges in preparing, recruiting, and retaining STEM teachers? [1A]

FACILITATOR SCRIPT: As you may know, the Noyce Program provides funding to higher education institutions via (a) scholarships, (b) stipends, and (c) other means of programmatic support to recruit and prepare science, technology, engineering, and mathematics (STEM) majors and other professionals to become K-12 teachers and teacher leaders.

4. If the Noyce Program could fund anything (the sky is the limit), what would you suggest that NSF fund to recruit, prepare, and retain STEM teachers and teacher leaders in high-need school districts?[1]

   1. Probe: Can you identify specific areas within STEM education that are currently underserved or overlooked that could benefit from additional funding? [1A]

   2. Probe: Are there specific populations of individuals in STEM education that are currently underserved or overlooked that you think could benefit from additional funding? [1A]

   3. Probe: How could the Program be expanded to further support the professional development of STEM teachers and teacher leaders? [1A]

   4. Probe: Which entities did you collaborate with in relation to your Noyce project? [IF NEEDED: NON-PROFIT PARTNERS THAT PROVIDED PROFESSIONAL DEVELOPMENT, INTERNSHIPS, RESEARCH EXPERIENCES; OTHER INSTITUTIONS; OTHER DEPARTMENTS; LOCAL SCHOOL DISTRICTS; ETC.] What role did they have? [2A]

   5. Probe: Which entities would you say were missing from your Noyce project’s partnerships that would help support your participation? [2A]

5. Looking back on when you first started with your institution’s Noyce program, what motivated you to participate as a Noyce scholar/fellow? What factors influenced your decision? In what ways did these factors influence your decision? For example, what factors existed within society, the larger field, at the institution level, and/or at the college, department, and faculty level? [1C, 2A]

6. What are the barriers to becoming a Noyce scholar/fellow? [IF NEEDED: FOR EXAMPLE, TO WHAT EXTENT ARE OBTAINING PARTICIPATION INFORMATION, FUNDING AMOUNTS, ETC. BARRIERS TO BECOMING A NOYCE SCHOLAR/FELLOW?] [1C]

   1. Probe: Did you have any challenges with the application process? Please explain. [IF NEEDED: WRITING AN ESSAY, TEST SCORES, GPA, RECOMMENDATIONS, ETC.]

7. What do you think could be learned from the Noyce Program and applied to other similar STEM teacher preparation and teacher leader initiatives? [1D]

Engagement with Noyce Program Staff

FACILITATOR SCRIPT: In this next section, we would like to discuss your experiences with staff and collaborators for your Noyce project. As a reminder, your responses will not be tied to your name and all responses will be combined with other focus group participants’ answers before they are reported.

8. How would you describe your experience working with the person leading your Noyce project (called a Principal Investigator or PI, and is often a faculty member or administrator at your college/university)?

   1. Probe: How frequently did you directly connect or interact with your PI? What kind of interactions did you have?

   2. Probe: What kind of support and resources did they provide to help you succeed in your role?

   3. Probe: How could PIs further support current and former Noyce scholars/fellows?

9. How, if at all, have you interacted with the National Science Foundation (NSF)?

   1. Probe: How would you describe your overall experience working with NSF? What kind of interactions did you have?

   2. Probe: What challenges, if any, did you experience when connecting with NSF?

   3. Probe: How could NSF further support current and former Noyce scholars/fellows?

Perceptions of the Noyce Program

FACILITATOR SCRIPT: In this next set of questions, we would like to explore your perceptions about the Noyce Program overall and its impact as well as any gaps in its reach.

10. What is your overall perception of the Noyce Program? [2A]

    1. Probe: What is your perception of current Noyce scholars and fellows? [2B]

    2. Probe: What is your perception of former Noyce scholars and fellows? [2C]

FACILITATOR SCRIPT: As you know, one of the main intentions of the Noyce Program is to address the critical need for recruiting, preparing, and retaining highly effective elementary and secondary mathematics and science teachers and teacher leaders who persist as classroom teachers in high-need school districts.

11. What is your definition of a high-need school district in K-12 education? [2A]

1. Probe: How closely does/did your teaching requirement placement align with your definition of high-need schools or districts?

12. What is your perception of the impact of the Noyce project at your institution (i.e., after implementation)? For example, how does/did the Noyce project at your institution address STEM teacher shortages and/or retaining STEM teachers in high-need schools? [2D]

    1. Probe: What factors contribute to your perception of the impact of the Program?

    2. Probe: What are the gaps in the Noyce Program's reach and impact to addressing the critical need for recruiting, preparing, and retaining highly effective STEM teachers and teacher leaders in high-need school districts.)? [2D]

Wrap Up

13. As we wrap up our time together, would you like to share any lessons learned during your involvement with the Noyce Program or any growth opportunities that could help improve the Program? [5]

FACILITATOR SCRIPT: Thank you for your time and for sharing your experiences and perceptions. This will be extremely helpful for NSF as they identify areas in Noyce that are working well and opportunities for growth. As I mentioned at the beginning of our focus group, all information that you provided will remain confidential, and any identifying details will be removed before any information is shared with external parties. Finally, all of the recordings will only be made accessible to the WKC Evaluation Team.