Site visit interview and focus group scripts

National OCEANIC AND ATMOSPHERIC ADMINISTRATION

Evaluation Support Services

Request for Clearance
Supporting Statement and
Data Collection Instruments


Attachment C: Site Visit Instruments & Contact Letters


Project Officer: Meka Laster

April 10, 2015

Attachment C.1
EPP-CSC NOAA-Funded Recipients (focus group)

INTRODUCTION TO STUDY

[Focus group facilitators will explain that they are the external evaluators conducting a study of EPP, and will lead group introductions. Facilitators will also pass out consent forms.]

Thank you for participating in our focus group about [name of CSC]. My name is XX and I am from IMPAQ International, a research organization based in Maryland. You were selected for the focus group because you are studying the ______________ sciences at [name of CSC], and are funded by the National Oceanic and Atmospheric Administration (NOAA). The purpose of our meeting is to get your feedback on your [name of CSC] experience, and if it has affected your academic and career choices. This focus group is part of a 3-year independent evaluation of NOAA’s Educational Partnership Program (EPP). We will be here for about 90 minutes today.

Everything you say in this focus group will be kept confidential – we will not report your name in our study. We want you to feel free to speak openly, so we also ask that you respect others in the group by keeping everything you hear in our conversation today confidential. We also ask that you respect the opinions of everyone in the room. As you saw in the consent form, we are going to record the focus group to assist us with our notetaking. We will use the recording only for our analyses and will not give access to the recording to anyone outside of the research team. Does anyone have any questions before we get started?

1. Let’s start by talking about your background and your experiences with [name of CSC]. What do you study and what year are you in the program? What attracted you to [name of CSC]?

2. Can you describe the academic opportunities that have been made available to you because of NOAA funding? [Probe for coursework and choosing major.]

3. What about research opportunities? [Probe for conference travel, research funding, collaboration with faculty, work with partner institutes, community outreach, mentoring, etc.]

4. What are your career goals? In what ways (if at all) have your academic and career choices been affected by your experience at [name of CSC]?

5. What do you like best about [name of CSC]?

6. What challenges have you experienced at [name of CSC]?

7. Have you received any other funding during your program? [Probe for other programs that provide similar experiences.]

8. Are there any areas or activities in [name of CSC] that are in need of greater time or financial resources?

9. Are there any other comments or suggestions you would like to make regarding your experience with [name of CSC]?

Attachment C.2
CSC Directors (interview)

INTRODUCTION TO STUDY

Thank you for participating in our interview about [name of CSC]. As you know, this interview is part of a 3-year independent outcome evaluation of NOAA’s Educational Partnership Program (EPP) that provides funding to [name of CSC]. We are not evaluating individual centers but instead are looking at outcomes across the entire EPP and its program components. Everything you say will be kept confidential – we will not report your name in our study. Your responses will be de-identified and will not be shared directly with individuals outside of the research team. Findings from the interviews will be reported in summary form, but your role and the CSC you work with may be identified in the report. Before we begin, do you have any questions about the purpose of the evaluation or our confidentiality policy?

We anticipate that this interview will take approximately 1 hour.

We are going to record the interview to assist us with our notetaking. We will use the recording only for our analyses and will not give access to the recording to anyone outside of the research team. Do I have your permission to record? (Yes/No)

1. Let’s start by talking about your background and your experiences with [name of CSC]. How long have you been at [name of school]? You’ve been the director of [name of CSC] since [year], is that correct?

2. Can you describe the research opportunities that have been made available to your faculty and students because of NOAA funding? [Probe for conference travel, research funding, collaboration with students and other faculty, work with partner institutes, etc.] Can you give me some specific examples? [Probe for new/groundbreaking research, increase in publications and/or dissemination of research.]

3. Can you describe the academic opportunities that have been made available to your school and students because of NOAA funding? [Probe for position funding, courses/majors made available, etc.]

4. Can you describe the community outreach opportunities that have started because of NOAA funding? How has NOAA funding allowed students and faculty to participate in community outreach?

5. Can you describe the relationship with [partner institutions]? What are the results of your collaboration? What are the strengths of the partner institutions? What are the benefits and challenges of working with them?

6. Can you describe the relationship between [name of CSC] and [name of school]? What is the value-added of the CSC to [your home institution]? And vice versa? [Probe for financial value added, such as funding for scholarships; faculty outcomes, such as funding for faculty positions, opportunities for research, and student outcomes, such as more NOAA-related courses/majors offered, leading to an increase in students studying in NOAA-related fields.]

7. What are the characteristics of the relationship between [name of CSC] and [community outreach partners]? What are some of the common activities? What are the benefits and challenges of working with them?

8. Can you describe the relationship between [name of CSC] and NOAA? What are the benefits and challenges of working with NOAA?

9. Are there any areas or activities in [name of CSC] that are in need of greater time or financial resources?

10. In what ways do you think [name of CSC] is successful at training and building a diverse workforce in NOAA-related fields and working with students from underrepresented communities?

11. In what ways do you think [name of CSC] provides resources on best practices in science education and training (particularly in NOAA-related fields)?

12. What do you like best about [name of CSC]

13. What challenges have you experienced as the director of [name of CSC]?

14. Are there any other comments or suggestions you would like to make regarding your experience with [name of CSC]?

Attachment C.3
Faculty/Administrators (interview)

INTRODUCTION TO STUDY

Thank you for participating in our interview about [name of CSC]. As you may know, this interview is part of a 3-year independent outcome evaluation of NOAA’s Educational Partnership Program (EPP) that provides funding to [name of CSC]. Everything you say will be kept confidential – we will not report your name in our study. Your responses will be de-identified and will not be shared directly with individuals outside of the research team. Findings from the interviews will be reported in summary form, but your role and the CSC you work with may be identified in the report. Before we begin, do you have any questions about the purpose of the evaluation or our confidentiality policy?

We anticipate that this interview will take approximately 1 hour.

We are going to record the interview to assist us with our notetaking. We will use the recording only for our analyses and will not give access to the recording to anyone outside of the research team. Do I have your permission to record? (Yes/No)

1. Let’s start by talking about your background and your experiences with [name of CSC]. How long have you been at [name of school]? What is your role with [name of CSC]?

2. Can you describe the opportunities that have been made available to you because of NOAA funding? [Probe for position funding, conference travel, research funding, collaboration with students and other faculty, work with partner institutes, community outreach, etc.]

3. How does [name of school]’s partnership with NOAA affect student outcomes? [Probe for outcomes that lead to more students majoring in NOAA fields, more students taking NOAA-related courses, and more students entering NOAA fields postgraduation.]

4. What do you like best about [name of CSC]?

5. What challenges have you experienced at [name of CSC]?

6. Can you describe the relationship between [name of CSC] and [name of school]? What is the value-added of the CSC to [your home institution]?

7. Are there any areas or activities in [name of CSC] that are in need of greater time or financial resources?

8. In what ways do you think [name of CSC] is successful at training and building a diverse NOAA-related workforce and working with students from underrepresented communities?

9. Are there any other comments or suggestions you would like to make regarding your experience with [name of CSC]?

Attachment C.4
Partner Institutions (phone interview)

INTRODUCTION TO STUDY

Thank you for participating in our interview about [name of CSC]. As you may know, this interview is part of a three-year independent outcome evaluation of NOAA’s Educational Partnership Program (EPP) that provides funding to [name of CSC]. Everything you say will be kept confidential – we will not report your name in our study. Your responses will be de-identified and will not be shared directly with individuals outside of the research team. Findings from the interviews will be reported in summary form, but your role and the CSC you work with may be identified in the report. Before we begin, do you have any questions about the purpose of the evaluation or our confidentiality policy?

We anticipate that this interview will take approximately 1 hour.

We are going to record the interview to assist us with our note taking. We will use the recording only for our analyses and will not give access to the recording to anyone outside of the research team. Do I have your permission to record? (Yes/No)

1. Let’s start by talking about your background and your experiences with [name of CSC]. How long has [name of school] been involved as a partner of [name of CSC]? What is your role in the partnership?

2. Can you describe the opportunities that have been made available to your faculty and students because of NOAA funding? [Probe separately for position funding, conference travel, research funding, collaboration with students and other faculty, work with partner institutes, community outreach, etc.]

3. Can you describe the relationship with [name of the lead institution]? What are some of the results of your collaboration? What are the strengths of [the lead institution]? What are the benefits of this partnership? What are some of the challenges?

4. Can you describe the relationship with [other partner institutions]? What are some of the results of your collaboration? What are the strengths of [the partner institutions]? What are the benefits of these partnerships? What are some of the challenges?

5. What is the value-added of the CSC to [your home institution]? And vice versa?

6. Are there any areas or activities in [name of CSC] that are in need of greater time or financial resources?

7. In what ways do you think [name of CSC] is successful at training and building a diverse workforce in NOAA-related fields and working with students from underrepresented communities? In what ways do you think [name of CSC] provides resources on best practices in science education and training (particularly in NOAA-related fields)?

8. What do you like best about [name of CSC]?

9. What challenges have you experienced with [name of CSC]?

10. Are there any other comments or suggestions you would like to make regarding your experience with [name of CSC]?

Attachment C.5
CSC Community Partners (phone interview)

INTRODUCTION TO STUDY

Thank you for participating in our interview about [name of CSC]. As you may know, this interview is part of a 3-year independent outcome evaluation of NOAA’s Educational Partnership Program (EPP) that provides funding to [name of CSC]. Everything you say will be kept confidential – we will not report your name in our study. Your responses will be de-identified and will not be shared directly with individuals outside of the research team. Findings from the interviews will be reported in summary form, but your role and the CSC you work with may be identified in the report. Before we begin, do you have any questions about the purpose of the evaluation or our confidentiality policy?

We anticipate that this interview will take approximately 1 hour.

We are going to record the interview to assist us with our notetaking. We will use the recording only for our analyses and will not give access to the recording to anyone outside of the research team. Do I have your permission to record? (Yes/No)

1. Let’s start by talking about your background and your experiences with [name of CSC]. What is your role in the partnership? When and how did the partnership begin?

1. Can you describe the activities and opportunities that your institution/organization has conducted in partnership with [name of CSC]?

2. Can you describe the relationship with [name of CSC]?

3. Are there any areas or activities that you would like to pursue with [name of CSC] in the future?

4. In what ways do you think [name of CSC] is successful at training and building a diverse workforce in NOAA-related fields and working with students from underrepresented communities?

5. What do you like best about [name of CSC]?

6. What challenges have you experienced with [name of CSC]?

7. Are there any other comments or suggestions you would like to make regarding your experience with [name of CSC]?

Attachment C.6
Participant Consent Form for Focus Groups

You are invited to take part in a study of the Educational Partnership Program (EPP) for Minority-Serving Institutions, which funds the [name of CSC] with which you are associated. This 3-year independent evaluation is being carried out by Insight Policy Research and IMPAQ International under contract to the National Oceanic and Atmospheric Administration’s Office of Education (NOAA OEd). The evaluation seeks to determine whether the EPP is meeting its goals for its participants as well as to gather information on program implementation to inform future NOAA programming. As part of this study, you are being asked to participate in a focus group that will last about 90 minutes, with a short break in the middle. The focus group will ask for your feedback on your [name of CSC] experience and your views as to how it may be affecting your academic and career choices. Your consent is needed if you are to participate, and we strongly urge you to take part in what we hope will be an interesting discussion of the CSC experience with your peers. Below are several pertinent points about your participation.

Voluntary Participation

It is your choice whether to participate in this research. If you do choose to participate, you may change your mind and leave the focus group at any time. Refusal to participate or stopping your participation will involve no penalty and no loss of benefits.

Risks

Strenuous efforts are being made to ensure that only Insight Policy Research and IMPAQ International project staff will have access to recordings of the focus group (interview) sessions and any associated transcripts or notes. Apart from the very small possibility that someone other than the researchers would hear a recording or see a transcript, there are few other risks associated with participation.

Confidentiality

The data from the focus group will not be intentionally shared with anyone, and will be kept on a password-protected computer accessible only to designated staff. We will ask participants in the focus group not to reveal what is discussed in the group to anyone outside the group, but we cannot absolutely guarantee focus group confidentiality. Your name will not appear in any report or other publication, responses will be summarized in a way that will never identify anyone personally, and the data will be used for research and educational purposes only.

Whom to Contact

For more information about the evaluation and your participation in it, or if you would like to withdraw from participation in the focus group, please get in touch with [Maria DiFuccia], whose contact information can be found below:

Maria DiFuccia, IMPAQ International

Evaluation Site Visit Coordinator

202-774-1948

[email protected]

Consent

I have read the information in this consent form. All my questions about the research have been answered to my satisfaction.

Your signature below indicates your permission to take part in this research.

________________________________________________________

Printed name of participant

________________________________________________________ ____________________

Signature of participant Date

Attachment C.7

CSC Contact Letter for Site Visit

To: Cooperative Science Center Director

CC: Key CSC Staff [such as those helping to schedule site visit]

Subject: Evaluation of NOAA’s Educational Partnership Program – site visit

Dear XXX,

As part of the evaluation of the NOAA’s Education Partnership Program (EPP), Insight Policy Research, Inc. and IMPAQ International are conducting site visits to interview or conduct focus groups with key Cooperative Science Center (CSC) leaders, administrators, community partners, and students. The site visits are part of a three-year independent outcome evaluation of NOAA’s EPP that provides funding to [name of CSC]. We are not evaluating individual centers, but instead are looking at outcomes across the entire EPP and its program components. We are conducting the site visits to gather feedback from multiple sources about their experience with the CSCs, including CSC contributions to [name of institution], products generated by the CSCs, participants’ perceptions of the CSCs, and activities in need of greater time or financial resources. We appreciate your assistance with scheduling and planning these interviews and focus groups during our site visit.

Each individual interview will take approximately 60 minutes, and will be audio recorded with the respondents’ permission. The students will be interviewed together in a focus group, which will take approximately 90 minutes. The information provided will be used for program evaluation purposes only, and will only be reported in aggregate. No individual names will be reported.

We would like to schedule a call with you to discuss selecting a sample of participants for the interviews and focus group. During the call, we would also like to discuss more specific logistics about the two –day site visit. Please let us know when you are available in the next two weeks for a phone conversation.

Thank you for your participation!

[Site Visitor’s Name]

Attachment C.8

CSC Contact Letter to Institutional Partners for Interviews

To: Institutional Partner

Subject: Evaluation of NOAA’s Educational Partnership Program – request for interview

Dear XXX,

As part of the evaluation of the NOAA’s Education Partnership Program (EPP), Insight Policy Research, Inc. and IMPAQ International are conducting interviews with key Cooperative Science Center (CSC) leadership, administrators, community partners, and students. These interviews are part of a three-year independent outcome evaluation of NOAA’s EPP that provides funding to [name of CSC]. We are not evaluating individual centers, but instead are looking at outcomes across the entire EPP and its program components. We are conducting these interviews in order to gather feedback from multiple sources about their experience with the CSCs, including CSC contributions to [name of institution], products generated by the CSCs, participants’ perceptions of the CSCs, and activities in need of greater time or financial resources. As someone who works closely with [name of CSC], your input is critical to our study. Therefore, we would like to conduct a telephone interview with you.

Each telephone interview will take approximately 60 minutes, and will be audio recorded with your permission. The information provided will be used for program evaluation purposes only, and will only be reported in aggregate. No individual names will be reported.

Could you please provide me with some dates and times when you are available for a phone interview within the next two to three weeks?

Please let me know if you have any questions.

Thank you for your participation!

[Interviewer’s Name]