Sexual Risk Avoidance Education Grantee COVID-19 Interviews

SRAENE Grantee Interviews, Impact of COVID-19 on Programming, Draft Interview Protocol

SRAENE Grantee Interviews

Impact of COVID-19 on Programming

Draft interview protocol

An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information subject to the requirements of the Paperwork Reduction Act of 1995, unless it displays a currently valid OMB control number. The OMB # is 0970-0531 and the expiration date is 07/31/2022.

INTRODUCTION

Thank you for taking the time to complete the virtual interview. My name is [interviewer name], and I’ll be guiding our discussion through QualBoard. My organization, Mathematica, is conducting interviews with Sexual Risk Avoidance Education (SRAE) grantees, on behalf of FYSB, to provide context and deeper understanding of the impact of COVID-19 on SRAE grantee operations and programming.

The purpose of the interview is to learn about how grantees responded to the pandemic, what changes were made to programming, and how the pandemic and programming changes impacted grantees and youth served. Impacts of the pandemic on programming could be related to local, state and national restrictions. There are no right or wrong answers. I’m hoping to learn as much from you as possible, so the more detail you can provide, the better!

I want to share definitions for a few common terms that can help with these questions:

• When I ask about your program or programming, I mean all of the activities funded by your FYSB SRAE grant. This includes not just the programming that youth receive, but also the staff and staffing structure, the administration, processes to enroll youth and keep youth engaged, and any performance measurement and evaluation that occurs.

The QualBoard will be open for five days, and we estimate the total time you may spend responding will be up to 60 minutes. You can log in and out of the virtual interview on QualBoard at times that are most convenient to you. After you respond to the initial questions, you may receive some follow up questions from the interviewer, asking you to clarify or provide more details on a response. We ask that you log back into the QualBoard at least one more time after you initially answer questions to make sure you see and respond to the follow up questions.

We will keep the information that you share with us private. Your responses will be used only to characterize the general experiences of your state/organization, without attribution to you or any other specific individual.

Do you agree to participate in this study today? [If yes, continue]

Changes to the workspace and work environment

A. Remote working during and post-pandemic. The first six questions ask about staff working remotely during the 2020-2021 school year/and plans for remote work in the 2021-2022 school year.

If your program does not operate on a school year calendar, please answer according to the federal fiscal year for 2020-2021 and 2021-2022 which starts in October of each year.

1. At the start of the 2020-2021 academic school year, how many full- or part-time staff members worked for your SRAE program?

2. Currently, how many full- or part-time staff members work for your SRAE program?

3. During fall 2020, what percentage of the staff worked remotely either full time or most of the time?

4. During spring 2021, what percentage of the staff worked remotely either full time or most of the time?

5. Please tell us about what, if anything, worked well with work environments, either remote or in-person, over the last year.

6. Please tell us about what, if anything, was challenging about work environments, either remote or in-person, over the last year.

7. Thinking about fall 2021 what percentage of staff [will be/are] work[ing] remotely either full time or most of the time?

2. COVID-related staffing changes: We are interested in learning about the impact of COVID-19 on program staffing.

1. Did you have to reduce staff at any time because of the pandemic? (Close-ended response: Yes/No)

1. [IF B1= YES] Why did you need to reduce staff? For example, was it related to reductions in programming, staff childcare issues, staff safety concerns, etc.

2. [IF B1= YES] Did you later rehire or replace these staff members? (Close-ended response: Yes/No)

3. [IF B1b= NO] Do you have plans to rehire or replace these staff members? (Close-ended response: Yes/No)

2. Did any staff members choose to leave or reduce their hours because of the pandemic? (Close-ended response: Yes/No)

3. Please use this space to describe how staffing changes varied over time from fall 2020 to spring 2021. For example, has staffing been a consistent issue over the course of the last year, or was it worse at different times? If staffing changes did not vary over time from fall 2020 to spring 2021, select the box below to proceed. (Close-ended response: Check box. Staffing changes did not vary over time from fall 2020 to spring 2021)

4. Please use this space to describe how staffing changes affected programming during the 2020-2021 school year. If staffing changes did not affect programming, select the box below to proceed. (Close-ended response: Check box. Staffing changes did not impact programming)

5. Do you have enough staff to meet your needs for fall 2021? (Close-ended response: Yes/No)

1. [IF B5=NO] Please describe how you expect to adapt your programming in fall 2021 in response to staffing issues.

3. Plans to ensure safety in the physical workspace: We are interested in learning about workplace policies or protocols that you might have implemented to ensure health and safety during the pandemic.

1. What health and safety workplace policies or protocols did you establish to ensure safety during the pandemic for both employees and youth (if programming was offered hydrid or in-person)? (For example, revised cleaning standards, additional physical barriers, requirements to wear masks, to socially distance, or to get tested or comply with contact tracing.)

2. What policies, activities, or factors did you take into consideration to establish health and safety guidelines for your employees and/or youth? (For example, the concerns of high-risk staff or staff with high-risk family members; local, school, state, and Centers for Disease Control and Prevention guidelines and policies, etc.)

3. What challenges did you face when implementing workplace modifications or safety protocols? (For example, staff or partner adherence to protocols, lack of funding, difficulty in meeting state requirements, etc.)

4. Changes in staff communication. The next questions focus on staff communication during the 2020-2021 school year.

1. During fall 2020, how did you meet with staff? (Closed-ended response: Select one only) (1) All in-person meetings, (2) a combination of in-person and remote/virtual meetings, (3) all remote or virtual meetings

2. During spring 2021, how did you meet with staff? (Closed-ended: response: Select one only) (1) All in-person meetings, (2) a combination of in-person and remote/virtual meetings, (3) all remote/virtual meetings

3. Thinking of fall 2021, how do you currently meet with staff? (Closed-ended response: Select one only) (1) All in-person meetings, (2) a combination of in-person and remote/virtual meetings, (3) all remote/virtual meetings

4. What ways of communicating with staff worked well during the pandemic?

5. What challenges did you face communicating with staff during the pandemic?

6. Please use this space to provide any communication strategies that you implemented in response to the pandemic that you will continue to use after the pandemic, and discuss how you plan to use the strategies. (For example, increased use of virtual meetings, email, etc.)

5. Changes in use of technology. We are interested in learning about new uses of technology during the 2020-2021 school year.

1. Did your SRAE program need to start using any new technology because of the pandemic in order to operate or deliver your program? New technology can include laptops, integration of virtual technology such as Zoom, Microsoft Teams, Google Hangouts, WebEx, or any school-/school district–specific technology. (Close-ended response: Yes/No) [IF NO SKIP TO SECTION F].

   1. [IF E1=YES] Please describe which technologies your organization started using.

   2. [IF E1=YES] Did staff receive any training in the new technology? (Close-ended response: Yes/No)

      • [IF E1b=YES] Please describe any training staff received in the new technology, and whether you thought it was sufficient.

   3. [IF E1=YES] Please describe what worked well in using these new technologies.

   4. [IF E1=YES] Please describe what was challenging in using these new technologies.

   5. [IF E1=YES] Which new technologies, if any, will you continue to use post-pandemic, and why? If you do not plan to continue using any new technologies post-pandemic, select the box below to proceed. (Close-ended response: Check box. Not planning to continue use of new technologies post-pandemic)

6. Changes to staff supervision. Next, we ask about changes to staff supervisions during the 2020-2021 school year.

1. Did you modify the way staff were supervised during the pandemic? (Close-ended response: Yes/No)

   1. [IF F1=YES] How did you modify the way staff were supervised?

   2. [IF F1=YES] What was successful about the modifications you made to staff supervision?

   3. [IF F1=YES] What modifications to staff supervision will remain post-pandemic?

2. What was most challenging about supervising staff during the 2020-2021 school year?

Program design and adaptations

1. Virtual, hybrid, and in-person delivery. In this section, we ask questions about changes to your SRAE program delivery due to the pandemic.

1. Did any local, state or national restrictions impact your program plans? For example, travel restrictions, group size limits, etc) (Close-ended response: Yes/No)

   1. [IF A1=YES] What local, state or national restrictions impacted your programming?

2. How many sites were implementing programming during fall 2020? Please include programming that occurred in-person, remotely/virtually, or a combination of in person and remotely/virtually. If no sites implemented programming during fall 2020, enter 0.

3. [SKIP IF A1=0] Thinking about the sites that implemented programming in fall 2020, how were the programs delivered? (Close-ended response: Select all that apply) (1) In-person, (2) Remote/virtual, (3) Combination of in-person and remote/virtual.

4. How many sites were implementing programming during spring 2021? Please include programming that occurred in-person, remotely/virtually, or a combination of in person and remotely/virtually. If no sites implemented programming during spring 2021, enter 0.

5. [Skip if A2=0] Thinking about the sites that implemented programming in spring 2021, how were the programs delivered? (Close-ended response: Select all that apply) (1) In-person, (2) Remote/virtual, (3) Combination of in-person and remote/virtual.

6. How many sites [are currently implementing/will probably be implementing programming in fall 2021]? Please include programming that occurs in-person, remotely/virtually, or a combination of in person and remotely/virtually. If no sites are implementing programming during fall 2021, enter 0.

7. [Skip if A3=0] How are programs [planning to be delivered/delivered] this fall 2021? (Close-ended response: Select all that apply) (1) In-person, (2) Remote/virtual, (3) Combination of in-person and remote/virtual.

8. Compared to fall 2021, do you anticipate the number of sites able to implement programming in spring 2022 to increase, decrease, or stay about the same?

9. Please describe the things you found challenging about changing modes of program delivery throughout the year. If mode did not vary over time from fall 2020 to spring 2021, select the box below to proceed. (Close-ended response: Check box. Programming mode did not vary over time from fall 2020 to spring 2021)

10. Thinking about the 2020-2021 school year, did the mode of program delivery (in person, remote/virtual, combination of in-person and remote/virtual) vary based on any of the following? (Close-ended response: Select all that apply) (1) Site characteristics, such as participant ages and/or grade levels (specify: comment box to add details), (2) Curriculum, (3) School district requirements, (4) Other requirements of the program setting, or (5) Something else-specify).

11. [SKIP IF All in-person programming: A2 & A4 & A6=1] Among the programs you implemented virtually or as a combination of in person and remotely or virtually, did you make any adaptations to the curricula and their activities? (Close-ended response: Yes/No) For example, did you use break-out rooms in Zoom, modify activities or assignments, or make changes to interactive components of the curricula? (Close-ended response: Yes/No)

1. [IF A10=YES] Please describe adaptations made to the curricula and their activities due to virtual programming.

12. This school year (2021-2022), will you continue to use any of these adaptations or modifications made to the curricula? (Close-ended response: Yes/No)

1. [IF A12=YES] Please describe adaptations or modifications made to the curricula that you will continue with this school year.

13. [SKIP IF A1=0] In fall 2020, how many sites decreased the intended dosage delivery from what they had planned?

14. [SKIP IF A3=0] In spring 2021, how many sites decreased the intended dosage delivery from what they had planned?

15. [SKIP IF A13 & A14=0] Why did the dosage decrease? (Closed-ended response: Select all that apply) (1) To fit the school virtual implementation schedule, (2) to fit to the new delivery mode of the program (online, in person, etc.), or (3) other reasons–specify).

16. Please use this space to describe facilitators’ reactions to changes in program delivery over the last year.

17. Thinking about fall 2021, how does the expected dosage compare to pre-pandemic levels? (Closed-ended response: (1) Same dosage, (2) Decreased dosage, or (3) Increase dosage.

2. Staff training on pandemic-related changes to program delivery. The next questions ask about staff training due to pandemic-related changes to program delivery.

1. Describe any other training you have not already discussed that staff received specifically in response to the pandemic. For example, training in how to deliver programming virtually, how to engage youth, etc. If no other trainings occurred, select the box below to proceed. (Close-ended response: Check box. No other trainings occurred)

2. Will you continue using any of these trainings in the future? (Close-ended response: Yes/No)

   1. [IF B2=YES] Which of these trainings do you plan to continue in the future?

3. Please use this space to describe any specific staff training needs for the current 2021-2022 school year that are related to changes made to the program or delivery of the program in response to the pandemic. If there are no specific staff training needs, select the box below to proceed. (Close-ended response: Check box. No specific staff training needs)

3. Youth enrollment, attendance, engagement, and retention. In the next series of questions, we ask about changes to youth enrollment, attendance, engagement and retention associated with the pandemic. When responding to this series, please note in your answer if successes or challenges affected different subgroups of the youth more than others.

1. Between fall 2020 and spring 2021, did the number of youth the program intended to serve increase, decrease, or stay about the same? (Closed-ended response: Select one only) (1) increased, (2) decreased, (3) stayed about the same

2. Compared to the number of youth served during the 2020-2021 school year, [does the program intend to serve/is the program currently serving] more, fewer, or about the same number of youth this current school year? (Closed-ended response: Select one only) (1) more, (2) fewer, (3) about the same

3. Please describe any successes related to enrolling youth in programing during the 2020-2021 school year. If there were no successes related to enrolling youth during the 2020-2021 school year, select the box below to proceed. (Close-ended response: Check box. No enrollment successes in 2020-2021)

4. Please describe any challenges related to enrolling youth in programing during the 2020-2021 school year. If there were no challenges related to enrolling youth during the 2020-2021 school year, select the box below to proceed. (Close-ended response: Check box. No enrollment successes in 2020-2021)

5. Please describe any successes related to program attendance and youth retention in programing during the 2020-2021 school year. If there were no successes related to youth attendance and retention during the 2020-2021 school year, select the box below to proceed. (Close-ended response: Check box. No attendance and enrollment successes in 2020-2021)

6. Please describe any challenges related to program attendance and youth retention in programing during the 2020-2021 school year. If there were no challenges related to youth attendance and retention during the 2020-2021 school year, select the box below to proceed. (Close-ended response: Check box. No attendance and enrollment challenges in 2020-2021)

7. Please describe any successes related to youth engagement in programing during the 2020-2021 school year. If there were no successes related to youth engagement during the 2020-2021 school year, select the box below to proceed. (Close-ended response: Check box. No youth engagement successes in 2020-2021)

8. Please describe any challenges related to youth engagement in programing during the 2020-2021 school year. If there were no challenges related to youth engagement during the 2020-2021 school year, select the box below to proceed. (Close-ended response: Check box. No youth engagement challenges in 2020-2021)

   1. Were different modes of programming (virtual, hybrid, in-person) associated with differences in youth enrollment, attendance and retention, and/or engagement? (Closed-ended Yes/No)[IF C9=YES] Please use this space to describe how the different modes of programming were associated with changes in program implementation. For example, how did the different modes affect enrollment, attendance, and/or engagement?

9. Thinking about the 2020-2021 school year, did you use any types of information to measure youth engagement with programming? For example, feedback from facilitators, observing virtual programming to see how youth and staff interacted with one another, or tracking session attendance, length of engagement with materials, or submission of assignments. (Closed-ended response: Yes/No)

   1. [IF C10=YES] Describe the types of information you used to measure youth engagement with programming.

   2. [IF C10=YES] Did your engagement measures differ from previous years? (Closed-ended response: Yes/No)

      • [IF C10b=YES] Describe how these engagement measures differed from those of previous years.

• [IF C10b=YES] Do you plan to use any of these newer engagement measures in the future? (Closed-ended response: Yes/No)

Changes to data collection and evaluation activities because of the pandemic

1. Performance measures: The next series of questions ask about how your performance measure and local evaluation data collection activities changed due to the pandemic.

1. Did the pandemic change how you collected performance measures data from sites and youth during the 2020–2021 school year? (Closed-ended response: Yes/No)

1. [IF A1=YES] Please describe how the collection of performance measure data from sites and youth changed.

2. Local evaluation

1. Did you plan to collect other data, beyond the performance measures data, for your own evaluation purposes during the 2020–2021 school year? (Close-ended response: Yes/No)

2. [IF B1=YES] Were you able to collect the other data you planned to, beyond the performance measures data, for your own evaluation purposes during the 2020–2021 school year? (Close-ended response: Yes/No)

1. [IF B2=YES] Did the pandemic change how you collected other data for your own evaluation purposes? For example, did you collect data in a different way or using different measures because of the pandemic? (Close-ended response: Yes/No)

   • [IF B2a=YES] Describe how the pandemic changed how you collected other data for your own evaluation purposes.

     • [IF B2a=YES] Are there changes to your data collection approach that you will retain post-pandemic? (Closed-ended response: Yes/No)

• [IF B1a=YES AND YES TO RETAINING DATA COLLECTION APPROACH] Please describe the changes to your data collection approach you will retain and your reasons for retaining them.

2. [IF B2 = NO] Please describe why you were not able to collect the other data you planned to, beyond the performance measures data, for your own evaluation purposes during the 2020-2021 school year.

3. [IF B2 = NO] Do you plan to resume data collection for your own evaluation purposes during the 2021–2022 school year? (Close-ended response: Yes/No)

1. [IF B3=YES] Please describe the data you plan to collect and how you plan to collect them.

4. Please use this space to describe any specific technical assistance needs you have related to upcoming data collection and evaluation activities. If you have no additional technical assistance needs related to upcoming data collection and evaluation activities, select the box below to proceed. (Close-ended response: Check box. No specific technical assistance needs for data collection and evaluation activities)

The future of programming

1. Lessons learned

1. As you think about how your program responded to the pandemic, were there any lessons learned that changed your plans for programming for the current 2021-2022 school year that you have not shared previously? (Closed ended response: Yes/No)

   1. [IF A1=YES] Please describe the lessons learned and how they will affect or have affected changes in your programming for the current 2021-2022 school year.

B. Impact of pandemic on youth served by program

1. How have the youth you served been most affected by the pandemic? (For example, impacts on mental health, physical health, behavior, family, etc.)

2. What do you think will be most challenging for youth in the coming year? (For example, gaps in knowledge or learning, risky behaviors, mental health, etc.)

Closing

1. Please discuss any additional training or technical assistance needs you foresee needing in the 2021-2022 school year, not already discussed. If you have nothing additional to add, select the box below to proceed. (Close-ended response: Check box. Nothing additional to add)

2. Please provide any additional information not discussed in this interview that you would like us to know, as you think about the 2021-2022 school year. If you have nothing additional to add, select the box below to proceed. (Close-ended response: Check box. Nothing additional to add)

Thank you for participating in this interview.

Mathematica 7