Attachment B_Tier 1 TPP_InterviewGuide_PARTNER_5.23.2022

Form Approved

OMB No. XXXX-XXXX

Exp. Date XX/XX/20XX

TPP Tier 1: Partner Interview Guide

Introduction & Confidentiality Information

[Lead interviewer introduces self and introduces note-taker (“who will be taking detailed notes and asking a few questions”). Thank interviewee for participating, and for filling out the pre-interview form (if they did)]

As you may know, we/[Abt Associates and our partner DIR] have been contracted by the Office of Population Affairs (OPA) to conduct the TPP20 Tier 1 and Tier 2 Implementation Evaluation. As an important part of that effort, we are talking to all 62 Tier 1 grantees and some partner organization staff for about half of the Tier 1 grantees. These interviews are meant to help us understand and document your experiences and processes as a partner in building and implementing [grantee name]’s TPP project. This is not a compliance assessment or audit of any kind. The information we gather will help OPA, practitioners, and other TPP partners shape future strategies for preventing teen pregnancy, reducing rates of STIs, and improving adolescent health.

Some questions may ask for your opinions based on your experience implementing aspects of the TPP project as a partner, and some may ask for your assessment of how the project is working overall. If there are any questions you do not feel comfortable answering or want to skip just let us know and we will move on to the next question. Your participation is voluntary. You may choose not to discuss any topic or to end the discussion at any time. The information we collect during this interview and information gathered from program documents will be combined with data collected from other Tier 1 TPP projects to identify implementation themes across grantees, communities, and target populations. Your names will not appear in the public-facing final report. That report also will include our findings from interviews with TPP Innovation and Impact (Tier 2) grantees. We will also produce a set of one-page site profiles describing each grantee, which will include basic information about the project, including the names of the grantee and major partner organizations (but no individuals’ names). The notes we take during this discussion will not be shared outside of the research team—that is, not with OPA, with other grant staff, partners, or community members. With your permission, we would also like to record the interview to ensure that we capture all of the details correctly. Do I have your consent to record the interview?

Before we start, do you have any questions?

I’m going to start with a few questions about the development of the TPP project, followed by some questions about your partnerships and project structure, which settings the project is operating in and its planned reach, the evidence-based interventions being implemented, the systems and services included in the project, and how the community has been engaged in the project. We’ll wrap up with a few questions about lessons learned.

*begin recording*

Community and Organizational Context

I’d like to start off with a few questions about your organization and its role in the development of TPP project.

1. Can you tell me a little bit about [your organization] and your role within the organization?

2. What is [your organization]’s role in implementing the (current) TPP project. [Confirm from grantee’s description.]

3. And what is your role on the TPP project?

4. How and when did your organization become involved in the TPP project? (Probes: Did it grow out of other initiatives, collaborations, or concerns in your community? Did you have a role on other current or past TPP related programs with the grantee and/or in the community?)

5. How does the TPP project fit in with your organization’s work or mission?

6. What are the challenges to addressing teen pregnancy, STIs, and overall adolescent health in your service area? These could be cultural factors, services, institutions, relationships, or priorities. (probe for how/why they are helpful)

   1. What will the TPP Project do to help address these challenges?

7. Thinking more about the community context, are there particular challenges for implementing a TPP effort in your community(ies)? (Interviewer note: Skip if answered in Q6.)

8. Overall, how has the COVID-19 pandemic affected implementation of the TPP project?

   1. What changes have affected the project planning and implementation?

   2. How has the project had to adapt because of the pandemic?

   3. Has your organization’s role on the TPP project changed because of the pandemic?

Partnerships and Project Structure

9. On the TPP project, in what ways does your organization collaborate with [grantee]? Please provide details about your role in the planning and implementation of the TPP project.

1. Has your role changed over time, and if so, in what ways?

2. How and how often does your organization meet or communicate with [grantee]? Has this changed over time?

10. Are there other TPP project partners your organization works with regularly as part of the project? How often do you talk with or meet other partners?

Community Reach and Settings

We’d like to understand a bit more about the participants you serve, the systems you are working in, and settings you are working to reach.

11. If applicable: I understand you are working with the following populations on the TPP project: [XXX]. Is that correct?

12. Were you involved in selecting the target population for the TPP project? Are there other target populations you would have liked to see included given the needs in the community?

13. If applicable: I understand you are working in [XXX] settings for the TPP project. Is that correct?

14. Does your organization recruit youth for TPP project services? If so, how are participants first engaged or recruited to participate in EBIs and other programming? If not, what is your understanding of how participants are recruited?

    1. Who’s involved in engaging participants?

    2. What steps does the project take to engage new participants in programming?

    3. In what settings does the project engage participants?

15. What is your understanding of how the settings for delivering interventions were chosen? Was your organization involved in selecting these settings? [Ask following based on org’s role]

    1. What features made your/the setting(s) a good fit for the services?

    2. How did you/how did the grantee choose settings?

    3. What stakeholders did your organization involve/were involved in choosing settings?

    4. What relationships, if any, did your organization need to establish or strengthen with providers or host sites? (Probe for how they went about establishing these relationships)

16. If applicable: Do you deliver programming in the sites and settings originally planned?

    1. If NO: What has changed? (probe for new settings added as well as settings removed)

    2. If NO: What prompted these changes?

    3. If NO: How have these changes affected your ability to deliver TPP programming?

17. To what degree do you think the project has been able to achieve the scale originally planned? (Note: Acknowledge that the scale and reach goals may have changed based on pandemic conditions)

    1. What factors have affected scale? (probe for supportive factors and challenges).

    2. Does your organization have its own service targets to meet (e.g., number of participants or referrals) for the TPP project?

18. Are you aware of any challenges in reaching specific populations the project originally hoped to serve?

    1. What factors have affected the project’s ability to reach these populations?

    2. Is it difficult for some youth to participate in EBIs or other programming the TPP project is offering?

    3. What youth experience barriers to participation? (If not already answered: What do those barriers look like?).

    4. How if at all has your organization been involved in addressing these barriers? (e.g., is there an approach the project put in place from the start or something you are figuring out as you notice new challenges?)

Evidence-Based Interventions

My next set of questions are about the evidence-based interventions (or EBIs) that are being implemented as part of the TPP project.

19. Is your organization involved in delivering or hosting EBIs, and if so, which EBIs and in which settings? [Interviewer note, prefill based on pre-interview form responses.]

20. Was your organization involved in choosing or vetting EBIs?

    1. [If YES]: How did you determine if a program was considered “evidence-based”? (Probes: What was the process? How were you involved? What were the most important criteria? Who else was involved in the decision?—setting providers, partners delivering EBIs, youth stakeholders, grantee only, other stakeholders? Did you use interventions the grantee, partners, or communities were familiar with, or take a new look at what might be a good fit for the community?)

    2. Did you look at an evidence-review list to find specific programs? And, if so, did the existing evidence-reviews have programs that were a good fit for your populations, communities, and settings?

    3. How was it determined that the EBIs would be a good fit for the TPP project’s target populations, communities, and settings?

21. If applicable/not answered in questions 19 or 20: As part of the TPP project, have you been able to deliver the EBIs you originally planned?

    1. [If YES]: Were you able to implement these in the sites and settings you originally planned?

    2. [If NO]: What changed? (EBIs offered, who’s offering them, etc) What led to the change in plans?

22. Have you been involved in deciding how or whether to adapt EBIs as a result of the COVID-19 pandemic, or other local context/needs?

    1. How, if at all, have you had to adapt EBIs as a result of the COVID-19 pandemic?

    2. How did you decide on these adaptations?

    3. Have you had to adapt EBIs for any reasons other than the pandemic?

    4. Do you think these adaptations for COVID-19 affect fidelity to the original model? Why or why not?

    5. What are the downsides or challenges you’ve seen with these adaptations?

    6. What are the upsides or benefits of these adaptations?

23. In your experience, how has the community responded to the EBIs?

    1. Which, if any, EBIs have the most community or institutional support?

    2. Which EBIs have the most youth engagement—either participants, or other youth stakeholders and advisors?

    3. With which EBIs have you been able to serve the most participants?

    4. Are some (or none, or all) EBIs particularly effective in serving or reaching specific populations?

    5. [If not addressed]: What factors—of the interventions, your approach, or the community—have contributed to this experience?

Systems and Services

Now, I’d like to ask a few questions about the systems and services being provided as part of the TPP project.

24. In your experience, how does the TPP project aim to improve youth friendly service availability, services, or linkages? (By linkages, we mean making connections among service providers, not necessarily directly connecting youth to a service.)

25. Does the project support linkages to other services beyond youth-friendly healthcare? If YES, What services, and how?

26. [If applicable]: What has your experience been in providing these services or linkages to youth?

    1. Have there been any challenges? (If YES, In what ways? How have you tried to address those challenges?)

    2. What kinds of benefits do you see in providing this support?

    3. To what degree are services and support integrated with or related to the EBIs?

27. The TPP grant program was designed with a “systems-thinking” approach in mind. What do you see as the most important systems in your area? [If needed: Systems thinking is the idea that different systems, agencies, major organizations, and institutions in communities are inter-related and that programs that support participants across multiple systems or points of contact can have a bigger impact.]

28. What are the main systems and leverage points that the TPP project targets? [If needed: “Leverage points” are places within a larger system or process where interventions can change a process, knowledge, or experience to improve outcomes for people who receive an intervention directly or who are effected by the system.]

    1. In what ways do you see these systems and intervention points as related to teen pregnancy prevention?

    2. To what degree do you think looking at a system-wide or multiple-system approach is helpful or necessary in achieving positive outcomes? (If not addressed: In what ways is it helpful?)

Community Engagement

29. To what degree was your organization involved in designing the TPP project, or refining it? (If not addressed: to what degree were you involved?)

    1. What has this process looked like? (If unclear, probe for more detail)

    2. What other stakeholders have been involved?

30. To your knowledge, how have youth participants been involved in designing or refining the project?

    1. In what ways were youth involved, and at what points in the project’s development? Did you help engage youth? (Probes: In choosing settings and EBIs; reviewing or adapting EBIs; conducting outreach to the public; identifying service needs; supporting peers; gathering ongoing feedback on the interventions and services?)

    1. In what ways were youth successfully engaged in the project design phase specifically? How was that engagement facilitated?

31. Are there any formal groups involved in the project planning or implementation? This could be community advisory groups, youth advisory groups, or related coalitions? [Probe for how/when/for what purpose these groups formed, their relationship to the grantee/subawardees].

    1. Are you or other members of your organization part of one of these groups?

    2. What are their role(s) on the project? (If applicable: How often/when does the TPP project consult them?)

    3. Tell me a bit more about this group – what is its role overall?

    4. Was it created for the this TPP project, or did it already exist?

Lessons Learned

32. What do you see as the most successful aspects of the project so far?

    1. What factors do you think have facilitated the project’s accomplishments to date? (These could be actions the project took, factors in your community, or both)

33. In your experience, what have been the biggest challenges in implementing this project? Is there anything you would do different from the start, if you had it to do over again? [Interviewer note: Probe to understand what they would do differently for the whole project, not just specific pieces.]

34. [If not fully addressed]: How did your approach overall change because of the COVID-19 pandemic?

35. If you were to design a TPP grant from scratch, how, if at all, would it differ from the currect TPP Tier 1 grant program?

Wrap-up

Those are all of the questions we have. I realize we’ve covered a lot, but is there anything we haven’t asked that you want to make sure we understand?

Thank you so much for being so generous with your time. If we need to clarify something in our notes later, we may get in touch to make sure we got it right.

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is XXXX-XXXX. The time required to complete this information collection is estimated to average 60 minutes per response, including the time to review instructions and gather the data needed, to review and complete the information collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: U.S. Department of Health & Human Services, OS/OCIO/PRA, 200 Independence Ave., S.W., Suite 336-E, Washington D.C. 20201, Attention: PRA Reports Clearance Officer