Attachment A_Tier 1 TPP_InterviewGuide_GRANTEE_5.23.2022

Form Approved

OMB No. XXXX-XXXX

Exp. Date XX/XX/20XX

TPP Tier 1: Grantee Interview Guide

Introduction & Confidentiality Information

[Lead interviewer introduces self and introduces note-taker (“who will be taking detailed notes and asking a few questions”). Thank interviewee for participating, and for filling out the pre-interview form (if they did)]

As you may know, we/[Abt Associates and our partner DIR] have been contracted by the Office of Population Affairs (OPA) to conduct the TPP Tier 1 and Tier 2 Implementation Evaluation. As an important part of that effort, we are talking to all 62 Tier 1 grantees and some partner organization staff for about half of the Tier 1 grantees [including yours]. These interviews are meant to help us understand and document your experiences and processes as you planned and implemented your TPP project. This is not a compliance assessment or audit of any kind. The information we gather will help OPA, practitioners, and other TPP partners shape future strategies for preventing teen pregnancy, reducing rates of STIs, and improving adolescent health.

If there are any questions you do not feel comfortable answering or want to skip just let us know and we will move on to the next question. Your participation is voluntary. You may choose not to discuss any topic or to end the discussion at any time. The information we collect during this interview and information gathered from program documents and the pre-interview form will be combined with data collected from other Tier 1 TPP projects to identify implementation themes across grantees, communities, and target populations. Your names will not appear in the public-facing final report. That report also will include our findings from interviews with Tier 2 grantees. It will also compile one-page profiles created for each Tier 1 and Tier 2 grantee, which will include basic information about your project, including the names of the grantee and major partner organizations (but no individuals’ names). The notes we take during this discussion will not be shared outside of the research team—that is, not with OPA, with other grant staff, partners, or community members. With your permission, we would also like to record the interview to ensure that we capture all of the details correctly. Do I have your consent to record the interview?

[If applicable: As I mentioned in the e-mail, we are also doing interviews with grant partners to gather additional context. These will be slightly shorter interviews but on similar themes. We can talk about how to pick the right partner or partners for these after the interview.]

Before we start, do you have any questions?

I’m going to start with a few questions about the development of the TPP project, followed by some questions about the project’s partners and structure, the project’s settings and overall reach, the evidence-based interventions being implemented, the services included in the project, and how the community and youth have been engaged in the project. We’ll wrap up with a few questions about your accomplishments to date and lessons learned.

*begin recording*

Community and Organizational Context

I’d like to start with a few questions about your TPP project and how it got started.

1. According to our records, your TPP Tier 1 project is called “[name from application].” Is that how you refer to it? Or, if not known: Is there a name you call the Tier 1 TPP project within your community?

2. How did the TPP project come about? (Probe: Why did you apply for the grant? What were you hoping to achieve? Did it grow out of other initiatives, collaborations, or concerns in your community?)

3. As the lead organization, what is [grantee name]’s role in implementing the TPP project? (confirm from application/pre-interview form – intermediary, provider, technical assistance provider/trainer, etc).

   1. What is your role? (PD, program manager, coordinator, etc.)

4. We understand your service area is [insert service area/communities served]. Why did you select this service area? (Probes: based on population, grantee relationship with the community or partners, specific local priorities, etc.?)

5. What resources or characteristics of the community or service area are helpful in implementing a TPP project? These could be cultural factors, services, institutions, relationships, or priorities. (Probe for how/why they are helpful)

6. What characteristics of the community or service make implementing the TPP project more challenging?

7. [For FY21 Grantees Only] What modifications did you need to make to the TPP project to implement it in two years rather than the three you had planned?

8. Overall, how has the COVID-19 pandemic affected implementation of the TPP project?

   1. How has the project’s planning and implementation had to adapt because of the pandemic?

Partnerships and Project Structure

My next few questions are about how your TPP project is structured and what role formal partnerships play. By formal partnerships, I mean organizations that have received grant funds or that have an MOU with your organization.

9. [If pre-interview form not completed] How many formal partnerships do you have? What organizations do you have formal partnerships with to implement the TPP Project? What did you look for in identifying formal partners? (e.g., referral source, service provider…) Are there particular partners that you consider to be primary or “anchor” partners?

   1. What types of organizations are these (e.g., non-for profit/community-based organization, secondary education institutions, post-secondary institution, health clinic, etc.)?

10. [If pre-interview form is completed:] Thank you for completing the pre-interview form. I understand that you have [#] formal partners on the TPP Project. What did you look for in identifying partners? Are there particular partners that you consider to be primary or “anchor” partners?

    1. [If YES]: Please tell me a bit about their roles.

11. Are there any other organizations that are not formal partners—i.e., your organization does not have an MOU with them or they are have not received grant funds—but have a substantial role on the TPP project? [If YES, ask interviewee to list these partners and their roles].

12. Have there been any changes in partners or their roles from what you originally planned?

    1. [If YES:] What led to these changes? (Probe on challenges/opportunities that contributed to these changes)

13. The TPP grant program was designed with a “systems-thinking” approach in mind. What do you see as the most important systems in your area? [If needed: Systems thinking is the idea that different systems, agencies, major organizations, and institutions in communities are inter-related and that programs that support participants across multiple systems or points of contact can have a bigger impact.]

14. What are the main systems and leverage points that your TPP project targets? [If needed: “Leverage points” are places within a larger system or process where interventions can change a process, knowledge, or experience to improve outcomes for people who receive an intervention directly or who are effected by the system.]

    1. In what ways do you see these systems and intervention points as related to teen pregnancy prevention?

    2. To what degree do you think looking at a system-wide or multiple-system approach is helpful or necessary in achieving positive outcomes? (If not addressed: In what ways is it helpful?)

    3. Did you experience any barriers to engaging key systems or agencies in your approach? If so, please describe which systems or agencies and what the barriers encountered were.

Community Reach and Settings

We’d like to understand a bit more about your target population(s), the systems you are working in, and settings you are working to reach. By “systems” I mean organizations, institutions, or networks (for example, a school system), by “settings” I mean the types of location (for example, a middle school class) and by “sites” I mean a specific local (for example, a specific middle school).

15. For the TPP project, how does your organization think about concepts like scale and reach? (If needed: Generally, “reach” refers to how many people an intervention serves directly, and scale refers to how many people the interventions serve relative to how big and broad the target population is).

16. I understand your TPP project focuses on reaching [target population(s)]. How was/were these population(s) chosen? (Probes: were these selected based on disparities identified in a needs assessment, partner or setting focus populations, or grantee and partner organization focus and mission).

17. How are participants engaged to participate in evidence-based interventions (EBIs) and other programming?

    1. Who’s involved in engaging participants?

    2. What steps does the project take to engage participants?

    3. Where does the project engage participants?

18. We have that your TPP project implements programming in [settings for programming]. Are there any other settings the project is implementing EBIs and other programming in?

19. How were settings for programming chosen?

    1. What features made them a good fit?

    2. What was the process for choosing settings and specific sites within those settings (formal or informal)?

    3. Who was involved in choosing settings and sites? (Probe for organizations/community members/individuals outside of the grantee organization)

    4. What relationships did you or partners need to establish with setting or site staff? (Probe: How did you go about establishing these relationships?)

20. Do you deliver programming in the sites and settings originally planned? (Probe for if sites have been added or original ones dropped?)

If changes were made:

1. What has changed? (probe for new settings added as well as settings removed)

2. What led to these changes?

3. How have these changes affected your ability to deliver TPP programming?

21. To what degree have you been able to achieve the scale you had hoped? (Interviewer note: acknowledge that the scale and reach goals may have changed based on pandemic conditions) [Interviewer note, check performance measures for intended scale prior to interview.]

    1. What factors affect have affected scale? (Probe for supportive factors and challenges).

22. To what degree have you been able to reach the specific populations the project originally hoped to serve? [Interviewer note, check preliminary data review for intended service population.]

    1. What factors have affected your ability to reach these populations?

    2. Is it difficult for some youth to participate in EBIs or other programming your TPP project is offering? Which youth experience barriers to participation? (If not already answered: What do those barriers look like?).

    3. How do you try to address these barriers? (Probe: Is there an approach the project put in place from the start or something you are figuring out as you notice new challenges?)

Evidence-Based Interventions (EBIs)

So now I’d like to ask some questions about the evidence-based interventions, or EBIs, being implemented.

23. I understand that your TPP project is implementing the following EBIs in the following settings:

[List EBIs and the associated settings].

1. Who delivers these EBIs? (e.g., grantee, sub-awardee(s), setting providers, some combination)

24. How were the EBIs chosen?

    1. What were the most important criteria?

    2. Who was involved in the decision? (Probes: setting providers, partners delivering EBIs, youth stakeholders, grantee only, other stakeholders)

    3. Did you use only interventions that your organization or partners were familiar with, or take a new look at what would be a good fit for the community?

    4. How did you determine that the EBIs you chose were a good fit?

25. How did you identify “evidence-based” interventions?

    1. Did you select from an evidence-review list? And, if so, did the existing evidence-reviews have programs you deemed were a good fit for your populations, communities, and settings?

26. As part of the TPP project, have you been able to provide all of the EBIs you originally planned?

    1. [If YES:] In the settings and sites you originally planned?

    2. [If NO:] Did you add EBIs or not implement certain EBIs? What led to the change in plans?

27. How, if at all, have you had to adapt EBIs as a result of the COVID-19 pandemic?

    1. Are these adaptations that will continue after the pandemic has ended?

    2. Have you had to adapt EBIs for reasons other than the pandemic?

    3. How did you decide on these adaptations?

    4. How do you think these adaptations may have affected fidelity to the EBI, if at all?

    5. What are the challenges you’ve experienced with these adaptations?

    6. What are the facilitators of these adaptations?

    7. What are the benefits and drawbacks of the adaptations?

28. What has the community response been like for the EBIs? (Interviewer probes: which had youth engagement, served the most participants, reaching target populations)

    1. Which do you feel have the most community or institutional support?

    2. Which have the most youth engagement—either participants, or other youth stakeholders and advisors?

    3. With which have you been able to serve the most participants?

    4. Are some EBIs particularly effective in serving or reaching specific populations?

    5. [If not addressed]: What factors—of the interventions, your approach, or the community—have contributed to the community’s response?

Other Services

My next set of questions are about the systems the EBIs are being implemented in and any other services being offered.

29. In addition to delivering EBIs to youth, I understand your project also [insert understanding of other services and interventions offered from applications, progress reports, and pre-interview survey]. Is that right? Is there anything I’m missing?

    1. If not addressed: How does the project aim to improve availability, services, or linkages to youth friendly healthcare?

    2. If not addressed: Does the project support linkages to other services beyond youth-friendly healthcare? If YES, what services, and how?

    3. What kinds of benefits do you see in providing this support?

30. Who provides these services (e.g., grantee, formal partner, other partner). How are participants connected to services (e.g., provide on site with EBIs, referrals to partners, etc.)?

31. What has your experience been in providing these services or linkages?

    1. Were you able to provide all services planned (either directly or through referrals)?

    2. Have there been any challenges? (If YES, in what ways? How have you tried to address them?)

    3. To what degree are services and support integrated with or related to the EBIs?

Community and Youth Engagement

My next set of questions ask about how the TPP project is involving the community.

32. Were community groups or members involved in designing the TPP project, or in refining it? (Probes: This may include formal or informal community groups, partners, or individual community members.)

    1. Who was involved? What kinds of organizations or individuals?

33. Were youth participants involved in designing or refining the project?

    1. In what ways were youth involved, and at what points in the project’s development and implementation?

34. Are there any formal groups involved in the project? This could be community advisory groups, youth advisory groups, or related coalitions. [Probe for how/when/for what purpose these groups formed, their relationship to the grantee/subawardees.]

    1. What are their roles on the project? (If applicable: How often/when do you consult them?)

35. [If not already addressed]: To what degree are youth engaged in each step of the TPP project? (Probes: in choosing settings and EBPs, reviewing or adapting EBPs, conducting outreach to the public, identifying service needs, supporting peers, or gathering ongoing feedback on the interventions and services?)

Lessons Learned

In the last few minutes of our time together, I’d like to ask about some of your lessons learned in implementing the TPP project.

36. What do you see as the most successful aspects of the TPP project so far?

    1. What makes these successful?

    2. What factors do you think have allowed the project to accomplish what you have so far? (These could be actions the project took, factors in your community, or both)

37. What have been the biggest challenges in implementing this project? Is there anything you would do different from the start, if you had it to do over again? [Interviewer note: Probe to understand what they would do differently for the whole project, not just specific pieces.]

38. [If not fully addressed]: How did your approach overall change because of the COVID-19 pandemic?

39. If you were to design a TPP grant from scratch, how, if at all, would it differ from the current TPP Tier 1 grant program?

Wrap-up

Those are all of the questions we have. I realize we’ve covered a lot, but is there anything we haven’t asked that you want to make sure we understand?

Thank you for being so generous with your time. If we need to clarify something in our notes later, we may get in touch to make sure we got it right.

[If applicable: If you have a few minutes, we can talk about the partner interviews (if not, we can do this by e-mail).]

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is XXXX-XXXX. The time required to complete this information collection is estimated to average 120 minutes per response, including the time to review instructions and gather the data needed, to review and complete the information collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: U.S. Department of Health & Human Services, OS/OCIO/PRA, 200 Independence Ave., S.W., Suite 336-E, Washington D.C. 20201, Attention: PRA Reports Clearance Officer