Form Approved
OMB No. XXXX-XXXX
Exp. Date XX/XX/20XX
[Lead interviewer introduces self and introduces note-taker (“who will be taking detailed notes and asking a few questions”). Thank interviewee for participating, and for filling out the pre-interview form (if they did)]
As you may know, we/[Abt Associates and our partner DIR] have been contracted by the Office of Population Affairs (OPA) to conduct the TPP20 Tier 1 and Tier 2 Implementation Evaluation. As an important part of that effort, we are talking to all 13 Tier 2 Network grantees and partner organization staff from each Tier 2 Network grantee. These interviews are meant to help us understand and document your experiences and processes as you built your innovation and impact network and explored, developed, tested, refined, and evaluated innovative interventions to improve adolescent health and prevent teen pregnancy and sexually transmitted infections (STIs) in your selected priority area. This is not a compliance assessment or audit of any kind. The information we gather will help OPA, practitioners, and other TPP partners shape future strategies for preventing teen pregnancy, reducing rates of STIs, and improving adolescent health.
If there are any questions you do not feel comfortable answering or want to skip just let us know and we will move on to the next question. Your participation is voluntary. You may choose not to discuss any topic or to end the discussion at any time. The information we collect during this interview and information gathered from program documents will be combined with data collected from other Tier 2 projects to identify implementation themes across grantees, communities, and priority areas. Your names will not appear in the public-facing final report. That report also will include our findings from interviews with the other tier of grantees awarded funding around the same time (the TPP Tier 1 grantees). We will also produce a set of one-page site profiles describing each grantee, which will include basic information about the project, including the names of the grantee and major partner organizations (but no individuals’ names). The notes we take during this discussion will not be shared outside of the research team—that is, not with OPA, with other grant staff, partners, or community members. With your permission, we would also like to record the interview to ensure that we capture all of the details correctly. Do I have your consent to record the interview?
As I mentioned in the e-mail, we are also doing interviews with grantees’ partners to gather a bit of additional context. These will be slightly shorter interviews but on similar themes. We can talk about how to pick the right partners for these after the interview.
Before we start, do you have any questions?
I’m going to start with a few questions about the development of the TPP project, followed by some questions about how you’ve been implementing the project, your engagement of partners and youth, how you have had to change and adapt the project over time, and your program evaluation plans. We’ll wrap up with a few questions about your accomplishments to date and lessons learned.
*begin recording*
I’d like to start off by asking a few questions about the development of the TPP project.
We understand your Tier 2 teen pregnancy prevention project is called “[name from application/pre-interview form].” Is that how you refer to it?
As the lead organization, what is [grantee organization name]’s role in planning and implementing the TPP project? (Confirm role from preliminary data review – intermediary; establish and support the network; explore need, resources, interventions; develop new interventions; test interventions; refine interventions; evaluate interventions; dissemination, etc.)
What is your role? (PD, program manager, coordinator, etc.)
How did the TPP project come about? What was the impetus for submitting the grant application? (Probes: Did it grow out of other initiatives, collaborations, or concerns in your community?)
Who was involved in the initial planning efforts for the project? (Interviewer note: this includes the proposal stage and the first 6-ish months of project start up post award)
(Probes: community based groups, coalitions, committees? Youth? Sub-awardees? Individuals from priority area?)
a. How were they involved?
What factors led to the selection of [priority area from application] as the priority area for the TPP project?
Who was involved in the priority area selection process and how?
[Confirm service area from the preliminary data review and, if available, completed pre-interview form.] We understand your project is operating in [service area]. What went into the decision to focus your efforts on this/these community(ies)? (Probes: were these data-driven decisions based on demographics or statistics on teen pregnancy rates, existing relationships built to support current efforts, a request from the community, etc.)
I understand you have [# from Performance Measures/pre-interview form] formal partners working with you on the TPP project. Is that correct? What criteria did you use to identify and engage partners for the TPP project?
How did you identify and recruit youth to participate in the project planning and implementation?
What roles do your network partners and youth have in implementing the TPP project?
When planning the network, how did you identify the systems that could be leveraged to make an impact in the key priority area? Did you use particular tools? (Probe: environmental scan? Stakeholder map?) If so, please describe.
In developing and implementing the TPP project network, did you use an overarching framework? Has this framework changed over time? If so, how has it changed and why?
Next, I’ll be asking a few questions about implementation of the TPP project.
I understand from your most recent Intervention List that you have developed [##] interventions that are in the following stages of development: [list interventions and current phase of development]. Is this correct?
Can you talk through how you went about exploring, selecting, and developing these interventions?
For each intervention, ask:
How did you identify that the intervention was needed?
Who was involved in the exploration and development process and how?
Was the intervention newly developed or adapted from an existing intervention?
From your perspective, what makes the intervention innovative?
Which types of settings are you implementing interventions in (e.g., clinics, juvenile detention facilities, schools, etc.)?
Are interventions being implemented in-person or remotely? (Probe for whether the modality had to change because of the pandemic, and if so how it changed and whether the change affected expected participation or outcomes in any way.)
What factors went into your decisions to select these settings? Who was involved in those decisions?
Have you considered ways to make interventions more flexible so they can be implemented in multiple settings or modalities (e.g., in-person/virtual/hybrid or modifying interventions to be more accessible in different settings)? If so, describe.
How did you move interventions from the exploring and developing phase to the testing, refining, and evaluating stages?
How did you know or assess when an intervention was ready to move to the testing stage?
What types of modifications, if any, did you make to interventions during the testing and refining stages? How did you determine what types of modifications to make? Who was involved in those decisions?
Did you decide to terminate any interventions before moving them on to the next stage? Which ones? How did you make that determination?
What challenges did you face in moving from the exploring and developing phase to the testing, refining, and evaluating phase? Did challenges differ by intervention?
Were you able to overcome those challenges? If so, how?
Looking back, are there things you would do differently?
What advice would you give to a new Tier 2 grantee about how to do this process?
For interventions that have entered delivery (testing, refining, and evaluating), how has the network’s role changed? Probe for:
How the grantee works with partners to monitor progress, and how frequently (e.g. intervention check ins, trainings, team/network updates, in-depth reviews)
Challenges the network has encountered during the delivery phase
How the network adapted when challenges arose
How have you had to adjust your implementation of the TPP project due to challenges surrounding the COVID-19 pandemic? (We’ll have another question specifically about how the COVID-19 pandemic affected your plans for partnering a little later.)
Thinking about the community context, have you experienced any challenges, other than the COVID-19 pandemic, in implementing the TPP project? If so, please describe. (Probes: e.g., cultural, socioeconomic, political, geographic, public health, social norms/values).
How does the TPP project benefit from the capacity and strengths of the community you are operating in?
Have you encountered barriers to progress during implementation, other than those you’ve already mentioned? If so, what were the barriers? Probe for:
How the barrier was addressed
The role of the grantee in removing barriers
The role of network members in removing barriers
The role of youth in removing barriers
Our next set of questions are about your engagement of partners and youth in the project.
As the grantee, what is your role in supporting and coordinating the network?
Has the composition of your network changed over time? If so, what led to those changes?
What strategies do you use to engage your formal partners in the network?
Do you use different strategies to engage informal partners in the network?
Which strategies have worked best for partner engagement?
How have you involved participants from the priority area in planning and implementation?
What role has your organization played in engaging these participants?
Have you had any concerns about equity or access issues related to engaging participants from the priority area? If not, what has the network done to ensure equity and access?
Has your approach to partnering changed over time?
What led to these changes? (Probe: additional subject matter expertise needed; needs changed as interventions began to be implemented)
In what ways, if at all, has the network affected the way agencies collaborate and coordinate to develop new interventions to reduce teen pregnancy and STIs?
Have you had to make any changes to your plans for partnering because of the COVID-19 pandemic? If so, please describe what changes you made and for what purpose.
How has the pandemic affected equity or access issues?
How have you been reaching youth during pandemic?
Did the COVID-19 pandemic have any unintended positive consequences on your partnering or youth engagement (e.g., expanded reach)?
What lessons have you learned about establishing an innovation and impact network? (Probe: staffing and managing a successful team, developing initial partnerships, educating stakeholders about the network.)
If you were starting from scratch, what would you do differently?
If you encountered resistance to partnering, how did you overcome that challenge?
Thinking about the overall makeup and structure of your network, what factors do you think have been most helpful?
Now, I’m going to ask a couple questions about adaptations you’ve needed to make to the TPP program.
What types of pivots did your network need to make in response to the COVID-19 pandemic?
What factors do you think made it possible for you to adapt to changes required by the pandemic? What worked well?
What challenges did you have?
How did you try to overcome these challenges?
How adaptable are the interventions developed to changing conditions that might be required to respond to safety precautions around the COVID-19 pandemic? (Probe: Any changes to implementation settings?)
My next set of questions are about program evaluation.
What approach did your network take to producing a program evaluation plan?
What partners were involved in the process? What was your role?
How did you determine when an intervention was ready for evaluation? Dissemination?
Who was involved in the decision?
What methods or metrics do or will you use to evaluate the intervention’s effectiveness or readiness for evaluation?
How many interventions have you moved into the evaluation phase? How many into the dissemination phase?
How are you measuring impact in your selected priority area?
What impacts have you seen to date?
Are these impacts what you were expecting to see at this point? If not, explain.
Can you describe the dissemination activities you’ve completed for interventions determined to be effective? (Probe: how were interventions disseminated so they are accessible to others?)
Who is involved in the dissemination activities?
What are your target audiences for dissemination?
What feedback, if any, have you received on the disseminated interventions?
Before we finish up, I’d like to ask a few questions about your accomplishments and lessons learned.
What would you say have been your two or three most important accomplishments?
What do you consider to be the key factors in your accomplishments date? Probe for:
What specifically about the network has contributed to these accomplishments?
What specifically about engagement of individuals from the priority area contributed to your success?
Overall, what have you learned about developing and delivering innovative new interventions to prevent teen pregnancy and STIs?
What conditions are necessary to enable a community to move beyond status quo approaches to preventing teen pregnancy?
What did you do that you thought was particularly effective? Should be replicated?
Is there anything you would do different from the start, if you had it to do over again?
Those are all of the questions we have. I realize we’ve covered a lot, but is there anything we haven’t asked that you think would be particularly helpful in understanding the TPP project?
Thank you for being so generous with your time. If we need to clarify something in our notes later, we may get in touch to make sure we got it right. If you have a few minutes, we can talk about the partner interviews (if not, we can do this by e-mail).
If they have a few minutes left to discuss partners:
As I mentioned, we would also like to interview one or two of your partners. Those interviews will cover the same topics, where applicable, but will be a bit shorter (no more than an hour).
We have a couple of guidelines for deciding on the partners, but these are somewhat flexible:
The partner(s) should be organizations with which you, as the grantee, have a formal relationship – either a subgrant or a Memorandum of Understanding.
Ideally, the partner(s) should be from an organization that is substantially involved in the network, in the development, testing, or refining of new interventions, in the community advisory group, or was involved in the vision for the project, or design and implementation of the TPP project. We would also be interested in speaking with a partner organization that is delivering one or more of the newly developed interventions for the project in one or more settings.
Do you think you have one or two partners who fit this description and could put us in touch with the right contact?
[If YES]: Great! Just let me know what’s the best way to get us in touch. Either you could give us an e-mail introduction or let them know we’ll be in touch and give us their contact information to get in touch directly. Whatever works best for you and them.
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is XXXX-XXXX. The time required to complete this information collection is estimated to average 120 minutes per response, including the time to review instructions and gather the data needed, to review and complete the information collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: U.S. Department of Health & Human Services, OS/OCIO/PRA, 200 Independence Ave., S.W., Suite 336-E, Washington D.C. 20201, Attention: PRA Reports Clearance Officer
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Tanya de Sousa |
| File Modified | 0000-00-00 |
| File Created | 2023-08-26 |