Grantee Interview Protocol

ATTACHMENT C

Telephone Interview Guide for Use with PAF Grantee Coordinators

Telephone Interview Guide for Use with PAF Grantee Coordinators

Department Name:

Location:

Individual Interviewed (name and title):

Individual’s Contact Information (phone and email):

Contractor Staff Member Administering Survey:

Date of Interview:

Introduction

Thank you for taking the time to talk with me about the Pregnancy Assistance Fund (PAF) in your state/agency!

As you know, PAF is a Federal effort that is providing funding to states and tribes (usually distributed to sub-awardees) for the implementation of programs for expectant and parenting teens, women, fathers, and their families. The program encourages the use of evidence-based or evidence-informed and culturally and linguistically appropriate practices.

The Office of Adolescent Health (OAH; within the federal Department of Health and Human Services), which oversees PAF, is interested in learning about how PAF-funded programs are operationalized in the field and how selected PAF-funded programs impact behavior and health outcomes for youth. Therefore, OAH has hired Mathematica Policy Research to conduct a Feasibility and Design Study for the PAF programs. As part of the PAF Feasibility and Design Study, all PAF grantees will be part of a descriptive cross-grantee study by sharing their program design decisions and implementation experiences. Also, as a separate component of the PAF Feasibility and Design Study, OAH expects to engage two or three grantees in a rigorous impact and in-depth implementation evaluation. OAH has contracted with Mathematica Policy Research to lead the preliminary study efforts through the Feasibility and Design Study.

This interview will contribute to the cross-grant descriptive study, also called the Design and Implementation Analysis. The data collected through the interviews we are conducting will be reported in two ways. First, a summary profile will be created for each grantee that will contain facts about PAF program design decisions – for example, the selected program models, populations to be served, number of providers, types of implementation settings, and number of total youth to be served. This information will be reported for each grantee receiving PAF funding. While we will not attribute this factual information to a specific respondent, it will be attributed in aggregate to a specific grantee. Second, we will produce a report that discusses themes emerging from responses to “how” and “why” PAF program decisions were made. Such responses will not be attributed to any one grantee, but will instead be analyzed and reported as part of overall trends across all grantees.

This interview should take about 2 hours. I’m going to ask you questions in three areas: background information, program administration, and program design. Your participation in this interview is voluntary. In addition, if you do not feel comfortable answering any specific question, we can skip that question and move on to the next. Before we begin, do you have any questions or comments?

Do I have your consent to begin the interview?

INTERVIEWER IS TO STOP HERE IF THE RESPONDENT DOES NOT PROVIDE VERBAL CONSENT.

Module A: Background Information

[Key Facts]

1. I want to start with just a few basic facts about your state’s/tribe’s PAF grant:

   1. You’re implementing a Category [insert #(s) per application] PAF grant – is that correct?

   2. And you [are/are not – per application] a repeat PAF grantee, correct?

      1. [If repeat grantee] Can you briefly describe how the new funds are being used, versus the prior grant? (Probes: Are you expanding services to new providers? Are you supplementing the program with new materials?)

2. Please describe your current position and role on the PAF grant.

Probes: What are your current responsibilities? How long have you been in your current position (# of months)? Is your position newly created to support the PAF program in your state? What certifications, degrees, or other credentials do you hold that are relevant to your current position? How long have you been working on the issues surrounding expectant and parenting youth? If repeat grantee: How did your role change in the new grant cycle?

[Approach/Strategy for the Grant]

1. How did your agency become aware of the availability of PAF funds?

2. Why did your agency decide to apply for PAF funds?

3. Did you seek assistance in developing your proposal for PAF funds? If so, who provided this assistance and what type of assistance did they provide?

4. How would you describe your state’s/tribe’s overarching approach or strategy to supporting expectant or parenting youth – please describe the state’s/tribe’s broad strategy, including PAF and other programming?

1. What are the primary objectives of this state-wide/tribal-wide strategy?

2. What approaches are you using to achieve those objectives?

3. How did your state/tribe develop this overall approach/strategy?

4. How does the programming you are offering through PAF fit into this broader approach/strategy?

5. Is there any other background information related to your agency’s PAF program approach/strategy that you think we should know?

[Miscellaneous]

6. This study will give us the opportunity to learn a great deal about how grantees design and implement their PAF programs. From your perspective, what do you think are the most valuable questions that we could answer through this study?

1. What would you most like to learn about other programs and their implementation plans?

2. What information would help you to improve, sustain, or scale-up your program?

Module B: Program Administration

[State Agency Structure/Administration]

(Note: We will likely know much of this information from grant documents, but this question will verify/update and confirm that the respondent understands the issues.)

1. What state/tribal agency is administering the PAF program?

1. To what extent is your agency directly providing PAF services to youth?

2. Who is the primary contact with provider entities?

3. Can you describe the various program staff roles and responsibilities at your agency with respect to the PAF program?

2. Please tell me about the cost of administering the PAF program in this state.

1. What is the annual grant amount in Year 1? Year 2?

2. Of this total, how much of the funding goes to administrative costs in year 1? Year 2?

3. How much goes to direct program provision (for example, in the form of sub-awards to providers) in Year 1? Year 2?

4. How much is directed toward training and technical assistance to providers in year 1? In year 2?

5. How much is going to program monitoring and evaluation in year 1? In year 2?

   1. Beyond the required monitoring activities, will the state’s/tribe’s PAF program be formally evaluated in any way?

   2. If so, how will it be evaluated? Will it be on a state-wide or a provider level? Who will conduct the evaluation?

3. What other funding streams, if any, are being integrated with PAF funds? If other streams are integrated with PAF funds,

1. What other government entities (state, county, city, or federal) are providing this funding?

2. What proportion of the total budget for your PAF program comes from other sources?

3. To what extent are the other funding agencies playing a role in PAF program administration?

4. What is the total number of youth that you plan to serve over this grant period?

5. Is there any other information related to the state agency or administrative structure of your PAF program that you think we should know?

[Sub-Award Structure, Criteria, and Process]

1. Will funds be allocated to providers/ sub-awardees to provide PAF programming? If yes, please describe the process used to identify and fund PAF providers in your state.

1. What entities are eligible for sub-awards to provide PAF services?

2. What specific target population(s) are covered by these providers? (if applicable)

3. How does the award process work? What are the criteria for selection?

4. What is the performance period for sub-awards?

5. What is the average award amount (if applicable) to sub-awardee providers?

6. What is the range in the number of youth intended to receive PAF-funded services per provider sub-award?

7. What is the total number of sub-awards made to program providers in Year 1? Year 2?

2. Is there any other information related to the provider sub-award process that you think we should know?

[State and Provider Relationships]

1. Please describe the extent to which your agency is overseeing the PAF provider entities that have received sub-awards.

Probes: How closely is the state agency overseeing the PAF program working with provider entities? How often and in what ways do state and local provider entities communicate?

2. What process has been established for monitoring and evaluation by the state agency? Who is responsible for these evaluation and monitoring activities?

3. Please describe the support that the state offers to provider entities (either through the state itself, or services the state has contracted for). For each type of support, please explain who is responsible for providing related services:

1. General guidance regarding the PAF Program

2. Training and technical assistance

3. Other supports

4. Please describe the relationship among PAF provider entities. To what extent do they have formal or informal interactions with each other? What role does the state/tribe play in fostering these interactions?

5. Is there any other information related to the state/tribe relationship with PAF providers that you think we should know?

Module C: Program Design

[Program Model Selection and Design]

1. Please describe the primary program approaches that your state/tribe is implementing under the PAF grant.

2. Please describe the process by which you determined what programs or program components to emphasize in your state’s/tribe’s PAF program. Specifically, to what extent did you:

1. Conduct a needs and resource assessment of the services, policies, and systems available to help pregnant and parenting youth in your state?

1. Who was involved in this process? How were they involved?

2. What data did you use?

3. How did the needs assessment influence your program design? Did it influence your selection of (or guidance related to) program models? If so, how?

4. How did your state assess the needs of expectant and parenting youth? What were the major findings of the assessment? What improvement goals did you set as part of PAF?

5. How did your state assess resources (health, education, housing, etc.) available to expectant and parenting youth? What were the major findings of the assessment? What improvement goals did you set as part of PAF?

6. How will your state continue to assess these different kinds of needs and resources throughout the PAF grant period? How will you use the assessment to inform ongoing program development and improvement? How will you use it to identify or connect with partner organizations?

2. Use evidence-based or evidence-informed approaches?

   1. Did your state identify one or more evidence-based or evidence-informed program model(s) for implementation? Why or why not? How did you identify these approaches? (Probes: To what extent did your state review available evidence as part of applying for PAF funds? How did you use the evidence to inform program selection or implementation?)

   2. Which program models were selected and why? How would you characterize these programs, as evidence-based or evidence-informed? (Probes: What do you know about the research on these programs? How important do you think it is that you implement the program in the same way as it is described in the research? To what extent are you incorporating elements of various programs?)

   3. To what extent are providers allowed flexibility in how they implement your chosen evidence-based or evidence-informed program models?

      1. Did the state create a process for approving/rejecting program model adaptations? If so, what is this process?

      2. To what extent is the state/tribe concerned with fidelity to the evidence-based program model or to planned program implementation? What training or guidance does the state provide about adherence to the program model?

      3. Does your state plan to monitor program fidelity? If so, how?

3. Use culturally and linguistically appropriate approaches?

   1. Did your state deem any program model(s) or characteristics of program models inappropriate for your state? Why or why not?

4. Build formal and informal partnerships to ensure the availability and success of wrap-around services for pregnant and parenting youth? By “partners,” we mean non-providers who help the state/tribe to establish or operate the PAF program. “Formal” partners are those with an explicit role in the program – often formalized through a memorandum of understanding (MOU) or some other agreement; “informal” partners are those whose role is less well defined and no formal agreement exists for their participation.

   1. Did your state/tribe establish any formal partnerships for the PAF program? (For example, partnerships with other agencies or community organizations for service referrals, or partnerships between high schools and colleges to support participants’ education.)

      1. Do you have MOUs in place with these entities?

      2. To what extent have formal partners (with MOUs) played the roles expected of them?

   2. Have any informal partners played significant roles in implementing PAF in your state? If yes, please describe their roles.

   3. What made you want to work with these formal and informal partners?

   4. What was the process for bringing them on board?

   5. How have you worked with them over time?

   6. To what extent did your state build on existing partnerships or develop new ones for the PAF grant?

   7. Are any partners involved in the administration or implementation of the PAF program? If yes, in what ways?

3. Earlier, you described [NUMBER] main PAF program approaches: [LIST APPROACHES HERE]. I’m going to ask you for some key details about each of these. Let’s start with [APPROACH 1 NAME]. For this approach, please describe:

   1. What are the key components?

   2. What types of organizations are the providers and what are their names?

   3. What are the types of provider settings—that is, where services are actually delivered—and the number of sites for each setting?

   4. What is/are the target population(s)?

   5. What is the intended dosage?

[REPEAT a-e FOR EACH PROGRAM APPROACH]

4. How has your state addressed the medical accuracy of programming?

5. How, and by whom, were final decisions related to the PAF programmatic approach(es) made?

6. Is there any other information related to program model selection or design that you think we should know?

[Guiding Considerations]

1. In designing the overall PAF program in your state, did you:

1. Assess the current organizational infrastructure and capacity in your state? If so, how did these considerations influence your program design?

2. Secure buy-in from key stakeholders (for example, school district or college administrators, community leaders, elected representatives, parent organizations)? If so, who were these stakeholders and how did you engage with them?

3. Consider the sustainability of PAF-funded programs in the future if federal funds do not continue? If so, what are your plans to sustain the programs?

Q2. Was there a primary program leader (or “champion”), that is, someone who saw this PAF opportunity and really led the charge (e.g. held meetings, influenced decision-makers) to formulate it and make it a reality?

Module D: State/Tribal Context

1. Please describe broadly, from the beginning, the process that your state went through in order to determine which category or categories of PAF funding to pursue.

2. In your opinion, what particular issues or problems for pregnant and parenting teens are unaddressed in your state, and how is the PAF grant attempting to address them?

Probes: What is the teen pregnancy rate for your state/tribe/reservation? What is the teen birth rate? What communities or populations (racial, economic, geographical, vulnerable) are most at-risk?

3. Are there specific policies in place in this state that impact the kinds of programming that can be offered to expectant and parenting youth? For example, has the legislature passed laws which either mandate or restrict specific approaches to teen pregnancy prevention or sex education programming?

Probes: How have these laws influenced the PAF programming decisions that have been made in your state.

4. Please describe programs and activities – other than the PAF program – that are currently operating in your state that are aimed at supporting expectant or parenting youth.

1. How well do you think the available programming serves youth in the state?

2. What % of the targeted youth population do you think are receiving these services?

   1. To what extent are services uniformly available to all eligible youth throughout the state/reservation? Are there areas where services are more or less available?

   2. To what extent will the PAF grant fill in any gaps in services?

3. In what settings do the programs operate? (Probes: For example, are they provided in religious-based organizations, community-based organizations, schools or colleges, etc?)

4. How are these programs funded? (Probes: What programs are funded with federal dollars? State dollars? Non-profits, foundations, and other organizations?)

5. To what degree is there coordination in the state around service delivery to expectant and parenting youth? In what ways are the different program administrators or operations staff in communication with each other?

5. To what extent are PAF programs integrated with other programming for expectant and parenting youth in the state/tribe?

   1. Have any programs been eliminated or absorbed into PAF?

   2. To what extent has the PAF grant allowed the state/tribe to expand its supports for expectant and parenting youth?

6. Is there any other basic background information about youth parenting, pregnancy or related issues, or the PAF program in your state that you think we should know?

[Miscellaneous]

1. To your knowledge, how has the PAF program been received by local community stakeholders?

[Additional Information, Informants, and Follow-up]

1. Is there anything else that we should know about your state’s PAF program?

2. Is there anyone else who you think would be a good informant for this study (i.e. a descriptive study on how PAF programs were designed and implemented)? Could we have that person’s title and contact information?

3. We may want to follow-up with you in a year or so to see how the state’s PAF programs are being implemented. What is the best way to reach you? If we can’t reach you via this method, are there other people who will know how to reach you? Could we have their contact information?