Form 1 Program Directors Process Evaluation Form

OMB No.: XXXX – XXXX

Expiration Date: XX/XXXX

Evaluation of OWH-Girls At-Risk for HIV/Juvenile Delinquency Programs

Program Directors: Process Evaluation Questionnaire

Introduction: The Office on Women’s Health (OWH), Department of Health and Human Services has engaged GEARS, Inc. to conduct an evaluation of its HIV prevention program for girls at-risk for HIV/juvenile delinquency. You are being asked to participate in this interview because you are the Director of a program that has received a grant from OWH to provide intervention services to girls who are at-risk for HIV and juvenile delinquency. The interview will last approximately 1.5 hours. It will include questions about your experiences with program development and implementation of your OWH funded HIV/AIDS and juvenile delinquency prevention program. The information we gather from you may help to improve our understanding of the issues which confront HIV/AIDS primary prevention programs for girls who are at-risk for juvenile delinquency. All information that you provide to GEARS staff is for evaluation purposes only. It will be kept private to the extent permitted by law.

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0990-XXXX. The time required to complete this information collection is estimated to average 1.5 hours per response including the time to review instructions, search existing data resources, the gather data needed, and complete and review the information collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to:

U.S. Department of Health & Human Services
OS/OIRM/PRA
200 Independence Ave., S.W., Suite 531-H
Washington D.C. 20201

Attention: PRA Reports Clearance Officer.

Interview date
Name of implementing agency
Name of interviewee
Program title
Interviewee’s telephone number
Interviewee’s mailing address
Interviewee’s email address
Name if interviewer
Interview starting time
Interview ending time
Length of interview

Interviewer Instructions:

Please ask the following questions of the respondent in person or over the phone.  Probes are provided to ensure that the type of information needed is collected from all respondents.  If the respondent does not provide the content pertaining to probes in her or his original response, please ask the probe.  Probe questions are not optional.  They represent information that is required for the process evaluation.\

Background Information:

1. What is your organizational mission?

Probe:

• When was your organization founded?

2. What are your organization's main services?

Probe:

• How long has your organization been providing HIV prevention services?

• How long has your organization been providing juvenile delinquency services?

• Who is the primary beneficiary of the services your organization provides?

Program Description

1. What are the goals of your OWH HIV/JD prevention program?

2. What are the objectives of your OWH HIV/JD prevention program?
   Probe:

   • If participant has difficulty answering question, ask, “What do you expect to achieve with your HIV/JD prevention program?”

3. Are there certain characteristics of your program participants that you took into

account when designing your HIV/JD prevention program?

Probe:

• If respondent doesn’t understand question, say, “For example, attitudes towards HIV/JD, cultural and traditional factors, etc.”

4. Are there certain characteristics of the locality that you took into account when

designing your HIV/JD prevention program?

Probe:

• If respondent doesn’t understand question, say, “For example, regional attitudes towards HIV/JD, access to transportation, access to computers, email accounts, etc.”

5. Have any of your objectives changed from your original program plan?

Probe:

• If yes, please state the reason for changes?

6. Please describe your HIV/JD prevention program for girls?
Probes:

• What is your HIV/JD prevention program approach (e.g., train the trainer/peer mentorship, etc.)?

• What is the theoretical basis underlying your prevention approach (e.g., health belief model, etc.)?

  • If participant has difficulty answering the above question, ask, what is the logic behind your HIV/JD prevention approach?

• What are the components of your HIV/JD prevention program? By program components we are referring to the different aspects of your program such as outreach, recruitment, prevention education, referral, etc.

• Are your program components based on evidence-based practices/curricula?

• What do you see as the most important components of your HIV/JD prevention program?

• Among these program components, which one is gender specific?

• Why was this gender specific approach/strategy chosen for this program component?

• Interviewer: If participant has more than one program component with gender specific strategies, repeat the question until participant provides gender specific strategies for all program components.

• Was the chosen intervention modified for the racial/ethnic group(s) you serve?

• If yes, how was it modified?

• How were you trained to provide the HIV/JD prevention program?

• What were the cultural, linguistic, and developmental features of the approaches/strategies used in developing the HIV/JD prevention program?

7. How was fidelity of the program maintained?

Probes:

• How long does it take to implement each program component? (Interviewer, please note that the respondent may speak of sessions, modules, or activities instead of using program component. Please make sure that GEARS captures how the entire intervention is structured.)

• How many training modules are in your primary prevention education component for your participants?

• How many sessions make up each training module?

• How long is each session?

Are there specific sessions or modules that a participant must finish in order to be counted as completing the program?

Probe:

• If yes, what are they?

• Have your HIV/JD prevention program components (such as outreach, recruitment, etc) changed in any way?

• If yes, how were they changed?

• Have your HIV/JD gender specific strategies changed in any way?

• If yes, how were they changed?

8. Does your agency implement the HIV/JD prevention program or is the intervention implemented by other partners through a sub-contract?

Probe:

• Who are these partners?

• What roles do these partners and collaborators play in the delivery of your intervention services?

If participant has difficulty answering the above question, ask, which component(s) of the prevention program do they implement?

• How do you monitor the quality of services provided by sub-contractors?

• For your program, who are the most important partners and collaborators?

9. What organizational support system (meeting space, computers, equipments,

etc.) do you access from your partners and collaborators to implement your HIV/JD prevention program?

10. Over the course of the program have you made changes in implementing your

program, including the individual sessions? If yes, what changes have you made

and what prompted you to make them?

11. How (i.e., what is the process that you go through) do you make changes in your

program including program design, logical framework, etc.?

Probe:

• Who is involved in making these decisions?

12. Where do you implement your program?

Probe:

• Is this the only place you implement your program?

13. How do your participants get to your program? (Do they walk, parents drive them,

bus, etc?)

HIV/JD Prevention Program Participants

1. Please describe the girls in your intended (target) population.

Probes:

• What is the average age of your participants?

• What is the age range for your participants?

• How do they do in school?

• What kinds of grade do they get?

• Do they attend school regularly?

• Describe the neighborhoods they live in?

• Are you trying to reach girls with particular risk factors? If so, what are they?

• What is their family situation like? (E.g. do they come from single-parent households, foster-care, etc)

• Where do they live?

2. Is this the same group of girls as those described in your initial proposal to OWH?

Probe:

• If the characteristics of your girls have changed, would you describe the reason(s)?

3. How do you recruit girls to participate in your HIV/JD prevention program?
   
   

4. How many girls did you plan to serve at the beginning of each program cycle?
   
   

5. How many girls did you actually serve?
   Probe:

Note to Interviewer: If programs served more or less girls than expected ask respondent the following question:

• “How do you account for the difference in your expected number of girls served and the actual number served?”

Program Resources

HIV/JD Prevention Program Staff:

1. What are the key positions for this program?

(Interviewer: for each key position ask the following questions)

Probe:

• Is this position filled?

• What is the minimum training required for this position?

• What is the minimum work experience required for this position?

• Is there a job description for this position?

• Is it possible to get a copy of the job description for this position?

2. Has the role of any staff position changed?

Probe:

• Has the role of any staff position changed?

• Have any staff left?

• Have any staff been added?

• Have staff roles changed in any way (e.g. peer educator changed to outreach staff, etc.)?

6. What training (if any) is provided to staff in HIV prevention and education on an

ongoing basis?

Probe:

• How many hours of training is provided?

• Is an orientation considered as part of the training?

• Do staff have the opportunity to work with mentors?

3. Is there any additional training or technical support needed to successfully implement

your HIV/JD prevention program?

Funding:

1. What is/are the funding sources for your HIV/JD prevention program?
   Probe:

   • OWH monies only?

   • OWH monies and additional funding?

     1. What type of funding?

   • In kind services?

2. Is your program sufficiently funded?
   Probe:

Note to interviewer: If the respondent answers “no” to this question, ask:

• What modifications have you made in the implementation of your program to meet funding needs?

• What components of your program require additional funding?

3. Do you have any problems accessing your OWH monies to implement your HIV/JD program?

4. Do you currently have a mechanism in place that allows you to look for additional funding?

Management Information System:

1. Other than GEARS’s evaluation, do you collect data on your HIV/JD prevention program on a regular basis?

2. If yes, what type of information do you collect (e.g. participants’ characteristics, program activities, participant feedback, etc.)?

3. Do you have standardized instruments or forms to collect these data?

4. Do you have a system in place to store, manage, and analyze local data for your HIV/JD prevention program?
   Probe:

   • If yes, please tell me the type of software you use to store and analyze data.

5. How do you use the data you collect?

Evaluation

1. Are you carrying out a local evaluation of your HIV/JD prevention program?
   Probes:

• Is the evaluator internal or external? What is your evaluation design?

• What type of evaluation is carried out? (process, outcome)

• Do you have a standardized instrument to collect evaluation data?

• What type of information do you routinely collect?

• What type of software do you use to store, manage and analyze evaluation data?

2. Was the HIV/JD prevention program’s evaluation plan developed and approved by the majority of programs’ stakeholders?

3. To what degree is there consensus among key stakeholders about program’s need, design, implementation and evaluation plans?

4. How do you use your local evaluation data?
   Probes:

   • Decision making

   • Monitoring program activities

   • Documenting best practices, etc.

   • Shares with stakeholders

HIV/JD Prevention Program Implementation Facilitation and Hindering Factors

1. What factors or aspects of your organization help facilitate implementation of your HIV/JD prevention program?

Probes:

• In thinking about your answer, consider how your organization functions, its traditions and resources

• Also consider your local area. Are there traditions, values, ways of doing things, and resources (e.g., transportation, health services, entertainment) that helped in implementing your HIV/JD prevention program?

• Also consider history of providing services, collaborative partnerships.

2. Were these facilitating factors later incorporated into the program approach and design of your HIV/JD prevention program? If yes, how were they incorporated?
   

3. What factors hindered or challenged the implementation of your HIV/JD prevention program?
   

Probes:

• In thinking about your answer, consider how your organization functions, its traditions and resources?

• Also consider your local area. Are there traditions, values, ways of doing things, and resources (e.g. transportation, health services, entertainment) that hindered in implementing your HIV/JD prevention program?

4. What was done to overcome these hindering factors and were these efforts successful?

5. Is there anything that you would like to see changed in this program? If so, please describe what you would like to see changed.

Comments:

Are there any additional comments that you would like to share about your program or your work as a Program Director?

Thank you for participating in this interview.

Developed by GEARS, Inc. Page 9 of 9