Prevention Demonstration Project

Instructions for completing the

Adolescent Family Life Care and Prevention Demonstration End of Year Report Templates

Who completes the AFL Care and Prevention End of Year Reports?

All Adolescent Family Life grantees funded by the Office of Adolescent Pregnancy Programs (OAPP) complete this report. The project director of the agency that receives federal funding directly from the Office of Adolescent Pregnancy Programs is responsible for the accuracy and completeness of the submitted report. Organizations which provide both OAPP-funded pregnancy prevention services and OAPP-funded care services must complete End of Year Reports and Enclosures separately for each program. In addition to the program and evaluation reports and enclosures described here, projects must submit a Financial Status Report (FSR).

For the End of Year Report for Project Years 1-4, projects should complete the report, including Enclosures A, B, & C, using data from that specific project year (e.g. October 1, 2008-September 30, 2009).

At the end of Project Year 5, use this End of Year Report template and complete the program progress report and evaluation report covering content for the project over all 5 years. For program statistics, complete Enclosure A twice, once for year 5 participants and once for cumulative participants over all 5 years. Enclosure C should be completed only for year 5.

Sections of the AFL Care and Prevention End of Year Reports:

Cover page:

This page is completed once and reproduced as the first page of the subsequent two sections.

Project number: This number is the AFL-assigned number that identifies your project, for example APH PA 2000.

Applicant organization: This is the name of the agency that directly receives federal funding to provide services through the project. Subcontracting agencies are not listed here.

Title of project: This is the descriptive title or name of the project, for example Young Mothers/Healthy Families, Teen Parent Program, etc.

Project period: This is the 12 month period that indicates the beginning and end of the funding period, for example July 1, 2005-June 30, 2006.

Project year: AFL projects are funded in 5 year cycles. Indicate what year of the 5 year cycle the project is reporting on. This may be different than the project year that the project is currently in, for example the end of year report is for year 2 and the project is currently in year 3.

Project director: The person designated by the agency as responsible for the project. Include his or her name, title, street or mailing address, email address, and telephone and fax numbers. According to the special terms and conditions signed by the program, if a new project director is appointed, a CV or resume must be sent in to the OPHS grants management specialist and OAPP project officer for prior approval.

Program coordinator: In some projects, the person who is responsible for the day-to-day operations of the demonstration project is different than the project director (the agency-designated responsible party). In this case, include the name of the program coordinator, title, email address, and telephone and fax numbers.

Project evaluator: As designated by the Title XX legislation, each demonstration project must have an independent, university-affiliated evaluator from his or her home state. Provide the evaluator’s name, title, street or mailing address, email address, and telephone and fax numbers. According to the special terms and conditions signed by the program, if a new project evaluator is appointed, a CV or resume must be sent in to the OPHS grants management specialist and OAPP project officer for prior approval.

Combined Progress and Evaluation Report

This new report template combines the information formally separated into the progress and the evaluation report. It is usually completed by the project director or program coordinator, but will require information from the evaluator for section V Evaluation. Please contact your OAPP project officer if there are any questions regarding particular parts of the report. Note that the descriptive part of the report may take a significant amount of time in the first year; however in subsequent years of the project some parts may be the same, thus the time to complete this section will decrease. Instructions regarding how to complete the report are included throughout the template.

I. Executive Summary (Abstract).

This provides the reader with a concise description of your project that should cover the program, intervention, population served, setting, comparison group, research design, data collection, outcome and process results, and implications of the findings.

II. Detailed description of the demonstration model for the previous year.

This section should be a complete description of the project that provides complete information about the intervention, theoretical model, curricula used, intervention strategies and activities, intensity of strategies, the Logic Model, challenges encountered, changes in the project since the previous progress report, and unique accomplishments.

For more information on creating a logic model, see the following resources, or contact your OAPP project officer.

Logic Model References

1. Health Promotion Planning Resources. The Health Communication Unit. The University of Toronto. http://www.thcu.ca/infoandresources/planning_resources.htm#tp

2. Taylor-Powell, E. “

Logic Model: A Program Performance Framework,” Cooperative Extension, University of Wisconsin. http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html#more

3. Kirby, D., “BDI Logic Model: A Useful Tool for Designing, Strengthening and Evaluation Programs to Reduce Adolescent Sexual Risk-Taking, Pregnancy, HIV, and Other STD’s,” ETR Associates, June 15, 2002. http://www.etr.org/recapp/BDILogicModel062002.pdf

4. W.K. Kellogg Foundation. Evaluation Handbook, 2004. http://www.wkkf.org/Pubs/Tools/Evaluation/Pub3669.pdf

5. Rush, B, and Ogborne A. “Program Logic Models: "Expanding their Role and Structure for Program Planning and Evaluation." The Canadian Journal of Program Evaluation. vol. 6, No.2, 95-106, 1991.

6. United Way. Outcome Measurement Resource Network. Web site for outcomes measurement. United Way of America. http://national.unitedway.org/outcomes.

7. Wong-Rieger, D., and David, L. "Using Program Logic Models to Plan and Evaluate Education and Prevention Programs." In A. Love (Ed). Evaluation Methods Sourcebook II. Canadian Evaluation Society, University of Ottawa, 1995.

IV. Grantee's financial sustainability plan and activities that support continuation of services at the termination of this Federal funding.

IV. Grants management, program or evaluation issues not otherwise addressed in the report.

V. Evaluation

This part of the report is usually completed by the project evaluator with input from the project director. It should offer a complete explanation of the evaluation design including the research objectives (in SMART terms), the process evaluation and outcome evaluation, data collection and analysis, results, discussion of findings, recommendations, and dissemination activities. The template provides specific probes in each section to guide what you should cover in this section. The probes were developed from criteria used to score the EOY reports and following them should help you produce a better report.

Additional consultation can be obtained through contacting your project officer and referral to OAPP-contracted evaluation consultants.

Note that the descriptive part of the report may take a significant amount of time in the first year; however in subsequent years of the project some parts may be the same, thus the time to complete this section will decrease.

Your report should be complete but not redundant. You should aim for a report that is no more than 40-50 pages (excluding enclosures). Writing a concise report will allow you to use sections of the report for dissemination purposes.

Enclosure A

Which clients should be included in the AFL Care Demonstration Project End of Year Report?

All participants involved in the AFL program during the year that is being reported (for example July 1, 2005-June 30, 2006), regardless of how long or the level of involvement in the program's activities, should be included in the reporting of demographic information captured on Enclosure A. To the best of your ability, please provide unduplicated numbers of clients seen during the project year.

In the 5^th year end of project report, complete Enclosure A twice, once for year 5 participants and once for cumulative participants over all 5 years.

Categories of participants:

• Pregnant and parenting adolescent women: Include all pregnant and parenting adolescent females that received services from the AFL Care project who entered the project at the age of 18 or younger but are now over 18 due to being enrolled in the project for a follow-up period.

• Adolescent fathers and male partners: Include all adolescent fathers and male partners that received services from the AFL Care project.

• Infants and children: Include in this category infants or children of the pregnant or parenting adolescent who received services from the AFL project.

• Siblings: Indicate in this category siblings of the pregnant or parenting adolescent who received services from the AFL project.

• Parents/grandparents: Indicate in this category parents or grandparents of the pregnant or parenting adolescent who received services from the AFL project.

• Other Care Services Recipients: Include in this category other people who received services through the Care project, for example other family members such as aunts/ uncles/ other caregivers, or people attending an instructional course or presentation but not routinely accessing services through the project. Please provide a note of explanation to describe what services these participants received.

Race/ethnicity categories:

Please note, each participant should have designated both an ethnicity and a race category, thus there should be equal numbers of participants in the ethnicity and race tables.

The following ethnicity category description, defined in October 1997, is required for all Federal reporting, as mandated by the Office of Management and Budget. (For more information, go to http://www.whitehouse.gov/omb/inforeg/statpolicy.html#dr). If the participant did not indicate his or her ethnicity, include them in the “unknown/ unreported” category. Please provide unduplicated counts, with cumulative numbers and percentages in each category in the last row or column.

• Hispanic or Latino: A person of Cuban, Mexican, Puerto Rican, South or Central American or other Spanish culture or origin, regardless of race. The term “Spanish origin,” can be used in addition to “Hispanic or Latino.”

The following racial category descriptions, defined in October 1997, are required for all Federal reporting, as mandated by the Office of Management and Budget. (For more information, go to http://www.whitehouse.gov/omb/inforeg/statpolicy.html#dr). If a participant has checked more than one race, include them in the “more than one race” category. If the participant did not indicate his or her race, include them in the “unknown/ unreported” category. Please provide unduplicated counts, with cumulative numbers and percentages in each category in the last row or column.

• American Indian or Alaska Native: A person having origins in any of the original peoples of North and South America (including Central America), and who maintains tribal affiliation or community attachment.

• Asian: A person having origins in any of the original peoples of the Far East, Southeast Asia, or the Indian subcontinent including, for example, Cambodia, China, India, Japan, Korea, Malaysia, Pakistan, the Philippine Islands, Thailand, and Vietnam.

• Black or African American: A person having origins in any of the black racial groups of Africa.

• Native Hawaiian or Other Pacific Islander: A person having origins in any of the original peoples of Hawaii, Guam, Samoa, or other Pacific Islands.

• White: A person having origins in any of the original peoples of Europe, the Middle East, or North Africa.

Enclosure B

Care and Prevention projects in years 2-5 should complete Enclosure B, Performance Measures for AFL Care or Prevention Projects, by using the data from the AFL Core Follow-Up Questionnaire for All Clients, as described on Enclosure B.

In the 5^th year end of project report, complete Enclosure B twice, once for year 5 outcomes and once for cumulative outcomes over all 5 years (or all years for which you collected the baseline and follow-up questionnaires).

Enclosure C

The Efficiency Measure should be completed using the following instructions:

Projects should complete the efficiency measure (Enclosure C) for years 2-5. Projects do not need to complete Enclosure C for year 1.

In the 5^th year/ end of project report, complete Enclosure C only for year 5.

Program costs: FSR Line 10D (Net outlays—this Period) minus Evaluation costs.

The Financial Status Report (FSR), completed annually, details the budgetary amounts expended by the project yearly. The amount listed on Line 10D of the FSR (Net Outlays—this period) is the base amount expended by the project, including both the federal and non-federal (matching) shares. The evaluation costs allocated in the budget, including both the federal and non-federal (matching) shares should be subtracted from Line 10D so that only the program costs are included in the numerator.

Number of client service hours delivered

This number is derived by tabulating the total number of service hours delivered to clients through the AFL project. There are a variety of activities that may be included in this number. All activities counted in this measure should be directly related to the AFL program’s goals and objectives. When calculating the number of hours, note that it does not matter what proportion of time is spent on any one activity. For example, a 3 hour after school program may include tutoring, an arts and crafts program, and a health education lesson, however it is not necessary to split those out; rather the program can just list a 3 hour after school program as one complete activity counted as 3 client service hours.

These are activities conducted by the AFL project (not referral agencies, unless they are paid for or are in-kind services under the grant) that include, but are not limited to:

• AFL-related face to face individual, group, or family activities (note that each client should be counted individually for group activities)

  • Mentoring

  • Tutoring, Educational classes

  • Vocational/ occupational activities

  • Supportive counseling

  • Transportation to activities/ appointments related to program goals

  • Attendance at program-goal-related events for the client, client assessments

  • Home, school, or office visits, attempted and completed

  • Group education (classroom health education, support groups, curricula delivery, tutoring, etc.)

  • Developmental assets-related group or individual activities (e.g. trips to colleges, volunteer activities, community service, sports activities, cultural events, arts and crafts, etc.)

  • Family activities: meetings, group education, supportive counseling, etc.

  • Child care or developmental activities for children of parents attending the AFL project activities or AFL program participants

• AFL-related Phone/email individual, group, or family activities, for example

  • Supportive counseling or case management by phone, coordination of services, referral communication

• AFL-client-related collaboration/ advocacy, for example

  • Advocacy with courts, schools, health/ social service organizations, etc. (this could be by phone or in person)

  • Case staffing for individual clients or families

OAPP requests that you report on 2 types of client service hours delivered in the program: 1) Client service hours delivered to individuals; and 2) Client service hours delivered in group or family settings, in which multiple participants are present for the intervention. This is important so that additional information regarding the types of client service hours delivered is indicated in the efficiency measure.

Programs may develop their own mechanism for tabulating this information, although examples of tables to calculate the information are included below. The intent is not that these examples be used as a form for the information collection, but as examples that the grantee may use when developing a mechanism for tabulating this information. These tabulation forms should not be submitted with the End of Year Report. Rather, you will only report the total number of Client Service Hours delivered individually, the total number of Client Service Hours delivered in group or family settings, and the sum of Individual and Group/ Family Client Service Hours.

Sample Tabulation Form for Client Activities

Client name	Date	Activity	Ind Hours	Group Hours	Total
Pam	10/14/05	Individual home visit	1.5
Marie	10/15/05	Group education session		1.0
Janna	10/15/05	Group education session		1.0
Kim	10/15/05	Group education session		1.0
Pam	10/15/05	Group education session		1.0
Toni	10/15/05	Group education session		1.0
Maria	10/17/05	Case staffing re Maria	1.0
Janna	10/18/05	Advocacy-court with Janna	5.0
		Total	7.5	5.0	12.5

On Enclosure C, you would report 7.5 for Individual Client Service Hours, 5.0 for Group/ Family Client Service Hours, and 12.5 for Total Client Service Hours.

Alternately, the program could create a tabulation form for each client, for example:

Client: Pam

Date	Activity	Ind. Hours	Group Hours	Total
10/14/05	Individual home visit	1.5
10/15/05	Group education session		1.0
10/21/05	Mt with school counselor	1.5
10/25/05	Attempted home visit	.5
	Total	3.5	1.0	4.5

On Enclosure C, you would report 3.5 for Individual Client Service Hours, 1.0 for Group/ Family Client Service Hours, and 4.5 for Total Client Service Hours.

Another way to calculate the number of hours is to multiply the service by the number of AFL participants present, for example, if 5 students attend a 1 hour class: 5*1= 5 group client service hours for that intervention. However, the benefit of the previous methods for calculating the number of hours is that the project evaluator could easily calculate client dosage (number of hours of service any given client receives) and include that as a potential predictor variable.

Date	Activity	Hours	Participants	Ind Hrs	Group Hrs	Total
10/14/05	After school program	3	10		30
10/15/05	Group education session	1.0	5		5
10/21/05	Mt with school counselor	1.5	2	3
10/22/05	Individual client sessions	1.0	3	3
10/25/05	Attempted home visit	.5	3	1.5
				7.5	35	42.5

On Enclosure C, you would report 7.5 for Individual Client Service Hours, 35 for Group/ Family Client Service Hours, and 42.5 for Total Client Service Hours.

Enclosure D

The purpose of this form is to collect information from AFL grantees to provide a detailed description of AFL demonstration projects and how they may be replicated in other settings. The information collected with this form will be used to quantitatively describe the characteristics of program implementation across demonstration projects and will also be linked with effectiveness data among demonstration projects who are participating in the national cross-site evaluation.

The staff member who is most familiar with the project components and activities should complete this form after the process evaluation section of the end-of-year report is completed.

• Complete this form using data only from the project year that has most recently ended. If you are in the first year of your grant, please answer the questions about your project to date.

• Please complete one enclosure for each treatment group if your AFL demonstration project has more than one.

• If you are completing a hard copy of this form, use a pencil to mark an “X” in the box for your answer or write your answer in the lines provided. Do not use any other marks. Erase cleanly any answer you wish to change.

• If you are completing this form electronically, click the appropriate box with your mouse to mark an “X” or place your cursor in the appropriate field to type your answer.

• Not all response options will apply to every project; however, please answer every question unless the instructions direct you to skip a question. If none of the printed response options apply exactly to your project, mark the option that best fits your situation.

• If you do not recall something exactly, please provide your best guess.

• If you mark an “other” response option that prompts you to describe or specify your response, please complete these fields with your best estimate and/or necessary clarifying information.

9