Qpr

UNITED STATES DEPARTMENT OF LABOR

Division of Older Worker Programs

SCSEP
QPR Handbook
The Charter Oak Group, LLC

Rev.5 (5/19/10)

SCSEP Quarterly Progress Report Handbook
I. Introduction
The Quarterly Progress Report has been used to report SCSEP performance for many
years. It was extensively revised effective July 1, 2004, to report both the new SCSEP
performance measures required by the Older Americans Act amendments of 2000, the
SCSEP Final Rule, and Older Worker Bulletin 04-06; and many of the new data elements
captured by the SCSEP Data Collection System. The current QPR, effective May 20,
2010, has been revised to reflect performance system and programmatic changes made by
the 2006 amendments to the Older Americans Act. These changes are also reflected in
SPARQ Release 4.3.
The QPR is produced for all levels of the SCSEP system: nationwide, grantee-level, subgrantee level, and by states for the national grantees. All authorized SPARQ users have
access to all final QPRs through the SPARQ home page. All users also have access to
their own QPRs at any time during the quarter and, therefore, do not need to wait for final
QPRs to monitor their progress.
Although the QPR is the official report for SCSEP, it is not the only tool for managing
performance. Furthermore, while the QPR can identify potential performance issues, it
rarely is sufficient to tell you the cause of a problem or how to fix it. For that, you will
need to consult other sources. SPARQ provides all grantees and sub-grantees a variety of
management reports through the SPARQ home page. (These reports are particularly
important for the Common Measures because the QPR does not provide any information
about quarterly performance on the Common Measures until the first day after the quarter
has ended.) In most cases, when the QPR suggests that you are having a problem with a
measure, the management reports will provide you the detailed information you need to
analyze the problem. The management reports will be significantly enhanced in future
releases of SPARQ. In addition, all levels of the SCSEP system receive customer
satisfaction reports that provide customer evaluations of the services they have received
and offer guidance for program improvement.
This Handbook is designed to provide SCSEP practitioners a plain-language explanation
of the performance measures and data elements reported in the QPR, along with practical
suggestions for how these elements can be used to manage operations and improve
performance. The Handbook is not intended as a technical explanation. The complete
specifications are posted at the Mathematica web site: http://SCSEP.MathematicaMPR.com.
The QPR has six sections. Sections A-D contain information about the grant, customer
flow, community service assignments, and participant characteristics. The core
performance measures are reported in Section E. The additional performance measures
are reported in Section F.
A participant can have multiple enrollments, sometimes referred to as participations.
Enrollments are distinguished by a combination of personal identification (PID) number
1
The Charter Oak Group, LLC

Rev. 5 (5/19/10)

and application date. In most instances, the QPR uses enrollments, not participants, as
the unit of analysis. The QPR usually counts active enrollments, meaning those who
were active at any time during the reporting period. This includes participants who were
active before the reporting period and remained so during the entire reporting period;
those who enrolled in the program during the reporting period and remained active during
the remainder of the reporting period; and those who became active either before or
during the reporting period and who exited during the reporting period. In a few
instances, the QPR reports only those participants who were active on the last day of the
reporting period. Those instances are noted in the description of the QPR measure or
element.
There are five QPRs in each program year: one for each quarter and the Final QPR,
which is the official report of performance for the program year. Each QPR uses four
different reporting timeframes: current quarter, program year-to-date (YTD), last four
quarters (L4Q, sometimes referred to as rolling four quarters), and last nine quarters
(L9Q, for element B13 only). Nearly all of the elements in Sections B and C use the
current quarter and YTD. All or nearly all of the elements and measures in Sections D, E
and F use the last four quarters. (Because of a transition in timing rules required by the
performance system, for PY 2007, the last four quarters timeframe was not reported for
the Common Measures. In addition, the last four quarters are not reported for the
community service or service level measures because of the change in the modified
position count from program year to program year.)
The different time periods provide a comprehensive picture of performance. The current
quarter tells you how you have done in the most recent three-month period. The YTD
timeframe looks at performance during this program year only and is most useful for
monitoring how you will be evaluated at the end of the program year when the Final QPR
is generated. During the early quarters of the program year, YTD does not provide a
sufficient timeframe to show trends. Therefore, the last four quarters are shown so that
you can see the direction in which your performance is heading over an entire 12-month
period. This is critical for managing performance. For example, in the second quarter
QPR, a program may be meeting its goal on the entered employment measure for the
current quarter and YTD. However, the last four quarters show that entered employment
has steadily declined from 55% to 53% to 51% to 50%. This trend suggests that unless
corrective action is taken, the entered employment measure may continue to deteriorate
and the Final QPR may show entered employment under 50%.
Note on Completeness of QPRs. Users may view their own QPR at any time during a
quarter. Users must regenerate their QPR before it will reflect the most recent data.
Grantees must regenerate all sub-grantee QPRs in order for the grantee QPR to reflect the
most recent data. At the official close of each quarter, all QPRs are automatically
regenerated and frozen. Data submitted after the close of a quarter will not change the
QPR for that quarter but will be reflected in the YTD and in the L4Q of future QPRs for
that program year.

2
The Charter Oak Group, LLC

Rev. 5 (5/19/10)

II. Section A. Grantee Information
Description. This section contains identifying information about the grantee and subgrantee. All of the data are provided by the national office.

A. GRANTEE INFORMATION
1a. Grantee:

1b. Sub-Grantee:

2. Grantee Code:

3a. Report Period Ending:
_________________
(MM/DD/YYYY)

3b. Program Year(s):

4. Number of Authorized
Positions:

___________________

5. Number of Modified
Positions:

Element A4. Number of Authorized Positions. This is the official number of authorized
positions. SCSEP depends on grantees to provide the national office with the number of
authorized positions assigned to each sub-grantee. If the grantee does not keep this
information current, the sub-grantee QPRs will be inaccurate or may not contain the
service level and community service measures at all. The authorized positions are
reported in Section A4. SPARQ uses modified positions rather than authorized positions
for calculating vacancies in Section B and the service level and community service
measures in Section E. Modified positions, which are based on higher state minimum
wages, are reported in element A5.
Element A5. Number of Modified Positions. The number of authorized positions is
modified to account for higher state minimum wages. The national office determines
modified positions based on minimum wage information provided by the grantees.
Higher state minimum wages are prorated for the entire program year so that the number
of modified positions does not have to be revised each quarter. This means that the count
of modified positions is accurate for YTD but will not be accurate for a given quarter if
the state minimum wage increases during the program year.
III. Section B. Participation Levels
Description. This section displays customer flow and placement information. For YTD
at the grantee or sub-grantee level, elements B1-B4 will always balance: carried over
participants plus new participants minus exiters will equal current participants. Element
B5a, vacancies, shows modified positions minus current participants minus rejected
records. A negative number means that a grantee is over-enrolled. Element B5b is the
number of records rejected by SPARQ and not included in the QPR. Element B6 reports
the number of individuals on the waiting list, and elements B7 reports the number of
participants who are co-enrolled with another program. Elements B8a-B8c show exit
reason at the time of exit. Element B8d shows exclusions discovered after exit.
Elements B9a-B9d display the movement of participants into or out of the project due to
transfers or a change of sub-grantee. Elements B10a-B10e report placements that have
begun, without regard to the exit reason or performance outcomes. Element B10f counts
3
The Charter Oak Group, LLC

Rev. 5 (5/19/10)

placements that did not result in the achievement of entered employment. Element B11
reports the number of hours of paid training. Element B12 reports the percentage of
participants working in community service between 18 and 22 hours per week on
average. Element B13 reports the average duration for all the enrollments that were
active at any time during the reporting period in a project.
B. PARTICIPATION LEVELS
1. Carried Over from the
Previous Program Year:

2. New Participants:

3. Total Exits:

Q ____________

Q ____________

YTD __________

YTD __________

4. Current
Participants:

5a. Vacancies:
5b. Rejected Records:

Q1 ___________
YTD ___________
6. Number of Persons on Waiting List:

7. Co-Enrollments:

Q ____________

Q _____________

YTD __________

YTD _________________

8a. Exits Due to
Unsubsidized
Employment:

9a. Transferred
Into Project:

9c. Change of
Sub-Grantee
Into Project:

Q __________

Q __________

Q __________

YTD ________

YTD ________

YTD ________

9b. Transferred
Out of
Project:

9d. Change of
Sub-Grantee
Out of
Project:

8b. Other Reasons for Exit:
Q __________
YTD ________
8c. Exclusions:

Q __________

Q __________

YTD ________

10a. Placements:

10d. Percent of Placements
with Benefits:

Q __________

Q __________

YTD ________

L4Q ________

10b. OJE Placements:

10e. Average Hours per Week
in Placement:

Q __________

Q __________

YTD ________

L4Q ________

10c. Average Starting Wage
in Placement:

10f. Placements without
Entered Employment:

Q __________

Q __________

L4Q ________

YTD ________

Q __________
YTD ________

YTD ________
8d. Exclusions after Exit:
Q __________
YTD ________
11. Number of Hours of Training:

2. % Participants Assigned 18-22 Hours per Week:

13. Average Project Duration:

Q _______ YTD _______

Q ________ YTD ________

Q ________ L9Q ________

Element B1. Carried Over from the Previous Program Year. This is the number of
participants who were active, i.e., had an assignment date but no exit date, at the start of
the program year. The data for the quarter will be frozen when the quarter officially
closes. The YTD number may change during the year if the grantee changes the status of
any participants who were counted as active at the start of the program year.
4
The Charter Oak Group, LLC

Rev. 5 (5/19/10)

Element B2. New Participants. This is the number of participants who first became
active, i.e., have an assignment date, during the reporting period. This includes anyone
who re-enrolls during the reporting period but not anyone counted in element B1. It also
includes, at the grantee and sub-grantee level, anyone who transferred into the project
(element B9a) during the reporting period. Participants moved into the sub-grantee from
another sub-grantee (element B9c) are not included in the count of new participants.
Element B3. Total Exits. This includes everyone who exited, i.e., had an exit reason and
an exit date, within the reporting period. It includes all true exits (choices i-vi) in field 6
of the Exit Form and choices ii and iv in field 6a, non-exit reasons for closing the record.
It does not include choices i and iii in field 6a. It thus includes at the grantee and subgrantee level anyone who transferred out of the project during the reporting period
(element B9b). Participants moved out of the sub-grantee to another sub-grantee
(element B9d) are not included in the count of total exits.
Element B4. Current Participants. This is the number of active participants, i.e., those
with an assignment date but no exit date, on the last day of the reporting period. It is
always YTD. For YTD at the grantee, sub-grantee, state, and nationwide levels, element
B1 plus element B2 minus element B3 will always equal element B4.
Element B5a. Vacancies. This is the number of modified positions (element A5) minus
the number of current participants (element B4) minus the number of rejected records
(element B5b). Over-enrollment is reflected as a negative number in element B5a.
Element B5b. Rejected Records. This is the number of enrollments not included in the
QPR. Because only data quality issues affecting eligibility or performance cause a record
to be rejected, a high rejection rate may render your QPR unreliable. The number in this
field may be lower than the number of rejection messages in the DQR because a single
enrollment can be rejected for multiple reasons.
Element B6. Number of Persons on Waiting List. This is the number of applicants on the
waiting list at any time during the reporting period. It includes all applicants who have
been determined eligible, have not withdrawn their applications, and have been either
placed on the waiting list or determined eligible 14 or more days before the end of the
reporting period and not yet assigned to community service. Since this measure excludes
those who have been assigned, it is not a count of all applicants who have been placed on
the waiting list since the start of the program year.
Element B7. Total Co-enrollments. This is the count of all participants who were active
during the reporting period and have been co-enrolled in any other program.
Element B8a. Exits Due to Unsubsidized Employment. This is a count of all participants
who exited during the reporting period where field 5 of the Exit Form lists unsubsidized
employment as the reason for exit. Elements B8a, B8b, and B8c are unrelated to the
performance measures. They report the reasons for exit at the time of exit, not the
5
The Charter Oak Group, LLC

Rev. 5 (5/19/10)

ultimate outcome. Participants who exited for unsubsidized employment and are
included in element B8a may not achieve entered employment and may not be included
in measure E2. Conversely, participants who exit for personal reasons and are included
in element B8b may subsequently obtain unsubsidized employment and may be included
in measure E2.
Element B8b. Other Reasons for Exit. This is a count of all participants who exited
during the reporting period where field 6 of the Exit Form lists a reason other than
unsubsidized employment: 6(i)-6(vi). It includes only true exits and excludes non-exit
reasons for closing the record (field 6a of the Exit Form.) It does not include reasons for
exit that constitute an exclusion from the performance measures. See element B8c.
Element B8c. Exclusions. This is a count of all participants who exited during the
reporting period where field 6 of the Exit Form lists a reason that constitutes an exclusion
from the performance measures: 6(vii)-6(x).
Element B8d. Exclusions after Exit. This is a count of all exited participants whose
exclusions were discovered after exit. Such exclusions are recorded in field 9a of the
Exit Form or fields 28c(viii), 29c(viii), 29e(viii), or 30c(viii) of the Unsubsidized
Employment Form.
Element B9a. Transferred into Project. This is a count of all participants who
transferred into the project from another grantee. It is based on field 17 of the Participant
Form. At the grantee and sub-grantee level, this number is included in the count of new
participants (element B2).
Element B9b. Transferred Out of Project. This is a count of all participants who
transferred out of the project into another grantee. It is based on field 6a(ii) of the Exit
Form. At the grantee and sub-grantee level, this number is included in the count of total
exits (element B3).
Element B9c. Change of Sub-grantee into Project. This is a count of all participants who
entered the project from another sub-grantee within the same grantee. It is based on field
17a of the Participant Form. This number is not included in the count of new participants
(element B2). This element is not currently reported in the QPR.
Element B9d. Change of Sub-grantee Out of Project. This is a count of all participants
who have left the project for another sub-grantee within the same grantee. It is based on
field 6a(iii) of the Exit Form. This number is not included in the count of total exits
(element B3). This element is not currently reported in the QPR.
Element B10a. Placements. This is a count of all enrollments where any placement
(unsubsidized employment) started during the reporting period and there is no exclusion
from Common Measures discovered at or after exit. It is based on the existence of an
Unsubsidized Employment Form. It does not reflect whether the placement has resulted
in entered employment. A single enrollment can have only one count in element B10a.
6
The Charter Oak Group, LLC

Rev. 5 (5/19/10)

Therefore, if the placement count is higher than the count of exits for unsubsidized
employment (B8a), that means that someone who exited for other reasons has started a
placement in this reporting period or that someone who exited for unsubsidized
employment in a prior reporting period has started a placement in this reporting period.
Element B10b. OJE Placements. This is a subset of regular placements. It includes all
placements where OJE was indicated in field 20 of the Community Service Assignment
Form and field 8 of the Unsubsidized Employment Form.
Element B10c. Average Starting Wage in Placement. This is the average starting wage
of all participants who started unsubsidized employment within the reporting period,
regardless of whether they achieved entered employment. It includes all unsubsidized
employment for those participants who have had more than one job.
Element B10d. Percent of Placements with Benefits. This is the percentage of
unsubsidized employments that started within the reporting period and provided one or
more fringe benefits, regardless of whether the participant achieved entered employment
in that employment. It includes all unsubsidized employment for those who have had
more than one job.
Element B10e. Average Hours per Week in Placement. This is the average hours per
week of all participants who started unsubsidized employment within the reporting
period.
It is based on the information entered into field 18 of the Unsubsidized
Employment Form. Participants expected to work full-time are counted as working 40
hours per week.
Element B10f. Placements without entered employment. This is a count of all
participants who exited in the prior quarter and started unsubsidized employment but did
not achieve entered employment with some employer by the end of the current quarter.
Subsequently recorded entered employment causes the YTD count for this element to
decrease. Multiple employments for the same participant only count once.
Element B11. Number of Hours of Training. This is the total of all paid training hours
during the reporting period.
Element B12. % of Participants Assigned 18-22 Hours per Week. This field counts the
number of participants who are paid for an average of 18 to 22 hours per week of
community service (based on the number of days of enrollment and total hours paid
during the quarter) divided by the number of participants active during the quarter.
Element B13. Average Project Duration. This field reports the average time in months
that all participants have been enrolled with the grantee or sub-grantee from July 1, 2007,
to the end of the reporting period or to date of exit, whichever is earlier. Approved
breaks in participation are not included in this average. The average includes those still
active at the end of the reporting period and those who exited during the reporting period.

7
The Charter Oak Group, LLC

Rev. 5 (5/19/10)

Use. Section B of the QPR contains two kinds of information: customer flow and
outcomes. The outcomes are reported in raw numbers without relation to the grantee’s
goals. This section of the QPR, when used in conjunction with the performance measures
in Sections E and F and the management reports, is extremely valuable for helping
grantees and sub-grantees manage day-to-day operations and predict performance.
Elements B1 through B5a provide an important view of customer flow and turnover. In
order to reach the desired service level, grantees must exit and enroll sufficient numbers
of participants. If the service level and count of current participants are not where they
should be for the reporting period, this will likely be reflected in a low exit rate. It may
also be reflected in a negative or low customer flow, i.e., new participants minus exiters,
or in a significant number of vacancies. Since vacancies are calculated based on
modified positions, this number, although a snap-shot in time, may be an indicator of
under-spending and potential performance problems.
Element B5b is a critical indicator of the quality of your data. If the number of rejected
records is 2.0% or more of the total number of records submitted (elements B3 plus B4
plus B5b), the data may not be sufficiently reliable to be used for baseline purposes. The
data are still extremely useful for management purposes, but a rejection rate of 2.0% or
more tells you that you must pay serious attention to the quality of your data. In SPARQ,
the EDQP enables you to see the rejection messages for each record before you leave that
record. The DQR tells you all of the errors in all of the records in your database.
Consequently, you should be able to keep your rejection rate close to zero.
Element B6 tells you the number of applicants on the waiting list. The management
report will tell you who the applicants still on the waiting list are and what their
characteristics are. You can use this information to evaluate how effective your
recruitment efforts are and to decide which priority candidates to enroll.
Elements B8 and B10 begin to tell you a complete story about exits, placements, and the
quality of the jobs into which participants are placed. Elements B8a, B8b, and B8c show
the reasons for exit. Although nearly all exits can result in a successful entered
employment, participants who exit for unsubsidized employment are far more likely to
achieve success. You will want to watch the ratio between element B8a and element
B8b, knowing that elements B8c and B8d are excluded from the Common Measures.
You can use the management reports to look at all exiters and see which of the
participants who exited for other reasons might have been candidates for placement if
approached differently or might still be candidates if their personal circumstances change.
If element B10a is higher than element B8a, that means that participants who exited for
other reasons subsequently started a placement or that participants who exited for
unsubsidized employment in a prior period have gotten a new placement in this period.
You can compare elements B10a and B10f to see what the success rate is for participants
who start unsubsidized employment. Finally, you can use elements B10c and B10d to
evaluate the quality of the jobs into which you are placing participants. B10c, in
conjunction with B10e, the number of hours worked per week, is the best predictor of
8
The Charter Oak Group, LLC

Rev. 5 (5/19/10)

average earnings (measure E4). With B10b, the number of OJE placements, and B11,
hours of paid training, you can begin to look at the effect of training on entered
employment.
Element B12 is designed to tell you the percentage of participants working in community
service between 18 and 22 hours per week. Community service must be part-time,
meaning less than 40 hours per week. Although there is no rule setting forth the number
of hours per week that are permissible, the national office has recommended that
participants be provided 20 hours per week of community service.
Element B13 tells you the average duration in the project of all participants. This
element will enable grantees to manage the requirement that they maintain an average
project duration of 27 months (or 36 months with an extension).

IV. Section C. Community Service Assignments
Description. This section of the QPR summarizes community service activity by
combining community service assignment codes G1-G14 and E1-E14 (field 17 of the
Community Service Assignment Form) into service to the general community (elements
C1 and C2) and service to the elderly community (elements C3 and C4), and then
combining these two broad categories into the general category of community service
(elements C5 and C6).
Community service is reported both by the total number of hours worked (field 19 of the
Community Service Assignment Form minus field 21) for all community service
assignments that were active at any time during the reporting period and by the number
of participants who were active in an assignment during the reporting period. A
participant can have more than one active assignment in the reporting period, but the
assignments cannot overlap.

C. COMMUNITY SERVICE ASSIGNMENTS

Q

YTD

1. Number of participants providing service to the general community
2. Total number of hours worked in service to the general community
3. Number of participants providing service to the elderly community
4. Total number of hours worked in service to the elderly community
5. Total number of participants providing community service
6. Total number of hours worked in community service

9
The Charter Oak Group, LLC

Rev. 5 (5/19/10)

Use. This summary information about the amount of community service performed is
primarily for reporting to stakeholders on the impact of SCSEP. It has no management
purpose except as a way to check on the total number of community service hours
provided. The community service measure itself is reported in measure E1.

V. Section D. Participant Characteristics
Description. This section reports all participant characteristics from the Participant
Form, including in the last sub-section entitled Additional Measures, the priorities of
service and the waiver factors for the durational limit that are components of the most-inneed measure (measure E6). Age and education have been grouped into standard
demographic categories. For each characteristic, both the number and the percent of
active participants represented by that number are displayed.
Most characteristics are determined at the time of enrollment. Age in elements D3
through D7 is calculated from the date of the eligibility determination. None of the
characteristics that are priorities of service and are used in the most-in-need measure are
permitted to be updated in the data system after the determination of eligibility.
However, the characteristics that are waiver factors for the durational limit (elements D36
through D39) can be entered into SPARQ whenever they occur. The waiver factors are
reported in Section D and counted in the most-in-need measure only in the program year
in which they were first entered and in any subsequent year in which they are updated.
Age 75 and over (element D31) is a waiver factor and is calculated each time the QPR is
generated.

10
The Charter Oak Group, LLC

Rev. 5 (5/19/10)

D. PARTICIPANT CHARACTERISTICS
Q
No.
Gender

Q
%

YTD
No.

YTD
%

L4Q
No.

1. Male
2. Female
3. 55-59
4. 60-64

Age at
Enrollment

5. 65-69
6. 70-74
7. 75 & over

Ethnicity

8. Hispanic, Latino, or Spanish origin
9. American Indian or Alaska Native
10. Asian

Race

11. Black or African American
12. Native Hawaiian or Pacific Islander
13. White
14. 8th grade & under
15. 9th grade – 11th grade
16. High School diploma or equivalent
17. 1 – 3 years college

Education

18. Post-secondary certificate
19. Associate’s degree
20. Bachelor’s degree or equivalent
21. Some graduate school
22. Master’s degree
23. Doctoral degree
24. Family income at or below the poverty level
25. Individuals with disabilities
26. Individuals with limited English proficiency
27. Individuals with low literacy skills
28. Individuals residing in rural areas

Additional
Measures

29. Individuals with low employment prospects
30. Individuals who failed to find employment
after using WIA Title I
31. Individuals age 75 and over at date of report
32. Individuals who are homeless or at risk of
homelessness
33. Displaced homemakers
34. Veterans (or spouses)
35. Individuals receiving public assistance
36. Individuals with severe disability
37. Individuals who are frail
38. Individuals old enough for but not receiving
SS Title II
39. Individuals with severely limited employment
prospects in areas of persistent
unemployment

11
The Charter Oak Group, LLC

Rev. 5 (5/19/10)

L4Q
%

Use. The information in this section is important for understanding the composition of
your participant pool. It can provide insight into the strengths and needs of your
participants, and it will help you monitor your progress toward the most-in-need goal. If
your performance on this measure is not what you desire, elements 24 through 35 of
Section D will tell you whether you are enrolling sufficient numbers of individuals with
barriers to employment. Elements 31 and 36 through 39 will tell you whether your
participants have the more serious barriers to employment that are used to determine
waiver factors. The information in this section will also allow you to monitor your
compliance with SCSEP priorities.

VI. Section E. Core Performance Measures
Description. Presented in this section are the six core performance measures, three
Common Measures and three SCSEP measures. Participants whose exit reasons are
6(vii)-(x) or whose exclusion is discovered after exit are not included in the Common
Measures. Also not included in the Common Measures are participants who return to the
program or who re-enroll within 90 days of exit. For each performance measure in
Section E, both the goal for the program year and the performance for the reporting
period are given. The performance for each measure includes the number of records in
the denominator and numerator.
The rules for determining the quarter and the program year in which measures are
reported have been simplified with the adoption of the current performance measures on
July 1, 2007. Each Common Measure has an activity period and a reporting period. The
activity period is the quarter or quarters in which the activity being measured must occur.
The reporting period is the quarter in which the success or failure of the measure will be
reported. Data for a measure can be entered into SPARQ as early as the first day of the
reporting period, but it will not be reflected in the QPR until the reporting period has
ended. The last day for entering data is the last day of the “fifth quarter” of the program
year in which the reporting period falls, i.e., the September 30 following the end of the
program year. For entered employment (measure E2), the activity period and the
reporting period are both the first quarter after the exit quarter. For retention and average
earnings (measures E3 and E4), the activity period is the second and third quarters after
the exit quarter, while the reporting period is the fourth quarter after the exit quarter.
A complete explanation of the timing rules for the reporting of the performance measures
is included in Topics 13 and 14 of the Unsubsidized Employment Form Guide of the
Data Collection Handbook.

12
The Charter Oak Group, LLC

Rev. 5 (5/19/10)

E. CORE PERFORMANCE MEASURES
MEASURE
1. Community Service

2. Common Measures
Entered Employment

3. Common Measures
Employment
Retention

4. Common Measures
Average Earnings

5. Service Level

6. Service to Most-inNeed

DESCRIPTION

GOAL

Q
RATE

YTD
RATE

L4Q
RATE

The number of hours of community service
in the reporting period divided by the number
of hours of community service funded by the
grant minus the number of paid training
hours in the reporting period

N=

N=

D=

D=

Of those not employed at the time of
participation, the number of participants
employed in the first quarter after the exit
quarter divided by the number of participants
who exit during the quarter

N=

N=

N=

D=

D=

D=

Of those participants who are employed in
the first quarter after the exit quarter, the
number employed in both the second and
third quarters after the exit quarter divided by
the number of participants who exit during
the quarter

N=

N=

N=

D=

D=

D=

Of those participants who are employed in
the first, second, and third quarters after the
quarter of program exit, total earnings in the
second and third quarters after the exit
quarter, divided by the number of exiters
during the period

N=

N=

N=

D=

D=

D=

The number of participants who are active
on the last day of the reporting period or who
exited during the reporting period divided by
the number of modified community service
positions

N=

N=

D=

D=

Average number of barriers per participant.
The total number of the following
characteristics: severe disability, frail; age 75
or older, old enough for but not receiving SS
Title II, severely limited employment
prospects and living in an area of persistent
unemployment, limited English proficiency,
low literacy skills, disability, rural, veterans,
low employment prospects, failed to find
employment after using WIA Title I, and
homeless or at risk of homelessness divided
the number of participants who are active on
the last day of the reporting period or who
exited during the reporting period

N=

N=

N=

D=

D=

D=

13
The Charter Oak Group, LLC

Rev. 5 (5/19/10)

Measure E1. Community Service. This measure reports the amount of community
service provided compared to the amount of community service funded by the grant. The
number of hours of community service (total hours paid minus paid training hours) is
divided by the number of modified positions times 21 hours per week times the number
of weeks in the reporting period, minus the number of paid training hours.
Measure E2. Common Measures Entered Employment. This measure reports the
employment outcomes for all participants who were not employed at the time of
participation (field 13 of the Participant Form). The number of exiters who have any
wages in the first quarter after the quarter of exit (field 28(c) of the Unsubsidized
Employment Form) is divided by the total number of participants whose first quarter after
the exit quarter is during the reporting period.
Measure E3. Common Measures Employment Retention. This measure reports the job
retention of those exiters who had wages in the first quarter after the quarter of exit, i.e.,
who entered employment. The number of exiters who have any wages in both the second
and third quarters after the quarter of exit (fields 29(c) and 29(e) of the Unsubsidized
Employment Form) is divided by the number of exiters who have any wages in the first
quarter after the quarter of exit (field 28(c) of the Unsubsidized Employment Form), i.e.,
who entered employment.
Measure E4. Common Measures Average Earnings. This measure reports the average
earnings during the second and third quarters after the quarter of exit for all exiters who
were employed in the first, second, and third quarters after the quarter of exit, i.e., who
achieved retention as defined by the Common Measures. The total amount of wages for
the second and third quarters after the quarter of exit (fields 29(d) and 29(f) of the
Unsubsidized Employment Form) is divided by the number of exiters who have any
wages in the second and third quarter after the quarter of exit, i.e., who achieved
retention. This measure is reported solely for participants who were enrolled into the
program on or after July 1, 2005.
Measure E5. Service Level. This is the number of participants active at any time during
the reporting period (element B3 plus element B4) divided by the number of modified
positions (element A5). The numerator for this measure is cumulative, but the
denominator is fixed.
Measure E6. Service to Most-in-Need. This measure reports the average number of
specified barriers to employment per participant. The total number of the specified
priorities of service (elements D25 through D30, D32, and D34) and waiver of duration
factors (elements D31 and D36 through D39) is divided by the number of participants
active during the reporting period.
Use. This section of the QPR reports performance on the core SCSEP measures. It
allows you to see your current performance, your performance for the program year to
date, and the last four quarters, which will provide the trend for each measure even at the
start of the program year. The trend may be the most important management tool
14
The Charter Oak Group, LLC

Rev. 5 (5/19/10)

because it tells you whether you are likely to achieve your goal under current conditions.
When evaluating the measures reported in the QPR, you will obviously have to take into
account any extraordinary conditions that may be affecting performance in one direction
or another. For example, if entered employment has lagged in the current quarter because
your job developer was out sick but the trend line has been generally positive, the failure
to meet your quarterly goal should not be disturbing. On the other hand, even if you have
met your goal, if the trend is negative, you should consider systematic changes that will
reverse the trend.
Measure E1. Community Service. This measure reports how effectively you are using
your grant to provide community service. You are held harmless for any paid training
provided. The numerator is primarily affected by the number of participants enrolled
(elements B3 and B4) and the total number of hours of community service provided
(element C6). Because this measure does not reflect the distribution of community
service hours among your participants, element B12 tells you the percent of participants
who are receiving the optimum number of community service hours per week.
You can improve performance on this measure by enrolling more participants, reducing
turnover time when enrolling new participants, assigning participants to more hours of
community service per week, limiting leaves of absences, and ensuring that all paid hours
are accurately reported in SPARQ each quarter. An edit in SPARQ requires you to enter
total hours paid for each quarter in which a participant was active.
NOTE: While the grantee QPR will always contain accurate data for this measure, subgrantee QPRs will not be accurate for this measure unless their grantees have reported to
DOL the proper count of authorized positions for each sub-grantee.
Measure E2. Common Measures Entered Employment. The QPR does not report
success or failure on this measure until the reporting period (the first quarter after the
quarter of exit) has ended. Therefore, while the QPR will always provide you with
accurate information about this measure, measure E2 cannot be used during the quarter
for management purposes. For that, you will need to consult the management report,
Started Employment But Not Yet Achieved Entered Employment. This report tracks all
participants who have an unsubsidized employment record but for whom success or
failure in entered employment has not yet been reported in SPARQ. The summary
section of this report contains a version of the entered employment rate that provides realtime information on the entered employment rate. On the first day of the reporting
period, the denominator of the entered employment rate is fully populated while the
numerator starts at zero for that quarter’s cohort. Throughout the quarter, as each followup is done and successful entered employments are entered into SPARQ, the numerator
rises. This report does include records that are rejected by the QPR. The Exited report
also tells you whether each exiter has achieved entered employment.
The follow-up reports will help you schedule and track follow-ups that are pending. If
you do not complete the follow-up, the QPR cannot report a successful placement. You
should also pay attention to rejected records (element B5b). Most rejections involve
15
The Charter Oak Group, LLC

Rev. 5 (5/19/10)

performance outcomes; you get no credit for rejected records until you correct them in
SPARQ. The Exited report will identify all participants who have exited for any reason
and whether they have achieved entered employment. You can use this report to target
your resources to those participants (including those who might have exited for personal
reasons) with whom you still have a chance to achieve entered employment.
Information in Section B of the QPR is also very helpful for managing the entered
employment rate. Total exits (element B3) will let you determine how many successful
entered employments you will need to reach your goal. Since the vast majority of those
who exit for unsubsidized employment achieve entered employment, the ratio of exits for
unsubsidized employment (element B8a) to total exits (element B3) is important. You
should also pay attention to the ratio of exclusions (element B8c) to total exits since
exclusions do not count for any of the Common Measures. Total placements (element
B10a) compared to element B8a will tell you how many participants who exit for
personal reasons nonetheless start unsubsidized employment and how many participants
have multiple placements. Element B10f will tell you how many participants who started
unsubsidized employment failed to achieve entered employment. These unsuccessful
placements may be due either to a negative outcome for the placement having been
entered into SPARQ or to Follow-up 1 not having been done. The management reports
will tell you if there are undone Follow-up 1’s. If so, you have until September 30
following the end of the program year to complete these follow-ups and receive credit in
your entered employment rate.
You can use the QPR to deconstruct your entered employment rate for the current report
quarter and to predict your entered employment rate for the next report quarter. For the
current report quarter:
 Measure E2 gives you the counts for the numerator and denominator of the
Entered Employment Rate.
 The denominator comes from last quarter’s exiters. Subtract exclusions (elements
B8c and B8d) from total exits (element B3).
 Any changes in data since the freezing of the prior quarter, e.g., correction of
rejects, could cause this calculation to be a little different from the actual
denominator in measure E2.
 The numerator starts with all exiters from the prior quarter who began a
placement in the prior quarter (element B10a). This is close to your highest
potential numerator.
 For the prior quarter, the difference between element B10a and element B8a may
tell you how many participants who exited for personal reasons nonetheless
started a placement in the prior quarter. If you continue to track exiters from the
prior quarter who left for personal reasons and they get a job in the report quarter,
that would increase the potential numerator. There is no QPR field for this, but
you can find these participants in the management report, Started Employment But
Not Yet Achieved Entered Employment.
 Subtract failed placements (element B10f) in the report quarter QPR from total
placements (element B10a) to get the numerator for the report quarter. Element
16
The Charter Oak Group, LLC

Rev. 5 (5/19/10)

B10f reports placements where the exit occurred in the prior quarter and no
entered employment was recorded by the end of the report quarter.
 Failed placements can be due to follow-ups not being done or not being properly
entered into SPARQ, or to the actual failure of the placement to last into the
report quarter and a new placement not being started in the report quarter.
 Use the follow-up report to see how many undone follow-ups you have. Count
any pending Follow-up 1 where the exit date was in the prior quarter.
 Undone follow-ups can be completed and entered into SPARQ until September
30 following the close of the program year.
To calculate an estimate of your entered employment rate for the next report quarter:
 The denominator will be total exits (element B3) in the report quarter minus
exclusions (elements B8c and B8d) in the report quarter.
 The potential numerator is the total of all placements started in the report quarter
(element B10a) plus any placements started in the next report quarter. (There is
no QPR field for this.) You can track the successful entered employments for the
new quarter by using the Started Employment But Not Yet Achieved Entered
Employment or Exited management reports for the quarter.
 The potential numerator will be reduced by any placements that do not last into
the next report quarter. Assuming that all Follow-up 1’s will be completed and
properly entered into SPARQ during the report quarter, calculate the true YTD
failed placement rate by subtracting from element B10f those failures that are due
to undone follow-ups and divide this result by the total placements YTD in
element B10a. You can apply the resulting rate to the potential numerator to
determine what the final numerator may be if the same rate of failed placements
prevails.
If you are having trouble with the entered employment rate, the QPR and the
management reports will help you identify which of these causes might be at work:






Follow-up 1 not completed or not properly entered into SPARQ
Too many rejected records
Too many participants exiting for reasons other than unsubsidized employment
Too many participants exiting for other reasons not identified as exclusions
Too few participants who exit for other reasons subsequently starting
unsubsidized employment
 Too many participants starting unsubsidized employment but failing to be
employed with some employer in the quarter after the exit quarter
Measure E3. Common Measures Employment Retention. As with the entered
employment rate, the management reports are extremely useful for managing
performance on this measure. There are reports that track pending follow-ups, and a
report of all participants who have achieved entered employment but have not yet
achieved retention. Like the report on Started Employment But Not Yet Achieved Entered
Employment, this report has a real-time counter for the retention measure. The
denominator will be fully populated on the first day of the reporting period, and the
17
The Charter Oak Group, LLC

Rev. 5 (5/19/10)

numerator will grow with each successful Follow-up 2 recorded. You will want to stay in
close touch with the participants in this report to make sure that they are getting the
support they need on their current jobs or to find them new jobs if the old jobs are not
working out. Case management follow-up should be done early in each quarter
regardless of when you do the follow-up for performance reporting. This is a measure
where you cannot afford to be passive. If you do not contact the participant until you do
Follow-up 2 for performance purposes, you will be too late to have any effect on the
outcome.
Measure E4. Common Measures Average Earnings. Average earnings are reported only
for those participants who achieve retention. The primary factors affecting this measure
are average starting wage (element B10c and Unsubsidized Employment Form field 16),
average number of hours of work per week (element B10e and Unsubsidized
Employment Form field 18), and number of weeks of employment in the second and
third quarters after the exit quarter (not recorded). Average hours per week and average
starting wage should be a useful predictor of the average earnings measure.
Measure E5. Service Level. Service level is a measure that you can diagnose from the
QPR. If the service level is lagging, you can see what the problem is by examining
elements B1-B5a. Generally, you cannot enroll new participants unless you are exiting a
sufficient number of existing participants. The ratio of new to carried over participants is
also helpful for this measure, as is the number of vacancies. Management reports for
pending applicants, applicants who have been determined eligible but not yet assigned,
and waiting list can also be helpful in diagnosing service level problems.
NOTE: While the grantee QPR will always contain accurate data for this measure, subgrantee QPRs will not be accurate for this measure unless their grantees have reported to
DOL the proper count of authorized positions for each sub-grantee.
Measure E6. Service to Most-in-Need. Section D of the QPR will tell you how you are
doing on each of the characteristics that comprise the most-in -need measure. You can
use this information to modify your enrollment practices if you need to increase the
number of most-in-need factors per participant. This is also a good way to verify the
accuracy of your data collection. You can use the management reports to see the
characteristics of each participant. Sometimes staff persons do not pick up barriers that
could count toward the measure. Remember that you will receive most-in-need credit for
the priorities of service as long as the participant is enrolled, but the waiver factors will
only count for the program year in which they were first entered unless the data are redocumented in subsequent program years.

VII. Section F. Additional Performance Measures
Description. Section F presents the two additional performance measures for which there
are no negotiated goals: retention at 1 year; and the satisfaction of employers,
participants, and host agencies. The timing rules for retention at 1 year are similar to
18
The Charter Oak Group, LLC

Rev. 5 (5/19/10)

those for the Common Measures. The activity period and the reporting period are both
the fourth quarter after the quarter of exit. As with the Common Measures, no successes
or failures for this measure are reported until the reporting period has ended.
F. ADDITIONAL PERFORMANCE MEASURES
MEASURE
1. Retention at 1 Year

2. Customer Satisfaction

DESCRIPTION
Of those participants who are employed in the first
quarter after the exit quarter: the number of
participants who are employed in the fourth quarter
after the exit quarter divided by the number of
participants who exit during the quarter

Q
RATE

YTD
RATE

L4Q
RATE

N=

N=

N=

D=

D=

D=

Average ACSI for employers

Average annual ACSI for participants

Annual average ACSI for host agencies

Measure F1. Retention at 1 Year. This measure reports the percent of participants who
entered employment and who are employed approximately one year after exit. The
number of participants employed in the fourth quarter after the quarter of exit is divided
by the number of participants who entered employment.
Measure F2. Customer Satisfaction. This measure, which is in three parts, reports the
average American Customer Satisfaction Index (ACSI) score for employers, participants,
and host agencies. The index combines the scores on the first three questions of each
survey and assigns weights to those scores based on a proprietary methodology. The
survey for employers is administered throughout the program year. The participant and
host agency surveys are administered only once each year. The ACSI scores for
employers, participants and host agencies are only reported once each program year, after
the nationwide surveys have been analyzed.
Use. Unlike the measures in Section E, the measures reported in this section have no
goals. However, grantee performance on these measures is required to be published at
19
The Charter Oak Group, LLC

Rev. 5 (5/19/10)

the end of the program year. Like Section E, this section allows you to see your current
performance, your performance for the program year to date, and the trend for each
measure.
Measure F1. Retention at 1 Year. This measure extends the follow-up period to
approximately 15 months after the date of exit. The only participants subject to this
measure are those who entered employment. Although there is not yet a management
report specifically for this measure, the existing follow-up reports will tell you if a
participant is in the pool for this measure and needs to have Follow-up 3 completed. You
may conduct Follow-up 3 at the same time as Follow-up 2.
Measure F2. Customer Satisfaction. The ACSI scores reported here are not sufficient by
themselves to tell you how to improve services. Grantees receive separate reports for
each of the customer satisfaction surveys. These reports contain detailed analyses that
identify those services that have the greatest effect on overall satisfaction.

20
The Charter Oak Group, LLC

Rev. 5 (5/19/10)