Monthly Wage Pilot Findings02-2007

MonthlyWagePilotFindings02-2007.pdf

Monthly SSI Wage Reporting

Monthly Wage Pilot Findings02-2007

OMB: 0960-0715

⚠️ Notice: This form may be outdated. More recent filings and information on OMB 0960-0715 can be found here:

Document [pdf]

Download: pdf | pdf

Supplemental Security Income
Second Telephone Monthly Wage Reporting Pilot
Evaluation

Office of Quality Performance

February 2007

SUPPLEMENTAL SECURITY INCOME (SSI)
SECOND TELEPHONE MONTHLY WAGE REPORTING PILOT EVALUATION
EXECUTIVE SUMMARY
The first SSI telephone monthly wage reporting pilot conducted in CY 2003 established that
monthly wage reports made through an automated telephone system were a cost-effective
method of decreasing improper SSI payments by preventing both overpayments and
underpayments. The purpose of the second SSI telephone monthly wage reporting pilot was to
test new methods of recruiting and authentication. In the first pilot, volunteers were recruited by
mass mailings. The response rate to the first mass mailing was low and a second mass mailing
with some telephone follow-ups was needed to get a sufficient number of participants.
Consequently, it was decided that the second pilot would recruit participants by asking people
who were in the field office (FO) to conduct other business if they would like to volunteer. In
the first pilot, people who volunteered to participate were asked to call the national 800 number
and complete a two-step automated process to get a password. The person’s Social Security
Number (SSN) and password were used to authenticate the caller before a wage report was made.
However, half the volunteers were unable to follow the instructions to get a password even with
assistance from their local FO. Consequently, it was decided that the second pilot would use
knowledge based authentication (KBA) with the caller being asked to provide his or her SSN,
name and date of birth before reporting wages.
Between January and September 2006, a total of 394 individuals participated in the second SSI
telephone monthly wage reporting pilot by making one or more successful wage reports using
the automated telephone system. One-third of the wage earners were SSI recipients and twothirds were deemors. These callers were asked to speak their wage reports but could use the
telephone keypad if the system had difficulty understanding them.
The biggest hurdle was tuning the automated telephone reporting system to recognize the names
of the callers. Although progress has been made, additional improvement is possible.
Highlights of Findings

•

In-office recruiting of volunteers for the second pilot was easier and took less time than using
mass mailings to recruit volunteers for the first pilot.

•

Over 70 percent of the volunteers in the second pilot were able to pass authentication and
report wages using KBA. This is an improvement over the first pilot where only 50 percent
of the volunteers were able to pass authentication by obtaining and using a password.

•

There is a proposal to pass the numident name to the vendor for the duration of the call that
could increase the telephone system’s ability to recognize names by as much as 20 percent.
This would further increase the number of volunteers who are able to pass authentication.

•

Twenty percent of the second pilot volunteers reported for only 1 or 2 months.

•

The majority of FOs that participated in both SSI telephone monthly wage reporting pilots
said the second pilot was easier overall than the first pilot.

•

Receipts were issued to all disabled individuals who successfully reported their wages
through the telephone reporting system.

Conclusions

The new recruiting and authentication procedures for the second pilot were an improvement over
the recruiting and authentication procedures used for the first pilot. However, the automated
system provided by the vendor had some difficulty recognizing the names of the reporters.
Therefore, we think it is important that improvement in name recognition be pursued.
A survey was conducted of the 34 FOs that participated in both the first and second SSI
telephone monthly wage reporting pilots. Their majority view was that the second pilot was
easier overall and took less time than the first pilot.
Although we designed it to be as simple as possible, not all pilot volunteers were up to the
challenge of using the automated telephone system. Approximately 20 percent reported for only
1 or 2 months. Feedback from FOs indicated that participants dropped out of the pilot because
they thought it was too difficult or they preferred to mail their pay slips to the FO.
Getting people who are not reporting their wages to report by telephone will improve SSI
payment accuracy. Recruiting and training telephone wage reporters will be a new workload for
FOs. Therefore, we support the Agency’s recent decision to eliminate verification of wages
reported monthly through the automated telephone system unless there is a match alert which
indicates a discrepancy; i.e., an S2, K6 or K7 alert. This expedient is a reasonable trade-off.
Recommendations

We recommend that SSI monthly wage reporting using an automated telephone system be made
available as a reporting option to SSI recipients, deemors and representative payees nation-wide.
Although some screening by FOs will be necessary, this is a viable reporting option that should
accommodate a large number of individuals who are obligated to report wage changes.
We also recommend the following enhancements:
•

There is a proposal that would improve the recognition of unusual names. For all calls
received by the telephone system, the vendor would pass the SSN and date of birth given by
the caller to SSA. The corresponding name would be obtained from the Numident file by
SSA and sent back to the vendor. The vendor would add the Numident name to their
database of names for the duration of the call with no preference being given to the SSA
provided name. (The Numident name would just become another option for matching
against the caller provided name.) This systems enhancement could increase name
recognition by as much as 20 percent. We would prefer to see this enhancement made before
a national roll-out.

•

When a volunteer is recruited, he or she should be told by FO staff what name to use when
reporting wages. For example, if a person uses the name Mike but is shown on SSA records
as Michael, he should be told to use the name Michael when reporting.

•

Issue receipts to all wage reporters not just disabled individuals. Although SSA is only
required to issue receipts to disabled individuals, this step is necessary to assure callers that

their wage reports were successful. Otherwise, they tend to call the FO and ask if their
reports went through.
•

Allow concurrent TitleII/Title XVI beneficiaries to report their wages through the automated
telephone system.

•

Create a unique systems code to identify wage reports made through the automated telephone
system. Whether a new code for an existing field or a new field, this would serve multiple
purposes. It would allow the FO to know with certainty which reporters were telephone
reporters. It would allow for improved management information. Also, it would provide a
means of prioritizing the wage alert workload.

If the roll-out of telephone monthly wage reporting occurs before name recognition is improved,
consideration should be given to testing whether the volunteers can pass authentication while
they are still in the FO. This could be accomplished in one of two ways. The vendor could be
asked to set up a test system that would allow volunteers to practice making a report before they
leave the FO. This test system could be available throughout the month. If a test system is not
feasible, then we could restrict recruiting new volunteers to days of the month when the
telephone system is operational; i.e., from the first of the month until the Goldberg/Kelly cutoff
date. This way volunteers could try to make a real telephone wage report before leaving the FO.
In either case, we would avoid having a large number of people fail authentication after they go
home and try to report their wages.
A claims representative may be uncertain whether the volunteer will be able to report the correct
amount of wages because of a cafeteria plan or other complication. In these cases, we
recommend that the volunteer be asked to report by telephone and mail pay slips to the FO for a
two month test period. If the volunteer passes the test, they would be asked to stop mailing their
pay slips. If they fail the test, they would be asked to stop reporting by telephone.

SUPPLEMENTAL SECURITY INCOME (SSI)
SECOND TELEPHONE MONTHLY WAGE REPORTING PILOT EVALUATION
Background

SSI Stewardship payment accuracy findings point to wages as the second largest category of SSI
overpayment (O/P) and the largest category of SSI underpayment (U/P) over the past 5 years.
On average, approximately $400 million a year in O/Ps and $175 million in U/Ps is attributable
to wages. Stewardship data for FY 2005 indicate that approximately $190 million of these O/P
dollars and $120 million of these U/P dollars are the result of fluctuating income and failure to
report timely an increase or decrease in wages to SSA. Changes in the amount of wages received
by an SSI recipient or deemor (i.e., ineligible spouse or parent) affect the recipient’s payment
amount or eligibility status.
Much of this O/P and U/P could be prevented by large scale monthly wage reporting involving
tens of thousands of SSI recipients and deemors if it can be done in a quick and accurate manner.
Consequently, we piloted a new automated monthly wage reporting process using voice
recognition and touch-tone-telephone technology for wage reporting.
The first SSI telephone monthly wage reporting pilot was conducted in CY 2003. It established
that timely monthly wage reports made through an automated telephone system were a costeffective method of decreasing improper SSI payments by preventing both O/Ps and U/Ps. An
August 2004 report evaluating the first pilot is available on OQP’s intranet website in the
Title XVI report area. The direct address for this report is:
http://quality.ba.ad.ssa.gov/hq/oqareports/2004/OAIPQ_MonthlyWageReprtFindings.html. Also,
see Appendix G for a note concerning the issue of a control group or time period for the pilot.
Purpose of the Second Pilot

The purpose of the second SSI telephone monthly wage reporting pilot was to test new methods
of recruiting and authentication. A method of issuing receipts to disabled individuals was also
added to the automated system.
In the first pilot, volunteers were recruited by mass mailings. The response rate to the first mass
mailing was low and a second mass mailing with some telephone follow-ups was needed to get a
sufficient number of participants. Consequently, it was decided that a second pilot would recruit
participants by asking people who were in the field office (FO) to conduct other business if they
would like to volunteer. In the first pilot, people who volunteered to participate were asked to go
through a two-step automated process to get a password. The person’s Social Security Number
(SSN) and password were used to authenticate the caller. However, half the volunteers were
unable to follow the instructions to get a password even with assistance from their local FO.
Consequently, it was decided that the second pilot would use knowledge based authentication
(KBA) with the caller being asked to provide an SSN, name and date of birth before being
permitted to report wages.
Between January and September 2006, a total of 394 volunteers participated in the SSI telephone
monthly wage reporting pilot by making one or more successful wage reports using the
automated telephone system. One-third of the wage earners were SSI recipients and two-thirds
were deemors. These callers were asked to speak their wage reports but could use the telephone
keypad if the system had difficulty understanding the amount.

Second Pilot Business Process

The pilot was national in scope. A total of 51 FOs were selected to participate in the pilot with at
least 5 from each of the 10 regional offices. These FOs recruited SSI recipients, deemors and
representative payees to report wages by telephone while they were in the FO to conduct other
business; i.e., initial claims, redeterminations, limited issues and other posteligibility transactions.
The volunteers were told how to report their wages while in the FO and were given written
instructions to take home with them.
The telephone system was available from the first of the month until the Goldberg/Kelly cutoff
which varies from the 6th to the 9th day of the month. To avoid confusion and ensure reports
were timely, all volunteers were asked to report wages during the first 6 days of the month. The
volunteers called a special toll free number and reported the amount of wages they received in
the prior month by listening to recorded instructions and speaking their reply or pushing buttons
on their telephone keypad. They were asked to do this once per month for a 6 month period
starting in January 2006. (The pilot has since been extended through May 2007.)
People reporting wages they themselves earned were asked to:
1. Call a special toll free number for wage reporting;
2. Speak or key in their SSN;
3. Speak and spell their first and last names (e.g., “Smith, S-M-I-T-H”);
4. Speak or key in their date of birth;
5. Speak or key in the amount of wages received in the month just ended.
People reporting wages earned by another person (e.g., a representative payee) were asked to:
1. Call a special toll free number for wage reporting;
2. Speak or key in the wage earner’s SSN;
3. Speak or key in the caller’s SSN;
4. Speak and spell the caller’s first and last names (e.g., “Smith, S-M-I-T-H”);
5. Speak or key in the caller’s date of birth;
6. Speak or key in the amount of wages received in the month just ended.
Note: After each answer, the system spoke back the SSN, name, date of birth or amount of
wages and asked if it was correct. (The names were both spoken and spelled.)
Identifying information given by the individual was electronically transmitted to SSA by the
contractor, Verizon. The SSN, name and date of birth were checked for authentication purposes.
If the call passed authentication, the wage amount was obtained and transmitted to SSA. Once
per day (overnight), wage amounts were uploaded to the MSSICS pending file and the SSR.

A receipt for the wage report was mailed to all disabled individuals. If the wage report resulted
in a change in SSI payment, the recipient also received a written notice before the change took
place. The notice included appeal rights.
The data for the January through September 2006 reporting periods was analyzed by OQP to
determine whether the new authentication and recruiting procedures were an improvement over
the procedures used in the first pilot. This report is the result of that evaluation.
Procedural Differences Between First and Second Pilots

Procedure
Recruiting Volunteers
Authentication
Receipts for Wage Reports

First Pilot
Mass Mailings
Passwords
None

Second Pilot
In-Office Recruiting
KBA
All disabled individuals

Comparison of Recruiting Procedures

The first pilot recruited volunteers by mass mailings. Starting in March 2003, recipients and
deemors with wages were sent letters asking them to contact their local FO if they were
interested in volunteering to participate in the SSI telephone monthly wage reporting pilot. The
positive response rate to the mailing was about 6 percent. Consequently, we had only about half
the number of reporters we needed when the pilot started on May 1, 2003 and additional efforts
had to be made to recruit volunteers. A second mass mailing was sent out in June 2003 and FOs
followed-up by telephone to recruit a sufficient number of volunteers.
The second pilot did not recruit by mail. SSI recipients, deemors and representative payees were
asked if they would like to participate in the pilot while they were in the FO to conduct other
business; i.e., they were in the FO to file an initial claim or to complete a redetermination,
limited issue or other post-eligibility transaction. Most of the recruiting took place in
December 2005. When it was determined that 56 of the people who unsuccessfully attempted to
report in the first month of the pilot (January 2006) should not have been recruited, FOs were
asked to find replacements. Otherwise, additional efforts to recruit volunteers were not needed.
A survey was conducted of FOs participating in both the first and second telephone monthly
wage reporting pilots. A majority of these FOs thought recruiting was easier in the second pilot
than in the first pilot. Although they were able to meet their recruiting goals, some FOs had a
hard time finding volunteers who met the criteria for the second pilot among the people who
were in the office to conduct other business. For example, FOs with a large Spanish speaking
population had difficulty finding English speakers with wages. (See Appendix A for the criteria
list used by FOs and Appendix B for a list of systems exclusions.)
Comparing the two, it is clear that FOs were able to recruit volunteers much faster and more
efficiently using the in-office procedure of the second pilot. Even the FOs who reported some
difficulty finding eligible volunteers met their recruiting goals in a much shorter period of time.
Comparison of Authentication Procedures

The first pilot used password authentication. All volunteers called the national 800 number to
obtain a password that they used every time they reported their wages.

Volunteers were required to go through a two step process to obtain a password. First, they
called the national 800 number and selected the menu options to ask for a temporary password
known as a password request code. This was mailed to their address of record. After they
received the password request code in the mail, they called the national 800 number again and
selected different menu options to change their temporary password request code into a new
permanent password. This was confusing and difficult for the SSI recipients and deemors who
volunteered for the pilot.
Obtaining passwords was the biggest problem encountered during the first pilot. FO staff
reported that a great deal of time was spent assisting volunteers to obtain passwords. Despite
these efforts, half the volunteers were not able to obtain them.
Another limitation was that passwords could only be issued to SSI recipients and deemors.
Authentication rules forbid them from being issued to representative payees. This excluded
about one-third of potential reporters from the telephone wage reporting system.
The second pilot used KBA. A password was not used to gain access to the telephone wage
reporting system. Instead, all volunteers were asked to provide their SSN, first name, last name
and date of birth each time they called to report wages. If the caller was not the wage earner
(e.g., a representative payee) they were also asked to provide the wage earner’s SSN.
A higher percentage of volunteers were able to report using KBA. Approximately 72 percent
(394 out of 548) of the volunteers in the second pilot successfully passed authentication one or
more times versus 50 percent of the volunteers for the first pilot. By itself, this is sufficient
evidence that KBA was an improvement over password authentication.
A survey of the FOs that participated in both SSI telephone monthly wage reporting pilots found
a majority thought they spent less time on the second pilot than on the first pilot. The FOs said it
was easier to instruct the volunteers how to call the telephone system and report their wages for
the second pilot. A major part of these instructions involved how to pass authentication.
Although KBA was an improvement over password authentication, it was not perfect. If the
caller was not the wage earner, they would often get confused about when to enter their own
SSN and when to enter the wage earner’s SSN. Some callers also gave name and date of birth
information for the wrong person. Consequently, the call flow was changed to ask up front, “Are
you reporting wages for yourself? Please say Yes or No.” Answering “No” now takes the caller
down a separate path that specifies whose information is being requested.
A remaining problem with KBA is name recognition. The vendor’s database of names does not
recognize unusual names and we seemed to have many of them in the SSI population. This is
not a problem with the SSI population; it is a software limitation. Some improvements were
made to the software as the pilot progressed but more improvement is recommended. (See
Recommendations)
Despite the name recognition issue, it is clear that KBA is an improvement over password
authentication. It is easier and a higher percentage of reporters were successful using it.

Comparison of Receipts and Notices

The first pilot issued a notice only if the amount of wages resulted in a change. In addition to
issuing these notices, the second pilot issued a receipt for all wage reports made by disabled
individuals in accordance with SSA policy. A systems check confirmed that receipts were being
issued to disabled individuals whenever a wage report was passed to SSA from the telephone
wage reporting system.
Reporting Persistence

Calls tapered off after the first 4 months partly because of a systems problem in May and partly
because the volunteers were asked to report their wages for 6 months (the original planned length
of the pilot).

Month
Called
01/2006
02/2006
03/2006
04/2006
05/2006
06/2006
07/2006
08/2006
09/2006

Successful
Reports
290
332
329
299
259
277
233
233
227

The automated telephone system was available to accept wage reports from the first of the month
until the Goldberg/Kelly cutoff date which varied between the 6th and the 9th day of the month.
All volunteers were asked to report during the first 6 days of the month to avoid any
misunderstanding about the cutoff date.
The system was very reliable with one notable exception. The automated telephone system was
unavailable on the first two days of May 2006 due to an SSA systems configuration issue. As a
result, hundreds of calls could not be processed. Not all reporters called again when the system
was working. This was an unplanned and unexpected problem that adversely affected the ability
of volunteers to report. No similar problem occurred during the first pilot.

Months
Successfully
Reported
1
2
3
4
5
6
7
8
9
Totals

Number of
People
50
29
30
25
22
28
45
74
91
394

Percent of
Total
13%
7%
8%
6%
6%
7%
11%
19%
23%
100%

Approximately 20 percent of the people reporting wages stopped reporting after only 1 or 2
months. Although they had volunteered to participate, they changed their minds and dropped out
of the pilot. Feedback from FOs indicated these participants thought telephone reporting was too
difficult or preferred to report by mailing their pay slips to the FO. In the first pilot,
approximately 15 percent of the people stopped reporting after only 1 or 2 months.
Approximately 60 percent of the people reporting wages made reports for 6 or more months.
Originally, we asked people to report for a 6 month period ending in June 2006. The pilot was
extended as the workgroup continued to seek ways to improve reporting success. However,
some reporters dropped out after June.
Excluding May 2006 when the automated telephone system was not always available, 23 percent
of the people reporting wages had a break in reporting. They successfully reported wages then
skipped one or more months before successfully reporting again.
At various times during the pilot, improvements were made to the automated system to make it
easier for volunteers to report. For example, after a person gave their name, the system would
speak back the name and asked if it was correct. However, people with names that could be
spelled more than one way sometimes failed authentication. To improve the situation, the
system was modified to both speak back and spell the person’s name when asking if it was
correct. Because during this pilot improvements were made to the system as problems were
identified, it is not surprising that many reporters either stopped at some point or had a break in
reporting.
Summary of FO Survey Results

The 51 FOs that participated in the second SSI telephone wage reporting pilot were asked to
complete a survey in October 2006. They all cooperated and completed the survey
questionnaires. One questionnaire asking about their experience with the second pilot was
completed by all 51 FOs. A second questionnaire was completed by the 34 FOs that participated
in both the first pilot conducted in 2003 and the second pilot conducted in 2006. These 34 FOs
were asked to compare the two pilots.
The 34 FOs that participated in the two SSI telephone monthly wage reporting pilots thought the
second pilot was an improvement over the first pilot. A majority of these FOs thought the
second pilot was easier overall and took less time than the first pilot. Very few FOs thought the

second pilot was harder in any respect. Only one FO thought the second pilot was harder overall
and only two FOs thought the second pilot took more time.
A majority of the 51 FOs that participated in the second pilot recommended that SSA offer
telephone monthly wage reporting in the future. The dissenters expressed doubts that the people
in their service area had the skills to report properly or questioned the reliability of the pilot
telephone system.
Although recruiting was considered easier in the second pilot, it was still rated difficult by most
of the 51 FOs participating in the second pilot. FO staff had to screen potential reporters against
a list of criteria on a case by case basis. (See Appendix A) Some FOs had a harder time finding
people who could be recruited because of the composition of the populations they serve; e.g., a
high percentage of people who do not speak English. In remarks, they said they would like
fewer exclusions so more people could be recruited. For example, some offices indicated they
would like to be able to ask people who speak Spanish or people who are concurrent
Title II/Title XVI beneficiaries to participate in telephone wage reporting.
Most FOs thought instructing the volunteers was easier in the second pilot. This is a direct result
of the new method of authentication. The biggest complaint in the first pilot was how difficult it
was and how much time it took to instruct and assist the volunteers to get passwords. In the first
pilot only, they needed a password before they could call the automated telephone system. The
difficulties encountered in the second pilot centered on assisting volunteers who were running
into problems using the automated telephone system. The inability of the system to recognize
the reporter’s name was often cited. This was generally because the caller had an unusual name
(e.g., Blinda) or spoke with an accent. Another complaint was dropped calls. Although details
were not given, we know that people who called to report wages that were earned by another
person (e.g., a representative payee calling to report wages earned by an SSI recipient) found the
voice prompts confusing and often gave the wrong information. This resulted in the call being
dropped before the wage report could be completed. A separate path with new voice prompts
was created for people who were reporting wages earned by another person to avoid confusion.
Summary of 51 FO Responses to Questionnaire about the Second Pilot:
Issue
Recruiting Volunteers
Instructing Volunteers

21
31

Easy
41.2%
60.8%

30
20

Yes

Difficult
58.8%
39.2%

Issue
Had to Give Assistance
to Volunteers
Recommend Future Use
of Telephone Reporting

86.3%

13.7%

58.8%

41.2%

Issue
Overall Satisfaction

Good
33.3%

Fair
31.4%

Poor
35.3%

Summary of 34 FO Responses to Questionnaire Comparing Second Pilot to First Pilot:

Issue
Recruiting Volunteers
Instructing Volunteers
Assisting Volunteers
Overall Comparison

Easier
18
52.9%
23
67.6%
12
35.3%
18
52.9%

About the Same

Issue
More or Less Time

Less Time
19
55.9%

About the Same

14
9
22
15

41.2%
26.5%
64.7%
44.1%

38.2%

Harder
2
5.9%
2
5.9%
0
0.0%
1
2.9%
More Time
2
5.9%

See Appendixes E, F and G for more details about the responses to these two questionnaires.
Conclusions

recognition by as much as 20 percent. We would prefer to see this enhancement made before
a national roll-out.
•

•

Issue receipts to all wage reporters not just disabled individuals. Although SSA is only
required to issue receipts to disabled individuals, this step is necessary to assure callers that
their wage reports were successful. Otherwise, they tend to call the FO and ask if their
reports went through.

•

Allow concurrent TitleII/Title XVI beneficiaries to report their wages through the automated
telephone system.

•

If the roll-out of telephone monthly wage reporting occurs before name recognition is improved,
consideration should be given to testing whether the volunteers can pass authentication while
they are still in the FO. This could be accomplished in one of two ways. The vendor could be
asked to set up a test system that would allow volunteers to practice making a report before they
leave the FO. This test system could be available throughout the month. If a test system is not
feasible, then we could restrict recruiting new volunteers to days of the month when the
telephone system is operational; i.e., from the first of the month until the Goldberg/Kelly cutoff
date. This way volunteers could try to make a real telephone wage report before leaving the FO.
In either case, we would avoid having a large number of people fail authentication after they go
home and try to report their wages.
A claims representative may be unsure whether the volunteer will be able to report the correct
amount of wages because of a cafeteria plan or other complication. In these cases, we
recommend that the volunteer be asked to report by telephone and mail pay slips to the FO for a
two month test period. If the volunteer passes the test, they would be asked to stop mailing their
pay slips. If they fail the test, they would be asked to stop reporting by telephone.

APPENDIX A

WHO COULD BE RECRUITED FOR THE SSI TELEPHONE MONTHLY WAGE REPORTING PILOT
CONDUCTED IN CY 2006

Requirements for All Reporters:
1. Must speak English
2. Must be able to do simple arithmetic; i.e., add wage amounts
3. Must be able to understand and follow the reporting instructions
Additional Criteria for Participating in the Pilot:
1.
2.
3.
4.
5.
6.

Type of master record must not be a blind individual, couple or child.
The wage earner must have continuing wages.
The wage earner must not have 2 or more employers at the same time.
The wage earner must not be a Title II beneficiary.
If a MSSICS case, there is an IWAG screen for the current employer.
At the time you recruit the reporter, the record the wage earner is on must be in C01
payment status.
7. At the time you recruit the reporter, the record the wage earner is on must not have any of
the following:
a. Manual Deeming
b. Type T or D income
c. Centrally Stored Information (CSI) mismatch

Notes: These criteria apply to the second SSI telephone wage reporting pilot conducted in 2006.
Recipients with representative payees could not participate in the first pilot because
authentication rules did not permit either the recipient or the representative payee to be issued a
password. By using knowledge based authentication instead of password authentication,
representative payees were able to participate in the second pilot.
Recruitment criteria may be modified in the future. For example, people who speak Spanish
may be able to use the system if the resources become available to make this addition to the
automated system.

APPENDIX B
EXCLUSION CRITERIA USED IN CY 2006
BY THE SSI TELEPHONE MONTHLY WAGE REPORTING PILOT SYSTEM

Wage reports received by the SSI telephone wage reporting system in 2006 were not processed if
the record had any of the following exclusions:
1.
2.
3.
4.
5.
6.
7.
8.
9.

RIC T (Terminated record)
PSY M01, M02, N12, or N27 - N54
Manual Deeming Cases -- Type V income present in reporting month
IRWE or PASS in reporting month, or master file type of blind for wage earner
No wages on SSR for reporting month
Wage earner has continuing T2 income
Centrally Stored Information (CSI) Mismatch present in MSSICS
More than one employer in MSSICS for reporting month
No wages on MSSICS for reporting month

Note: These systems exclusions correspond to “Additional Criteria for Participating in the Pilot”
found in Appendix A. Exclusions 1 through 6 were checks against the SSR. Exclusions 7
through 9 were checks against MSSICS.
These systems exclusions may be modified in the future. If our recommendations are adopted,
then item 6, Wage earner has continuing T2 income, would be eliminated.

APPENDIX C

WHO PARTICIPATED IN THE SECOND PILOT

The Office of Income and Security Programs set up a database to track the number and type of
people who were recruited by field offices (FO) to participate in the second pilot. FO staff
entered information to this web based application. Although not perfect, the data allows useful
analysis.
Reporters
During the second pilot, 584 unique individuals were recruited from all 10 regions to report
wages by telephone During December 2005 and early January 2006. After the first reporting
period was over on January 9, it was determined that 56 of these individuals should not have
been recruited because they did not meet the criteria for the pilot. (See Appendix A for the
criteria.) A request was made to obtain replacements. Consequently, some recruiting took place
after the initial recruiting period.
Month Recruited
December 2005
January 2006
February 2006
Subsequent Months

Percent Volunteers
85%
10%
4%
1%

Representative payees were not allowed to report wages in the first pilot because of the method
of authentication. However, they were allowed to report wages in the second pilot. The
volunteer database shows approximately 18 percent of the reporters were representative payees
reporting wages for another person; i.e., an SSI recipient or deemor.
Wage Earners
Approximately one-third of the wage earners were SSI recipients and two-thirds were deemors.
Almost all the SSI recipients were disabled individuals. Only 3 were aged individuals. Most of
the deemors (80 percent) were parents of disabled children eligible for SSI. The remaining
deemors (20 percent) were ineligible spouses of adult SSI recipients.
Was The Sample Representative?
For two reasons steps were not taken to ensure that the individuals recruited were representative
of the SSI population in general or of that segment of the SSI population with wages. First, this
pilot had the objective of testing specific processes. For this purpose, it was not necessary for
the sample to be representative. Second, one of the processes being tested was the method of
recruiting volunteers. Therefore, this method could not be changed to ensure a representative
sample. Nevertheless, we did have at least five FOs from each region participate in the pilot to
ensure that the pilot was national in scope and there were enough FOs for a post pilot survey.

APPENDIX D

FY 2006 SSI Telephone Monthly Wage Reporting Pilot
Survey of All Participating Field Offices

Recruiting Volunteers for the Telephone Pilot
Thinking about how easy or difficult it was to recruit SSI recipients, deemors and representative
payees to volunteer for the telephone pilot in December and January, how would you rate your
experience recruiting them?
Recruiting
FO Count
Very Easy
1
Easy
8
Somewhat Easy
12
Somewhat Difficult
15
Difficult
11
Very Difficult
4

Pct
2.0%
15.7%
23.5%
29.4%
21.6%
7.8%

Instructing Volunteers How To Report Wages
Thinking about how easy or difficult it was to explain to the volunteers how to make telephone
wage reports, how would you rate your experience instructing them?
Instructing
FO Count
Very Easy
2
Easy
14
Somewhat Easy
15
Somewhat Difficult
13
Difficult
5
Very Difficult
2

Pct
3.9%
27.5%
29.4%
25.5%
9.8%
3.9%

Assisting Volunteers After They Started Using The Telephone
Did you have to answer questions or otherwise assist any of the volunteers after they started
reporting wages by telephone?
Assisting
Yes
No

FO Count
44
7

Pct
86.3%
13.7%

If Yes,
Approximately how many volunteers required assistance? 227
Approximately how many volunteers said the system did not recognize their name? 131
Approximately how many volunteers called your office to confirm that their successful
wage report was received by SSA? 103
In remarks, please explain the types of problems reported and whether the person told
you he or she would stop reporting by telephone.

Future Use of Telephone Wage Reporting
Do you think that telephone monthly wage reporting should be offered to SSI recipients,
deemors and representative payees as a reporting option?
Future Use
Yes
No

FO Count
30
21

Pct
58.8%
41.2%

Overall Satisfaction With The Pilot
Please indicate your overall satisfaction with the SSI Telephone Monthly Wage Reporting Pilot
conducted in FY 2006.
Overall Satisfaction FO Count
Very Good
5
Good
12
Fair
16
Poor
13
Very Poor
5

Pct
9.8%
23.5%
31.4%
25.5%
9.8%

APPENDIX E
FY 2006 SSI Telephone Monthly Wage Reporting Pilot
Survey of Field Offices That Participated in Both the 2003 and 2006 Pilots

Offices that participated in both the 2003 and 2006 telephone wage reporting pilots should
complete this questionnaire in addition to completing the questionnaire for all participating field
offices. For each question, please compare your experience in the FY 2006 telephone pilot to
your experience in the first telephone pilot during 2003.

Recruiting Volunteers for the Two Telephone Pilots
In the 2003 pilot, a mass mailing was sent out asking SSI recipients and deemors to contact the
FO if they wanted to volunteer for the telephone pilot. In 2006, you recruited people while they
were in the FO to conduct business; e.g., to complete a redetermination. Comparing the two
pilots, how would you rate the 2006 pilot?
Recruiting
Much Easier
Easier
About the Same
Harder
Much Harder

FO Count
5
13
14
2
0

Pct
14.7%
38.2%
41.2%
5.9%
0%

Instructing Volunteers How To Report Wages
In the 2003 pilot, you had to instruct people how to get a password as well as how to call and
report their wages. In 2006, no password was needed but reporters were told to enter their SSN,
name and date of birth before reporting wages. Comparing the two pilots, how would you
rate the 2006 pilot?
Instructing
Much Easier
Easier
About the Same
Harder
Much Harder

FO Count
11
12
9
2
0

Pct
32.3%
35.3%
26.5%
5.9%
0%

Assisting Volunteers After They Started Using the Telephone
In the 2003 pilot, you may have had to assist some volunteers get a password. In the 2006 pilot,
there were no passwords but you may have had to assist some volunteers with problems
involving the system recognizing a name or date or birth. In both pilots you may have had to
assist them with problems involving the amount of wages they reported.
Comparing the two pilots, how would you rate the 2006 pilot?
Assisting
Much Easier
Easier
About the Same
Harder
Much Harder

FO Count
2
10
22
0
0

Pct
5.9%
29.4%
64.7%
0%
0%

Amount of Time Devoted to Pilot Activities
Please consider all pilot activities; e.g., recruiting, instructing and assisting volunteers.
Comparing the two pilots, how would you rate the 2006 pilot?
Time
Much Less Time
Less Time
About the Same
More Time
Much More Time

FO Count
4
15
13
2
0

Pct
11.8%
44.1%
38.2%
5.9%
0%

Overall Comparison Rating
Please consider all pilot activities; e.g., recruiting, instructing and assisting volunteers.
Comparing the two pilots, how would you rate the 2006 pilot?
Overall Comparison
Much Easier
Easier
About the Same
Harder
Much Harder

FO Count
4
14
15
1
0

Pct
11.76%
41.18%
44.12%
2.94%
0%

APPENDIX F
TOP SUGGESTIONS AND REMARKS FROM THE SURVEY QUESTIONNAIRES
Suggestions:
1.
2.
3.
4.
5.

Have fewer exclusions to participation in the pilot.
Allow Spanish speakers to report wages by telephone.
Extend the reporting period beyond the Goldberg/Kelly cutoff date.
Use fewer voice prompts to make it easier.
Give FOs the ability to check the system as soon as calls come in.

Most Common Remarks:
1. The biggest problem was name recognition. More FOs commented about this than any
other issue. Many volunteers complained to the FO that they could not make a wage
report because the system did not recognize their name. In some cases the FO said the
individual had an accent or speech impediment.
2. Many volunteers complained about calls being dropped but neither the volunteer nor the
FO could explain why. (We can speculate that some had to do with name recognition
and some involved 3rd party reporters who tried to report before a separate path was
created for them.)
3. Many volunteers called the FO to verify that their wage report had been processed
correctly.
Reasons Why Volunteers Dropped Out of the Pilot:
1.
2.
3.
4.
5.
6.
7.

Difficulty understanding instructions
Confused following voice prompts
Too time consuming
Couldn’t figure out correct amount to report (earned versus paid, etc.)
Didn’t like telephone reporting
System dropped their call (name recognition, etc.)
System not available when called

APPENDIX G
CONTROL FOR FIRST SSI TELEPHONE MONTHLY WAGE REPORTING PILOT
After the evaluation for the first SSI telephone monthly wage reporting pilot was completed and
the report was issued, the question came up whether there was a control established to compare
monthly wage reporters to non-reporters. The answer was that it was not practical to design a
control into the pilot. Instead, our improper payment reduction findings were based on a
comparison of the telephone wage reports to the pre-pilot wage estimates for the pilot period.
One control option would have been to select a control group. This would have required us to
sample a similar but separate group of SSI recipients and deemors who were not making monthly
wage reports and verify their wages at the conclusion of the pilot. The limited resources
committed to the pilot did not permit this. Another option would have been to use a control time
period. This would have required us to ensure that the individuals we sampled had not
previously reported their wages to a field office on a monthly or periodic basis. This was not
feasible. The sample selection program could not make this distinction. Also, we had difficulty
recruiting a sufficient number of pilot participants without this additional restriction.
Nevertheless, the data for the first pilot was reexamined to determine if, by happenstance, a
control time period could be established. We identified 275 monthly reporters who received
wages in the four month pilot period, June through September 2003 and the pre-pilot period June
through September 2002. These four months were chosen because they were the peak reporting
period for the first pilot. Including additional months would have resulted in fewer monthly
reporters being available for comparison. We found the following:
Period Comparison
Control Period: June – September 2002
Pilot Period: June – September 2003
Decrease
Percent Decrease

Total Wage Overpayments
$16,879.19
$13,333.58
$3,545.61
21%

The results show there was 21 percent less wage overpayment for the pilot months as compared
to the pre-pilot months. Unfortunately, we had only 275 reporters for this comparison and the
difference is not statistically significant. Therefore, we cannot use this finding to estimate or
project that SSI telephone monthly wage reporting will reduce wage overpayments by 21 percent.
The second pilot conducted in 2006 was designed to test new procedures, not to retest the
accuracy or effectiveness of telephone monthly wage reports. Therefore, wages were not
verified and no estimates of improper payment reduction could be made with or without a
control.

File Type	application/pdf
File Title	Supplemental Security Income
Author	244251
File Modified	2007-02-27
File Created	2007-02-27