FY 2010 Hearing Process Report Card Survey Description and Survey Correspondence

DOCUMENTATION FOR THE GENERIC CLEARANCE
OF CUSTOMER SATISFACTION SURVEYS
TITLE OF INFORMATION COLLECTION:

Fiscal Year (FY) 2010 Hearing Process
Report Card Survey

SSA SUB-NUMBER: D-01
DESCRIPTION OF ACTIVITY:
BACKGROUND
Social Security benefits applicants who do not agree with the Social Security Administration’s
(SSA) initial decision on their claim are legally entitled to appeal that decision. As part of the
appeals process, claimants can file a request for a hearing before an administrative law judge
(ALJ), who decides whether or not benefits should be awarded.
In FY 2008, we began conducting a satisfaction survey with two groups of applicants who
recently participated in an ALJ hearing. The first group consists of applicants who received a
favorable decision from the ALJ, and the second group includes applicants with denied appeals
for Social Security benefits. The annual survey assesses overall satisfaction with the hearing and
addresses various aspects of service that can have a bearing on satisfaction with the process.
FY 2010 is the third year in which SSA will conduct this survey.
SURVEY
Description of Survey
We developed two questionnaires: one for awarded and one for denied hearing applicants. The
two surveys are identical with the exception of one question related to notice clarity, which was
tailored to address the different outcomes experienced by these two groups.
The survey covers a variety of topics relating to the service SSA provides during the hearing
process, including the following:


The ease of finding information about the hearing request;



The quality of information explaining the process;



Ease of contacting SSA and the helpfulness of the information provided about the hearing
process;



Whether the person chose to have a representative handle their hearing;
0960-0526 Generic Clearance
FY 2010 Hearing Report Card Survey
2009
1



Whether the person had a hearing with a judge face-to-face or by video conference;



Satisfaction with the location of the office where the hearing was held;



Satisfaction with the length of the various stages of the hearing process: from date hearing
requested to date hearing held, from date hearing held to date decision received, and the
overall period from date hearing requested to date decision received;



Satisfaction with the judge’s performance, including clarity of explanations, preparedness to
discuss the case, courtesy, and time spent with the claimant;



Satisfaction with notice clarity, including the explanation of the judge’s decision, the
explanation of the amount of benefits (for those awarded benefits), and the explanation of
how to appeal the judge’s decision (for those denied benefits);



Satisfaction with SSA staff including their helpfulness, courtesy, and job knowledge;



Overall ratings of the hearing experience and SSA’s service.

We have attached a copy of the questionnaire and the survey correspondence to this submission.
The majority of the questions remain the same as those used in the prior surveys. Based on our
analysis of the survey data, however, we decided to improve the way we ask about the length of
the hearing process.


The prior questionnaire contained two non-sequential questions addressing satisfaction with
the length of the hearing process. Question 7 rated the elapsed time from the date the
hearing was requested to the date the hearing was held. Question 13 rated the overall time
from the date the hearing was requested to the date the decision was received. However, the
survey did not ask participants to rate how long it took to receive the decision after the
hearing was held.



To make it easier for responders to consider the waiting time issue, we created a separate
section in the FY 2010 survey titled “Waiting for the hearing and decision.” The new
section includes the original questions 7 and 13 (now numbered as question 12 and 14), as
well as a new question rating the time from the date the hearing was held to the date the
decision was received.

0960-0526 Generic Clearance
FY 2010 Hearing Report Card Survey
2009
2

Statistical Information

Sample Selection
In FY 2010, SSA expects to issue about 608,000 hearing decisions on disability claims,
consisting of approximately 72 percent favorable (award) and 28 percent unfavorable (denial)
decisions. Because of the relatively small size of these universes, we select decisions at three
intervals over the fiscal year (October 2009, January 2010 and April 2010) in order to stratify the
sample by outcome (award/denial) and geographic area. We select 500 awarded and 500 denied
cases from each of SSA’s ten regions in each phase of the survey for an overall regional sample
size of 1,500 awards and 1,500 denials. This method yields 5,000 awarded and 5,000 denied
applicants at each interval for a combined total of 15,000 awards and 15,000 denials.
Regional results are weighted to arrive at a national overall satisfaction rating that represents the
universe of hearings decisions processed by SSA in FY 2010. The following table displays the
anticipated volumes of hearing decisions for each of SSA’s ten regions.

Region
Boston
New York
Philadelphia
Atlanta
Chicago
Dallas
Kansas City
Denver
San Francisco
Seattle
Nation

Estimated Yearly
Hearing Awards
17,000
39,000
52,000
121,000
66,000
56,000
21,000
14,000
41,000
15,000
442,000

Estimated Yearly
Hearing Denials
6,000
14,000
19,000
40,000
26,000
27,000
8,000
4,000
17,000
5,000
166,000

Methodology
An SSA-approved contractor conducts the survey by mail. The contractor mails all sampled
individuals a brief scannable questionnaire accompanied by a cover letter that explains the
purpose of the survey and encourages individuals to participate. We include both the Paperwork
Reduction Act and the Privacy Act statements in the initial cover letter. About two weeks later,
the contractor sends a follow-up postcard to every sampled individual. Two weeks after the
mailing of the follow-up postcard, we send a duplicate survey package (questionnaire and
follow-up cover letter) to those sampled individuals who have not responded at that point.

0960-0526 Generic Clearance
FY 2010 Hearing Report Card Survey
2009
3

Response Rate
SSA takes the following steps to maximize the response rate for this survey:


The questionnaire is short and has an eye-catching title. The scannable version is designed
for ease of use by a disabled population, e.g., font sizes are larger, difficult fill-in bubbles
are not used;



The first survey package includes a cover letter signed by an Agency official that encourages
the individual to respond by emphasizing the importance of the survey;



We make two follow-up contacts with participants: a follow-up postcard and a follow-up
survey package;



We send a Spanish version of the survey form to those participants identified in the sample
selection process as having a preference for doing business in Spanish;



We provide SSA’s toll-free national 800 number on all letters so participants can call if they
have any questions.

Using the same methodology in the FY 2008 survey, SSA achieved a combined response rate of
50 percent at the national level. Among the regions the combined response rate ranged from
46 percent to 53 percent. Considering our prior experience, we anticipate that a response rate of
at least 50 percent is also attainable in the FY 2010 survey. (Note we are referencing the
FY 2008 survey results because data from the FY 2009 survey is still being validated. As part of
its data validation, SSA conducts a nonresponder analysis to evaluate any potential impact of the
response rate on survey findings.)
Sampling Variability
The key variable for this survey is the percent of hearing applicants who are satisfied with SSA
service overall (i.e., those who rate service excellent, very good, or good (E/VG/G)). In
FY 2008, the overall rating was 74 percent E/VG/G for awarded applicants and 34 percent
E/VG/G for denied applicants. Nationally, our proposed sample size of 10,000 cases in each
phase is large enough, assuming a 50 percent response rate, to provide a sampling variability at
the 95-percent confidence level equal to:
Stratum
Awards
Denials

National Key Variable (E/VG/G Rating)
74%
34%

Sampling Variability
+/- 1.7%
+/- 1.9%

The table below shows the range of the E/VG/G overall service ratings among the regions in the
FY 2008 survey. Based on an anticipated response rate of 50 percent from the regional samples
0960-0526 Generic Clearance
FY 2010 Hearing Report Card Survey
2009
4

of 1,500 awarded and 1,500 denied cases, we estimate that the sampling variability will fall in
the ranges shown. These ranges are acceptable given the intended purpose of the survey.
Stratum
Awards
Denials

Regional Key Variable (E/VG/G Rating)
67% to 80%
27% to 40%

Sampling Variability
+/- 3.5% to +/- 3.0%
+/- 3.5% to +/- 3.6 %

SSA’s Office of Quality Performance (OQP) will perform all sampling and data analysis.
Statistical support will be provided by Daniel Zabronsky, Director of SSA’s Division of
Modeling in OQP. He can be reached at (410) 965-5953.
IF FOCUS GROUP MEMBERS WILL RECEIVE A PAYMENT, INDICATE AMOUNT:
N/A
USE OF SURVEY RESULTS:
We first conducted the Hearings Process Report Card Survey in FY 2008 to establish a baseline
measure of customer satisfaction with the hearing process. We plan to continue it on an annual
basis to track any changes in perceptions that may result from agency initiatives to reduce the
hearing backlog and improve efficiency.
BURDEN HOUR COMPUTATION (Number of responses (X) estimated response time (/60) =
annual burden hours):
Number of Responses:
Estimated Response Time:
Annual Burden Hours:

30,000
5 minutes
2,500 hours

NAME OF CONTACT PERSON: Faye I. Lipsky
TELEPHONE NUMBER: 410-965-8783

0960-0526 Generic Clearance
FY 2010 Hearing Report Card Survey
2009
5