FY 2010 Internet Report Card Survey

DOCUMENTATION FOR THE GENERIC CLEARANCE
OF CUSTOMER SATISFACTION SURVEYS
TITLE OF INFORMATION COLLECTION: Fiscal Year (FY) 2010 Internet Report Card
Survey
SSA SUB-NUMBER: D-01
DESCRIPTION OF ACTIVITY (give purpose of activity, provide specific information; i.e.,
date(s) of survey, number of focus groups, locations, etc.):
BACKGROUND:
The Social Security Administration (SSA) annually surveys individuals who do business with us
to assess their satisfaction with the specific mode of contact they used. Because Internet
transactions have grown into an important method for doing business with SSA, we want to add
the satisfaction of Internet users into the calculation of our performance indicator. As part of this
effort, we will conduct the first Internet Report Card Survey to assess the satisfaction of the
people who use our online transactional services.
We have designed the survey, which we will conduct on an incremental basis, to 1) assess public
perception of the Internet transaction services provided by SSA and 2) collect information that
will help SSA understand the customer experience behind satisfaction ratings. The Internet
Report Card Survey will become one of four annual surveys (along with the 800 Number Caller,
Field Office Caller and Office Visitor Surveys) that SSA uses to produce its performance
indicator, “percent of individuals doing business with SSA who rate overall service excellent,
very good or good.”
The FY 2010 Internet Report Card Survey will include individuals who completed SSA’s online
applications for one of the following 1) retirement benefits, 2) Title II disability benefits, or 3)
Medicare Part D Subsidy. Online applications constitute the largest and most significant
category of SSA’s Internet transaction workload which is why we selected these applicants to
assess. In subsequent years we plan to expand the Internet Report Card Survey to include other
categories of transactions.
SURVEY:
Description of Survey
The Internet Report Card Survey uses a brief, scannable questionnaire that SSA mails out to
selected respondents. When SSA receives the completed surveys, we forward them to our
contractors, who scan them and prepare a data file of survey results. SSA then computes data
tabulations and conducts an analysis of the survey findings.

0960-0526 Generic Clearance
FY 2010 Internet Report Card Survey

1

To ensure a higher response rate, SSA will send a Pre-Notice postcard letting participants know
they will soon be receiving a survey. Some time after we send the survey, we will mail out a
Follow-Up postcard to remind participants about the survey and encourage them to complete it.
The survey covers aspects of online services that can have an influence on satisfaction. Specific
questions include the following:


How the person learned about SSA’s online applications;



Whether the person had help completing the application and who helped them;



Satisfaction with the ease of finding and using the online application, including navigation,
appearance, and explanations provided;



Satisfaction with the ease of completing the online application, including answering
questions, getting clarification, and the length of time it took;



Satisfaction with explanations about the application process/benefits and the website’s
security and privacy;



Overall opinion of SSA’s online application;



Likelihood to recommend SSA’s online application to others and to use SSA’s website for
future business.

We have attached a copy of the questionnaire and the accompanying survey correspondence to
this submission.
Statistical Information
Sample Selection
SSA maintains data files of all Internet transactions. We will use those files to select a random
sample of individuals who complete an online application during a two-week period in October.
We will stratify the sample to select an equal number (2,000 each) of retirement benefit, Title II
disability benefit, and Medicare Part D subsidy applications.

0960-0526 Generic Clearance
FY 2010 Internet Report Card Survey

2

The following table shows the online applications that SSA processed in FY 2008 based on
SSA’s e-Service Usage Statistics. To produce a combined Internet satisfaction rating, we will
weight the survey data to reflect the actual universes recorded for each online application during
the sample period.
Online Application
Retirement Benefits
Disability Benefits
Medicare Part D Subsidy

Yearly Universe
450,000
260,000
280,000

Methodology
For the sake of consistency, we plan to follow the same methodology we use for the Office
Visitor Survey. First, we will send a Pre-Notice postcard to all sampled individuals advising
them to expect the survey questionnaire. Within three business days, we will mail the
questionnaire with a cover letter explaining the purpose of the survey. Five business days later,
we will send a follow-up letter and another copy of the questionnaire to all sampled individuals.
Response Rate
SSA will take the following steps to maximize response rates:


To minimize the time between the customers’ service experience and the survey, we will
select the sample listing shortly after the individual completes the online application.



We will mail out a Pre-Notice postcard within a few days after sample selection to inform
participants that 1) we have selected them for the survey and 2) they should be on the
lookout for the envelope containing the questionnaire. Using a postcard format allows the
recipient to quickly see that the survey is sanctioned by SSA. Additionally, the postcard will
identify the contractor who is conducting the survey for SSA, and should increase the
likelihood that participants will open the envelope when they receive the questionnaire.



We will mail out the actual survey questionnaire within three business days after sending the
Pre-Notice postcard. An SSA official signs an enclosed cover letter, which encourages the
individual to respond by emphasizing the importance of the survey and the confidentiality of
their response.



Five business days after the first mailing, we will mail a follow-up letter and a duplicate
questionnaire to all individuals in the sample. We will not send more than one follow-up
because responses for service satisfaction surveys must be obtained shortly after the date of
the contact (while respondents' perceptions are still fresh) to ensure their validity and utility.



The questionnaire is short and has an eye-catching title. We have designed the scannable
questionnaire for ease of use by an aged and disabled population; e.g., font sizes will be
larger, difficult fill-in bubbles are not used.
0960-0526 Generic Clearance
FY 2010 Internet Report Card Survey

3

Using methods similar to these, we have achieved a consistent response rate of approximately 50
percent with past surveys (e.g., Office Visitor Survey). There response rates are the highest
possible considering that time permits only a limited number of contacts. We expect to achieve
an equally good response rate in this survey.
Sampling Variability
The key variable in the Internet Report Card Survey is overall satisfaction with the Internet
transaction (i.e., percent giving a rating of excellent, very good or good). Based on earlier
surveys conducted by SSA with these same populations, we estimate the overall satisfaction
rating will be 98 percent for retirement applicants, and 88 percent for both Title II disability and
Medicare Part D subsidy applicants.
The sample size of 2,000 for each online application is large enough, assuming a 50 percent
response rate, to provide a sampling variability at the 95-percent confidence level of
+/-1.4 percent for retirement applications and +/-1.9 percent for disability and Medicare Part D
subsidy applications.
The Office of Quality Performance (OQP) will perform all sampling and data analysis. Dan
Zabronsky, Director of SSA’s Division of Modeling in OQP, provides statistical support and is
available for discussion of statistical-related issues.
IF FOCUS GROUP MEMBERS WILL RECEIVE A PAYMENT, INDICATE AMOUNT
We will not compensate participants for this survey.
USE OF SURVEY RESULTS:
SSA will use the results of this survey to evaluate and improve various aspects of its Internet
services. In addition, we will use the study results in combination with results from the
800 Number Caller, Field Office Caller, and Office Visitor surveys to report a combined
performance indicator of customer satisfaction for the agency each year.
BURDEN HOUR COMPUTATION (Number of responses (X) estimated response time (/60) =
annual burden hours):
Number of Responses: 6,000
Estimated Response Time: 5 minutes
Annual Burden Hours: 500 hours
NAME OF CONTACT PERSON: Deborah A. Larwood
TELEPHONE NUMBER:

410-966-6135

0960-0526 Generic Clearance
FY 2010 Internet Report Card Survey

4