FY 2011 Internal Report Card Survey

DOCUMENTATION FOR THE GENERIC CLEARANCE

OF CUSTOMER SATISFACTION SURVEYS

T ITLE OF INFORMATION COLLECTION: Fiscal Year (FY) 2011 Internet Report Card (IRC) Survey

SSA SUB-NUMBER: D-02

DESCRIPTION OF ACTIVITY (give purpose of activity, provide specific information; i.e., date(s) of survey, number of focus groups, locations, etc.):

BACKGROUND:

The Social Security Administration (SSA) annually surveys individuals who do business with us to assess their satisfaction with the specific mode of contact they used. Currently, SSA uses the results of three ongoing surveys of 800 number callers, field office callers and office visitors to produce its key performance indicator, “percent of individuals doing business with SSA who rate overall service excellent, very good or good.” Because Internet transactions have grown into another important method for doing business with SSA, we recently added the satisfaction of Internet users into the calculation of our performance indicator.

In light of workload and budget constraints, SSA is implementing the IRC Survey on an incremental basis. In FY 2010, the IRC Survey sample included individuals who filed online for either retirement benefits, Title II disability benefits or a Medicare Part D Subsidy. In FY 2011, SSA expanded its online application to individuals applying for Medicare Part B only. Therefore, we are adding a Medicare Part B sample to the online applicant group in the upcoming survey.

In addition, in FY 2011 we will expand the survey to include individuals who used SSA’s online change of address or direct deposit service. In subsequent years we plan to add the other two categories of transactions SSA currently offers on its website: requests for information and disability report forms.

The IRC Survey assesses public perception of the transactional services provided on SSA’s website and collects information to help SSA understand the customer experience behind satisfaction ratings.

SURVEY:

Description of Survey

We conduct the IRC Survey by mail using a brief, scannable questionnaire. The survey covers aspects of online services which we know, based on SSA’s earlier surveys of these populations, can have an influence on satisfaction. There are two questionnaires, one for online applications and one for online change of address or direct deposit reports.

The following questions apply to both groups:

• How the person learned about SSA’s Internet transaction service;

• Whether the person had help to complete the Internet transaction and who helped them;

• Satisfaction with the ease of finding and using the Internet transaction service, including navigation, appearance, and explanations provided;

• Satisfaction with the ease of completing the Internet transaction, including answering questions, getting clarification and the length of time it took;

• Satisfaction with explanations about the Internet transaction process/benefits and about the website’s security and privacy;

• Overall opinion of SSA’s Internet transaction service; and

• Likelihood to recommend SSA’s online Internet transaction service to others and to use SSA’s website for future business.

The following questions apply only to individuals who used the online change of address or direct deposit service:

• Whether the person verified their identity by using a password or knowledge-based authentication; and

• Satisfaction with the ease of obtaining a password to use SSA’s change of address or direct deposit service.

We have included a copy of both questionnaires along with the survey correspondence (pre-notification postcard and cover letters) with this submission.

Statistical Information

Sample Selection

SSA maintains data files of all Internet transactions. We will use those files to select a random sample of individuals who complete an online application or use the online change of address/direct deposit service during a two-week period in October 2010.

The following table shows the estimated volume of online applications processed in a year based on SSA’s eService Usage Statistics. We will stratify the sample to select an equal number (2,000) of each group.

Online Application	Estimated Yearly Universe
Retirement Benefits	930,000
Disability Benefits	699,000
Medicare Part B	210,000
Medicare Part D Subsidy	482,000

The following table shows the estimated volume of online change reports that are processed in a year based on SSA’s eService Usage Statistics. We will stratify the sample to select an equal number (3,000) of each group.

Online Change Report	Estimated Yearly Universe
Change of Address	420,000
Direct Deposit	180,000

To produce a combined Internet satisfaction rating, we will weight the survey data to reflect the actual universes recorded for each Internet transaction during the sample period.

Methodology

For the sake of consistency, we plan to follow the same methodology we use for the other performance measure survey that is conducted by mail (i.e., the Office Visitor Survey). First, we will send a pre-notification postcard to all sampled individuals advising them to expect the survey questionnaire. Within three business days, we will mail the questionnaire with a cover letter explaining the purpose of the survey. And finally, five business days later, we will send a follow-up letter and another copy of the questionnaire to all sampled individuals.

Response Rate

SSA will take the following steps to maximize response rates:

• To minimize the time between the customers’ service experience and the survey, thereby ensuring salience, we will select the sample listings shortly after the individual completes either an online application or a request for a change of address/direct deposit.

• The contractor will mail a pre-notification postcard within two days after they receive the sample listing. The postcard informs participants that 1) they have been selected for the survey and 2) they should be on the lookout for the envelope containing the questionnaire. Using a postcard format allows the recipient to quickly see that the survey is sanctioned by SSA. Additionally, the postcard will identify the contractor who is conducting the survey for SSA, and should increase the likelihood that participants will open the envelope when they receive the questionnaire.

• We will mail the survey questionnaire within 3 business days after sending the pre-notification postcard. An agency official signs the survey cover letter and encourages the individual to respond by emphasizing the importance of the survey.

• Five business days after we send the first questionnaire, we will mail a follow-up letter and a duplicate questionnaire to all individuals in the sample. We will not send more than one follow-up because responses for service satisfaction surveys must be obtained shortly after the date of the contact (while respondents' perceptions are still fresh) to ensure their validity and utility.

• The questionnaire is short and has an eye‑catching title. The scannable questionnaire will be designed for ease of use by an aged and disabled population, e.g., font sizes will be larger, difficult fill-in bubbles will not be used.

Using the same methodology in the FY 2010 IRC Survey, SSA achieved a combined response rate of 46 percent. This response rate is the highest possible considering that time permits only a limited number of contacts. We expect to achieve an equally good response rate in the upcoming survey.

Sampling Variability

The key variable in the IRC Survey is overall satisfaction with the Internet transaction (i.e., percent giving a rating of excellent, very good or good). Our proposed sample sizes of 2,000 cases for online applications and 3,000 cases for change reports are large enough, assuming a 50 percent response rate, to provide a sampling variability at the 95‑percent confidence level equal to:

Stratum	Key Variable (E/VG/G Rating)*	Sampling Variability
Applications
Retirement	96%	+/- 1.4
Disability	86%	+/- 2.2
Medicare Part B	90%	+/- 1.9
Medicare Part D	91%	+/- 1.9
Change Reports
Change of address	96%	+/- 1.4
Direct deposit	96%	+/- 1.4
* The E/VG/G ratings for retirement, disability and Medicare Part D reflect the actual ratings obtained in the FY 2010 IRC Survey. Satisfaction for the other strata are estimated.

OQP will perform all sampling and data analysis. Dan Zabronsky, Director of SSA’s Division of Modeling, will provide statistical support, and is available for discussion of statistical-related issues. He can be reached at (410) 965-5953.

IF FOCUS GROUP MEMBERS WILL RECEIVE A PAYMENT, INDICATE AMOUNT (No more than $25 can be authorized under OMB rules):

We will not compensate participants for this survey.

USE OF SURVEY RESULTS:

SSA will use the results of this survey to evaluate and improve various aspects of its Internet services. In addition, the Agency will use the findings in combination with results from the 800 Number Caller, Field Office Caller, and Office Visitor surveys to compute the agency’s combined performance indicator of overall satisfaction.

BURDEN HOUR COMPUTATION (Number of responses (X) estimated response time (/60) = annual burden hours):

Number of Responses: 14,000

Estimated Response Time: 5 minutes

Annual Burden Hours: 1,167 hours

0960-0526 Generic Clearance

FY 2011 Internet Report Card Survey

6