Description - Internet Report Card Survey FY 12

DOCUMENTATION FOR THE GENERIC CLEARANCE

OF CUSTOMER SATISFACTION SURVEYS

T ITLE OF INFORMATION COLLECTION: Fiscal Year (FY) 2012 Internet Report Card (IRC) Survey

SSA SUB-NUMBER: D-02

DESCRIPTION OF ACTIVITY:

BACKGROUND:

The Social Security Administration (SSA) annually surveys individuals who do business with us to assess their satisfaction with the specific mode of contact they used. As part of this effort, we conduct the Internet Report Card Survey, in which we survey individuals who completed their business on SSA’s website.

In light of workload and budget constraints, SSA is implementing the IRC Survey on an incremental basis. In FY 2010 and 11, the IRC Survey sample included individuals who filed an online application for retirement benefits, Title II disability benefits, Medicare Part B or a Medicare Part D Subsidy. The FY 2011 sample also included individuals who used SSA’s online change of address or direct deposit service.

In FY 2012, we will expand the survey to include individuals who completed one of three online Disability Reports: (OMB No. 0960-0144) i3368 for adult disability applications; (OMB No. 0960-0577) i3820 for childhood disability applications; and (OMB No. 0960-0441) i3441 for an appeal of a previously denied disability benefit application.

The IRC Survey assesses public perception of the transactional services provided on SSA’s website and collects information to help SSA understand the customer experience behind satisfaction ratings.

SURVEY:

Description of Survey

We conduct the IRC Survey by mail using a brief, scannable questionnaire. The survey covers aspects of online services which we know, based on SSA’s earlier surveys of these populations, can have an influence on satisfaction. We use separate questionnaires for each category of Internet transactions.

The following questions apply to all categories:

• How the person learned about SSA’s Internet transaction service;

• Whether the person had help completing the Internet transaction and from whom;

• Satisfaction with the ease of finding and using the Internet transaction service, including navigation, appearance, and explanations provided;

• Satisfaction with the ease of completing the Internet transaction, including answering questions, getting clarification and the length of time it took;

• Satisfaction with explanations about the Internet transaction process/benefits and about the website’s security and privacy;

• Overall opinion of SSA’s Internet transaction service;

• A person’s likelihood to recommend SSA’s online Internet transaction service to others and to use SSA’s website for future business; and,

• Whether a person needed special accommodations because of a medical condition, and the level of satisfaction on how SSA met the person’s needs.

The following questions apply only to individuals who used the online change of address or direct deposit service:

• Whether participants verified their identity by using a password or knowledge-based authentication; and,

• Satisfaction with the ease of obtaining a password to use SSA’s change of address or direct deposit service.

The following questions apply only to individuals who completed an online Disability Report:

• Whether participants completed the online Disability Report in more than one sitting and how easy it was to get back to the report they worked on earlier.

Statistical Information

Sample Selection

SSA maintains data files of all Internet transactions. We will use those files to select a random sample of individuals who complete an online transaction or use the online change of address/direct deposit service during a 2‑week period in November 2011. We will select the sample for the online Disability Reports from a 3‑week period in January-February 2012.

The following table shows the estimated volume of online applications SSA processes in a year based on our eService Usage Statistics. We will stratify the sample to select an equal number (2,000) of each group.

Stratum	Estimated Yearly Universe	Sample Size
Online Application
Retirement Benefits	1,000,000	2,000
Medicare Part B	250,000	2,000
Disability Benefits	940,000	3,000
Medicare Part D Subsidy	380,000	3,000

The following table shows the estimated volume of online change reports SSA processes in a year based on our eService Usage Statistics. We will stratify the sample to select an equal number (3,000) of each group.

Online Change Report
Change of Address	438,000	2,000
Direct Deposit	188,000	2,000

The following table shows the estimated volume of online disability reports that SSA processes in a year based on our eService Usage Statistics. We will stratify the sample to select an equal number (2,666) of each group.

Online Disability Report
i3368	785,000	2,666
i3820	115,000	2,666
i3441	715,000	2,666

To produce a combined Internet satisfaction rating, we weight the survey data to reflect the actual universes recorded for each targeted Internet transaction during the sample period.

Methodology

For the sake of consistency, we plan to follow the same methodology we use for the other performance measure survey we conduct by mail (i.e., the Office Visitor Survey). First, we will send a pre-notification postcard to all sampled individuals advising them to expect the survey questionnaire. Within three business days, we will mail the questionnaire with a cover letter explaining the purpose of the survey. Finally, five business days later, we send a follow-up letter and another copy of the questionnaire to all sampled individuals.

Response Rate

SSA will take the following steps to maximize response rates:

• To minimize the time between customers’ service experience and the survey, thereby ensuring salience, we will select the samples shortly after individuals complete the online transaction.

• The SSA-approved contractor will mail a pre-notification postcard to all sampled individuals. The postcard informs them that 1) they have been selected for the survey and 2) they should be on the lookout for the envelope containing the questionnaire. Using a postcard format allows individuals to quickly see that SSA sanctions the survey. Additionally, the postcard identifies the contractor who is conducting the survey for SSA, and should increase the likelihood that individuals will open the envelope when they receive the questionnaire.

• We will mail the survey questionnaire within 3 business days after sending the pre-notification postcard. An SSA official signs the survey cover letter and encourages individuals to respond by emphasizing the importance of the survey.

• Five business days after we send the first questionnaire, we will mail a follow-up letter and a duplicate questionnaire to all individuals in the sample. We will not send more than one follow-up because responses for service satisfaction surveys must be obtained shortly after the date of the contact (while individuals’ perceptions are still fresh) to ensure their validity and utility.

• The questionnaire is short and has an eye‑catching title. The scannable questionnaire is designed for ease of use by an aged and disabled population, e.g., font sizes are large, difficult fill-in bubbles are not used.

Using the same methodology in the FY 2011 IRC Survey, SSA achieved a combined response rate of 53 percent in the online application category and 56 percent in the online change report category. These response rates are the highest possible considering that time permits only a limited number of contacts. We expect to achieve equally good response rates in the upcoming survey.

Sampling Variability

The key variable in the IRC Survey is overall satisfaction with the Internet transaction (i.e., percent giving a rating of excellent, very good or good). Assuming a 50 percent response rate, our proposed sample sizes are large enough to provide a sampling variability at the 95‑percent confidence level equal to:

Stratum	Key Variable (E/VG/G Rating)*	Sampling Variability
Online Application
Retirement	95%	+/- 1.4
Medicare Part B	92%	+/- 1.9
Disability	88%	+/- 1.6
Medicare Part D	91%	+/- 1.6
Online Change Report
Change of address	95%	+/- 1.4
Direct deposit	98%	+/- 1.4
Online Disability Report
i3368	80%	+/ 2.3
i3820	80%	+/ 2.3
i3441	70%	+/ 2.6
* The E/VG/G ratings for online applications and change reports reflect the actual ratings obtained in the FY 2011 IRC Survey. Satisfaction for the online Disability Report strata are estimated.

OQP will perform all sampling and data analysis. Dan Zabronsky, Director of SSA’s Division of Modeling, will provide statistical support, and is available for discussion of statistical-related issues. He can be reached at (410) 965-5953.

IF FOCUS GROUP MEMBERS WILL RECEIVE A PAYMENT, INDICATE AMOUNT:

We do not compensate participants for this survey.

USE OF SURVEY RESULTS:

SSA uses the results of this survey to evaluate and improve various aspects of its Internet services. In addition, SSA uses the findings in combination with results from the 800 Number Caller, Field Office Caller, and Office Visitor surveys to compute the agency’s combined performance indicator of overall satisfaction.

BURDEN HOUR COMPUTATION:

Number of Responses: 22,000.

Estimated Response Time: 5 minutes.

Annual Burden Hours: 1,833 hours.

NAME OF CONTACT PERSON: Deborah A. Larwood

TELEPHONE NUMBER: 410-966-6135

Generic Clearance FY 2012 IRC Survey

July 2011