Description Internet Report Card FY 2013

DOCUMENTATION FOR THE GENERIC CLEARANCE

OF CUSTOMER SATISFACTION SURVEYS

TITLE OF INFORMATION COLLECTION: Fiscal Year (FY) 2013 Internet Report Card (IRC) Survey

SSA SUB-NUMBER: D-02

DESCRIPTION OF ACTIVITY:

BACKGROUND:

The Social Security Administration (SSA) annually surveys individuals who do business with us to assess their satisfaction with the specific mode of contact they used. As part of this effort, we conduct the IRC Survey, in which we survey individuals who completed their business on SSA’s website.

In light of workload and budget constraints, SSA implemented the IRC Survey on an incremental basis. In FY 2012, the IRC Survey sample included three categories of Internet transactions:

• online applications (iClaim for retirement benefits, title II disability benefits, Medicare Part B or i1020 for a Medicare Part D Subsidy);

• online change (address or direct deposit) reports; and,

• online Disability Reports (i3368 for adults and i3820 for childhood disability applications, i3441 for an appeal of a previously denied disability benefit application).

In FY 2013, we will expand the survey to include a fourth category of transactions: online requests for information (proof of income letter, a replacement SSA-1099 and a replacement Medicare card).

The IRC Survey assesses public perception of the transactional services provided on SSA’s website and collects information to help SSA understand the customer experience behind satisfaction ratings.

SURVEY:

Description of Survey

We conduct the IRC Survey by mail using a brief, scannable questionnaire. The survey covers aspects of online services that we know can have an influence on satisfaction. Although we tailored the survey content for each category of Internet transactions, all questionnaires contain the same core satisfaction questions.

The following questions apply to all categories:

• How the person learned about SSA’s Internet transaction service;

• Whether the person had help to complete the Internet transaction and from whom;

• Satisfaction with the ease of finding and using the Internet transaction service, including navigation, appearance, and explanations provided;

• Satisfaction with the ease of completing the Internet transaction, including answering questions, getting clarification and the length of time it took;

• Satisfaction with explanations about the Internet transaction process/benefits and about the website’s security and privacy;

• Overall opinion of SSA’s Internet transaction service;

• A person’s likelihood to recommend SSA’s online Internet service to others, to use SSA’s website for future business, and to access the website using a wireless handheld device; and,

• Whether the person needs special accommodations because of a medical condition, and satisfaction with how well SSA met the person’s needs.

The following questions apply to two categories of transactions - online change (address, direct deposit) reports and online requests for information:

• Whether the person created a new online account (username and password) to take care of business on SSA’s website or whether the person used an online account created earlier; and,

• Satisfaction with the ease of creating the online account.

The following question applies to one type of transaction - online requests for information:

• Satisfaction with how long it took to receive the information in the mail.

The following questions apply to one type of transaction - online Disability Reports:

• Whether the person completed the online Disability Report in more than one sitting and the ease of returning to the report started earlier.

Statistical Information

Sample Selection

SSA maintains data files of all Internet transactions. We use those files to select a random sample of individuals who completed one of the targeted online transactions. We will select the samples of online applications and online Disability Reports from a 2‑week period in October 2012. We will select the sample for the online change (address, direct deposit) reports and the online information requests from a 3‑week period in February 2013.

The following table shows the estimated volume of transactions processed in a year based on SSA’s eService Usage Statistics and the sample sizes for each targeted transaction. We will select a random sample containing 2,000 records for 7 of the 12 transactions; we will identify a larger number of records for the remaining 5 transactions (3,000 – 3,333) because our experience has shown that these populations are less inclined to participate. The larger sample sizes will yield more robust data for these transactions.

Transaction	Estimated Yearly Universe	Sample Size
Online Application
Retirement Benefits	1,000,000	2,000
Medicare Part B	290,000	2,000
Disability Benefits	1,000,000	3,000
Medicare Part D Subsidy	340,000	3,000
Online Disability Report
i3368	815,000	3,333
i3820	115,000	3,333
i3441	735,000	3,333
Online Change Report
Change of Address	430,000	2,000
Direct Deposit	190,000	2,000
Online Information Requests
Proof of Income Letter	780,000	2,000
Replacement SSA-1099	180,000	2,000
Replacement Medicare Card	165,000	2,000

To produce a combined Internet satisfaction rating, we weight the survey data to reflect the actual universes recorded for each targeted Internet transaction during the sample period.

Methodology

SSA sends the sample files to a private contractor who conducts the survey by mail. For the sake of consistency, we follow the same methodology we use for the other performance measure survey we conduct by mail (i.e., the Office Visitor Survey). First, we will send a pre-notification postcard to all sampled individuals advising them to expect the survey questionnaire. Within three business days, we mail the questionnaire with a cover letter explaining the purpose of the survey. We include the Paperwork Reduction Act and the Privacy Act statements in the cover letter. Five business days later, we send a follow-up letter and another copy of the questionnaire to all sampled individuals.

Response Rate

SSA takes the following steps to maximize response rates:

• To minimize the time between customers’ service experience and the survey, thereby ensuring salience, we select the samples shortly after individuals complete the online transaction.

• We mail a pre-notification postcard to all sampled individuals. The postcard informs them of 1) their selection to participate in the survey and 2) they should be on the lookout for the envelope containing the questionnaire. Using a postcard format allows the sampled individuals to quickly see SSA sanctions the survey. Additionally, the postcard identifies the contractor conducting the survey for SSA, and should increase the likelihood that sampled individuals will open the envelope when they receive the questionnaire.

• Within 3- 4 days after sending the pre-notification postcard, we mail the survey questionnaire along with a cover letter encouraging sampled individuals to respond by emphasizing the importance of the survey.

• Five business days after we send the first questionnaire, we mail a follow-up letter and a duplicate questionnaire to all individuals in the sample. We do not send more than one follow-up because responses for service satisfaction surveys must be obtained shortly after the date of the contact (while the person’s perceptions are still fresh) to ensure their validity and utility.

• The questionnaire is short (one page - front and back), has an eye-catching title, and is easy to read and complete. The scannable questionnaire is designed for ease of use by an aged and disabled population, e.g., font sizes are large, difficult fill-in bubbles are not used.

• We provide SSA’s toll-free national 800 number so sampled individuals can call if they have any questions.

Using the same methodology described above in FY 2012, SSA achieved a combined response rate of 51 percent in the online change (address, direct deposit) report category, 42 percent in the online application category, and 30 percent in the online Disability Report category. Response rates are generally highest when sampled individuals have a significant business connection with SSA (e.g., currently receiving benefits) and lowest where the nature of the business reflects a negative interaction (e.g., filling out a Disability Report to appeal an unfavorable decision on an application for disability benefits).

The response rates we achieved in FY 2011 are the highest possible considering that time permits only a limited number of contacts. We expect to achieve equally good response rates in the upcoming survey. Note that SSA routinely conducts a non-responder analysis to identify any significant differences in the responder and non-responder populations and their potential impact on the survey results.

Sampling Variability

The key variable in the IRC Survey is overall satisfaction with the Internet transaction. We define satisfaction as a combined rating of excellent, very good, or good (E/VG/G). Assuming a 50 percent response rate, our proposed sample sizes are large enough to provide a sampling variability at the 95‑percent confidence level equal to:

Transactions	Key Variable (E/VG/G Rating)*	Sampling Variability
Online Application
Retirement	95%	+/- 1.4
Medicare Part B	92%	+/- 1.9
Disability	88%	+/- 1.6
Medicare Part D	91%	+/- 1.6
Online Disability Report
i3368	80%	+/ 2.0
i3820	80%	+/ 2.0
i3441	70%	+/ 2.2
Online Change Report
Change of address	95%	+/- 1.4
Direct deposit	98%	+/- 1.4
Online Information Requests
Proof of Income Letter	90%	+/- 1.9
Replacement SSA-1099	90%	+/- 1.9
Replacement Medicare Card	90%	+/- 1.9
* The E/VG/G ratings for two transactions - online applications and online change reports - reflect the actual ratings obtained in the FY 2011 survey. We estimated the satisfaction rates for the online Disability Report transactions (added in FY 2012) because we are still validating the data from the FY 2012 survey. We also estimated the E/VG/G ratings for the online information requests (new for FY 2013).

OQP performs all sampling and data analysis. Dan Zabronsky, Director of SSA’s Division of Modeling, provides statistical support, and is available for discussion of statistical-related issues. He can be reached at (410) 965-5953.

IF FOCUS GROUP MEMBERS WILL RECEIVE A PAYMENT, INDICATE AMOUNT:

We do not compensate participants for this survey.

USE OF SURVEY RESULTS:

SSA uses the results of this survey to evaluate and improve various aspects of its Internet services. In addition, the Agency uses the findings in combination with results from the 800 Number Caller, Field Office Caller, and Office Visitor surveys to compute the agency’s combined performance indicator of overall satisfaction.

BURDEN HOUR COMPUTATION:

Number of Responses: 30,000

Estimated Response Time: 5 minutes

Annual Burden Hours: 2,500 hours

NAME OF CONTACT PERSON: Deborah A. Larwood

TELEPHONE NUMBER: 410-966-6135

Generic Clearance FY 2013 IRC Survey

June 2012