Description FY 2014 Internet Report Card Survey

DOCUMENTATION FOR THE GENERIC CLEARANCE

OF CUSTOMER SATISFACTION SURVEYS

TITLE OF INFORMATION COLLECTION: Fiscal Year (FY) 2014 Internet Report Card (IRC) Survey

SSA SUB-NUMBER: D-02

DESCRIPTION OF ACTIVITY:

BACKGROUND:

The Social Security Administration (SSA) annually surveys individuals who do business with us to assess their satisfaction with the specific mode of contact they used. As part of this effort, we conduct the IRC Survey, in which we survey individuals who completed their business on SSA’s website.

The IRC Survey sample includes four categories of Internet transactions:

• online applications (iClaim for retirement and Medicare Part B benefits, or i1020 for a Medicare Part D Subsidy);

• online change (address or direct deposit) reports;

• online Disability Reports (combined iClaim/i3368 for disabled adult applicants, i3820 for childhood disability applications, and i3441 for an appeal of a previously denied disability benefit application); and,

• online requests for information (proof of income letter, a replacement SSA-1099 and a replacement Medicare card).

The IRC Survey assesses public perception of the transactional services provided on SSA’s website and collects information to help SSA understand the customer experience behind satisfaction ratings.

SURVEY:

Description of Survey

We conduct the IRC Survey by mail using a brief, scannable questionnaire. The survey covers aspects of online services that we know can have an influence on satisfaction. Although we tailored the survey content for each category of Internet transactions, all questionnaires contain the same core satisfaction questions.

The following questions apply to all categories:

• How the person learned about SSA’s Internet transaction service;

• Whether the person had help to complete the Internet transaction and from whom;

• Satisfaction with the ease of finding and using the Internet transaction service, including navigation, appearance, and explanations provided;

• Satisfaction with the ease of completing the Internet transaction, including answering questions, getting clarification and the length of time it took;

• Satisfaction with explanations about the Internet transaction process/benefits and about the website’s security and privacy;

• Overall opinion of SSA’s Internet transaction service;

• A person’s likelihood to recommend SSA’s online Internet service to others, to use SSA’s website for future business, and to access the website using a wireless handheld device; and,

• Whether the person needs special accommodations because of a medical condition, and satisfaction with how well SSA met the person’s needs.

The following questions apply to two categories of transactions - online change (address, direct deposit) reports and online requests for information:

• Whether the person created a new online account (username and password) to take care of business on SSA’s website or whether the person used an online account created earlier; and,

• Satisfaction with the ease of creating the online account.

The following question applies to one type of transaction - online requests for information:

• Satisfaction with how long it took to receive the information in the mail.

The following questions apply to one type of transaction - online Disability Reports:

• Whether the person completed the online Disability Report in more than one sitting and the ease of returning to the report started earlier.

Statistical Information

Sample Selection

SSA maintains data files of all Internet transactions. We use those files to select a random sample of individuals who completed one of the targeted online transactions. We will select the samples of online applications and online change (address, direct deposit) reports from a 2‑week period in October 2013. We will select the sample for the online Disability Reports and the online information requests from a 2‑week period in February 2014.

The following table shows the estimated volume of transactions processed in a year based on SSA’s eService Usage Statistics and the sample sizes for each targeted transaction. We will vary the sample sizes for each transaction. We will identify a larger number of records for the transactions where our experience has shown that the populations are less inclined to participate. The larger sample sizes will yield more robust data for these transactions.

Transaction	Estimated Yearly Universe	Sample Size
Online Application
Retirement Benefits	1,300,000	1,800
Medicare Part B	500,000	1,800
Medicare Part D Subsidy	160,000	3,000
Online Change Report
Change of Address	480,000	2,000
Direct Deposit	450,000	2,000
Online Disability Report
i3368	1,000,000	4,800
i3820	115,000	4,000
i3441	208,000	4,000
Online Information Requests
Proof of Income Letter	3,000,000	2,200
Replacement SSA-1099	240,000	2,200
Replacement Medicare Card	230,000	2,200

To produce a combined Internet satisfaction rating, we weight the survey data to reflect the actual universes recorded for each targeted Internet transaction during the sample period.

Methodology

SSA sends the sample files to a private contractor who conducts the survey by mail. For the sake of consistency, we follow the same methodology we use for the other performance measure survey we conduct by mail (i.e., the Office Visitor Survey). First, we will send a pre-notification postcard to all sampled individuals advising them to expect the survey questionnaire. Within three business days, we mail the questionnaire with a cover letter explaining the purpose of the survey. We include the Paperwork Reduction Act and the Privacy Act statements in the cover letter. Five business days later, we send a follow-up letter and another copy of the questionnaire to all sampled individuals.

Response Rate

SSA takes the following steps to maximize response rates:

• To minimize the time between customers’ service experience and the survey, thereby ensuring salience, we select the samples shortly after individuals complete the online transaction.

• We mail a pre-notification postcard to all sampled individuals. The postcard informs them that they will soon be invited to participate in the survey, and to be on the lookout for the envelope containing the questionnaire. Using a postcard format allows the sampled individuals to quickly see SSA sanctions the survey. Additionally, the postcard identifies the contractor conducting the survey for SSA, and should increase the likelihood that sampled individuals will open the envelope when they receive the questionnaire.

• Within 3 - 4 days after sending the pre-notification postcard, we mail the survey questionnaire along with a cover letter encouraging sampled individuals to respond by emphasizing the importance of the survey.

• Five business days after we send the first questionnaire, we mail a follow-up letter and a duplicate questionnaire to all sampled individuals. We do not send more than one follow-up because responses for service satisfaction surveys must be obtained shortly after the date of the contact (while the person’s perceptions are still fresh) to ensure their validity and utility.

• The questionnaire is short (one page - front and back), has an eye-catching title, and is easy to read and complete. The scannable questionnaire is designed for ease of use by an aged and disabled population, e.g., font sizes are large, difficult fill-in bubbles are not used.

• We provide SSA’s toll-free national 800 number so sampled individuals can call if they have any questions.

Using the same methodology described above in FY 2013, we achieved a combined response rate of 49 percent in the online change (address, direct deposit) report category, 45 percent in the online application category, 39 percent in the online request category, and 29 percent in the online Disability Report category. Response rates are generally highest when sampled individuals have a significant business connection with SSA (e.g., currently receiving benefits) and lowest where the nature of the business reflects a negative interaction (e.g., filling out a Disability Report to appeal an unfavorable decision on an application for disability benefits).

The response rates we achieved in FY 2013 are the highest possible considering that time permits only a limited number of contacts. We expect to achieve equally good response rates in the upcoming survey. Note that SSA routinely conducts a non-responder analysis to identify any significant differences in the responder and non-responder populations and their potential impact on the survey results.

Sampling Variability

The key variable in the IRC Survey is overall satisfaction with the Internet transaction. We define satisfaction as a combined rating of excellent, very good, or good (E/VG/G). Assuming a 50 percent response rate, our proposed sample sizes are large enough to provide a sampling variability at the 95‑percent confidence level acceptable for the intended purpose of the survey:

Transactions	Key Variable (E/VG/G Rating)*	Sampling Variability
Online Application
Retirement	95%	+/- 1.4
Medicare Part B	92%	+/- 1.9
Medicare Part D	89%	+/- 1.6
Online Change Report
Change of address	94%	+/- 1.4
Direct deposit	96%	+/- 1.4
Online Disability Report
i3368	87%	+/ 1.4
i3820	86%	+/ 1.6
i3441	75%	+/ 1.9
Online Information Requests
Proof of Income Letter	90%	+/- 1.9
Replacement SSA-1099	90%	+/- 1.9
Replacement Medicare Card	90%	+/- 1.9
* The E/VG/G ratings reflect the validated satisfaction rates for FY 2012 for the online application, online change report, and online Disability Report transactions. We estimated the ratings for the online request category because we included that category for the first time in FY 2013 and data tabulations are still in progress.

SSA’s Office of Quality Performance (OQP) performs all sampling and data analysis. Dan Zabronsky, Director of SSA’s Division of Modeling, OQP, provides statistical support, and is available for discussion of statistical-related issues. Mr. Zabronsky can be reached at (410) 965-5953.

IF FOCUS GROUP MEMBERS WILL RECEIVE A PAYMENT, INDICATE AMOUNT:

We do not compensate participants for this survey.

USE OF SURVEY RESULTS:

SSA uses the results of this survey to evaluate and improve various aspects of its Internet services. In addition, SSA uses the findings in combination with results from the 800 Number Caller, Field Office Caller, and Office Visitor surveys to compute the agency’s combined performance indicator of overall satisfaction.

BURDEN HOUR COMPUTATION:

Number of Responses: 30,000

Estimated Response Time: 5 minutes

Annual Burden Hours: 2,500 hours

NAME OF CONTACT PERSON: Deborah A. Larwood

TELEPHONE NUMBER: 410-966-6135

Generic Clearance FY 2014 IRC Survey

August 2013