Template Description TTW EN Beneficiary Satisfaction Survey 2014

DOCUMENTATION FOR THE GENERIC CLEARANCE

OF CUSTOMER SATISFACTION SURVEYS

TITLE OF INFORMATION COLLECTION: 2014 Ticket to Work (TTW) Employment Network (EN) Beneficiary Satisfaction Survey

SSA SUB-NUMBER: B-01

DESCRIPTION OF ACTIVITY (give purpose of activity, provide specific information; i.e., date(s) of survey, number of focus groups, locations, etc.):

Background

Ticket to Work (TTW) is a Social Security Administration (SSA) program that offers adults receiving Social Security disability insurance (SSDI) benefits or Supplemental Security Income (SSI) payments due to disability or blindness choices for receiving employment services. Under this program, SSA issues tickets to eligible recipients, who in turn, may choose to assign those tickets to an employment network (EN) or to their state vocational rehabilitation agency (SVRA) to obtain employment services, vocational rehabilitation services, or other support services necessary to achieve a vocational (work) goal. The EN or SVRA coordinates and provides appropriate services to help recipients find and maintain employment.

To ensure ENs are providing effective and quality service, SSA needs to assess SSDI beneficiaries and SSI recipients’ satisfaction with their ENs. In this survey, SSA proposes to conduct a customer satisfaction assessment of select recipients who are participating in the TTW program. The purpose of the survey is to assess participants’ satisfaction with their EN.

Description of Survey

We are planning to survey approximately 33,000 SSDI beneficiaries and SSI recipients or their representative payees. This sample will include 27, 0000 beneficiaries/recipients who have assigned their tickets with a specific EN for three months or more as of December 31, 2014; as well as 6,000 beneficiaries/recipients who decided to unassign their Ticket from their EN between January 1, 2014 and December 31, 2014.

The purpose of the EN Beneficiary Satisfaction Survey is to (1) collect information from beneficiaries who participate in the Ticket program with regards to their satisfaction with their ENs; and (2) provide feedback to ENs regarding the strengths and weaknesses of their operation, staff, and the services they provide, as perceived by their consumers. We will provide this information via the program’s website to help beneficiaries make informed choices when they select an EN from which they wish to receive services. The degree of accuracy needed is sufficient to identify meaningful differences among ENs’ ratings. For this purpose, we consider a difference of 10 percent sufficient to potentially affect a consumer’s decision to choose one EN over another.

The 2014 TTW EN Beneficiary Satisfaction Survey is a follow-up to a survey of the same population we conducted in 2013. We have updated the 2014 survey and have tasked our agency-approved contractor, Booz Allen Hamilton, to improve the validity of the “satisfaction” measurement, and to solicit new information from respondents. To ensure comparability, the 2014 questionnaire uses essentially the same language in the satisfaction items as the 2013 survey.

The survey questionnaire covers a variety of topics related to satisfaction with a current or former EN, areas of strength and weakness of the EN, service expectations, service needs, service fulfillment, the beneficiaries’ current employment situation, and an open-ended item examining areas for program improvement.

• New Questions 1 and 2 were included to assess whether respondents currently assigned their “ticket” to an EN. We added Question 2 to understand why Ticket holders unassigned their ticket from their EN. We only ask this question if respondents stated in the previous questions that they unassigned their ticket from an EN.

• Questions 3 - 9 measure respondents’ satisfaction with their ENs. We will use these items to report beneficiary satisfaction. These items are measured on a Likert 5-point scale ranging from “Completely Satisfied” to “Completely Dissatisfied.” A common, balanced scale should result in equal conceptual distances between scale points, more variation in responses, and more accuracy.

• Questions 3, 4, and 5 give respondents the opportunity to report their satisfaction with interactions they had with EN staff. These items are similarly worded to 2013 items.

• Questions 6 and 7 give respondents the opportunity to report their satisfaction with specific services they received from their ENs. These items are also similarly worded to 2013 items.

• Question 8 gives respondents the opportunity to rate their satisfaction with the EN’s ability to provide services to help them meet their employment goals.

• Question 9 is a measurement of respondents’ overall satisfaction with their EN. This item is similarly worded to a 2013 item.

• Question 10 explores respondents’ selection of their current EN, to help us determine whether they considered another service provider or multiple ENs before assigning their ticket. This item is similarly worded to a 2013 item.

• Question 11 asks the reason(s) why respondents selected their ENs. This item is similarly worded to a 2013 item.

• Question 12 solicits information that allows SSA and ENs to evaluate the ability to meet the service needs and expectations of TTW participants. We worded this item similarly to a 2013 item.

• Question 13 seeks information from respondents about which aspects of their ENs they liked and if there are any areas for improvement. We worded these items similarly to the 2013 questionnaire. We will also use the results to assess whether any of these factors affect satisfaction.

• Questions 14 - 21 solicit information about respondents’ employment. The results will provide insight into employment status, recency of employment and compensation, as well as insight into the reasons why respondents might be unemployed full-time. The results will provide us with insights of relationships between employment status and satisfaction with their EN.

• Question 14 asks about employment status. We worded this item similarly to the 2013 question except changing the wording “Are you currently full time or part-time employed?” to “Are you currently employed?” with response options providing different employment status categories.

• New Questions 15, 16, 17, and 18 were added to explore why the respondent is working part-time as opposed to full-time. We only solicit these items from the respondents that select part-time employment status in the previous question.

• Questions 19 and 20 solicit similar information as the 2013 items about the period at which respondents began working and whether they perceived that the EN had a role in getting them a job. We worded these items similarly to the 2013 questionnaire.

• Question 21 asks about compensation, and is similarly worded to the 2013 questions “What is your annual salary at your current position before taxes and benefits?”

• Question 22 assesses the extent to which beneficiaries anticipate reducing or eliminating reliance on disability cash benefits with earnings from work, with 5-pt scale from “Strongly Agree” to “Strongly Disagree.”

• Question 23 is an open-ended item that seeks information from the respondents about comments/suggestions to improve the Ticket to Work program. We worded this item similarly to the 2013 questionnaire.

The data collection process is mixed-mode. First, we will mail a postcard in an enclosed envelope to the entire sample. The postcard contains the web address and an individualized password/link to the web version of the survey. We will mail all potential respondents we invited who do not complete the questionnaire online within 10 days of receiving the initial postcard mailing a paper (e.g., self-administered) copy of the survey. The survey mailing includes an introductory letter explaining the objectives of the study and inviting respondents to either complete the survey online or complete the paper questionnaire enclosed in the mailing and return it in the postage-paid envelope included in the mailing. Approximately two weeks later, those invited respondents who do not respond to the pre-survey postcard or to the first mailing of the survey will receive a second survey mailing identical to the first.

We will select for phone-call reminders any invited respondents who did not respond to the initial mail survey from ENs with close to 25 complete responses (threshold for individual EN-level reporting). The anticipated size of the telephone follow-up reminder call sample is approximately 2,000 non-respondents. We will use the phone contacts to encourage completion of the paper or web-based surveys, but interviewers will administer the survey telephonically, if a respondent prefers to complete the survey by phone. We will make up to three attempts to contact those potential respondents. Following the telephone outreach effort, we will make a third mailing of the self-administered survey to individuals who were contacted by phone and who did not complete the survey online or did not return the survey as part of the second self-administered survey out-go mailing. As noted above, we are conducting this directed follow-up effort to respondents in ENs which are close to the reporting threshold of 25 completed cases, for the purpose of increasing the number of “reportable” ENs.

EurekaFacts, the SSA-approved contractor, will initiate the data collection within one month of receiving OMB approval.

The survey responses are strictly voluntary and anonymous. We will not provide any payments/stipend to participants.

USE OF SURVEY RESULTS:

We will use the results of the study to monitor and evaluate respondent satisfaction with their EN’s performance. Prospective participants in the TTW program will use the information to help them choose an EN. (Where sufficient data from the survey is available, SSA will also incorporate customer satisfaction information derived from the survey into the EN Profile, an online resource guide that contains performance-related data about all ENs.)

BURDEN HOUR COMPUTATION (Number of responses (X) estimated response time

(/60) = annual burden hours):

Number of Responses: 33,000

Estimated Response Time: 12 minutes

Annual Burden hours: 6,600

NAME OF CONTACT PERSON:

Quantitative Surveys:

For quantitative surveys, you will need to complete the questions below.

A. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Provide, in tabular form, data on 1) the number of entities in the universe covered by the collection, 2) the corresponding sample for the universe as a whole, and 3) each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If you have conducted the collection previously, include the actual response rate achieved during the last collection.

We are planning to survey two groups within population of SSDI beneficiaries and SSI recipients or their representative payees. Similar to the 2013 survey, we plan to survey approximately 27,000 SSDI and SSI recipients or their representative payees who assigned their tickets to ENs as of December 31, 2014. New to the 2014 survey, we are planning to survey approximately 6,000 SSDI and SSI recipients or their representative payees who previously unassigned their ticket from an EN within the past year (December 31, 2013 to December 31, 2014).

We will apply two sampling approaches to select samples from these two groups: a stratified random sampling of Ticket-assigned beneficiaries and random sampling of Ticket-unassigned beneficiaries.

The sampling universe for Ticket-assigned beneficiaries includes all SSDI and SSI recipients served by ENs under the TTW program for at least three months as of December 31, 2014. A mixed sampling method is proposed; we will survey (1) all beneficiaries served by all ENs meeting these criteria who have 500 or fewer beneficiaries, and (2) a randomly selected 500 beneficiaries from all ENs meeting these criteria who have more than 500 beneficiaries assigned to them.

	N (approximate)
Number of (Ticket-assigned) TTW beneficiaries	46,882
Sample	27,000
Strata	By EN size. We will divide the sample into 3 strata based on grouping ENs by number of beneficiaries with assigned tickets. The first strata will consist of large ENs defined as those with 101 beneficiaries or more. The second strata, mid-sized ENs, are defined as having between 26 and 100 beneficiaries. The third strata, small ENs, consist of those with 25 beneficiaries or less. At the strata-level, we will sample a sufficient number of beneficiaries from the large ENs to ensure that we are able to report on the maximum number of ENs in which the reporting threshold of 25 completed surveys can be met. Samples of approximately 7,900 and 2,400 beneficiaries, respectively, will then be selected from the second and third strata (mid-sized and small ENs). This will allow a sufficient number of completed cases to make-group level (i.e. sized-based comparisons) in levels of satisfaction across large, mid-sized and small ENs.

The expected response rate is 31 percent, which is typical for this type of survey and is based on the number of completed surveys yielded during the 2013 data collection. While the research plan includes some elements shown to increase return rate, i.e., multiple formats for completion, advance notice, follow-up (mail and phone) and postage-paid, addressed envelopes; it does not include any incentives for respondent participation.

We will use a simple random sampling technique to obtain a representative sample of TTW participants who chose to unassign their Ticket.

	N (approximate)
Number of (Ticket-unassigned)TTW participants	8,995
Sample	6,000

Based on results of another survey (the Outreach Effectiveness Survey) that SSA conducted with the Ticket-unassigned groups of the beneficiary population, we assume an overall response rate of 17 percent among respondents who unassigned their Ticket. To perform needed statistical analyses related to the unassigned Ticket population and to collect 1,000 completed survey responses, we require a sample size of 6,000 unassigned beneficiaries. In addition, the respondent universe includes the following segments:

1. Beneficiaries without representative payees listed in the sample

2. Beneficiaries with representative payees in the sample

3. Representative payees on behalf of beneficiaries

B. Describe the procedures for the collection of information:

The evaluation uses a mixed-mode, multi-stage approach to data collection that draws on the Dillman Tailored Design Method. Beneficiaries will receive an initial pre-notification post-card contained within a sealed envelope to ensure that the information contained on the post-card (beneficiary’s EN name, PIN and link to an online survey administration option) is maintained confidential for the intended recipient. All invited beneficiaries and recipients who do not complete the questionnaire online within 10 days of receiving the advance postcard will be mailed the paper (self-administered) survey. The self-administered survey mailing will include an introductory letter which will communicate the objectives of the study and invite the individual to either complete the survey questionnaire enclosed in the mailing and return the paper survey in a postage-paid envelope that will be provided or, to complete the survey online. Those invited respondents who do not respond to the pre-survey postcard or to the first mailing of the survey will receive a second survey mailing identical to the first. The second survey mailing will be sent approximately 2-3 weeks after the initial mailing.

A sub-sample of the larger non-respondent population will receive additional outreach in the form of a telephone out-reach effort and a third mailing of the self-administered survey. This effort will assist in increasing the total number of reportable ENs (e.g. those with a minimum of 25 responses which has been determined as the threshold for SSA to incorporate individual-EN level customer satisfaction data derived from the survey into the EN Profile, an online resource guide that contains performance-related data about all ENs.)

C. Describe methods to maximize response rates and to deal with the issues of non-response.

There are multiple efforts used to maximize response rates. As noted above, we will mail potential recipients a pre-notification postcard in an enclosed envelope explaining the purpose of the survey and encouraging them to participate online. To those who did not complete the survey online, we will also mail a paper version of the survey with an addressed, postage-paid envelope, as well as the URL and PIN for an online response option for the survey.

Within two to three weeks of the original mailing, we will mail non-respondents a reminder letter, another copy of the survey, a postage-paid envelope, URL and PIN for optional online completion. We will mail surveys in the language of preference (Spanish or English) on record with SSA.

In addition, to boost response rates in ENs which are near, but below, the required 25 completed response threshold needed for “breakout” reporting of satisfaction rates by individual ENs, we will follow-up the second mailing of the self-administered survey with a telephone reminder/outreach effort to be followed by a third mailing of the self-administered questionnaire. The phone contacts will primarily be used to encourage completion of the paper or web-based surveys, but respondents will be provided the option to complete the survey by phone. Up to three attempts will be made to contact beneficiaries.

There are no unusual problems requiring specialized sampling procedures. The research plan proposes annual data collection.

To increase the number of ENs with at least 25 respondents, we will follow-up by telephone with non-respondents of mid-sized/smaller ENs who have not responded to the three previous mailings. We will make the phone calls in the language of preference (Spanish or English) on record with SSA.

D. Describe any tests of procedures or methods to be undertaken. When possible, OMB encourages testing of procedures as an effective means of refining collections of information to minimize burden and improve utility. However, this is not always necessary.

EurekaFacts, an SSA-approved subcontractor which administered last year’s survey, has revised this year’s survey to better address beneficiaries’ satisfaction with ENs. The current survey has been reviewed by individuals with expertise with the TTW program, including contractor staff who are supporting the TTW program, and SSA staff.

E. Provide the name and telephone number of individuals you consulted on statistical aspects of the design. If you are using a contractor who will actually collect and/or analyze the data, provide their name as well.

Consulted on statistical aspects of the design and to collect and analyze the data:

Jorge Restrepo (240) 403-1636

Bohdana Sherehiy, PhD (240) 403-1637

Djass Mbangdadji (240) 403-1640

EurekaFacts, LLC (Subcontractor to Booz Allen Hamilton)

Michael Greenberg (240) 207-3467

Booz Allen Hamilton

7