Supporting Statement - 0806 (Revised)

Supporting Statement For

Supported Employment Demonstration (SED) Project

OMB No. 0960-0806

A. Justification

1. Introduction/Authoring Laws and Regulations

The Social Security Disability Insurance (SSDI) program provides benefits to disabled workers and their families. In 2015, SSA paid more than $143 billion in SSDI benefits to almost 11 million people. The Supplemental Security Income (SSI) program guarantees a minimum level of income support to financially needy individuals who are aged, blind, or disabled. In 2015, SSA paid nearly $55 billion in Federal SSI benefits to more than 8 million people. Given the large number of individuals who rely on the SSDI and SSI programs to make ends meet, and the interest in supporting employment efforts of those with disabilities, it is helpful for policymakers to have an evidentiary base from which to consider potential program improvements and innovations, which can strengthen the ability of individuals with disabilities to work. Part of SSA’s stewardship role involves finding ways to promote work and increase independence. Section 1110(a) of the Social Security Act (Act) gives the Commissioner of Social Security the authority to help fund research or demonstration projects relating to the prevention and reduction of dependency. SSA contracted with Westat to implement and evaluate the SED.

SSA previously studied this population in the Mental Health Treatment Study (MHTS), which showed the provision of supported employment services under the IPS model, including delivery of integrated behavioral and medical health care, had positive effects on employment, mental health, and lower hospital utilization. Currently we are interested in applying the intervention model earlier in the disablement process. Through the Supported Employment Demonstration (SED) project, SSA aims to study the impact of access to these services on outcomes such as benefit receipt; employment; medical recovery; and functioning among denied (SSDI or SSI) disability applicants.

The SED period of performance is August 2016 through August 2022. The renewal application for OMB approval is required after 3 years of activities. We are reapplying for approval as required, to continue the study until its conclusion in FY2022.

2. Description of Collection

The Consolidated and Further Continuing Appropriations Act of 2015 (Pub. L. 113-235) appropriated funds to design, develop, and implement an early intervention demonstration to test innovative strategies aimed at helping people with disabilities remain in the workforce. (See (https://www.ssa.gov/disabilityresearch/supported_employment.html.)

In August 2016, SSA awarded a contract to Westat, Inc. to implement and evaluate whether offering evidence-based interventions of integrated vocational, medical, and behavioral health services to individuals with behavioral health challenges can significantly reduce the demand for disability benefits and help individuals remain in the labor force.

The Supported Employment Demonstration (SED) uses a random assignment design to assign 3,000 participants to one of two treatment groups (Full Services or Basic Services) or to a control group (Usual Services). For 36 months, participants receives varying degrees of services based on their assignment. These services include systematic medication management, health care management and care-coordination services, and long-term employment services following the evidence-based Individual Placement and Support (IPS) model.

The SED demonstration offers the same services to SED participants available to disability beneficiaries participating in the earlier MHTS demonstration (i.e., supported employment services under the IPS model and integrated behavioral health and medical care). Unlike the SSDI beneficiaries enrolled in the MHTS demonstration, however, the SED participants are individuals whose initial applications to SSA for disability benefits were denied for mental health impairments provided in the Social Security Act under Title II (SSDI) or Title XVI (SSI) initiatives. SSA seeks to determine whether offering this evidence-based package of integrated vocational and mental health services to denied disability applicants fosters employment that leads to self-sufficiency; improved mental health and quality of life; and reduced demand for disability benefits. The SED is a research study that will use a randomized controlled trial to compare the outcomes of two treatment groups and a control group. One sample of 3,000 individuals was recruited for the study. Each participant receives services from the study for 36 months. Study participation begins on the day following the date of randomization to one of the three study groups. Participation is voluntary, and participants can withdraw from the study at any time without penalty; however, the services and other benefits offered through the study will then stop.

The SED study population consists of individuals aged 18 to 50 who apply for disability benefits alleging a mental illness, and receive a denial of benefits as the initial decision within the past 60 days. The SED will enroll up to 1,000 participants in each of the three study arms for a total of 3,000 participants: 40 participants in each of three study arms for the 20 urban sites equaling an n of 2,400 urban site participants, and 20 participants in each of three arms for the 10 rural sites equaling an n of 600 rural site participants.

We randomly select and assign each enrolled participant to one of three study arms:

• Full-Service Treatment (n=1,000). The multi-component service model from the MHTS comprises the Full-Service Treatment. At its core, it includes an Individual Placement and Support (IPS) supported employment specialist and behavioral health specialist providing IPS supported employment services integrated with behavioral health care. Participants in the Full‑Service Treatment group also receive the services of a Nurse Care Coordinator who coordinates Systematic Medication Management services, as well assistance with: out-of-pocket expenses associated with prescription behavioral health medications; work-related expenses; and services and treatment not covered by the participant’s health insurance.

• Basic-Service Treatment (n=1,000). The Basic-Service Treatment model leaves intact IPS supported employment integrated with behavioral health services as the centerpiece of the intervention arm. The Basic-Service Treatment is essentially the Full-Service model without the services of the Nurse Care Coordinator; Systematic Medication Management; and the funds associated with out-of-pocket expenses for prescription behavioral health medications.

• Usual Services (n=1,000). This study arm represents a control group against which we can compare the two treatment groups. Participants assigned to this group seek services as they normally would (or would not) in their community. However, at the time of randomization, each Usual Service participant will receive a comprehensive manual describing mental health and vocational services in their locale, along with state and national resources.

This study will test the two treatment conditions against each other and against the control group on multiple outcomes of policy interest to SSA. The key outcomes of interest include: (1) employment; (2) earnings; (3) income;

(4) mental status; (5) quality of life; (6) health services utilization; and (7) SSA disability benefit receipt and amount. SSA is also interested in the study take up rate (participation); knowing who enrolls (and who does not); and fidelity to evidence-based treatments; among other aspects of implementation.

Data collection for the evaluation of the SED consists of the following activities: baseline in-person participant interviews; quarterly participant telephone interviews; receipt of SSA administrative record data; and collection of site-level program data. Evaluation team members also conduct annual site visits involving: (1) pre-visit environmental scans to understand the local context in which we embed SED services; (2) independent fidelity assessments in conjunction with those carried out by state Mental Health or Vocational Rehabilitation staff; (3) key informant interviews with the IPS specialist, the nurse care coordinator, the case manager, and facility director; (4) focus groups with participants in the Full‑Service and Basic-Service Treatment groups; and

(5) ethnographic data collection consisting of observations in the natural environment, and person‑centered interviews with participants and non‑participants. The respondents are study participants and non-participants; IPS specialists; nurse care coordinators; case managers; and facility directors.

2. Use of Information Technology to Collect the Information

SSA’s contractor, Westat, has designed, developed, and maintains a high quality, secure, and cost-effective information technology (IT) solution to support the SED project. SSA designed, developed and maintained the Management Information System (MIS) used in the MHTS; however, more sophisticated security threats rendered the 10-year old, underlying MHTS technology obsolete. Therefore, Westat aggressively retooled its underlying software components with technology solutions that met the government’s exacting standards for efficiency and security. In accordance with the Government Paperwork Elimination Act (GPEA), Westat developed an integrated web-based MIS to maintain, track, and record all data collected during the demonstration. The MIS serves as the mission control center, and includes dashboards for the multiple sub-systems to monitor activities at all demonstration sites. A web-based MIS facilitates flexible tracking of demonstration activities, including recruitment and enrollment; provision of intervention services; and transition planning. Using a web-based MIS system also allows us to enter, update, and store data in real-time so multiple users can access the system to view and enter information about participants to complete and monitor study activities. Users of the MIS consist of staff working on the SED who can easily access the system from any computer with Internet access by simply typing in the URL address and using their secure login credentials. The MIS captures data on randomization results; insurance coverage; providers; and intervention services, and stores this data in one secure, centralized database. We programed the MIS to implement automated edit checks to minimize errors, ensure data quality across study sites, and established built-in security measures to maintain the confidentiality and privacy of participants. No Social Security Numbers exist in the MIS.

Both the Implementation Team and the Evaluation Team use data that the MIS captures and maintains. However, we designed the system to maintain the firewall between implementation and evaluation. For example, although the MIS tracks and reports progress on the status of quarterly follow-up interviews, no users of the system have access to the actual interview data. Furthermore, although the interviewers have access to contact information stored within the MIS to administer the follow-up interviews, they do not have access to randomization results or any other MIS subsystems, given they must remain blinded to the assigned group condition. The MIS assigns users specific access rights that determine which subsystems, components, and data they can access. These limited, role-based user privileges enforce the firewall between the implementation and evaluation components of the demonstration.

As discussed in additional detail below and in Supporting Statement B, many aspects of the information collection involve in-person interviews with participants that would not benefit from additional electronic submission capacity.

2. Why We Cannot Use Duplicate Information

The nature of the information we collect and the manner in which we collect it precludes duplication. SSA does not use another collection instrument to obtain similar data

5. Minimizing Burden on Small Respondents

This collection does not affect small businesses or other small entities.

6. Consequence of Not Collecting Information or Collecting it Less Frequently

The data we collect will provide SSA with the scientific evidence it needs to assess the value of the intervention activities. In addition, SSA would not be able to accurately develop additional ways to improve services to current and future beneficiaries based on the outcomes of the demonstration, as well as other sources. Quarterly participant interviews cannot take place less frequently due to concerns about recall or specificity. There are no technical or legal obstacles to burden reduction.

7. Special Circumstances

There are no special circumstances that would cause SSA to conduct this information collection in a manner inconsistent with 5 CFR 1320.5.

8. Solicitation of Public Comment and Other Consultations with the Public

The 60-day advance Federal Register Notice published on February 25, 2020, at 85 FR 10804, and we received no public comments. The 30-day FRN published on April 28, 2020 at 85 FR 23587. If we receive any comments in response to this Notice, we will forward them to OMB. We did not consult with the public in the development of this project

8. Payment or Gifts to Respondents

Receipt of mental health services is a subject that people generally do not want to engage in because of the perceived stigma associated with mental health problems. Therefore, it is imperative that we use available resources and evidence-based practices to encourage participation and achieve the highest response rates possible. Research unequivocally demonstrates that incentives improve response rates. SSA makes payments to SED respondents for participating in the following data collection activities:

• Participants received $50 for completing the baseline interview, taking approximately 45 minutes.

• Participants received $45 for completing the post-enrollment (Composite International Diagnostic Interview or CIDI) interview, if selected, taking approximately 75 minutes.

• Participants will complete quarterly interviews during the three-year intervention period for a total of 12 interviews and receive payments for these interviews as follows:

  • Participants receive $25 for completing each quarterly interview lasting approximately 20 minutes (Quarters 1, 2, 3, 5, 6, 7, 9, 10, and 11) for a total of 9 quarterly interviews totaling a payment of $225.

  • Participants receive $40 for completing an annual interview lasting approximately 30 minutes (Quarters 4, 8, and 12) for a total of 3 annual interviews totaling a payment of $120.

• Participants receive $40 for attending the focus group, if selected, lasting approximately 60 minutes.

• Participants and non-participants receive $40 for completing the person‑centered interview, if selected, taking approximately 60 minutes.

We implemented a tracking system has been implemented within the MIS for managing payments to respondents. At the end of the baseline interview, participants receive a reloadable Visa debit card. The MIS manages the incentive payments for the quarterly interviews and adds the appropriate monetary amount to the debit cards once respondents complete an interview.

8. Assurances of Confidentiality

SSA protects and holds confidential the information it collects in accordance with 42 U.S.C. 1306, 20 CFR 401 and 402, 5 U.S.C. 552 (Freedom of Information Act),

5 U.S.C. 552a (Privacy Act of 1974), and OMB Circular No. A-130. As such, SSA assures the respondents that the proposed collection of information is voluntary and we will keep the information we collect. We communicate this to respondents in the Overview and FAQ document, and in the informed consent language regarding their participation in study activities. Additionally, Westat has extensive experience conducting data collection in a secure environment. We store all study materials that Westat maintains in a secure project directory on the Westat network accessible only to project team members. For the SED project, SSA provides data files to Westat via secure File Transfer Protocol (FTP). Westat sanitized the SSA data files of Social Security Numbers (SSNs) and created random IDs for each potential demonstration participant. When we loaded contact information for each denied applicant into the MIS, we created another special ID. This ID provides an additional key that ties the contact information ID with the subsequent enrollment decision, other study data, and the individual’s SSN. Westat maintains a linking file that matches the sample file unique personal identifiers and the study IDs. Only the Westat IT security officer and his designee hold the original files containing SSN, along with other personal information, and the linking file in an independent secure file location with access.

With respect to activities conducted as part of the SED, reviewers conducting the independent fidelity assessments request that the study site remove participant names from all records examined by fidelity reviews. We do not record any personally identifiable information (PII) on the fidelity assessments and we only base fidelity ratings on aggregate data. We examine all medical charts on the premises of the study site and we do not remove any charts or photocopies from the site. We upload audio recordings from study-related interviews and focus groups to a secure Westat privacy secured website, as well as the Westat project directory accessible only to study staff, and erase them from the recording equipment. We store transcripts from the audio recordings in Westat’s secure project directory that is accessible only to study staff. In addition, we keep hard copy documents containing PII (e.g., scheduling notes) in a locked filing cabinet in a secure study office only accessible to research staff. SSA staff have access to redacted transcripts associated with the qualitative interviews, and we do not identify participants. Westat will destroy audio recordings, transcripts, notes, and any other identifying information associated with the data collection, once we finalize the study report and give approval for the destruction. Study-related reports, which Westat will submit to SSA, will not contain any PII. At the end of the project, Westat will deliver to SSA a final, de-identified report and a data package that includes a codebook, user’s guide, data dictionary, Public Use data files, and Restricted Use Files, along with a linking file that will allow SSA to re-identify participant data if desired. Westat will conduct a disclosure analysis to protect respondent anonymity in the presentation of group statistics. The Public Use data files that Westat will submit to SSA will not contain any PII, nor will they identify the study sites. Furthermore, Westat will also conduct disclosure risk analyses for the Public Use files, and will work with SSA’s Disclosure Review Board to employ statistical disclosure controls, such as variable suppression, recodes, top-codes, perturbation, and possible removal of especially high‑risk variables, to reduce disclosure risk. This project has an estimated end date of 2/28/2023. After completing the study, and after receiving SSA’s approval, Westat will destroy all identifiable data and provide documentation of the destruction to SSA. Westat will also remove the project directory from the Westat network server. Data destruction will occur no later than 1 year after the end of the contract or 2/28/2024.

8. Justification for Sensitive Questions

To estimate the prevalence of various mental impairments among the study population, we will invite selected enrolled participants to complete the World Health Organization (WHO), World Mental Health Composite International Diagnostic Interview (WMH‑CIDI) in-person or by telephone within two weeks following randomization. The CIDI instrument contains sensitive questions on topics such as depression, anxiety, phobias, psychosis, mania, or post-traumatic stress disorder. Individuals who enroll in the SED may also experience anxiety, fatigue, or frustration while completing other study-related interviews even though the other interviews do not contain sensitive questions, as does the CIDI. To mitigate the risk of respondents becoming psychologically distressed during their participation in any study-related activities, all of the community mental health centers (CMHCs) selected as sites for the SED will have existing crisis management services available. These services range from in-house services, to on-call service, to collaborations with local emergency rooms. Since all Full-Service and Basic Service participants in the SED will register at their local CMHC, they will be eligible for these existing crisis management services. Additionally, all staff conducting baseline and quarterly interviews; site visit interviews; and focus groups will receive training on how to recognize and handle a respondent who is experiencing psychological distress. Specifically, we will instruct staff to ask the respondents if they want to take a break, after which the respondents can decide whether they want to continue the interview. In some cases, it may be possible simply to skip the section that is causing distress and continue with the remainder of the interview or focus group protocol. Signs of severe psychological distress include respondents who are: (1) no longer making sense when speaking; (2) crying uncontrollably; or (3) stating plans or thoughts of harming themselves, or others. If respondents present a clear and immediate danger to themselves or others, Westat staff will end the session and follow established CMHC procedures, or call 911. In addition to the steps described above, immediately following the session Westat staff will notify the Westat Operations Director, who will in turn notify the Principal Investigators and the SSA Contracting Officer’s Representative or designee. The Westat Operations Director will also notify the Westat IRB in accordance with the appropriate reporting requirements.

8. Estimates of Public Reporting Burden

Modality of Completion	Number of Respondents	Frequency of Response	Number of Responses	Average Burden per Response (minutes)	Estimated Total Annual Burden (hours)	Average Theoretical Hourly Cost Amount (dollars)*	Total Annual Opportunity Cost (dollars)**
Competency and CIDI Screener	1,878	1	1,878	75	2,348	$10.22*	$23,996**
Baseline Interview	3,000	1	3,000	45	2,250	$10.22*	$22,995**
Quarterly Interview (Quarters 1, 2, 3, 5, 6, 7, 9, 10, and 11)	3,000	9	27,000	20	9,000	$10.22*	$91,980**
Annual Interview (Quarters 4, 8, and 11)	3,000	3	9,000	30	4,500	$10.22*	$45,990**
Fidelity Assessment Participant Interview	180	4	720	60	720	$10.22*	$7,358**
Key Informant Interview	120	4	480	60	480	$17.22*	$8,266**
Participant Focus Groups	600	2	1,200	60	1,200	$10.22*	$12,264**
Person -Centered Interview	180	4	720	60	720	$10.22*	$7,358**
Totals	11,958		43,998		21,218		$220,207**

*We based these figures on average DI hourly wages for both individual recipients and disabled recipients with children and spouses (based on SSA's current FY 2019 SSI data: https://www.ssa.gov/legislation/2019%20Fact%20Sheet.pdf); and the BLS.gov data for Social Worker's hourly wages (https://www.bls.gov/oes/current/oes_nat.htm).

** This figure does not represent actual costs that SSA is imposing on recipients of Social Security payments to complete this application; rather, these are theoretical opportunity costs for the additional time respondents will spend to complete the application. There is no actual charge to respondents to complete the application.

The total burden for this ICR is 21,217.5 burden hours (reflecting SSA management information data), which results in an associated theoretical (not actual) opportunity cost financial burden of $163,797. SSA does not charge respondents to complete our applications.

We base our burden estimates on current management information data, which includes data from actual interviews, as well as from years of conducting this information collection. Based on our current management information data, the current burden information we provided is accurate.

13. Annual Cost to the Respondents (Other)

This collection does not impose a known cost burden on the respondents.

14. Annual Cost To Federal Government

The estimated cost of intervention implementation; evaluation data collection efforts associated with the burden described in item 12 (estimate of burden); and analysis and reporting activities is $74,854,979. The estimated annual cost is as follows:

Year 1	$5,932,189
Year 2	$15,492,558
Year 3	$19,334,844
Year 4	$18,619,216
Year 5	$12,451,860
Year 6	$3,024,312
Total	$74,854,979

15. Program Changes or Adjustments to the Information Collection Request

When we last cleared this IC in 2017, the burden was 21,230 hours. However, we are currently reporting a burden of 21,217.5 hours. This change stems from a decrease in the number of CIDI administrations from 3,000 respondents as initially planned to 1,878, which includes 300 randomly selected participants plus 1,578 participants who completed the CIDI prior to the change reducing the number of administrations.

16. Plans for Publication Information Collection Results

As indicated in Figure 1 below, we are now in Year 3 of the project, and nearing Year 4. We recruited and enrolled 1,000 participants in each arm within 15 months of beginning this study. Thus, intervention implementation occurs at the selected demonstration sites in Years 2 through 6. Finally, during Year 6, we will complete all planned analyses; submit the final evaluation reports; and deliver to SSA data files (including a public-use file) with accompanying documentation.

Figure 1. Overview of project schedule

The demonstration will test the two treatment conditions against each other and the control group on multiple outcomes of policy interest to SSA. The primary outcome of interest is the receipt of SSA disability benefits. Additional outcomes of interest include the timing of benefits-related decisions; total award payments; employment; earnings; income; mental status; quality of life, and health services utilization. The evaluation will also assess participation (the take up rate of denied applicants); study implementation; mental health center and provider views of the study; and participant and non-participant views of their disability experience and association with the study. The ethnographic studies will add depth to our characterization of participants and non-participants and provide a deeper understanding of the complex interrelationship of social and economic contexts with individual experiences of mental illness and work among those applying for SSA disability benefits.

The evaluation will include five distinct components: early assessment analysis; participation analysis; process analysis; impact analysis; and a benefit-cost analysis:

1. Early Assessment Analysis. We did an early assessment of the study implementation (including recruitment and enrollment, service provision, etc.) to provide feedback on how the study was going. After approximately six months of experience with recruitment and enrollment, we visited all sites and assessed the implementation consistency within and across sites. The early assessment report described implementation across all sites. We observed recruitment efforts; investigated the quality of random assignment; and identified any issues that arose with implementation technical assistance. We are done with this early assessment.

2. Participation Analysis. The participation analysis assesses the take-up rate for the study. The analysis documents the recruitment process as a series of discrete steps, within which we document the outcome or disposition of each step for each prospective participant until they disengage from the recruitment process or enroll in the study.

3. Process Analysis. The process analysis describes implementation of the SED across all sites. The analysis provides a longitudinal account of demonstration activities to describe changes over time in response to contextual features of the implementation environments. For example, the process analysis describes how early program operations matured, and explain how each component of the SED evolved. The purpose of describing the implementation is to strengthen inferences of causality among program elements and outcomes. The goal of the process analysis is to provide a theory of how and why the SED program works. Ultimately, the process analysis will integrate data from multiple sources to assess implementation fidelity; identify aspects of operations or implementation that required technical assistance; and assess the feasibility of expanding the demonstration or scaling up some or all of its features as future policy.

4. Impact Analysis. The impact analysis assesses the effectiveness of the interventions by comparing the three conditions: Full-Service, Basic-Service, and Usual Services (control). Analyses includes cross-sectional comparisons between the groups at key time points (e.g., univariate comparisons, multi-level model-based comparisons, assessment of the drivers of outcomes); longitudinal analyses (e.g., duration comparisons, multi-level model-based comparisons, assessment of the drivers of outcomes); and subgroup analyses (e.g., age and claim type). We also assess the validity of random assignment by examining whether the treatment and control groups differ on baseline characteristics.

The goal of the SED is to promote delay of participation (to the point of non‑participation) in SSA disability benefits by obtaining and maintaining employment. Thus, the primary outcome for the impact evaluation is the receipt of SSA disability benefits. In addition, we will examine appeals and reapplication, the timing of benefits-related decisions, and total award payments. Other key outcomes concern employment; earnings; clinical; and quality of life outcomes, such as participants’ access to healthcare, healthcare costs, health and functioning, and wellbeing.

5. Benefit-Cost Analysis

In estimating net benefits of the treatment conditions, we will follow the basic principles of benefit cost analysis and apply standard procedures assessing employment programs. The benefits and costs we will assess include: resource costs for direct health service provision; IPS services program costs; reduction in health care expenses paid by public programs (e.g., Medicaid) or private funding; participants’ after-tax earnings and fringe benefits; taxes paid by participants; reduction in disability benefits (e.g., SSIDI); and reduction in other public dependency.

17. Displaying the OMB Approval Expiration Date

SSA is not requesting an exception to the requirement to display the OMB approval expiration date  

18. Exceptions to Certification Statement

SSA is not requesting an exception to the certification requirements at

5 CFR 1320.9 and related provisions at 5 CFR 1320.8(b)(3).

7