Supporting Statement A for the Youth Transition Exploration Demonstration (YTED)
OMB No. 0960-NEW
Justification
On July 27, 2022, the Social Security Administration (SSA) announced a new funding opportunity, the Interventional Cooperative Agreement Program (ICAP), in the Federal Register, 87 FR 45143. ICP allows SSA to enter into cooperative agreements to collaborate with states, private foundations, and other non-federal groups and organizations that have the interest and ability to identify, operate, and partially fund interventional research related to the Social Security Income (SSI) and Social Security Disability Insurance (SSDI). Section 1110(a) of the Social Security Act gives the Commissioner of Social Security the authority to help fund research or demonstration projects relating to the prevention and reduction of dependency. SSA is requesting clearance to collect data necessary to evaluate the Youth Transition Exploration Demonstration (YTED) under the ICAP. SSA awarded Mathematica a cooperative agreement to lead the YTED, which will help youth with disabilities transition successfully into the adult labor force and competitive, integrated employment. In addition, Mathematica is partnering with the Pennsylvania Office of Vocational Rehabilitation (OVR) to recruit youth and deliver intervention services and the University of Maryland’s Center for Transition and Career Innovation (UMD) to provide training and technical assistance to OVR.
Description of Collection
The YTED will provide SSA with empirical evidence on the impact of the intervention on youth in several outcome areas: (1) employment and earnings; (2) SSI and SSDI benefit receipt; and (3) other related outcomes, such as satisfaction and well-being. A rigorous evaluation of YTED is important to help SSA and other interested parties assess promising options to improve employment-related outcomes and decrease benefit receipts. Mathematica is directing YTED and evaluating it via a randomized controlled trial. The evaluation will assess the effect of TE services on employment and earnings, SSI and SSDI benefit receipt, satisfaction, and well-being. YTED will include the following components:
Implementation analysis: provides critical information about YTED’s rollout and administration, including its impact on analysis and relationship-building with community partners. Specifically, the implementation analysis will assess whether the intervention aligns with its intended design, identifies successes and challenges during service deliver; and evaluates how OVR develops new or strengthens existing collaborative relationships with community partners.
Impact analysis uses the experimental design provide rigorous quantitative estimates of YTED’s effects on enrollee outcomes. By applying random assignment, the study ensures the treatment and control groups share similar characteristics at enrollment. This approach allows the evaluation to provide an unbiased assessment of whether the intervention helps with youth with disabilities increase earnings or employments, reduce reliance on SSI or SSDI benefits, and experience greater satisfaction and well-being.
Benefit-cost analysis will assess whether the impacts of YTED on enrollee outcomes are large enough to justify the resources required to produce them. By placing a dollar value on each benefit and cost of the intervention, a benefit-cost analysis can summarize in one statistic all the intervention’s diverse impacts and costs.
Recruitment
OVR will recruit 700 youth into the YTED over a 24-month period during Years 2 and 3 of the YTED two-year period. Potential enrollees include: all residents in the city of Philadelphia and the surrounding four counties in Pennsylvania (Bucks, Chester, Delaware, and Montgomery) ages 16 to 24 who qualify for vocational rehabilitation (VR) services; have not signed an individualized plan for employment with OVR; and are capable of giving informed consent or assent. Once enrolled, youth assigned to the control group will remain eligible for standard VR services and receive information about how to apply to OVR. Youth assigned to the treatment group will open a case with OVR and receive enhanced Transition Exploration (TE) services. Treatment group members will receive services from a specially trained OVR counselor called a Vocational Rehabilitation Counselor-YTED (VRC-YTED) for up to 12 months. Key enhanced services provided by a VRC-YTED will include career goal development, referral to intensive training programs, referral to other employment-related resources, and intensive engagement and follow up. As part of the TE intervention, OVR will develop strong collaborative relationships with intensive training programs like Job Corps and Year Up to facilitate seamless referrals and cooperation. OVR will also provide counseling on SSI, SSDI, and other benefits with a goal of helping youth to access them temporarily while pursuing full-time work.
Recruitment will primarily occur in person when OVR staff are helping youth apply for Pre-Employment Transition Services (Pre-ETS) or VR services visiting schools to deliver Pre-ETS or attend Individualized Education Program meetings, or attending events to promote YTED. OVR staff will begin the recruitment process by explaining TED and answering any questions. If youth indicate they would like to enroll, OVR staff will confirm the youth’s eligibility and then help them complete the consent form, release form, and baseline survey. In addition, the OVR staff conducts a second recruitment method reaching out to youth referred by a community partner, received Pre-ETS in the past, or contact OVR in response to receiving a letter from the Social Security Administration or learning about YTED through the media or social media. OVR staff will contact these youth to confirm their eligibility and then ask them whether they would prefer to meet in person or have the consent form, release form, and baseline survey mailed or emailed to them. For youth who would prefer a meeting, OVR staff will meet with the youth and help them complete the consent form, release form, and baseline survey. For youth who would prefer mail or email, OVR staff will send the consent form, release form, and baseline survey to the youth for them to complete on their own and then mail back to OVR. As noted previously, OVR staff will conclude the recruitment process by entering information from the consent form, release form, and baseline survey into an online system hosted by Mathematica to conduct random assignment.
The YTED will consist of the following data collection efforts for the evaluation of YTED:
Recruitment materials and baseline survey: Mathematica will provide information to YTED participants in several alternative formats. The materials include an informational website; a toll-free phone number; and an email address.
Consent form: Before participating in the YTED Mathematica will provide the participants with a paper consent form in person and will request the participants complete and sign it. SSA requires a wet signature on the consent form; therefore, we do not accept an electronic signature. The consent form will explain the demonstration; how random assignment work; what information the YTED will collect for evaluation purposes; and will clearly state participation is voluntary, and by signing, participants agree to let researchers access information such as their SSDI benefit status. Mathematica requires parents or guardians to sign the consent form for participants under 18.
Baseline survey: Most youth will complete the 15-minute baseline survey on paper, either on their own or with help from OVR staff, and then OVR staff will enter their responses into the online system hosted by Mathematica to conduct random assignment. For some youth, OVR staff may read the questions to the youth in person or on the phone and enter their responses directly into the online system. The baseline survey includes questions about race and ethnicity, which Mathematica will use for certain subgroup analyses. The baseline survey will be available in English and Spanish.
Follow-up survey: Mathematica will administer a follow-up survey to treatment and control group members 12 months after their study enrollment date. The survey will collect information on important outcomes that Mathematica cannot measure using program data. The follow-up survey, together with an analysis of outcomes derived from OVR and SSA program data, will capture the experiences of treatment and control group members over a period of 12 months. The survey will collect information about education and training, employment history, earnings, and satisfaction, receipt of and satisfaction with standard or TE services, goals and expectations, health and well-being, and other contextual variables. A follow-up interval of this length is important to measure the impacts of YTED because the effects of the demonstration on individual behavior and well-being could take time to emerge. Mathematica will conduct the 20-minute follow-up survey via web and telephone from the third quarter of 2025 to the second quarter of 2027. Mathematica will survey the full sample of 700 YTED enrollees and will target an 80 percent response rate. Mathematica will mail all sample members an advance letter with a link to the web survey and their unique survey login information so they can easily access the web survey. The survey introduction (in the web and telephone instruments) will inform all sample members that participation is voluntary and that they may skip any question that they do not wish to answer. The web will ask sample members completing the survey to click a button indicating they consent to complete the survey. Mathematica will ask sample members completing the survey by phone to verbally consent to complete the survey and an interviewer will record their verbal consent by clicking a button. Mathematica will translate the instrument into Spanish, and train Spanish-speaking interviewers to administer the survey. Mathematica will also provide Spanish versions of the advance letters and other communications to sample members who need them.
All survey mailings will include a toll-free number that sample members can use to contact Mathematica with questions or concerns. Professionally trained interviewers will respond to these calls throughout the field periods.
Two rounds of qualitative data from YTED implementation and operations staff. Mathematica will conduct two site visits during the third and fourth years of the evaluation. During each visit, Mathematica will conduct semi-structured interviews with program staff, community partners, and other groups to collect information about their experiences and any changes made to the program during implementation. The interview guide will include questions tailored to the experiences of each group and a set of core questions to enable Mathematica to systematically collect information across all groups. Mathematica will also collect cost data via a review of program documentation.
Three rounds of qualitative data from treatment group members Mathematica will conduct semi-structured telephone interviews with treatment group members in the second, third, and fourth years of the demonstration. Each round of interviews will be with four treatment group members to learn about their experiences enrolling and participating in YTED. The interview guide (see Attachment C) will include questions to elicit in-depth information about treatment members’ experiences with YTED. In the first quarter of 2026 and 2027, Mathematica will conduct two multiple-day site visits to interview VRCs-YTED, other program staff, UMD staff and community partners by telephone or in person. During each site visit, Mathematica will interview up to four staff per organization. These conversations will shed light on implementation successes and challenges as well as considerations for scaling up or adopting the TE model elsewhere. Mathematica will also learn about the collaboration between OVR and its new partners and how service experiences might differ between treatment and control group members by speaking to staff familiar with services for both groups.
The goals of the first site visit are to (1) describe recruitment and enrollment processes and deviations from the planned processes, (2) describe how the TE model components are brought into operation, (3) identify factors that hindered and facilitated service delivery, and (4) understand the development of collaborative relationships with community partners. The goals of the second site visit are to (1) describe changes made to the topics covered in the first site visit, (2) describe plans for sustaining the model, (3) collect information about program costs, and (4) describe counterfactual services. Topics for the site visit interviews are in Attachment C.
Mathematica will gather information using a range of techniques and data sources to fully describe the programs and activities. Mathematica will use the Consolidated Framework for Implementation Research (Damschroder et al. 2009) to guide the collection, analysis, and interpretation of qualitative data. The Consolidated Framework for Implementation Research offers systematic assessment of the multilevel and diverse contexts in intervention implementation and describes the myriad factors that might influence intervention implementation and effectiveness.
In the second quarter of 2025, 2026, and 2027, Mathematica will conduct telephone interviews with four treatment group members. The interviews will elicit in-depth information about participants’ experiences enrolling and participating in the demonstration. The conversations will be an opportunity to learn firsthand from treatment group members about whether and why they engaged, their perceived effectiveness of the services, and any areas of satisfaction, dissatisfaction, or suggested improvement.
We identified the following psychological costs based on the requirements for this information collection:
Psychological Cost #1:
Requirement for Program: The baseline and follow-up surveys include questions on individual and household income and assistance, physical and mental health, and race and ethnicity.
Psychological Cost: Respondents may perceive these questions as private or sensitive and this may lead respondents to skip these questions or decline participation in the demonstration.
Psychological Cost #2:
Requirement for Program: The treatment group interviews include questions about employment goals and experiences.
Psychological Cost: Respondents may perceive these questions as private or sensitive and this may lead respondents to skip these questions or decline participation in the demonstration.
We understand these psychological costs may cause respondents to delay their completion of the information collection or cause them to abandon the information collection entirely. However, we require full completion of this collection to [continue to] receive benefits. Therefore, we have taken this potential psychological cost into account when calculating our burden in #12 below.
The respondents are youth with disabilities who qualify and who are selected by SSA to enroll in YTED.
Mathematica will use a range of technologies to manage data collection for the baseline and follow-up surveys, as well as qualitative interviews.
Baseline Survey: Youth participants will complete the baseline survey on paper, either independently or with assistance from OVR staff. OVR staff will input responses directly into Mathematica’s online system for random assignment. To improve data entry and quality, the system will offer features like drop-down response categories, dynamic questions, automated skip patterns, and logical rules to ensure appropriate responses.
Follow-Up Survey: Mathematica will administer the follow-up survey using several technologies:
Web-Based Questionnaires: Respondents can complete the survey online through Confirmit® software on various devices, with features like range and logic checks, dynamic text fills, and preprogrammed skips.
Computer-Assisted Telephone Interviewing (CATI): Professionally trained interviewers will manage phone surveys using Confirmit®, ensuring Mathematica contacts respondents at convenient times and recording notes after each contact attempt.
Sample Management System (SMS): This system will track and update respondent information in real time, ensuring mailings and follow-up efforts are accurate and directed appropriately. SMS will also handle language preferences and ineligibility documentation.
Toll-Free Number: A toll-free number will be available for respondents to ask questions or raise concerns, with trained interviewers responding throughout the field period.
The nature of the information we collect and the manner in which we collect it precludes duplication. SSA does not use another collection instrument to obtain similar data.
Some of the service providers that Mathematica will interview for the process analysis might be staff of small entities. Mathematica’s protocol imposes minimal burden on all organizations involved, and interviewers will keep discussions to one hour or less. Mathematica will collect the minimum amount of information required for the intended use and schedule interviews at times convenient to the respondents. In this way, Mathematica will minimize the effect on small businesses and other small entities.
If SSA did not evaluate the YTED, we would be unable to address important issues regarding potential and current SSI and SSDI beneficiaries’ success in finding, maintaining, and advancing in employment as a result of the intervention. The baseline survey is a one-time collection and necessary to conduct a credible evaluation. The data Mathematica will collect are not available from other sources, and the survey will collect a richer set of information than Mathematica can gather from program records alone. Program records do not offer details on job search activities. The baseline survey is a one-time survey and thus cannot occur less frequently.
There are no special circumstance that would cause SSA to conduct this information collection in a manner inconsistent with 5 CFR 1320.5.
The 60-day advance Federal Register Notice published on June 29, 2024, at
89 FR 53472, and we received no public comments. The 30-day FRN published on October 2, 2024, at 89 FR 80302. If we receive any comments in response to this Notice, we will forward them to OMB. We did not consult with the public in the development of this form.
Correction Notice: SSA published this information collection as Transition Exploration Demonstration (TED), on June 29, 2024, at 89 FR 53472. We are changing the name of the information collection to Youth Transition Exploration Demonstration (YTED).
SSA consulted with an interdisciplinary group of economists; disability policy researchers; survey researchers; and information systems professionals inform the design of the information collection effort for this evaluation.
SSA will not offer an incentive payment to youth who enroll in the YTED and who complete the baseline survey; however, all 700 respondents will receive a $5 prepaid incentive enclosed in the invitation to participate in the survey. Those respondents who complete the survey will receive an additional $50 postpaid incentive; and SSA will pay a $50 postpaid incentive to those who complete a semi-structured interview. Respondents who complete a total of $105 if they complete
10. Assurances of Confidentiality
The identity of enrollees and the nature of the information collected require strict confidentiality procedures. SSA will protect the information collected in accordance with 42 U.S.C. 1306, 20 CFR 401 and 402, 5 U.S.C. 552 (Freedom of Information Act), 5 U.S.C. 552a (Privacy Act of 1974), and OMB Circular No. A-130.
Descriptions of the detailed plans for informed consent and data security procedures follow:
All potential YTED enrollees will make a genuinely informed decision about participating in the demonstration. Mathematica will obtain the informed consent of each enrollee through a signed consent form (see Attachment A), which describes the demonstration, the process of random assignment, and the evaluation’s information requirements. The informed consent also informs the potential enrollees that participation is voluntary and that agreeing to participate means that they give permission for researchers to access information about them, such as their SSDI benefit status, from other sources. Mathematica will also provide the enrollees with a toll-free telephone number to call with questions about the study. Mathematica requires a parent or guardian to sign the consent form if the enrollee is under the age of 18.
Mathematica will clearly state the assurances and limits of confidentiality in the enrollee consent form and the materials used to conduct surveys, qualitative data collection with YTED enrollees, and qualitative data collection with YTED implementation and operations staff. The consent form will make clear the assurances and limits of confidentiality. The Paperwork Reduction and Privacy Act statements will appear on all study documents, including the baseline and follow-up survey instruments and the follow-up survey advance letter. The Privacy Act statement provides assurance that all information collected will be kept confidential, unless required by law, and will not be used in any way that would affect SSA benefit eligibility or payments. After collecting and analyzing the survey and qualitative data, Mathematica will not attribute responses to specific individuals in any public documents. Mathematica will also destroy all data in a secure manner at the completion of the evaluation.
SSA and Mathematica have procedures in place to appropriately safeguard data from unauthorized use and disclosure, including the use of passwords and encryption. Mathematica uses several mechanisms to secure data, including obtaining suitability determinations for designated staff, training staff to recognize and handle sensitive data, protecting computer systems from access by staff without favorable suitability determinations, limiting the use of personally identifiable information in data, limiting access to secure data on a need-to-know basis and to staff with favorable suitability determinations, and creating data extract files that exclude identifying information.
As stated in #2 above, we need to ask some questions which some respondent may perceive as sensitive in nature to participated in YTED. These include questions regarding their household income; questions about employment goals and experiences that well as physical and mental health. However, we must ask these questions to conduct a comprehensive evaluation of YTED and to understand how participants perceive YTED as affecting their employment goals and experiences
12. Estimates of public reporting burden
Modality of Completion |
Number of Respondents |
Frequency of Response |
Total Number of Respondents |
Average burden per response (in minutes) |
Total Annual Burden (hours) |
Average Theoretical Hourly Cost* |
Total Annual Opportunity Cost** |
Informed consent form and release form |
700 |
1 |
700 |
10 |
117 |
$7.25* |
$848** |
Baseline survey |
700 |
1 |
700 |
15 |
175 |
$7.25* |
$1,269** |
12-month follow-up survey |
560 |
1 |
560 |
20 |
187 |
$7.25* |
$1,356** |
Qualitative data collection with site staff |
18 |
2 |
36 |
68 |
41 |
$31.48* |
$1,291** |
Qualitative data collection with treatment group members |
4 |
3 |
12 |
45 |
9 |
$7.25* |
$65** |
Total |
1,982 |
|
2,008 |
|
529 |
|
$4,829** |
* We based this figure on the average minimum wage, as reported by Pennsylvania’s
data (https://www.dli.pa.gov/Individuals/Labor-Management-Relations/llc/Pages/Wage-FAQs.aspx)
*** This figure does not represent actual costs that SSA is imposing on recipients of Social Security payments to complete this application; rather, these are theoretical opportunity costs for the additional time respondents will spend to complete the application. There is no actual charge to respondents to complete the application.
Note: Mathematica will schedule an appointment to call the recipient at their preferred date and time; therefore, the respondents will not incur an average wait time.
Total Number of Respondents |
Frequency of Response |
Estimate Learning Cost (minutes) |
Estimated Total Annual Burden (hours) |
Total Annual Learning Cost (dollars)** |
700 |
1 |
3 |
35 |
$254**** |
*****We based this dollar amount on the Average Theoretical Hourly Cost Amount in dollars shown on the burden chart above.
NOTE: We included the total opportunity cost estimate from this chart in our calculations when showing the total time and opportunity cost estimates in the paragraph below.
We base our burden estimates on current management information data, which includes data from actual interviews, as well as from years of conducting this information collection. Per our management information data, we believe that 10,15,20,45, and 68 minutes accurately shows the average burden per response for learning about the program; receiving notices as needed; reading and understanding instructions; gathering the data and documents needed; answering the questions and completing the information collection instrument; scheduling any necessary appointment or required phone call; consulting with any third parties (as needed); and waiting to speak with SSA employees (as needed). Based on our current management information data, the current burden information we provided is accurate. The total burden for this ICR is 529 burden hours (reflecting SSA management information data), which results in an associated theoretical (not actual) opportunity cost financial burden of $6,475. SSA does not charge respondents to complete our applications.
This collection does not impose a known cost burden on the respondents
The annual cost to the Federal Government is approximately $1,124,119. This estimate accounts for costs for Mathematica to conduct the survey and analysis, and for SSA to oversee the contract over during the duration of YTED.
Description of Cost Factor |
Methodology for Estimating Cost |
Cost in Dollars* |
Designing and Printing the Form |
Design Cost + Printing Cost |
$75,710 |
Distributing, Shipping, and Material Costs for the Form |
Distribution + Shipping + Material Cost |
$2,036
|
Mathematica team survey administering information collections
|
Mathematica processing time for Survey administration; semi-structured interviews; follow-up surveys, qualitative data collection with site staff, and treatment group members |
$922,629 |
Full-Time Equivalent Costs |
Out of pocket costs and other expenses
|
0*
|
Systems Development, Updating, and Maintenance |
Costs to develop the SMS and Confirmit instruments, and to maintain the data inventory
|
$113,662 |
Quantifiable IT Costs |
Any additional IT costs |
10,082 |
Total |
|
$1,124,119 |
* We have inserted a $0 amount for cost factors that do not apply to this collection.
SSA is unable to break down the costs to the Federal government further than we already have. we used the figures above on the expected costs from our contract with Mathematica.
This is a new data collection that increases the public reporting burden. See #12 above for the burden figures
Data collection: Three years from 2024 to 2027
Data analysis: Four years from summer 2024 to 2028
Final report: 2028
Final report: The final evaluation report will document in detail the study methods and findings of the quantitative and qualitative analyses. The final report will synthesize the overarching findings from all analyses on the effectiveness of YTED at improving outcomes of interest for study enrollees. The report will discuss any relevant external factors such as changes in the local labor market or policy environments that might have affected the demonstration’s impact. In addition, the final report will recommend ways to expand the intervention to other populations and produce other policy recommendations for SSA that arise from the study results.
Analytic techniques. Due to the random assignment design, the impact analysis will focus on differences in the outcomes of enrollees between the treatment group and the control group using a regression framework to control for other explanatory variables. Mathematica will use regression-adjusted comparisons of treatment group members to control group members to estimate the impact of the intervention on enrollees’ education, employment, benefit, and other outcomes. Unadjusted comparisons of the treatment and control groups will also yield unbiased impacts estimates, but regression adjustment will improve the precision of the estimates and guard against potential small-sample imbalances. Mathematica will also analyze the information from the staff interviews to report the findings and their potential program and policy implications. Mathematica will not use complex quantitative techniques to analyze the qualitative data from these collections. Products resulting from information obtained in this data collection will provide SSA and its stakeholders with information about recruitment, enrollment, service provision, and the fidelity of implementation.
SSA is not requesting an exception to the requirement to display an expiration date.
SSA is not requesting an exception to the certification requirements at 5 CFR 1320.9 and related provisions at 5 CFR 1320.8(b)(3).
Berlin, Martha, et al. “An experiment in monetary incentives.” Proceedings of the Survey Research Methods Section of the American Statistical Association. Alexandria, VA: American Statistical Association, 1992.
Damschroder, Laura J., David C. Aron, Rosalind E. Keith, Susan R. Kirsh, Jeffery A. Alexander, and Julie C. Lowery. “Fostering Implementation of Health Services Research Findings into Practice: A Consolidated Framework for Advancing Implementation Science.” Implementation Science, vol. 4, 2009, pp. 50–65.
Dillman, D. A. “Is social exchange theory still relevant for explaining why people respond to surveys.” AAPOR Annual Conference, Anaheim, CA. 2014.
De Heer, W., and E. De Leeuw. “Trends in household survey nonresponse: A longitudinal and international comparison.” Survey nonresponse, vol. 41, 2002, pp. 41-54.
Jäckle, Annette, and Peter Lynn. “Respondent Incentives in a Multi-Mode Panel Survey: Cumulative Effects on Nonresponse and Bias.” Colchester, UK: Institute for Social and Economic Research, University of Essex, 2007.
James, Jeannine M., and Richard Bolstein. “The Effect of Monetary Incentives and Follow-Up Mailings on the Response Rate and Response Quality in Mail Surveys.” Public Opinion Quarterly, vol. 54, no. 3, autumn 1990, pp. 346–361.
Kay, Ward R. “The Use of Targeted Incentives to Reluctant Respondents on Response Rates and Data Quality.” Proceedings of the American Association for Public Research. Montreal, Canada: American Association for Public Opinion Research, 2001.
Mercer, Andrew, et al. “How much gets you how much? Monetary incentives and response rates in household surveys.” Public Opinion Quarterly vol. 79, no.1, 2015, pp. 105-129.
Schwartz, Lisa K., Lisbeth Goble, and Edward M. English. “Counterbalancing Topic Interest with Cell Quotas and Incentives: Examining Leverage-Salience Theory in the Context of the Poverty in America Survey.” Proceedings of the American Association for Public Research. Montreal, Canada: American Association for Public Opinion Research, 2006.
Singer, Eleanor, and Cong Ye. “The use and effects of incentives in surveys.” The ANNALS of the American Academy of Political and Social Science, vol. 645, no. 1, 2013, pp. 112-141.
Singer, Eleanor, and Richard A. Kulka. “Paying Respondents for Survey Participation.” Ann Arbor, MI: The University of Michigan, Institute for Social Research, Survey Research Center, 2000.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | OMB Part A: Justification |
| File Modified | 0000-00-00 |
| File Created | 2024-10-07 |