Revised Supporting Statement A - CAPE Youth Final

Center for Advancing Policy on Employment for Youth (CAPE-Youth) Data Collection

OMB Control Number 1230-0NEW

OMB Expiration Date:

SUPPORTING STATEMENT FOR

Center for Advancing Policy on Employment for Youth (CAPE-Youth) Data Collection

OMB CONTROL NO. 1230-0NEW

This ICR seeks to obtain approval for a new data collection to support the CAPE-Youth services.

1. JUSTIFICATION

1. Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

In FY 2020 DOL’s ODEP funded a four-year cooperative agreement for CAPE-Youth. The Council of State Governments develops and manages the CAPE-Youth services in collaboration with the K. Lisa Yang and Hock E. Tan Institute on Employment and Disability at Cornell University and the Transitions to Adulthood Center for Research at the University of Massachusetts Medical School. CAPE-Youth focuses on supporting State efforts to align State workforce systems to establish pathways toward careers and financial self-sufficiency for youth and young adults with disabilities (Y&YAD). The ultimate goal of CAPE-Youth is to improve transition and employment related outcomes for Y&YAD through the identification and dissemination of evidence-based practices. To help achieve this goal, CAPE-Youth is implementing a rapid cycle research strategy with three primary steps: (a) identify the needs of workforce systems; (b) conduct research and policy questions and assessment that will help address those problems; and (c) based on the findings of research and assessment, develop and disseminate resources to policymakers and key stakeholders.

CAPE-Youth is supporting Y&YAD in three main focus areas:

1. Youth and adult workforce systems collaboration;

2. Professional development for youth workforce practitioners; and

3. Career pathways and work-based learning such as apprenticeships programs for Y&YAD.

This information collection, which will be done by CAPE-Youth, is intended to accomplish initial information gathering to better support planned future efforts, and will build upon existing research to support the development of products and services for supporting training and employment for youth and young adults with disabilities. CAPE-Youth will use this data collection initially to identify the extent to which promising practices or barriers exist in: 1) Improving collaboration between participants and providers in the youth workforce system, 2) Identify potential Professional Development (PD) areas that practitioners need in their repertoire to serve youth, and 3) Identifying perceived factors or capabilities that may improve the provision of pre-Employment Transition Services (Pre-ETS) for Y&YAD. Understanding the perceptions of workforce system professionals will help identify barriers and promising practices, and allow CAPE-Youth to develop relevant tools, briefs, and technical assistance that better support their ability to build communities of practices and forge connections across States for the State clients they serve.

This package includes a variety of information collection instruments to help develop a formative understanding that will best support the generation of useful technical assistance and identification of promising practices:

Collaboration:

1. Introductory Email

2. Phase 1 Consent Form

3. Phase 1 Questionnaire

4. Phase 2 Consent Form

5. Phase 2 Questionnaire for Subset of Phase 1 Respondents

Professional Development:

1. Outreach Email

2. Participant Confirmation Email

3. Phase 1 Group Concept Mapping (GCM) Consent Form

4. Study Sign-Up

5. Demographic Questionnaire

6. Phase 1 Questionnaire

7. Phase 2 Consent Form

8. Phase 2 Instruction and Reminder Emails

Pre-ETS:

1. Phase 1 Recruitment Letter

2. Phase 1 Fact Sheet

3. Phase 1 Questionnaire

4. Phase 2 Recruitment Letter

5. Phase 2 Fact Sheet

6. Phase 2 Questionnaire for Subset of Phase 1 Respondents

2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.

The purpose of the CAPE-Youth data collection will be to better understand 1) perceptions of the extent and benefit of collaboration in the workforce system; 2) perceptions among youth workforce professionals about their professional development and training; 3) administrative data on student demographics to understand the general characteristics of program participants receiving Pre-ETS services and 4) the perceptions of the implementation of those administering Pre-ETS. CAPE-Youth will use this data to inform the development of products and services for States and service providers for Y&YAD in conjunction with ODEP. CAPE-Youth will develop products and briefs based on the data collected, and identify promising practices based on existing administrative data collections and research, and the responses from questionnaires included in this package. As noted above, the data collection will focus on three areas: 1) collaboration within States among workforce system agencies; 2) the implementation of Pre-ETS by State Vocational Rehabilitation Agencies and gaps and perceived strengths and weaknesses in the provision of services for youth with disabilities; and 3) the strengths and weaknesses of professional development for youth services providers.

1. Collaboration research: Workforce Innovation and Opportunity Act (WIOA) places renewed emphasis on meaningful collaboration between the public workforce development system, vocational rehabilitation (VR), and other entities to ensure that “a full range of services is available, regardless of disability or cultural background.” To appropriately respond to this renewed emphasis on meaningful collaboration, States have urgent needs for information, resource, and best practices in technical assistance to establish effective and efficient career pathways and financial self-sufficiency for Y&YAD.

It is important to acknowledge that although interagency collaboration in transition-to-adulthood is a commonly recommended practice, very little research has been done that develops an objective and solid empirical basis for recommending collaboration, including whether collaboration leads to improved outcomes, and identifying the specific agency and interagency policies and practices that lead to improved outcomes. In the area of transition to adulthood for youth with disabilities, interagency collaboration entails both formal and informal relationships between youth and adult systems, in which communications, information, and resource sharing help achieve joint transition goals and service coordination (Noonan, Morningstar, & Erickson, 2008; Test, Fowler, White, Richter, & Walker, 2009). There is currently little consistency in how collaboration occurs and who is involved, and there is a need to understand which approaches appear to be correlated with more favorable outcomes that would provide valuable, formative data to help guide ODEP and DOL’s Chief Evaluation Office in appropriately selecting models for more rigorous testing.”

The assumptions and constructs that underlie collaboration as a best practice still need refinement. In collaboration research more broadly, a recent systematic review revealed a tendency to focus more on antecedents and outcomes of collaboration, and less on the processes exhibited in strong collaborations (Gazley & Guo, 2017). Even task-oriented measures of collaboration often lack granular detail about effective interventions in practice, such as what minimal level of interagency communication is “frequent” or how best to structure formal collaborative agreements (such as memoranda of understanding/agreement) and resource sharing practices. This is a barrier to making specific, evidence-based recommendations on collaboration.

This formative study will gather and explore collaboration factors in more depth and determine if it is possible to identify factors and processes of strong collaborations. This study will gather information assessing the characteristics of relationships among required WIOA partners (VR, Workforce, and Education), as well as other essential systems like developmental disability and mental health agencies, juvenile justice, foster care, faith-based organizations, school re-engagement programs, and mentoring programs. The study will take place in two phases. Phase 1 will involve questionnaires provided to State staff in all fifty U.S. States, Washington D.C., and Puerto Rico. The questionnaires will leverage pre-validated “Levels of Collaboration” (LCS) measures (Frey et al., 2006), as well as additional Likert-type questions developed by the research staff to identify States with perceptions of successful collaborations. In Phase 2, this data will be compared with RSA-911 data on VR outcomes. The RSA-911 consists of VR case management data that are submitted quarterly by each State to the Rehabilitation Services Administration.

Phase 2 will build upon the results of the questionnaire data from Phase 1, to select up to ten States with perceptions of successful collaborations to participate in a series of more detailed questionnaires. The selection of the ten Phase 2 States will be based on the Phase 1 responses to the questionnaires, using criteria identified prior to Phase 2 project initiation. The selection of the ten States will include considerations of: (a) diversity of States/territories represented in the Phase 2 qualitative study (e.g., Federal region, State population characteristics, VR agency type); and (b) State systems that indicate higher levels of collaboration as well as employment/referral outcomes (as measured or proxied by RSA-911 measures, VR exits at employment, percentage referrals from other agencies, and levels of service receipt).

The questionnaires in Phase 2 will gather information from supervisors and practitioners at State level agencies to explore the practices and processes that embody successful collaborations, as well as challenges and strategies arising from the current COVID-19 pandemic and economic recession. The second phase of this project will be informed by Phase 1 of the study and explore State-level collaborations by focusing on supervisor-level stakeholder feedback, as well as local-level collaborations with community-based organizations and direct service providers from various agencies.

Ultimately, this study on collaboration will attempt to identify and initially describe existing levels of interagency collaboration, the presence of any common elements among States exhibiting success in coordination, any potential promising practices, and document perceptions of the respondents on the impact of COVID-19 and financial downturn on collaboration efforts.

2. Pre-ETS Research: The 2014 reauthorization of WIOA specified that State agencies of VR provide five Pre-ETS” to students with disabilities. Pre-ETS is a model of early short-term vocational readiness services designed to enhance competitive integrated employment opportunities for students with disabilities. A unique and important aspect of the law was the requirement that State agencies providing vocational rehabilitation services offer Pre-ETS not only to students who are eligible for VR services (their historical client base) but also to students who are potentially eligible for VR services by virtue of the student having a disability. WIOA mandated that VR reserve 15% of their Federal VR State grant allocation to provide Pre-ETS to students and potentially eligible students with disabilities.

This legislation has greatly expanded opportunities for youth with disabilities to receive employment readiness services. However, there is scant information related to VR’s outreach and delivery of Pre-ETS to various, eligible populations of students with disabilities across States, and if States are using promising practices to achieve comprehensive and inclusive service delivery and improve post-secondary outcomes.

Based upon currently available data and research on Pre-ETS, areas that benefit that need research and clarification include:

• whether and how Pre-ETS are delivered to underserved or under-resourced students,

• whether and how Pre-ETS are delivered to students who are eligible and potentially eligible for VR services;

• whether there are other disparities in service provision such as by race/ethnicity or disability and,

• what promising practices exist for post-school success and the identification and engagement of students in Pre-ETS

The purpose of this data collection using questionnaires and case studies is to address these gaps in knowledge. In addition, this study will leverage extant VR case data to supplement the needed information and further reduce burden.

Professional Development and training for youth service providers (YSP): The current domestic programmatic landscape consists of a variety of programs and services that support the educational, employment, and independent living goals of youth and young adults with disabilities. The services that each program delivers are contingent on funder requirements and can consist of activities such as school preparedness, mentoring, job skills training, job placement, on-the-job-training, long-term supports, and career development. For these programs to deliver comprehensive and effective services to youth with disabilities, youth serving professionals must be equipped with the knowledge, skills, and abilities to meet the needs of youth. The Guideposts for Success: Framework for the Future (Center for Advancing Policy on Employment for Youth [CAPE-Youth], 2020) indicates the components of service delivery that organizations and professionals need to provide to improve transition outcomes for youth and young adults.

However, further research is necessary to identify whether and how YSPs receive training in these competency areas during pre-service career preparation as well as through ongoing professional development (PD) and training provided through their employers, to effectively serve and support youth in preparing for, obtaining, and maintaining competitive integrated employment.

This research will identify the range and depth of the needs, demand, and priorities for PD, from the perspective of YSPs and other related professionals. Information gained through this proposed two-phase research study will help clarify what professionals perceive is successful in their context along with an improved understanding of relationship development, goal attainment and effectiveness; and the existence of any gaps in the content currently delivered.

Phase 1 will provide questionnaires to youth serving professionals and those with knowledge of the context from other perspectives. Phase 1 is expected to produce an initial understanding and present perspective of the value of each specific element of professional development. Participants will provide feedback on whether these elements are present or not in their experience. This set of results will serve as the basis for selecting participants for Phase 2 which will examine through questionnaires deeper issues including YSP engagement in PD development, access gaps among participants, sources of PD, and perceptions of PD.

Findings from this study will enhance the information available to develop relevant and timely resources to help strengthen and align preparation and ongoing PD across the myriad systems serving youth and young adults with disabilities. This will provide additional information for State and local policymakers and program developers to consider when selecting, implementing, and evaluating comprehensive and responsive PD programming. This information will enhance users’ ability to select, coordinate, and develop PD across diverse youth service professional settings, while also providing a framework to better meet the needs of youth and ultimately enhance the quality of employment outcomes for youth.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also, describe any consideration of using information technology to reduce burden.

The data collected via questionnaires will be done using an online platform. Questionnaire items will be entered in the Research Electronic Data Capture (REDCap) database, a HIPAA-compliant online data management system.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item A.2 above.

1. There are no existing national data on the perceptions of youth workforce systems professionals describing the extent to which they are fulfilling WIOA mandates to collaborate within the youth workforce system. There is no extant data on the perceptions of how successful any collaborations are. The only relevant data that exists is within the RSA-911 data collection, and pertains to collaboration and referral rates to other agencies. However, this data is insufficient alone to identify perceptions of successful collaborations among and within States. ODEP believes the qualitative data collected through this study will provide valuable context for policy makers and help identify promising practices and lessons learned by State level practitioners, and that this data would prove valuable in supporting data-informed decision making, policies and strategic planning.

2. While the RSA-911 data set provides valuable information, as an administrative database, there is limited descriptive data on Pre-ETS implementation. Notably, we are unable to examine characteristics of Pre-ETS recipients who are potentially eligible, a new population of students who are allowed to receive services since the 2015 reauthorization of WIOA. The RSA-911 data set does not provide insight into the steps VR agencies may be taking to reach out to partner agencies, schools, or local organizations to identify potentially eligible students. Gathering a broader base of data will provide a baseline on the scope of Pre-ETS implementation, which can be re-measured in later years to gain a qualitative sense of progress in serving potentially eligible students.

3. There is no existing national data on the perceptions of youth service providers as it pertains to the skills they need to be successful in serving youth compared to the professional development they receive. The qualitative data collected will provide valuable context for policy makers and help identify promising practices and lessons learned by State level practitioners in the implementation of professional development or training.

5. If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.

This data collection will not impact small business, the respondents are public employees in State and local workforce agencies.

6. Describe the consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

Without this data collection, State and local service providers will not be able to optimize their outreach, organizational structures and operations, and ODEP will be limited in its ability to provide high quality technical assistance to States to improve services and employment outcomes for youth and young adults with disabilities. A 2018 GAO report found that majority of States reported that additional assistance with identifying best practices for Pre-ETS would be useful to their agencies (Ref: https://www.gao.gov/assets/gao-18-502.pdf)

7. Explain any special circumstances that would cause an information collection to be conducted in a manner:

• requiring respondents to report information to the agency more often than quarterly;

• requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;

• requiring respondents to submit more than an original and two copies of any document;

• requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years;

• in connection with a statistical survey, that is not designed to produce valid and reliable results that can be generalized to the universe of study;

• requiring the use of statistical data classification that has not been reviewed and approved by OMB;

• that includes a pledge of confidentially that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or

• requiring respondents to submit proprietary trade secret, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information's confidentially to the extent permitted by law.

None

8. If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the agency's notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments. Specifically address comments received on cost and hour burden.

Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported.

Consultation with representatives of those from whom information is to be obtained or those who must compile records should occur at least once every 3 years -- even if the collection-of-information activity is the same as in prior periods. There may be circumstances that may preclude consultation in a specific situation. These circumstances should be explained.

The 60-Day Federal Register Notice inviting comment on this information collection was published on November 24, 2020 in 85 FR 75039 on pages 75039-75040. This publication can be found here: https://www.Federalregister.gov/documents/2020/11/24/2020-25937/agency-information-collection-activities-comment-request-center-for-advancing-policy-on-employment. No comments were received in response to this notice.

CAPE-Youth and ODEP conducted outreach to the Council of State Administrators of Vocational Rehabilitation and select State agencies on their perceptions of the suitability and availability of the data to be collected, and the burden associated with collecting the information. No comments were provided by any of the stakeholders in this study.

9. Explain any decision to provide any payments or gifts to respondents, other than remuneration of contractors or grantees.

None

10. Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy.

There is no assurance of confidentiality. Respondents will be informed of the voluntary nature of the questionnaires, and of the privacy provision in the initial cover letter and on the questionnaires, stating that their responses may be used for statistical purposes only and will not be disclosed, or used, in identifiable form for any other purpose except as required by law. Respondents are also informed of the general risk inherent in supplying information over the Internet. In addition, participants will be provided with the telephone number and e-mail addresses of the Project Manager and Principal Investigator, as well as the Institutional Review Board for Cornell and the University of Massachusetts, if they have any questions or concerns.

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private. This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

Not applicable.

12. Provide estimates of the hour burden of the collection of information. The Statement should:

• Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated. Unless directed to do so, agencies should not conduct special surveys to obtain information on which to base hour burden estimates. Consultation with a sample (fewer than 10) of potential respondents is desirable. If the hour burden on respondents is expected to vary widely because of differences in activity, size, or complexity, show the range of estimated hour burden, and explain the reasons for the variance. General, estimates should not include burden hours for customary and usual business practices.

• If this request for approval covers more than one form, provide separate hour burden estimates for each form.

• Provide estimates of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories. The cost of contracting out or paying outside parties for information collection activities should not be included here. Instead, this cost should be included in Item 14.

1

Estimated Annualized Respondent Cost and Hour Burden

Activity	No. of Respondents	No. of Responses per Respondent	Total Responses	Avg. Burden (Hours)	Total Burden (Hours)	Hourly Wage Rate*	Total Burden Cost
State Collaboration Research	1120	1	1120	0.76	851.2	$33.41	$28,438.59
Pre-Employment Transition Services Implementation Study	103	1	103	0.62	63.86	$33.41	$2,133.56
Professional Development for Youth Workforce Practitioners	400	1	400	1.63	652	$33.41	$21,783.32
Total	1623	1	1623	0.97	1567	$33.41	$52,355.47

* This is the hourly mean wage of Management Analysts employed by State governments based on the May 2020 Occupational Employment Statistics. https://www.bls.gov/oes/current/oes131111.htm

13. Provide an estimate of the total annual cost burden to respondents or recordkeepers resulting from the collection of information. (Do not include the cost of any hour burden shown in Items 12 and 14).

• The cost estimate should be split into two components: (a) a total capital

and start up cost component (annualized over its expected useful life); and (b) a

total operation and maintenance and purchase of service component.

The estimates should take into account costs associated with generating,

maintaining, and disclosing or providing the information. Include descriptions of

methods used to estimate major cost factors including system and technology acquisition, expected useful life of capital equipment, the discount rate(s), and the time period over which costs will be incurred. Capital and start-up costs include, among other items, preparations for collecting information such as purchasing computers and software; monitoring, sampling, drilling and testing equipment; and record storage facilities.

• If cost estimates are expected to vary widely, agencies should present ranges of cost burdens and explain the reasons for the variance. The cost of purchasing or contracting out information collection services should be a part of this cost burden estimate. In developing cost burden estimates, agencies may consult with a sample of respondents (fewer than 10), utilize the 60-day pre-OMB submission public comment process and use existing economic or regulatory impact analysis associated with the rulemaking containing the information collection, as appropriate.

• Generally, estimates should not include purchases of equipment or services, or portions thereof, made: (1) prior to October 1, 1995, (2) to achieve regulatory compliance with requirements not associated with the information collection, (3) for reasons other than to provide information or keep records for the government, or (4) as part of customary and usual business or private practices.

None

14. Provide estimates of the annualized cost to the Federal Government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses (such as equipment, overhead, printing, and support staff), any other expense that would not have been incurred without this collection of information. Agencies also may aggregate cost estimates from Items 12, 13, and 14 into a single table.

The total estimated cost to the Federal Government is expected to be $372,458 spent over three years, and therefore the estimated annualized cost for this information collection and the analysis and reporting is estimated to be $124,153 per year, over those three years. See the table below for more details:

15. Explain the reasons for any program changes or adjustments.

None, this is a new data collection.

16. For collections of information whose results will be published, outline plans for tabulations, and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

Data collection for this project will take place during 2021. CAPE-Youth will review the data and analysis results, in conjunction with ODEP, and develop a series of policy briefs, webinars, policy tools, and reports that would be published and disseminated until the end of the cooperative agreement. All of the data collected in questionnaires from the three studies will be used to help inform secondary phases of the research.

The Pre-ETS study, the Collaboration Study, and the Professional Development study will not require the use of statistical analysis or inference and will only be represented as aggregate and descriptive statistics. The data collected will be descriptive in nature and used to help determine a baseline for current policy implementation and perceptions of implementation.

17. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

None

18. Explain each exception to the certification Statement.

N/A

11