Summary of 60-day Federal Register Notice (FRN) Comments and Responses

WIOA PERFORMANCE ACCOUNTABILITY ICR ICR REFERENCE #xxxx-xxxx

OMB Control No. 1205-0521

60-Day FRN Public Comment and Agency Response

Department of Labor

Information Collection Request

Workforce Innovation and Opportunity Act (WIOA)

Performance Reporting System

Summary of 60-day Federal Register Notice (FRN) Comments and Responses

Executive Summary (Overview of Document)

The Department of Labor (DOL) published a 60-day Notice in the Federal Register (FRN) seeking public comments concerning proposed extension and revision for the authority to conduct the information collection request (ICR) titled, “Workforce Innovation and Opportunity Act (WIOA) Performance Accountability, Information and Reporting System.” on November 25, 2020 (OMB ICR Reference Number 1205-0521).

This document provides a summary of the 6 sets of public comments received in response to the 60-day comment Notice on the DOL only ICR and the Department’s responses to those comments. The Department has organized this document by issues raised by the particular forms of the ICR documents and by programs.

The comments are organized into 2 sections as outlined in the table of contents below. Each comment contains a comment number in the first column. The actual comment received appears in the second column. The third column provides the agency’s response. Finally, the fourth column indicates whether or not a revision to the ICR forms was made as a result of the comment. For the Apprenticeship comments, the Department’s Office of Apprenticeship responded to all of the comments in a separate word document which was embedded as a ‘response’ to the comments accordingly.

TABLE OF CONTENTS (Hyperlinked to each section)

DOL-ONLY PIRL

AMENDED DOL-Only ICR – COMMENT RESPONSES
#	COMMENT	DEPARTMENT RESPONSE	ICR Form Revision (if applicable)
DOL-ONLY PIRL
	(Texas) DOL has proposed a large number of changes to the 9172. While some are relatively minor clarifications to existing elements (updating details to match current practices), others are more substantive – including adding dozens of new data elements that will cost significant amounts of money to implement in grantee case management systems. Even the proposals to make existing elements required for additional programs will require programming changes minimally in the code that produces the PIRL and possibly in the case management system if an element is not currently tracked for that program and there is no existing way to enter it. We therefore urge DOL to consider whether the value of gathering additional data is worth the cost. In most cases, the cost of adding a new element to a case management system and to the PIRL coding is largely the same whether applicable to <1% of participants or to nearly all participants. We recommend that DOL reconsider whether these changes are really necessary and worth the cost given that while DOL estimates a cost to implement/comply with the ICR, it doesn’t provide additional money to grantees for this purpose. Between biennial updates to the PIRL, three-four times per year edit changes, and new quarterly data integrity improvement efforts that often require programming to change, grantees are having to dedicate ever-greater amounts to capturing and reporting data that may be of limited value and this comes at the expense of funding for other activities. We are particularly concerned about some of the larger changes, such as updating the ONET codes used in reporting occupational outcomes and the proposal new element for reporting whether individualized services were provided in person or remotely. While we see the value of both of these pieces of information, the fact is that many states may not have been tracking this and will not be able to report the data retroactively. Bigger changes like this have both programming and reporting challenges. On the question of in person vs. remote service, for example, our system doesn’t track in person vs remote service separately. If we report on a person who has exited prior to this going into effect, we won’t be able to answer this question affirmatively as yes or no. because of this. Talking to states that use some of the larger vendor products, it seems like we’re not alone, as many of those products don’t do this tracking either. As to the effective date of changes, we face a different problem. It appears that DOL intends to implement this proposed PIRL effective July 1, 2021 but we won’t know what the final requirements are until May or June given that this has to go back out for public comment again and those comments could result in removal, addition, or modification of proposed changes. We recommend that changes ultimately adopted through this ICR not be made mandatory until July 1, 2022 and that they not be applied retroactively to customers who exited prior to the changes going into effect. The delayed implementation will provide time for states to make changes and the non-retroactivity will ensure that states are not required to go back and try to gather data that would normally have been gathered during either the eligibility or service delivery phases of participation. What follows are comments on specific proposed changes including other recommendations for DOL PIRL elements that should be considered.	Thank you for your comment. DOL will issue additional guidance on the timing of the adoption of these changes.	N/A
	(Texas Comments #1-#53) Element 401 UC Eligible Status – ADDITIONAL RECOMMENDATION - DOL has indicated that they plan to administer the new Reemployment Service and Eligibility Assessment (RESEA) program measures using the PIRL. However, those measures depend in part on identifying who was an RESEA Participant. While Element 401 indicates if a person was referred by RESEA, it does not indicate if that person actually received all the RESEA-required services. A person might have come in and only received some of the services or might have been a Wagner-Peyser Participant prior to becoming a UI claimant and selected for RESEA and elected to not participate in RESEA. There are two ways to address this problem: Modify the response options in this element to indicate if RESEA Referred AND Participating or RESEA Referred but NOT Participating, OR Add a new RESEA Program Participation element to the 900 series of elements that specifies that the person was both referred for RESEA services AND received the required services.	The Department agrees with the commenter and has modified the definition of the PIRL data element.	ETA 9172- DOL only PIRL
	Element 808 MSFW – proposes element name change with additional language clarifying intent of the element and matches proposed Common PIRL change – Support Element 1301 – Support – clearly a typo correction and not change in the requirements. Element 1304 ETP Program of Study by Potential Outcome – Support – This is just an element Name change and has no functional impact on what and how the element is reported. Element 1406 Post Exit- Enrollment in PSE – Support – Non-substantive, clarifying change.	Thank you for your supportive comments.	N/A
	Element 915 TAA Petition Number – Need Clarification – the proposal is to remove language instructing states to create multiple PIRL rows on TAA customers who have more than one applicable TAA Petition Number (primarily when in the same Period of Participation). What’s not clear is if this is to bring the element in line with current reporting guidance (and thus not a substantive change) or if it represents a new policy for reporting. If the latter, please provide additional guidance or clarification about what to states should do when reporting a Participant who has more than one applicable TAA petition.	Thank you for your comment. This revision brings the element in line with current reporting guidance by removing erroneous reporting instructions provided under old reporting mechanisms. This does not represent a change in in reporting guidance from the current PIRL reporting.	N/A
	Making Elements 1000, 1001, and 1201 mandatory for TAA – Ambivalent – there may be cost to modifying PIRL coding to populate these elements for TAA-only Participants but this should be an easier change than some of the other proposals to make new data required for new programs. Making Element 1103 mandatory for REO Adult, 1004 and 1210 mandatory for REO Youth and 1205 mandatory for both REO programs – Ambivalent – there may be cost to modifying PIRL coding to populate these elements for REO-only Participants but these should be an easier change than some of the other proposals to make new data required for new programs. Element 1300 Received Training – Ambivalent – proposes to add “as defined by program specific guidance” to the “Received Training = Yes” definition – Support – This is largely as DOL has been operating and therefore doesn’t represent a meaningful change.	Thanks you for your comment. The Department strives to keep burden down while ensuring cross program alignment to facilitate easier common reporting.	N/A
	Modifying Element 1114 Referred to Jobs for Veterans State Grants Services to have an additional value for Vietnam-era veterans – Concerned – DOL has provided no context for this change. Why Vietnam-era veterans only? What about veterans of the original Gulf War or more recent military conflicts? This will likely required both IT case management costs (will need to now capture the date of entry into military service for people who left the military after May 1975 since they may have served during the Vietnam-era) and PIRL changes.	Veterans’ Program Letter No. 03-19 added “Vietnam-era veterans” as a sub-population who are eligible to receive services from Disabled Veteran Outreach Program specialists. As such, it is important for VETS to be able to separate out participants who fall into this category for analytic purposes.	N/A
	Elements 1306, 1311, 1316, 1612, and 1613 (related to updating ONET codes used in reporting) – Very Concerned – there could be significant costs to implementing this data in case management systems depending on how those systems were created and the degree to which there may be secondary systems or functions that are based on the older standards.	The Department appreciates the challenges associated with making this change and will therefore implement a phased implementation of this change in order to allow states time to determine how best to make these adjustments. To accomplish this, the Department will continue to allow older ONET codes via its edit checks during the transition period.	N/A
	Element 1322 – Not clear why the name is proposed to be changed to not align to the other “Date of Most Recent ____________ Service” data elements. Recommend either not changing or changing all the other applicable elements to this naming convention.	Accepting recommendation to align name to include “Date of…”	ETA 9172- DOL only PIRL
	Element 1328 Training Provided Virtual/Online (previously titled “Distance Learning”) – Very Concerned – we understand the value of this and while the addition of a “Mix of In Person and Online” option might seem easy – the reality is that it may not be. TWC has tracked remote vs in person training data but at the Eligible Training Provider Program-level. Students did one or the other. COVID-19 obviously caused many programs to shift from in person to online, but we don’t have a means to track that. Texas has hundreds of Eligible Training Providers with several thousand approved programs – trying to retroactively find out which classes were being offered in person vs online at different points in time is not feasible. We can only support this if the new “both” value is applicable to training conducted after the effective date of the element change.	DOL concurs with this element only be required for training occurring after the effective date of the element change, but states may voluntarily report “both” on prior training if the data is readily available.	N/A
	Element 1332 Participated in PSE during Participation – proposes to add “clarifying” language to the “No PSE” response and to eliminate the option to record “blank” if not applicable. The additional clarifying language is essentially restating the existing language to try to clarify intent – Support Removal of the option to leave blank if not Applicable – OPPOSE – This data element is used to drive the Credential Rate denominator. The Credential Rate is not applicable not applicable if a participant in a WIOA program is also enrolled in PSE unrelated to their services from workforce system (other than for Title I in-school Youth). For example, if we have a person enrolled in WIOA Adult who happens to be in PSE during participation but we are not paying for that education directly or with support payments and are instead only serving to help the person find employment while in school, then that person should not be in the Credential Rate denominator. The proposed change requires the state to record a yes or no on every person, even those for whom the element is not applicable such as in the example identified above. Without the blank if not applicable option, many states and locals may have confusion about how to respond. If they answer “Yes” to on a record like the above WIOA Adult example, then that will put the person in the Credential Rate denominator even if the participant was early in their education and only needed school-compatible employment assistance. This element should be entirely written as follows: Record 1 if the participant was in a postsecondary education program that leads to a credential or degree from an accredited postsecondary education institution at any point during program participation and the program provided services that directly supported that enrollment. Record 0 if the participant was not in a postsecondary education program that leads to a credential or degree from an accredited postsecondary education institution during program participation OR was in such education but the program did not provide services that directly supported that enrollment. Note: this includes if the participant was enrolled in a postsecondary education program that does not lead to a credential or degree from an accredited postsecondary education institution at any point during program participation. Note: This data element relates to the credential indicator denominator and those who are recorded as 1 are included in the credential rate denominator. This element is a subset of PIRL 1811. Do not record 1 if the participant was first enrolled in postsecondary education after exiting the program OR if the program was not directly supporting the post-secondary education.	The Departments have agreed to add back in a “leave blank” option into 1332 to ensure that all possibilities are appropriately accounted for. The language will read “Leave blank if the participant was not in a postsecondary education program, as defined in program specific guidance.”	ETA 9172- DOL only PIRL
	Making Elements 1403 and 1405 applicable to REO Adult and REO Youth programs and Elements 1408 and 1415applicable to REO Youth – Concerned – Are the required system change costs worth the value of these elements if grantees are not currently capturing this information on these programs?	Thank you for your comment. The purpose of making these items applicable to our adult and youth reentry programs is to have our grantees report on these items, so if the items are approved grantees will capture and report on them. PIRL 1403 is for Alternative School Services. This item will allow us to document in both our adult and youth reentry grants the number of participants who are provided high school equivalency classes. 40 percent of participants in our adult offender grants over the years did not have a high school diploma or equivalency and a quarter of participants in our recent youth offender grants that included data on school status were high school dropouts. PIRL 1405 is for work experience. Given the immediate need for many adult offenders and older youth offenders to find a job to support themselves work experience and transitional jobs are an important way the reentry grants can serve these individuals, and the DOL should be able to document how many individuals we have provided work experience and transitional jobs. PIRL 1415 is for post-secondary education and preparatory activities. DOL should be able to document the extent to which reentry programs have provided college bound services to young offenders. PIRL 1408 is for leadership development activities. PIRL 1408 is already in the youth offender schema and we are requesting that it also be applicable to the adult reentry grants because it is a valuable service, and the DOL should be able to document the extent it has provided this service to adult offenders.	N/A
	Propose to change Elements 1506, 1507, 1509, 1510, 1514, 1515, 1519, 1520, 1524, 1525, 1529, 1530, 1536, and 1538 from reporting amounts “paid” to “expenditures accrued” – Need Clarification – “Paid” and “accrued” have very different meanings in accounting. If the change is nonsubstantive and merely line up the language with the existing guidance, please provide reference to that guidance in response to public comments. However, if these change represent actual changes to what is to be reported (accrued vs. paid), we OPPOSE the change as being too costly to implement in accounting systems, benefit systems, case management systems, and PIRL coding which would all need to be modified for a change that does not clearly provide information that is essential and superior to the “paid” amounts reported currently.	This language change from “paid” to “expenditures” accrued is to align with current guidance on how to populate these fields throughout TAPR and PIRL. Accrual reporting is required under 20 CFR 618.860, TEGL 01-19, TEGL 06-09 (including Change 1 and Change 2), and has long been discussed in technical assistance materials such as webinars and reporting FAQ sheets available to states.	N/A
	Element 1542 A/RTAA Overpayment – OPPOSE – given that WIPS is a longitudinal data system, it is not clear why this change is necessary. It is possible for DOL to determine whether there was an overpayment in a POP by looking at each PIRL submission rather than 50+ states, territories, and Washington DC all having to make a programming change. It would be far cheaper for DOL to use the data they already get to track overpayments. DOL would only have to write or modify one query and apply it to all the PIRL submissions they’ve received.	The DOL understands this overpayment element to be necessary to track what kinds of participants are receiving overpayments to better provide guidance on how to avoid overpayments in a program that has long received concerns about overpayment risk. This element has been consistent for TAA participant reporting since FY 2009.	N/A
	Element 1608 Employment-Related to Training Q2 – Added the ability to record “unknown” in the element – STRONGLY SUPPORT – Few states receive occupational data as part of their wage records which means that the only way to report this element is through follow-up contacts with the Job Seeker or perhaps the Employer. Given that we must wait until the 2nd quarter after exit to contact the Exiter to obtain the information, sometimes we will not have current contact information and be unable to contact the Exiter. Contacting the employer to obtain information about the occupation the Exiter is employed in adds another 2+ months to the lag because of the delay in receiving wage records and TWC does not support contacting employers to obtain information about their employees absent a true statutory purpose like to determine unemployment eligibility information. Further, employers do not want to be contacted for WIOA performance accountability purposes. That was made perfectly clear to the Departments when they sought employer input on the WIOA “Effectiveness in Serving Employers” measure. While this element is unrelated to the employer measures, the underlying message – “don’t call us, we’ll call you” – is still applicable. HOWEVER, we also think that the “Leave Blank” provision needs to be clarified to state that we also leave it blank if the Exiter was not employed in the 2nd Quarter After Exit (Q2). While this may not be the Departments’ intent, the currently proposed set of options does not provide guidance on what to record (1, 0, or 9) if the Exiter was not found employed in Q2 post-exit. All three options include the condition “and obtained employment” which makes none of the 3 options appropriate if the person was unemployed in Q2.	Thank you for your comment.	N/A
	1700 Series – ADDITIONAL RECOMMENDATION – given there is significant seasonality in employment and earnings by calendar quarter (Q4 tends to have the highest number of people employed and many people get second, holiday jobs which raises earnings for the quarter), we recommend that DOL add an additional Element for the 4th Quarter Prior to Participation Quarter. This would improve pre-post participation earnings employment and comparisons by ensuring that there is a full 4 quarters of both pre and post participation earnings available for analysis.	Thank you for this recommendation. In the interest of minimizing burden, DOL will not be collecting this data at this time, but notes that states may collect additional information they deem important to their program management.	N/A
	Elements 1703-1706 Earnings in Xth Quarter after Exit Quarter – Changing the word “Wages “to “Earnings” Support – This change aligns the elements better with the reality that we work with people who are self-employed or contractors or gig workers. The term “wages” is more limiting. In addition, the data element definitions/instructions always used the word earnings. In addition, we have two related comments to offer: First, there is no need to report 999999.99 when wages are “not yet available” and we recommend removing this language. The Departments of Education and Labor have set out a reporting schedule which identifies when earnings for a given quarter are deemed to be available and the Departments have data edit checks to make sure that states are not reporting earnings as “not yet available” when they clearly should be available based on the calendar. Therefore, there is no need for states to code the “999999.99- data not available” data. Further, it makes it impossible to actually report $999,999.99 as “real earnings” on the off chance that it occurs – reserving what is otherwise a valid number in a monetary field to represent something other than a monetary value is not a good data practice – it makes the data harder to use by introducing an additional data cleansing step. States should report nulls until the data is mature and if there are no earnings once the data matures, they should record a 0. This then leads to our second recommendation – add an extra digit to these fields as well as to the earnings prior to participation data elements. Yes, it is very rare that we serve a person who earns more than $1M in a given quarter but it does happen. It happened during the Great Recession when a lot of very highly compensated finance workers and executives were laid off or their companies closed. This is likely happening again with participants in the COVID-19 period obtaining extraordinarily high levels of earnings pre and post-exit. To be clear, this is a rarity for most states but some states with high concentrations of very highly paid workers (Wall Street, Silicon Valley, etc.) could have this legitimately show up in their records. In the absence of adding a digit, we request that the instructions be modified to explicitly define what we should report in an instance where the Exiter legitimately had earnings of more than $1M in a given quarter. Of note is that Element 2203 which is to report HOURLY wage at placement is allows reporting of up to $9,999,999.99.	Thank you for your comment. The Departments are increasing the field length to address the Commenters concern, but the Departments are keeping the option to report all 9’s when wages are “not yet available.”	ETA 9172 DOL only PIRL
	Elements 1811 and 1813 Date Enrolled (or Completed) Education or Training during Program Participation – STRONGLY SUPPORT Removal of the word “Postsecondary” from the element titles will greatly reduce confusion about the applicability of these elements to people in secondary education. The detailed instructions on how to record the element always made it clear that this included people enrolled in secondary education, but the element name didn’t seem consistent with that guidance. This made training and consistent coding/application of the business rules more complicated.	Thank you for your comment.	N/A
	Elements 1902, 1909, and 1916 Category of Assessment – Ambivalent – Not clear why the change is proposed, particularly the removal the ability to report a person being assessed in both ABE and ESL.	Thank you for your comment. These changes were made to better align with the most recent version of these assessments.	N/A
	Element 1905, 1908, 1912, 1915, 1919, and 1922 Pre and Post Test Assessment Scores – Support – these changes simply update the language to match current reporting policy. Elements 2003 and 2004 related to DWG – Support Element 2126 – Support – nonsubstantive naming change. Addition of Individualized Services Provided Virtual/Online – Support subject to the conditions noted in the introduction about providing time to implement the automation changes to track this and that it not be applied retroactively to periods in which it was not possible to identify whether a service was provided in person or online.	Thank you for your comment.	N/A
	Elements 2109-2113 related to Type of Training – OPPOSE – Why would DOL remove Registered Apprenticeship as an option to report on these elements given the great interest in Registered Apprenticeship as well as the significant amounts of funding that have gone to Apprenticeship Expansion Grants? Yes, there is some redundancy to Element 2207, but the values are currently reported and there is no cost to continuing to report them.	Please see response from DOL’s Office of Apprenticeship in the document below.	ETA 9172 DOL Only PIRL
	Making Elements 2203, 2204, 2207, 2209, 2210, 2211, 2212 applicable to REO Adult program (and RIO Youth for 2210 – OPPOSE on basis of cost to modify IT systems and PIRL coding to record/report these items.	Thank you for your comments. We believe that the value of making these items applicable to our reentry programs outweighs any costs for modifying IT systems and PIRL coding. Items 2203 and 2204 are for hourly wages and weekly hours of jobs in which reentry grants placed participants. Finding a job is the main reason offenders apply for our programs and DOL should be able to document the wages and weekly hours of the jobs in which we place individuals. Items 2209, 2210, 2211, and 2212 are for supportive services relating to transportation, health care, child care, and housing. Many ex-offenders are in dire financial situations and are in need of supportive services. Over the course of the years, 75 percent of participants in our adult offender grants have received supportive services and we should be able to document what type of supportive services that they have received. Child care is relevant because women make up almost a quarter of participants in our adult offender programs. Similarly, the value of making 2210 applicable to youth reentry programs outweighs any costs modifying IT systems because DOL should be in a position to document the number of participants receiving health-related supportive services. As a benchmark, 20 percent of participants in our adult reentry grants have received health-related supportive services.	N/A
	Element 2212 Housing Services – Support the name change but don’t understand removal of the option to report “both” since a person might initially receive temporary housing assistance and then also receive permanent housing assistance when an opportunity arises.	The Department agrees with the commenter and has added the option to report “both” back in the data element.	ETA 9172 DOL only PIRL
	All Other Section E.11 Elements – OPPOSE – No justification was provided for the need to add these data elements and while they might be useful in some regard, it is not clear for what purpose or that the value of the data outweighs the cost of implementing them. These elements will require changes to case management systems and PIRL coding. In addition, the Wagner-Peyser registration requirements were substantially expanded under WIOA and require answering 100 questions (since not all elements can be asked in the PIRL format and still be understandable to lay people). Time spent gathering data is time that can’t be spent serving the customer so the data that is gathered has to be of highest value to make it worth the diversion of attention from the customer and the creation of additional hoops for them to jump through.	Thank you for your comment. The 4 elements added for REO are needed to populate the REO QPRs. The elements for virtual services, given the rapid expansion the use of virtual services, the Department believes it’s necessary to capture additional information. Individualized Services Provided Virtually: with the increasing capabilities of technology and the unfortunately circumstances of the pandemic, it seems possible and probable that individualized career services could be delivered virtually. This will be an important distinction as DOL analyzes performance outcomes. Transitioning Service Member Warm Handover/Housing Plan: As part of the ongoing partnership with the Dept. of Defense, it is necessary to ensure that the Warm Handover requirement (any military member who is deemed not career ready must be referred to the AJC) is being carried out as intended. Additionally, if someone is deemed to be at risk of homelessness, a housing plan is also required. Referred from VA: In a partnership with the Dept. of Veteran’s Affairs, it is important for DOL to determine what additional services a veteran has received in addition to the ones received at the AJC. To fully analyze the performance outcomes of veterans, DOL must also know as many non-DOL services received as possible.
	Suggestion related to PIRL Elements currently reported based on Status at Date of Program Entry but which can change during participation: We suggest that reporting on a participant’s Status at Exit would be more valuable for analyzing the success of program efforts. Ultimately, it doesn’t matter what the person’s status was at Date of Program Entry. Status at Exit reflects the challenges that participants face or advantages they have in navigating the labor market after their program participation ends. Do these characteristics change over time for a large portion of the population? Maybe not, particularly in a program like Wagner-Peyser that serves a very high volume of people and typically for relatively short periods. However, for smaller programs and programs where participation tends to occur over longer periods of time (like Youth and Vocational Rehabilitation), a greater proportion of Participants will have life changes that are important to note to tell the story of the workforce system, those we served, and what we together were able to achieve. Many of these data elements, particularly those related to what WIOA calls “Barriers to Employment” are utilized in the WIOA statistical adjustment models as well as to break out participants and result in the annual and quarterly report templates. As relates to the barriers, they statistically tend to correlate to lower outcomes – that is, all other things being equal, a person with a barrier is less likely to have as successful a set of outcomes as a person who is in every other way statistically similar but does not have any barriers. One example of this is if you had two people of the same age, demographic characteristics, gender, geographic location, work history, education levels, certifications, etc. but one with a felony conviction and the other without, we know that the person with a felony conviction is statistically less likely to have as strong a set of outcomes as the person without a felony conviction. In addition, some of the most common questions from elected and appointed officials, partners, and other stakeholders relate to the number of people served and their outcomes based on demographic and other characteristics. Some of the most requested characteristics are: Veterans People with a Disability Ex-Offenders Foster Youth Low Income Single Parents. When we’re asked these questions, they are not interested in a count based on status at date of participation. That is not relevant to the question. If the system begins serving an Active Military member before he or she musters out and is providing participatory level services, the current Joint PIRL requires this person to be reported as “Non-Veteran.” But the reason this person came to us was in preparation for leaving the military and could in fact do so during that same Period of Participation. Assuming the discharge type was not disqualifying for veteran status, this person is now a Veteran. We served a Veteran and yet we can’t report it in any of the data we report to the Departments of Labor and Education through our quarterly and annual reports because the requirements are to report based on status at Date of Participation. A similar example would be a person who had no disability when participation began but who had an accident during their Period of Participation and now has a disability – we have served a person with a disability but have no way to report it. That means that the Statistical Adjustment Models used to evaluate program performance based on the characteristics of those served and the economic conditions in which they were served does not get a true picture of who was served and therefore can’t account for that true casemix in evaluating performance at the end of the year. This undermines the legitimacy of the WIOA performance accountability system which is predicated on the creation of a model that can account for factors outside the program’s control that impact outcomes (positively or negatively). Further, for the Title I programs, the populations in question are much smaller when running local workforce development board performance and therefore the impact of each participant on the casemix becomes all the more important. If a local board has an unusual number of these kind of life changes in a group of exiters, the impact of not being able to update the record to reflect these changes will be much more pronounced. Our recommended solution is to change these data elements to not be limited to status at Date of Program Entry but rather to allow states to update values as appropriate to reflect the relevant characteristics of those we serve as they enter/reenter the labor market: 202 Individual with a Disability – As discussed above. 400 Employment Status – This one is slightly unique – what we’re particularly concerned with is the “negative” status on an element, the characteristics/statuses that are more likely to make a positive outcome harder to achieve. So in this case, we recommend that if at any point during participation, the participant becomes “Unemployed” we would report it as such and would not update that back to “Employed” if the person became employed during participation. This may seem inconsistent but from a reporting and service perspective, that person was unemployed and we served them while unemployed and we helped that person achieve a positive employment outcome (which will be reflected in wage records later). 402 Long-term Unemployed – In the current pandemic it is clear that a large portion of those we serve will not be “long term unemployed” at Date of Program Entry but will be unable to find employment for quite some time (new claimants from March will hit 27 weeks of unemployment by the end of October). If a person comes to us immediately after being laid off, they can still become long-term unemployed and employers may be more hesitant to hire them as they may have concerns that the person’s skills have atrophied through long term disuse. 407 Highest School Grade Completed / 408 Highest Education Level Completed – If in 11th grade at program entry but a HS or college grad or certificate holder at exit, the education level at exit is more relevant to their opportunities of success in the labor market. 704 Foster Care Youth Status – again, if we have a young person who was in their original family at participation and has a family tragedy that results in that person becoming a Foster Youth during participation, we have served a foster youth and that person is going to face all of the challenges that being a foster youth brings when it comes to completing their high school education, advancing to postsecondary education, and transitioning into the workforce. 800 Homeless Individual, Homeless Children and Youths, or Runaway Youth – as above, if a person had a stable family environment/home and then that changed during participation, that person is now going to face all the challenges that being homeless or a runaway bring. Their outcomes are statistically unlikely to be as good as those whose family/home status did not deteriorate during participation. 801 Ex-Offender – As discussed above. 802 Low Income – As with Long Term Unemployed discussed above, a person might come to the system with good income but over their period of participation, particularly through the pandemic, may have to apply for and receive public assistance (not unemployment insurance). There is keen interest in how the system helps people on public assistance to become self-sufficient. Cases that transition like this are not reportable in the existing system. 806 Single Parent – This status can certainly change during participation and creates an entirely new set of challenges for the participant. 807 Displaced Homemaker – as with single parent status, this can change during participation and impact outcomes and again, represents a priority population that WIOA requires us to report on. Most of the above elements-relate to “Barriers to Employment” under WIOA. The statute in its very first sentence says that “The purposes of this Act are the following: (1) To increase, for individuals in the United States, particularly those individuals with barriers to employment, access to and opportunities for the employment, education, training, and support services they need to succeed in the labor market.” It goes on to define these barriers in WIOA Sec.3 (24) and in WIOA 116(b)(3) requires the Departments of Labor and Education to develop a statistical model that accounts for the barriers (though it doesn’t call them barriers in WIOA Sec. 116(b)(3)(A)(v)(II)(bb), it does list many of the barriers as examples of factors to be considered by the model). Finally, WIOA Sec. 116(d)(2)(B) and (H) require performance data to be disaggregated by each subpopulation of individuals with barriers to employment. In short, WIOA defines a key part of its purpose to serve people with “Barriers to Employment,” requires the performance accountability system to account for the characteristics of those served including factors that may negatively impact performance – like “Barriers to Employment” – and, finally, requires reports to break performance data out by demographic characteristics and barriers to employment. However, the current Joint PIRL specs are not aligned with these statutory provisions because they focus on the person on at the beginning of participation and it is their status at the end of participation that will have the greatest influence on their outcomes in the vast majority of cases. In order to ensure that the national workforce system’s service to people with barriers as well as the outcomes achieved are fully reflected in the quarterly and annual reports as well as the statistical adjustment models, we urge the Departments to revisit the existing standards and modify them to be more reflective of the people we serve and the challenges they face during that period of service and transition to either Post-Secondary Education or Employment.	The Department thanks the Commenter for this suggestion and notes that not all of the referenced elements are based on status at date of program entry. For those that are based on status at date of program entry, it is acceptable to change information in an individual’s PIRL record based on new information if that new information reflects their status at program entry. If the participant’s status changes during participation, however, these elements must not be updated to reflect such changes. For example, if a participant discloses that they had been previously incarcerated but were uncomfortable discussing that when first entering the program, then PIRL DE801 (Ex-Offender Status at Program Entry) can be changed to a “1”, but if a participant who was unemployed starts working a part time job after entering the program then PIRL DE400 (Employment Status at Program Entry) cannot be changed. Additionally, elements that are not specifically designated as taking place at program entry, such as PIRL DE202 (Individual with a Disability) can be updated at any time to reflect the most accurate information about the participant. Special consideration for common reporting of co-enrollment. While ETA encourages the use of prior assessments when possible to determine participant eligibility, needs, etc. it may be the case that a participant in one program may have a change in status that leads to them becoming eligible for services in another program. In such a scenario, it is acceptable to update the relevant “at program entry” elements to reflect the participant’s status at entry into the subsequent program to better facilitate common reporting. The Department will not be making changes to the collection in response to this comment.	N/A
	(Pennsylvania #54-#55) Request to modify current definitions: The following proposals are being made to address definitional issues for the purpose of easing burden and improving reliability of the data: 413 - Migrant and Seasonal Farmworker Designation as defined at 20 CFR 651.10 808 - Eligible Migrant and Seasonal Farmworker Status (WIOA sec. 167) It is understood that 413 and 808 have definitional differences and that 808 is a common element across all core programs. However, we respectfully request a review of the two elements to identify a way to capture and report in a single element, including directions that only certain responses are applicable to Titles I and III. It is felt this would allow for a more robust look at MSFW's served by the workforce system. 907 - Recipient of Incumbent Worker Training The requirement to report multiple values identifying the program of funding is slightly redundant, given the values could be derived based on the programs of participation captured in other PIRL elements. 1306, 1311, 1316 – Occupational Skills Training Code – The requirement to align the codes in these fields with a specific O*NET SOC taxonomy will result in significant costs to adjust current systems to utilize the updated taxonomy. Not only will the cost be to incorporate the new taxonomy but also to ensure all prior recorded codes align with the current taxonomy. In addition, the realignment of these codes may lead to inaccuracies when analyzing the types of trainings being provided, as some codes stay the same but the definition changes. 1812 - School Status at Exit This element is applicable to Youth program participants only. As such, the definition should be modified in such a way as to indicate the information should be captured at the conclusion of youth services. 1814 - Date Attained Graduate/Post Graduate Degree (WIOA) This information should not be required for Title I and programs since a master's degree is not a program/performance focus.	Thank you for your comment. RE 413/808: There are statutorily reasons why the definitions exist, and the Department will provide additional technical assistance on reporting these 2 elements. RE 907: Due to different requirements for providing Incumbent Worker Training in the various programs and the possibility for combined reporting, the Department finds it necessary to collect this element using multiple values. RE 1306,1311,1316: The Department appreciates the challenges associated with making this change and will therefore implement a phased implementation of this change in order to allow states time to determine how best to make these adjustments. To accomplish this, the Department will continue to allow older ONET codes via its edit checks during the transition period. RE 1812: The Department disagrees with the commenter. This element is required for WIOA Youth, NFJP, INA, and REO Youth so it’s not just a youth element. Also, it’s important this element remain as status at exit because DOL compares school status at enrollment to school status at exit to assess the program’s impact on school status over the life of the program. RE 1814: This is a joint element and is included for title I in case grantees want to report this attainment for programs the individuals are co-enrolled in, and can be left blank if not applicable.	ETA 9172 DOL only PIRL
	Requests to remove data elements from the PIRL structure: 204 - 209 – Types of Disability-related services received – small, specialized populations Attempts to work with state Departments of Human Services to obtain crossmatches have indicated there is a lack of clarity at the state level as to what this means. Where clarity exists, extensive IT development is required to obtain the information from what research has shown to be a very small crossmatch of individuals. Information may be germane to service delivery/planning/goal setting but to make it mandatory data entry with coded values when applicable to such a small percentage of the population we serve is burdensome and costly. The collection of this information by workforce development front-line staff, who are unfamiliar with these services, require individual interpretation leading to inconsistencies and a lack of intended value. 308 – Homeless Veteran The inclusion of this data element is seen as an unnecessary data collection, as this demographic can be derived from two other elements (301 & 800) 315 – Other Significant Barrier to Employment Lack of clarity exists as to what would be captured here. Including a data element in the PIRL data file that is rarely populated does not fit with good data file design principles. 934 – Rapid Response Event Number While recognizing US DOL’s desire to retain this element to allow continued capture of data from those states that implemented an internal tracking system, the inclusion of this element adds unnecessary cost and burden to states who did not implement such a system. 935 – Accountability Exit Status (code value 3) – Discussions indicate the applicability of this code value is very limited across states, even those with large numbers of Trade participants. In addition, the time and cost associated with tracking and recording it from the case management system to the PIRL report is burdensome. 939 & 940 - Individual with A Disability Individualized Education Program/Disability Section 504 Plan Information may be germane to service delivery/planning/goal setting, but to make it mandatory data entry with coded values when applicable to such a small percentage of the populations is burdensome and costly. 1902 – 1922 – Educational Functioning Level testing information The inclusion of these data elements is seen as an unnecessary data collection, as the information is not necessary for determination of an EFL gain in relation to the measurable skill gains metric. In addition, significant changes to the assessment tools from which this data was captured has led to the misalignment of code values to information available.	While the Department understands the complexities of collecting and tracking data, it is important for front-line staff to learn new methods in which to collect data through professional development, as well as the reasons why the data provides critical information to the workforce development system. This data is useful to the system in multiple ways, including: 1. States and localities can improve programmatic accessibility for individuals by better understanding who is being served. 2. Multiple agencies and organizations serve participants under the Workforce Innovation and Opportunity Act (WIOA). Collecting common data elements enables the Department to learn who is being served, and by which agencies, allowing the Department to enhance collaboration among required and non-required partners. By expanding partnerships, WIOA programs can leverage funding to provide additional services and ultimately decrease service silos through better coordination. This also leads to improved services and outcomes for participants. 3. Tracking of disability data will help the Department get a better understanding of outcomes that are resulting under WIOA, including for new requirements such as those relating to Competitive Integrated Employment and other work options (such as Customized Employment). Analyzing these outcomes will also assist the DOL to identify needs to provide targeted technical assistance in areas where outcomes are lacking. The Department has released a suite of materials in the last fiscal year, including: A policy and analysis brief to explain the disability related data elements, and analyze state reporting across all these elements, which is available here: http://leadcenter.org/resources/report-brief/disability-related-reporting-participant-individual-record-layout-pirl. A WIOA PIRL Data Visualization Tool to help states understand their own levels of reporting on the disability-related PIRL data elements, available at http://drivedisabilityemployment.org/wioa-reporting. Two webinars to help state and local workforce systems better understand and collect these data. These webinars are available at: http://www.leadcenter.org/webinars/understanding-wioa-disability-related-reporting-tools-data-visualization and http://leadcenter.org/webinars/wioa-disability-related-reporting-deep-dive-participant-individual-record-layout-pirl-0. The Department is currently working on additional technical assistance materials relating to the PIRL disability data elements to be released later this fiscal year. RE 308: The Department agrees with the Commenter and will remove PIRL 308. RE 315: PIRL 315 is an important data element as JVSG requirements can change before PIRL revisions occur. For example, Vietnam-era veterans was added as a significant barrier to employment on February 7, 2019, but there was not an existing data element to capture this. PIRL 315 can be used in the interim until a permanent solution is implemented. RE 934: The Department notes this element is optional, so it adds no burden other than having a blank column in the file submission. This element has been used by states meaningfully to help improve tracking of rapid response events and according to a recent analysis, approximately 1/3 of states are utilizing this field meaningfully. While there may be no current plans to create a national system, when this data is available it provides important insights into the number of rapid response events. RE 935: DOL agrees with this comment that this element is particularly burdensome for TAA. The Department will remove code value 3. However, to track TAA participants who receive services under multiple petitions, PIRL 915 will be modified to accept a delimited list of values. The Department recommends only collecting this data when an individual indicates they have a disability. RE 939/940: The Department recommends only asking this question to participants that indicate they have a disability RE 1902 – 1922: The department agrees that it would be reasonable to removed elements 1902-1922 for the formula programs. DOL is going to keep these elements in the PIRL as required for YouthBuild, REO Adult, and REO Youth, Job Corps, NFJP as they have specific uses for these elements for monitoring and these elements are already included in their national case management systems so it’s not an additional burden for them to stay as required PIRL elements for those programs.	ETA 9172 DOL Only PIRL
	Error: Reference source not found
	(Pennsylvania) Request to remove the following elements as ‘required’ for Apprenticeship reporting The Apprenticeship program is unique from other workforce programs in that the individual is already employed through a connection outside the traditional workforce system. Required collection of this information would be burdensome to the employer who must provide this information to the grantee for tracking purposes and in some cases require the employer do an assessment to determine applicability to the apprentice: Element 108-A – ETA-assigned 1st Local Workforce Board Code Element 301 – Eligible Veteran Status Element 303 – Disabled Veteran Element 401 – UC Eligible Status Element 409 – School Status at Program Entry Element 804 – Basic Skills Deficient/Low Levels of Literacy at Program Entry Element 806 – Single Parent at Program Entry Element 807 – Displaced Homemaker at Program Entry Element 903 – Adult Program Participant Element 904 – Dislocated Worker Program Participant Element 905 – Youth Program Participant Element 910 – Adult Education Program Participant Element 911 – Job Corps Program Participant Element 913 – Indian and Native American Program Participant Element 914 – Veterans’ Program Participant Element 915 – TAA Petition Number Element 916 – Vocational Education Element 917 – Vocational Rehabilitation Element 918 - Wagner-Peyser Employment Service Element 919 – YouthBuild Element 938 – H-1B Element 1210 – Received Pre-Vocational Activities Element 1333 – Received Training from Programs Operated by the Private Sector Element 2413 – Incarcerated at Program Entry In addition, Element 1301 – Eligible Training Provider – Name – Training Service #1 should not be required because Registered Apprenticeships are not required to be included on the state Eligible Training Program list. In addition, not all Apprenticeship grantees have access to state Eligible Training Program lists, nor are they required to compile one.
	(From NC Community College) Based upon the information provided and the request to evaluate whether the collection of information is necessary for the proper performance of the functions of the agency, I have to say that it is not.  This is requiring 131 data elements that are to be collected for apprenticeship, many of which are not relevant to the apprentice.  Apprentices do not complete intake forms and are hired by the employer, through the means that the employer finds the most efficient. Employers are referred WIOA participants, but the company does not choose to hire them.  Therefore, this data collection burden is placed on the employer.  Employers do not have time to collect this additional data, especially now that they are dealing with the pandemic and continuing to run a company.  Employers are viewing this as a burden of additional paperwork and are more and more becoming uninterested in apprenticeship because of it. The information already collected is enough to follow the apprentice through training and after training.  We must minimize the burden of collecting information, not create more of a burden.  This does not follow the paperwork reduction act of 1995 (PRA), but is just the opposite.
	Montana The Montana Department of Labor & Industry (MTDLI) is committed to facilitating and developing high-quality registered apprenticeship (RA) programs in order to recruit, train and retain a diverse and highly skilled workforce. No other education and training system brings together government, education, business and labor so effectively to produce such high-performance outcomes. With that context, MTDLI respectfully asks that the RA Expansion grants be left off the list as DOL/ETA considers this extension of authority. RA is an industry-recognized, voluntary and employer-driven system. If DOL/ETA is to evaluate whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information will have practical utility, it should take into consideration the Fitzgerald Act and resultant and related State regulations put in place to ensure accountability and equal employment opportunity. Employer and Labor sponsors of RA programs are already subject to a myriad of legally binding provisions in both federal and state law. MTDLI sees no utility in applying WIOA, pre-employment-based performance metric requirements to an already well regulated, high-performing, employment-based, career building program. MTDLI is committed to a national System of Registered Apprenticeship. While we are encouraged by recent public investments in the model, we are also concerned that the proposed extension of data gathering authority to include Federal RA Expansion Grants underestimates the burden already imposed on registered sponsors and could negate the significant gains seen in recent years per that public investment.
		For comments pertaining to data collection, please see response from DOL’s Office of Apprenticeship in the document below. The comments pertaining to WIPS are outside of the scope of the ICR and will be addressed through technical assistance.

Revised 4/1/2021 2