Summary of Comments Submitted in Response to 30-day FRN and Agency Responses

DOL WIOA Performance Accountability, Information, and Reporting System

30-Day FRN Public Comment and Agency Response

ICR REFERENCE # 201604-1205-003

OMB Control No. 1205-0521

Expiration Date: 08/31/2019

Department of Labor

Information Collection Request

Workforce Innovation and Opportunity Act (WIOA)

Performance Accountability, Information, and Reporting System

Summary of 30-day Federal Register Notice (FRN) Comments and Responses

COMMENTS AND RESPONSES

1. Parallel Accountability/Reporting Systems

Comment: DOL proposes to operate the old and new performance reporting systems in parallel until “the last performance reporting requirements are satisfied for each program” under the old system. DOL currently requires states to report employment data through the 4^th quarter after Exit for the Workforce Investment Act (WIA) and Trade Adjustment Act (TAA) Participants. In addition, the old system will be applicable for those who exit thru June 30, 2016. Therefore, DOL is proposing states operate parallel reporting systems for nearly two years (more if they also insist on a final WIA Annual Report in the Fall of 2018)!

Operating the old and new reporting systems in parallel will cost considerably more than simply discontinuing the old system and implementing the new one, because the new system is substantially different in how Periods of Participation (POP) are structured (due to self-service not having any impact on Participants and Exit). If DOL did not plan to require parallel operation, states could just modify their systems to meet the new standards and make the jump all at once. Instead, states will have to maintain their old systems for two years and operate new systems in parallel, which will cost more than operating a single reporting system (more storage, higher CPU processing requirements, etc.).

The impact of this proposal goes beyond automation costs. Local Boards and other program providers will have to monitor two sets of measures in different ways for an extended period, which will make it harder for them to make the transition to the new system. Performance measurement and reporting is complicated enough without having to understand both models simultaneously. Local Board members, who serve on their boards in a part-time voluntary capacity and who are supposed to provide oversight over the service provider, will be particularly challenged in this regard.

Therefore, we strongly oppose this proposal and recommend that DOL discontinue the old reporting system in the Fall of 2016 with PY15 yearend reporting so that grantee can concentrate on making system changes necessary to implement the new performance accountability and reporting system. This way, local Boards and other service providers can focus on the future exclusively, rather than being distracted by the past.

Agency’s Response, June 2016: States will continue to be held accountable for WIA outcomes on exiters through 6/30/16. There is no expectation that States will continue to update/track any information on these exiters, but instead “hold” the exiter information to conduct a wage record match after all performance cohorts are available.

Action: None.

2. Expansion of Required Data Elements

Comment: One commenter previously commented on the issue of data elements to be required by DOL. In particular, this commenter was dismayed at the lack of program-by-program guidance indicating which elements were required for each program. This commenter also cautioned DOL to be extremely judicious in the elements they require to be reported. This commenter strongly recommended that each proposed new or modified data element be evaluated to determine whether it is required by statute and if not, to very carefully weigh the cost of gathering that information versus the benefit gained by having it, and to cull the list of non-statutorily‐required elements to only those of greatest value to the system.

In response, DOL stated that “most of the data elements were previously required under the WIA Standardized Record Data (WIASRD).” This seems to ignore the fact that the WIASRD included many of data elements that were not required for Wagner-Peyser in the Labor Exchange Reporting System (LERS). Requiring all WIASRD elements for all Wagner-Peyser participants will be a great increase in the amount of data to be collected in the workforce system. This is especially true in states like Texas where Wagner-Peyser Participants outnumber WIA Participants by well over 10-1.

On page 4 of Appendix A of the ICR, DOL agrees with a commenter that “it would not be realistic to collect the same depth and breadth of information from individual accessing Wagner-Peyser services relative to individual receiving training services under a different program.” And yet, the primary difference between the data elements DOL proposes to for Wagner-Peyser Participants and those required for Title I Participants is that Wagner-Peyser doesn’t require reporting on the Youth or Training-Specific elements.

Elements that were previously Youth only (like Education Status at Exit) are now proposed to be required for many other programs (particularly Adult, Dislocated Worker, and Dislocated Worker Emergency Grant). For Example, School Status at Exit was previously only required for WIA Youth. That is not the case in the proposal.

There is an entire section (E.08) at the end of the DOL-PIRL that has highly specialized data elements and yet are required for nearly every program. We object to requiring every Title I and Title III Participant from having to answer questions about whether they are or were in individualized education program/special education services in high school and many of these other elements. We can only support the gathering of the data element proposed to be required in this section if they were only applicable to a limited population – such as those who self-identified having learning disability or cognitive/intellectual impairment.

The general public seeking basic job search assistance is not looking for an old school case-management experience. Many of them are going to be turned off by a giant list of questions that they perceive as having little to do with their job search needs. Further, every minute spent gathering information for compliance purposes is one less minute that can be spent providing services.

We request that DOL revisit the proposed list of data elements to be required for each program and in each instance modify their proposal to NOT require reporting on elements that are not:

1. Required by statute,

2. Previously required for the specific program in question (in all circumstances)¹; or

3. Available through means other than asking Participants for the information (whether remotely or in person)².

Agency’s Response, June 2016: DOL carefully considered the inclusion of each data element and the value of the information. The Department disagrees on using the three criteria alone above. Statutory requirements do not specifically list all data elements that the Department believes necessary to provide a fulsome view of the programs through the data collected. For example, WIOA does not specify to collect any veteran-related data, which is a very important piece of information for most, if not all, DOL programs to collect.

At the same time, DOL realizes the impact of the collection of additional data elements on the system. ETA is working with colleagues in the Department’s Office of Disability Employment Policy to revise/reduce the number of data elements that will be collected in section E.08.

Action: Revise the data elements to be collected in section E.08.

3. Reportable Individuals

Comment: In addition to the program columns, the PIRL added a column indicating which elements were required for “Reportable Individuals”. However, the column was left blank, indicating that Reportable Individuals, who are not Participants, are not included in the PIRL. This is consistent with a question DOL received on the 60 Day ICR, which the Departments of Labor and Education (the Departments) answered by saying “PIRL data elements are to be collected on participants, with specific requirements determined by the program of participation.” We support the proposal that states and other grantees not be required to capture and report PIRL data elements on non-Participants.

Agency’s Response, June 2016: DOL does consider it important to be able to collect some data for reportable individuals and will indicate which data elements are required for reportable individuals. It should be noted that DOL agrees with the comment that the number of elements collected should be kept to a minimum.

Action: Add a column to the PIRL indicating which data element will be collected for reportable individuals.

Common Unique Identifier

Comment: We object to DOL’s proposal to develop a common unique identifier for Participants across all DOL programs. While we believe it is reasonable to expect a grantee to develop a common identifier across programs that the grantee operates (such as Title I and Title III, as well as the Trade Adjustment Assistance Act for this commenter), extending that identifier across grantees is not reasonable, except in instances where the grantees are sharing an automated system with a universal customer data model. We understand that often these other DOL programs are required WIOA partners. However, the technical challenges, costs, and potential legal/privacy concerns make connecting siloed data systems impractical. Even where both grantee are collecting Social Security Numbers (SSN) on 100% of clients (which most programs are not able to do) that only allows the grantees to identify which programs a Participant may be enrolled in across the grantees. It does not create an automatic common identifier. DOL is proposing not only significantly costly data sharing requirements, but also requiring potentially costly system modifications just so they (DOL) can un-duplicate across grantees.

Agency’s Response, June 2016 DOL considers it important to be able to track participants longitudinally over time as well as across multiple programs in order to fully assess impact of program mixes on outcomes. We also recognize that this is likely a lengthy process, and understand that this will likely take several years to realize.

Action: None.

4. Monitor Advocate Program

Comment: It is not clear how/why Monitor Advocates have their own separate columns for reporting. Historically the Monitor Advocates have been charged with ensuring comparability of services between Migrant Seasonal Farmworker (MSFW) and non-MSFW Participants. We support moving Migrant Indicators of Compliance reporting into the PIRL, but don’t understand why this would not be accomplished by simply reviewing Wagner-Peyser and Title I data. Further, we don’t understand (and don’t support) the additional data elements that DOL proposes gathering on MSFW Participants not served in Title I. We likewise don’t support the extensive follow-up tracking that would be required for MSFW Participants, such as whether the job they entered offered benefits. This may have been something that National Farmworker Jobs Program (NFJP) staff was expected to track, but it represents a significant increase in data to be gathered for MSFW Participants served outside of NFJP and it goes well beyond what WIOA requires to be reported.

Agency’s Response, June 2016: DOL agrees with this comment, and will remove the Monitor Advocate program column from the final PIRL. The Monitor Advocate program will utilize the same data elements as the Wagner-Peyser layout.

Action: Remove the Monitor Advocate column from the PIRL.

5. Changing Customer Characteristics

Comment: In response to the Joint Performance ICR published in July 2015, a commenter commented on the question of whether characteristics that can change over time should be reported based on their status as of the Date of Participation or whether they should be able to be reported based on updated information. In response to these comments, the Departments indicated that it would be too burdensome to require programs to update characteristic data related to barriers to employment. In response to the 30 Day version of the Joint Performance ICR published in April 2016, we requested that states be allowed to report based on updated information on relevant characteristics (repeated below):³

However, characteristics such as employment and disability status should be updatable. The statistical adjustment models are supposed to adjust performance expectations based on the statistically relevant characteristics of the Participants being served. Employment and disability status are both statistically relevant. A person who is employed and just looking for a second or better job, is much more likely to be employed in quarter 2 after exit than a person is unemployed – this is clearly evident in the data.

On the other hand, if a person heard rumors of potential layoffs and decided to proactively begin looking for work through the workforce system (thus becoming a Participant) and then that person is laid off a week later, (thus becoming “not employed”), their outcomes are more likely to be similar to a person who came to the workforce system unemployed already than one who was employed and remained employed during their POP.

Take this example out another level and assume that in addition to our proactive Participant we have another worker who is essentially identical to the first – same education, work experience, demographics, occupation, etc. – but this second worker didn’t think there was any way they would be laid off. This “reactive” worker was ultimately laid off at the same time as our proactive Participant and comes to the workforce system already unemployed at Date of Participation. So we have two Participants who were identical in every way, except one was proactive in responding to the layoff rumors and was thus still “employed at participation,”⁴ yet the two participants will have different impact on the targets through the Statistical Adjustment models.

We offer several additional examples of elements that it believes show the value of updating Participant characteristics. These elements were not included in its Joint ICR comments because they are only contained in the DOL PIRL:

1. Element 401 UC Eligible Status – DOL states that this element “is intended to determine how effective programs are at serving individuals that are eligible for [Unemployed Claimants] UC”. Imagine a person who hears rumors of pending layoffs and proactively begins searching for new employment and thus becomes a Participant before losing their job and filing for Unemployment Insurance. If that Participant loses their jobs as feared and they file for benefits, DOL’s restriction on updating claimant status will result in under-reporting the system’s service to claimants, and thus will undermine DOL’s stated intent.

2. Veteran Status – If a Participant begins service as a Transitioning Service Member and musters out before (thus becoming a veteran) before their POP ends, they will not be reported as a veteran served, which seems problematic given the heavy focus on service to and outcomes achieved by veterans.

Agency’s Response, June 2016 DOL will publish guidance on completing these variables, but overall, the Department considers that for simplicity and consistency among states it makes sense to collect these variables only one time. DOL will continue to discuss the gains and potential downfalls of allowing states to alter this data throughout the participation period.

Action: None.

6. Educational Functioning Levels (EFLs)

Comment: DOL PIRL 1904 and PIRL 1907 are used to report EFLs on Youth Participants, however the data elements allow reporting 8 different levels, while the Office of Career, Technical and Adult Education (OCTAE) has long used 11 levels. We recommend aligning Youth EFLs in the DOL PIRL with OCTAE’s EFL model. This will make tracking and reporting on young adults in AEFLA and Youth funded services easier.

Agency’s Response, June 2016: DOL agrees and will align the Educational Functioning Levels with OCTAE.  This revision will be made in the final PIRL.

Action: Revise the PIRL elements 1904 and 1907 to align with the OCTAE standards.

7. Pre/Post-Test Scores

Comment: We recommend deleting elements PIRL 1903 and PIRL 1906 for reporting pre/post-test scores. There are many different EFL test instruments and they don’t have identical scoring systems. That makes the score values of limited value. The important information is not the score, but that the score represents – the EFL.

Agency’s Response, June 2016: Having the actual test score does two things: it ensures that the pre-test and post-test taken were comparable to achieve an EFL gain, and it also aides in providing a stronger source of documentation when doing data validation and monitoring.

Action: None.

8. Youth Follow-up

Comment: Under WIA Youth follow-up services were required from the conclusion of Youth Services, even though DOL followed a Common Exit reporting model. We recommend continuing to do this given that “Exit” is a much less predictable concept than the completion of youth services (which has its own data element in the PIRL as it did in the WIASRD).

Agency’s Response, June 2016: That level of specificity will be covered in upcoming DOL guidance.

Action: None

9. Performance Report/Specifications

Comment: We note that the Program Performance Report specifications in Appendix C are entirely unchanged from the original even though there were changes in the measure definitions in the measure calculations included in the Joint ICR and changes in the exclusion reasons in the PIRL. For example, MSG only lists four ways to get a gain rather than five, still references “EXLUSIONARY REASONS” (which are now called OTHER REASONS FOR EXIT in the PIRL) and then references vales of 00, 01, and 98, even though 98 is no longer a valid code under OTHER REASONS FOR EXIT. Also, these specs are incomplete and don’t provide information on each data element on the report. There is no way to reasonably comment on the proposed quarterly report in the absence of a complete set of specifications.

Agency’s Response, June 2016: There was a version control issue with the 30-day version of the specifications; however, that has been corrected, and the updated version of the specifications will be included with the approved ICR.

Action: None.

10. Youth Service Reporting

Comment: In response to comments on the Joint ICR’s annual report, the Departments stated that youth services will be separated between career and training services through some kind of cross-walk, because this is required by statute. However, when faced with a similar suggestion on the DOL quarterly performance report, DOL declined to do so. We recommend that the Youth reporting be broken out the same way between the quarterly and annual reports to improve consistency.

Agency’s Response, June 2016: DOL agrees with this comment and will align the quarterly report in this ICR with the annual report in the joint WIOA performance ICR.

Action: Create youth-specific specifications in the quarterly reporting document.

11. Display of Performance Periods

Comment: A commenter suggested that the Annual Report be modified to display the performance periods to make it more understandable and provide more context for the data being reported. The Departments agreed with that proposal but DOL did not incorporate the suggestion in the quarterly report and we recommend they do so in the final quarterly report template.

Agency’s Response, June 2016: As mentioned in the 60-day response, DOL agrees with this comment and will add specific date placeholders on the quarterly report template.

Action: Add specific date placeholders on the quarterly report template which correspond to the report period.

12. Retroactive Data Gathering

Comment: In response to a comment raising a concern about the lack of Data Validation guidance and the importance of that information to fully develop data collection processes, DOL stated it “does not anticipate any retroactive data collection requirements.” This seems consistent a response to a question regarding when the new data element for category of disability will become mandatory to which DOL stated that the new elements will only be required for those who become Participants on or after 7/1/16. However, they also state that category of disability will expected to be collected and reported on all those who become Participants on/after 7/1/16.

The regulations and the ICRs are being developed through public comment process. Most aspects of what was proposed were not clearly outlined/required by statute and therefore potentially subject to some level of change. This commenter and many other grantees have been waiting for the final regulations and specifications to be issued before beginning many IT system changes in order to ensure that resources were not wasted implementing proposals, which may change in the final versions.

Many, perhaps most, grantees will not be able to complete modification of the automated systems consistent with the new reporting requirements until they are fully published and final. While the Joint and DOL ICRs may be final before 7/1/16, they will be far from complete. The Departments’ responses to 60 Day Joint ICR comments and DOL’s responses to comments on the 60 Day DOL ICR contain numerous instances where in response to a question or recommendation, the commenters are told that additional guidance will be provided in the future (assumedly after 7/1/16 and perhaps long after that date). In the absence of this guidance, states cannot complete the work that will allow them to capture this data in their reporting systems on 7/1/16.

We recommend that the requirement to gather new/modified data (or new/modified requirements relating to data already being captured) only be applicable to those served on or after 7/1/17). This can be done by not making Data Validation requirements applicable to those with a Date of Participation prior to that date.

Agency’s Response, June 2016: DOL agrees with the comment that the burden will simply be too high to retroactively collect source document and subsequently conduct data validation. The Department will publish guidance to the workforce system on data validation, including proper source documentation. This guidance will provide States sufficient time to prepare and to collect the required documentation from participants.

Action: None.

13. Primary Indicator of Performance: Credential Attainment

1. Comment: The Federal Partners appear to exclude participants enrolled in on-the-job training and customized training from the Credentials Indicator IF the programs are not designed to lead to a credential.   That seems a bit odd.  Apprenticeships would then be excluded?  

Agency’s Response, June 2016: Since most apprenticeship programs offer an industry-recognized credential upon program completion, apprenticeships would NOT be excluded from this indicator.

Action: None.

2. Comment: Participants who score at low levels of literacy are not included in Credential Indicator unless they are enrolled in programs that provide instruction at or above the ninth grade level.  What impact would this have on ESL learners enrolled in WIOA Youth? Final regulations will hopefully give further insights.

Agency’s Response, June 2016: This should not have any impact on ESL learners. This topic will be covered in future WIOA reporting guidance.

14. Primary Indicator of Performance: Measurable Skill Gains

Comment: Incarcerated participants appear to be excluded from the Employment Outcomes indicator. Under WIA, youth who were incarcerated, in a mandated residential facility, death, health care reasons, etc. were excluded from ALL measures.  Will they be excluded from all measures under WIOA?  These youth would then appear to be included in the Measurable Skills Gain indicator?  Clarification is needed.

Agency’s Response, June 2016: Participants in the WIOA Youth program are included in the measurable skill gains indicator.

Action: None.

15. Primary Indicator of Performance: Effectiveness in Serving Employers

1. Comment: The Departments have indicated that they intend to have States select two of the three options for measuring effectiveness in serving employers.  It is recommended that States only be required to report data element 1618 (Retention with the same employer in the 2^nd Quarter and the 4^th Quarter after exit) if the State elects this as one of their measures for effectiveness in serving employers. 

Agency’s Response, June 2016: Retention with the same employer is one of the three options for this indicator. If this is selected, states must also provide data on the Employer Penetration Rate OR the Repeat Business customers.

Action: None.

2. Comment: Under the Joint Performance Reporting ICR, Effectiveness Serving Employers is to be reported at the system level only, yet DOL is proposing to report it by program on its quarterly reports. We agree with the Joint guidance that these measures should be reported for the system as a whole and not broken down by program.

Agency’s Response, June 2016: DOL acknowledges that in the joint WIOA ICR, there was mention of utilizing this indicator as a “shared” measure between the core programs. DOL will offer further guidance on this indicator and specifically how it will be operationalized and how programs will be held accountable for outcomes.

Action: None.

16. Performance Reporting

1. Comment: As with any legislation there are expectations reporting needs are changing due to the new legislation, however, the overall scope of these modifications and the requirement for continued existing reporting as the transition to new reporting occurs there are large data collection requirements that need to be evaluated and system changes need to start happening quickly.

For example, under current Title III reporting, participant data (excluding services, outcomes wages) has 24 reportable items and under the new Title III, JVSG there are now 35 reportable items. Adding 11 new reportable items involves large system changes and staff training to be addressed so insure that staff are collecting and have a comprehensive knowledge of what is being asked and why of the individual.

Another example is under current reporting for Title I there have been 3 data elements defined as at participation. Under the PIRL, this has been expanded to 17, which are labeled at program entry, which is being interpreted as the replacement to “At Participation”.

Agency’s Response, June 2016: WIOA mandates that particular data elements and performance indicators be reported on. All of the PIRL elements in this ICR are required by statute. Therefore, the burden has been kept to a minimum.

Action: None.

2. Comment: On Page 12 of the 1205-0521 Supporting Statement, ETA has provided the following information:

“The Labor Exchange Reporting System (LERS), authorized under OMB Control Number 1205- 0240, is the only current mechanism for collecting performance information on the Wagner- Peyser Act Employment Service and Jobs for Veterans’ State grants’ activities. As such, this set of reports is necessary for tracking and reporting, to stakeholders, information on the usage, services provided, and performance of these programs. More specifically, these reports are used to monitor the core purpose of the program – mainly, tracking how many people found jobs; did people stay employed; and what were their earnings. It is expected that OMB Control Number 1205-0240 will be utilized for all Program Year 2015 reporting; this ICR will be discontinued after all required reporting is completed. Due to lag times in performance outcomes, this will likely be after the close of Program Year 2017.”

This is being interpreted that states will be required to produce multiple types of reports, balance the data collection as it relates to these various reports through at a minimum the final report of PY 2017, which is due to ETA in August, 2018. This is a large burden on states; states must maintain data collection in the old manner, and create new data collection processes, in what appears to be due by July 1, 2016.

Agency’s Response, June 2016: States will be expected to keep the participant records for exiters as of 6/30/2016 and submit a final “closeout report” for WIA and Wagner-Peyser programs. This file will not need to be appended after this date, and is only being kept to match against wage records to submit final outcomes late in 2017.

Action: None.

3. Comment: As with any legislation there are expectations reporting needs are changing due to the new legislation, however, the overall scope of these modifications and the requirement for continued existing reporting as the transition to new reporting occurs there are large data collection requirements that need to be evaluated and system changes need to start happening quickly.

For example, under current Title III reporting, participant data (excluding services, outcomes wages) has 24 reportable items and under the new Title III, JVSG there are now 35 reportable items. Adding 11 new reportable items involves large system changes and staff training to be addressed so insure that staff are collecting and have a comprehensive knowledge of what is being asked and why of the individual.

Another example is under current reporting for Title I there have been 3 data elements defined as at participation. Under the PIRL, this has been expanded to 17, which are labeled at program entry, which is being interpreted as the replacement to “At Participation”.

Agency’s Response, June 2016: WIOA mandates that particular data elements and performance indicators be reported on. All of the PIRL elements in this ICR are required by statute. Therefore, the burden has been kept to a minimum.

Action: None.

4. Comment: The Departments have indicated that they intend to collect data on outcomes in the first and third quarter after exit for all core programs.  It is unclear in the information if the VR Program is required to report employment data for the first and third quarter after exit or not.  Please clarify. 

If these elements are required to measure performance under the WIOA, it is recommended that this information be eliminated from the performance report.   This creates unnecessary costs associated with manual follow up with participants where UI data is not available and increased administrative costs associated with the data exchanges required to capture this information (for those with costs negotiated based upon the volume of transactions). 

Agency’s Response, June 2016: All core programs are expected to report on all data elements within the JOINT ICR. Program specific ICRs also indicate additional requirements separately for the core programs.

Action: None

5. Comment: After reviewing all of the documentation published to date regarding the WIOA common performance reporting, it is unclear who will be responsible for generating the state level reports. Under the current WIA process, states upload data to the EDVRS website which then generates the reports.  Will that process be carried over to the WIOA reporting, or will states be directly responsible for generating state level reports on their own?  If states are expected to generate their own reports under WIOA, additional lead time will be required between publishing the final reporting rules, and the deadline for implementation because states will need this time to budget and implement the supporting IT systems, updates to data intake, and business processes.

Agency’s Response, June 2016: The actual method of data submission will be detailed in further guidance.  For workforce programs, the file submission process will continue to be similar to that used under WIA, with an electronic system to accept individual files and aggregate reports.

6. Comment: Because NFJP grantees operate their own case management and data management programs, they can only reasonably be expected to report participation in other WIOA programs for individuals for whom they arrange co-enrollment. There is not consistency among one-stop operators from service area to service area or state to state relating to the amount of cooperation and data-sharing that states are willing or legally able to do with non-state agencies.

Action: No change

Rationale: In cases where a grantee does not arrange co-enrollment of a participant with another WIOA program, data can be collected from a participant via self-attestation or source documentation provided by the participant.

17. ETA-9173: Performance Report Template

1. Comment: What is Program Performance Report (ETA-9173) intended for? Will it be replacing reports like ETA-9002? Which reports will it replace? What USDOL systems will accept the new report formats? What will the timeline for implementation be?

Agency’s Response, June 2016: The ETA 9173 is designed to display the aggregated values from the state data submitted through the PIRL file. 

Action: ETA will discontinue the use of ETA 9002 and ETA 9090 once all WIA and Wagner-Peyser reporting requirements have been satisfied.

2. Comment: In estimating costs for this report it is unclear how the estimated burden and cost was produced because the Supporting Statement has the following statement:

In order to collect the participant level data that will be aggregated and displayed in the (Program) Performance Report (ETA-9173) and the Pay-for-Performance Report (ETA-9174) quarterly reports, States will use a standardized individual record file for program participants, called the DOL Participant Individual Record Layout (PIRL). The PIRL provides a standardized set of data elements, definitions, and reporting instructions that will be used to describe the characteristics, activities, and outcomes of WIOA participants. States and grantees will be required to collect participant information that corresponds with the data elements and descriptions delineated within the PIRL. Once collected, this information will then be aggregated according to the conditions outlined in the specifications found within the Program Performance Report spreadsheet. This document details the common data elements and technical specifications necessary for calculation of reporting elements under all the DOL programs listed in the paragraph below.

The specifications as defined above have the following issues:

a. The specification does not match the report sample

b. The specification has many cells with incomplete or missing data

c. The specification when compared to the ETA 9169 WIOA Statewide and Local Performance Report does not have the same calculations for the same reportable items.

Therefore, we feel that is cost estimate is completely underestimated in its overall value.

Agency’s Response, June 2016: After review, some specifications were omitted in error and have been added to the document. Additionally, these fields were accounted for in the burden/cost estimate initially. Therefore, no additional changes are necessary.

Action: None.

3. Comment: A commenter requested clarification on which reports will replace the ETA-9085.

Agency’s Response, June 2016: The Department will be using the ETA-9173 report as a general summary report for all ETA programs. However, the Department will have the ability to generate reports specific to the Native American program and also generate reports using various data elements in the PIRL that are being collected by the INA program.

Action: None

4. Comment: A commenter requested clarification on the ten miscellaneous data elements in Section E.08 and questioned the purpose of the data elements for the Native American program.

Agency’s Response, June 2016: As noted above, the Department is revising these data elements. All programs will be required to collect the information.

18. ETA-9174: Pay-for-Performance Report Template

At least two commenters shared the same concerns regarding the ETA-9174 Pay-for-Performance Report Template as follows:

Comment: Justifying the estimated burden for ETA-9174 it is problematic for multiple reasons:

• Specifications are not provided with the release of ETA-9174

• Calculating an estimated time burden based upon 57 respondents, with each providing one response, is an underrepresentation leading to undervaluing of the time estimate for the following reasons:

• P4P contracts are normally implemented at the local board level

• P4P contracts normally include multiple providers as parties to P4P contracts and local providers may be party to multiple P4P contracts

• Individual participants may obtain services from multiple P4P providers, which is problematic as PIRL 107 permits capture of only one provider ID for a participant - How will the Department harmonize P4P provider reports with participant data?

• ETA-9174 captures only a provider ID, so derived performance measures will not illustrate a true picture of performance outcomes under P4P contracts

• The 25-hour time estimate does not consider the fact that: - Local boards implement multiple P4P contracts on which reporting is required

Narrative reporting demands that local-level staff pull and analyze data and manually generate required report content (see comment below)

• The Department states in the ICR Supporting Statement: “Once the data has been aggregated, the outcomes of the PIRL data will be submitted by the ETA and then displayed according to the frameworks within the Pay-for-Performance Report.”

• The issue here is that the sample report provided seems to require text entry, which means aggregating a narrative report and generating Pay-for-Performance (P4P) numbers is not possible.

Agency’s Response, June 2016: The supporting statement should be updated to reflect a full narrative report, with no calculations. Additionally, the burden associated here is for each respondent (State) to submit all the reports annually (1) to the Federal government. Therefore, the burden estimate provided will not change from the previous iteration.

Action: Revise the supporting statement to reflect a narrative-only Pay-for-Performance report.

19. ETA-9172: Participant Individual Record Layout (PIRL)

1. Comment: In response to the original publication of the DOL-ICR for a 60 Day comment period on September 1, 2015, several commenters noted that there were instances where elements were in both the Joint PIRL and DOL PIRL, but that had differences in the specifications. In Appendix A of this 60 Day ICR, DOL stated these inconsistencies had been corrected; however, staff has identified a number of instances in this newest proposal where joint elements listed in this latest DOL-PIRL don’t match the information published in the Joint PIRL on April 26, 2016. For example, all the race/ethnicity elements in the Joint PIRL have a new note relating to Vocational Rehabilitation (VR) Participants in K-12 but those same elements in the DOL-PIRL do not have this note. While it is true that this VR-specific note may not be relevant to DOL programs, the Joint elements text need to match exactly. If Rehabilitation Services Administration (RSA), DOL, and perhaps someday the Office of Career, Technical, and Adult Education (OCTAE) are each able to individually clarify the joint elements for their programs, it increases the likelihood that they will end up accidentally making a substantive change to an element that is supposed to be common for all programs. Therefore, if additional guidance is needed on a joint element, that guidance should appear in all versions of the element. That will make it less likely that RSA, OCTAE, and DOL will accidentally make an unintended substantive change.

Agency’s Response, June 2016: The Departments have made every effort to keep the “shared” data element names and valid reporting values the same, making the reporting of these elements identical and comparable. However, there may be small nuances in a particular program that does not apply to the other programs. For example, if YouthBuild has a specific need for a special instruction in any given element, the Departments believe this instruction could confuse States if also placed in the Joint PIRL.

Action: None.

2. Comment: It is unclear in the published document whether the DOL PIRL (ETA-9172) will simply represent a data dictionary or be expanded into the required report layout.  Our evaluation of the PIRL indicated that the PIRL as it exists today would serve as a data dictionary may not be a good format for a report layout.  For example, if a participant receives training the PIRL includes data elements describing the training provider, but there is no way to include multiple training providers per participant per training period.  This is just one of many similar examples.

Agency’s Response, June 2016: The PIRL (ETA-9172) is not expected to be used as an actual reporting form, but instead represents a “data dictionary” including the data element name, definition, and valid values.

3. Comment: When will the initial data collection format (e.g., spreadsheet, comma delimited text file, or software API) be released for public review and comment?

Agency’s Response, June 2016: The data collection instrument is the PIRL.  This will be collected in a comma delimited format for workforce programs. Further detail will be provided in guidance when the ICR has been approved.

4. Comment: There are no definitions for the DOL PIRL column header titles, and it is not clear what programs each column represents.  Recommend providing a detailed definition or explanation for each. 

Agency’s Response, June 2016: The column headers in the PIRL represent the DOL-funded programs.  Each program is responsible for collecting all of the data elements marked with an “X” under that program’s column.

5. Comment: Section E.08 specifically has no legislative or statutory requirement therefore we feel that this is an overburden of data collection placed on States, Local Boards and staff to attempt to collect this information.

In reviewing the data elements proposed under this section there appears to be an attempt to collect information that apply to Vocational Rehabilitation and or Adult Education where these specific requirements may have a value under these specific titles. For example, PIRL 2702 has a variety of Fund Sources that staff within Title III, Title I would have no idea how to identify.

In addition, this appears to add data collection requirements that will add to the overall overhead of States, Local Boards to design new systems, train staff and provide the necessary tools to address as individuals move from a self-service (Registered Individual) into receiving significant staff assisted services (participant).

Agency’s Response, June 2016: DOL carefully considered the inclusion of each data element and the value of the information. Statutory requirements do not specifically list all data elements that the Department believes necessary to provide a fulsome view of the programs through the data collected. For example, WIOA does not specify to collect any veteran-related data, which is a very important piece of information for most, if not all, DOL programs to collect.

At the same time, DOL realizes the impact of the collection of additional data elements on the system. ETA staff is working with colleagues in the Department’s Office of Disability Employment Policy to revise/reduce the number of data elements that will be collected in section E.08. The section of data elements referenced here is NOT expected to be collected for Titles II and IV, and is only a part of the DOL-only ICR.

Action: None.

6. Comments: While we acknowledge the comments published in Appendix A (quoted below), this data collection does not appear to be based on legislative or statutory requirements. Again, this data collection is a burden for states, state and local boards, and staff where self-service individuals are transitioning to program participants requiring considerable staff-assisted services.

“PIRL 203 is intended to disaggregate disability types as these are self-disclosed by the individual. This data element is aligned with requirements in WIOA to ensure services to individuals with disabilities, and to prohibit discrimination based on disability. The Department will provide technical assistance regarding appropriate steps to gather and store information pertaining to this data element, although as is the case with all intake data elements, it is at the state or grantee’s discretion on how to collect the information.”

Section E.08 is not based on legislative or statutory requirements. This data collection is a burden for states, state and local boards, and staff. The proposed data elements seem to attempt to collect information specific to Title IV programs (Vocational Rehabilitation) and Title II programs (Adult Education). Title I and III programs would not be able to identify the required response to PIRL 2702 (fund sources). Further, this additional collection requirement will increase the overhead for states and local boards due to a need for new system design, staff training, and development of essential tools relating to the transition of self-service individuals to program participants requiring considerable staff-assisted services.

Agency’s Response, June 2016: DOL carefully considered the inclusion of each data element and the value of the information. Statutory requirements do not specifically list all data elements that the Department believes necessary to provide a fulsome view of the programs through the data collected. For example, WIOA does not specify to collect any veteran-related data, which is a very important piece of information for most, if not all, DOL programs to collect.

At the same time, DOL realizes the impact of the collection of additional data elements on the system. ETA is working with colleagues in the Department’s Office of Disability Employment Policy to revise/reduce the number of data elements that will be collected in section E.08.

7. Comment: The new PIRL has provided information as to what information is to be collected by each specific program and this information is extremely helpful in planning system modifications. The issue is when you balance the Appendix A Comments published on the PIRL to the newly proposed layout there are over 30 instances where the ETA has stated that “Additional Guidance, technical assistance….” will be provided or that common definitions are continuing to be developed and more information will be forthcoming. This again is an undue burden for States to start addressing new data collection and system modifications because of the limited scope of information and how to best address the modifications.

A large example of the issue is that the published excel spreadsheet of the ETA-9173 was published with three columns hidden. These three columns are:

a) Reportable Individual

b) Incumbent Worker and

c) Temporary Jobs

All three of these columns have no information identified for reporting and it is critical in determining modifications required to meet new reporting that this be known very early on so that planning of modifications can occur. By not having this information States will be forced into re-evaluating data collection and process changes continually and to insure something added does not have an adverse effect on already planned modifications.

Estimated time and costs cannot be accurate without full disclosure of all data collection requirements. The data collection drives reporting, as stated in the ICR Supporting Statement, reexamination of processes with each update with clarifying information extends time and costs associated with this ICR.

Agency’s Response, June 2016: The three hidden columns were not a part of the ICR and therefore have no bearing on the burden estimates. Additionally, this ICR is to request the ability to collect WIOA data. In some cases, HOW to collect this data requires additional guidance that is not typically included in the ICR itself.

Action: None.

8. Comment: PIRL 1800 and 1802 – Type of Recognized Credential 1 and 2: The available values are different and should be changed to match:

Type of Recognized Credential 1	Type of Recognized Credential 2
1 = High School Diploma/or equivalency	1 = High School Diploma/or equivalency
2 = AA or AS Diploma/Degree	2 = AA or AS Diploma/Degree
3 = BA or BS Diploma/Degree	3 = BA or BS Diploma/Degree
4 = Graduate/Post Graduate	4 = Graduate/Post Graduate
5 = Occupational Skills Licensure	5 = Occupational Skills Licensure
6 = Occupational Skills Certificate	6 = Occupational Skills Certificate
7 = Occupational Certifications	MISSING
8 = Other Recognized Diploma, Degree, or Certificate	7 = Other Recognized Diploma, Degree, or Certificate
0 = No recognized credential	0 = No recognized credential

Agency’s Response, June 2016: These variables should match; ‘Type of Recognized Credential 2’ will be corrected to match ‘Type of Recognized Credential 1’.

Action: ‘Type of Recognized Credential 2’ will be corrected to match ‘Type of Recognized Credential 1’.

9. Comment: PIRL 1606 – Employed in 4th Quarter after Exit Quarter

Reporting on an additional quarter of employment is extremely burdensome for programs such as NFJP that rely on data gathered by case managers supplied by participants and employers. The new extended period for retention will mean a redesign of system operations and may mean a refactoring of how grantees establish caseload-to-staff ratios to absorb the additional workload into existing budgets and staffing plans. This additional time requirement may mean that grantees spend less time providing direct services with program participants and more time spend on administrative reporting requirements.

Agency’s Response, June 2016: WIOA Section 167(c)(2)(C) requires the use of the primary indicators of performance described in sec 116(b)(2) A), including the “percentage of program participants who are in unsubsidized employment during the fourth quarter after exit from the program,” and “the percentage of program participants who are in education or training activities, or in unsubsidized employment, during the fourth quarter after exit from the program.” As these are WIOA statutory requirements, they cannot be waived.

Action: No change.

10. Comment: PIRL 1807 – 1810: Records Related to Measurable Skills Gains: We request additional information on whether defined assessment standards will be required. This new standard may mean partnering with another entity, and possibly paying for this service from a third party vender. This will add an additional reporting and tracking layer to an already filled case-manager list of tasks to accomplish for each new participant. We also request that DOL guidance for this measure takes into consideration both skills that can be quantified through assessment, and other types of job readiness skills that participants may gain through counseling and practices other than classroom instruction.

Agency’s Response, June 2016: The Departments intend to provide future guidance on the primary indicators of performance, including “measureable skill gains.”

11. Comment: PIRL 203 – Category of Disability: has no statutory requirement and appears to add overburden to staff to collect this data as individuals move into significant staff assisted services. In Appendix A of the published comments ETA provided the following response to this specific issue:

“PIRL 203 is intended to disaggregate disability types as these are self-disclosed by the individual. This data element is aligned with requirements in WIOA to ensure services to individuals with disabilities, and to prohibit discrimination based on disability. The Department will provide technical assistance regarding appropriate steps to gather and store information pertaining to this data element, although as is the case with all intake data elements, it is at the state or grantee’s discretion on how to collect the information.”

12. Comment: PIRL 202 – Individual with a Disability: is sufficient to meet the requirement to ensure services to individuals with disabilities, and to prohibit discrimination based on disability.

In addition, it is not useful to data integrity when the data element can only be collected when self-disclosed.

Adding identification of the type of disability adds overdue burden to States, Local Boards and Staff because that level of detailed identification of disability will require extensive staff training. In order to meet HIPPA and EEO confidentiality requirements, system security levels must also be added to insure that this information is displayed only to those staff that require this information.

At a minimum, we feel consideration should be given to maintaining the current reporting categories, which are the following:

1 = Physical Impairment

2 = Mental Impairment

3 = Both Physical and Mental Impairments

9 = Participant did not self-identify

0 = Not Disabled Blank = Not Applicable

In addition there is an added burden on State’s for system modifications in order to maintain current reporting under Title I, Title III and Trade that trying to balance old and new data collection becomes difficult at best and very confusing for staff.

Agency’s Response, June 2016: The Departments understand there is burden associated with the collection of each data element. However, as the commenter points out, this data element will be self-attested by the participant; therefore, overall burden should be low for the case managers within the core programs.

Action: None.

13. Comment: PIRL 601 – Exhausting TANF Within 2 Years (Part A Title IV of the Social Security Act) at Entry Level: DOL stated that they modified the PIRL to include a “not applicable” option, but neither the DOL PIRL nor the Joint PIRL show such an option.

Agency’s Response, June 2016: The version that was published did not have the “not applicable” option, but the final version will have this as option 9.

Action: Add “9 = Not Applicable” to PIRL 601.

14. Comment: PIRL 1812 – School Status at Exit: It appears that PIRL 1812 (School Status at Exit) will be required for WIOA Adult, WIOA Dislocated Worker, WIOA Youth, Dislocated Worker Grants (DWG), NFJP, Jobs for Veterans’ State Grants, and Monitor Advocate Program participants, not just Youth participants.  All of the Data Element Definitions/Instructions appear to be youth-related.  We are hopeful that this was an error and will be required only for WIOA Youth participants.

Agency’s Response, June 2016: Agree. This data element will not be collect for WIOA Adult, WIOA Dislocated Worker, Veteran’s State Grants and the Monitor Advocate programs. Collection will still be required for WIOA Youth, and NFJP (Youth).

Action: Revise the PIRL to reflect the collection of this data element from WIOA Youth and NFJP only.

15. Comment: Please explain the difference between PIRL 1003 – Most Recent Date Received Basic Career Services - Staff Assisted and PIRL 1004 – Date of Most Recent Career Service (WIOA)

Agency’s Response, June 2016: There is a distinction between the two data elements.  PIRL 1003 is the Most Recent Date an Individual Received Basic Career Services (Staff-Assisted), whereas PIRL 1004 is the Date of Most Recent Career Services.  PIRL 1004 can include both Basic and Individualized Career Services, and is the most recent data an individual receives either of these services.  Whereas, PIRL 1003 captures specific staff-assisted Basic Career Services, so the elements may be the same value but often would not be.

Action: No action for the ICR; DOL will provide guidance to the system.

16. Comment: Does PIRL 1004 eliminate the need for PIRL 1003? Please explain the difference between PIRL 1100 – Most recent date accessed self-service workforce information services and PIRL 1101 – Most Recent Date Accessed Self-Service Information-Only Activities).

Agency’s Response, June 2016: Data Element 1100 is intended to capture self-services that are focused specifically on workforce information services, which include labor market information that can be accessed via self-service or remotely.  Data Element 1101 is to focus on information on other partner programs, information on supportive services, and performance information.  As well any other information available virtually or via self-service at the one-stop center.

Action: No action for the ICR; DOL will provide guidance to the system.

17. Comment: PIRL 802 – Low Income: The ICR Supporting Statement referred to this comment regarding Low Income for WIOA:

Regarding paragraph (b) of this data element specification, a commenter recommended that the Department allow parents of youth who are eligible to receive a free or reduced price lunch to be considered low income for the Adult program, instead of it being just for a youth low-income determinant.

Department Agency’s Response, June 2016:  The Department has determined that the parents of youth who are eligible to receive free lunch and reduced price lunch are eligible to be served under the low income priority provisions in the Adult program. 

18. Comment: PIRL 802 – Low Income: One State indicated that it has districts that provide free and reduced lunch to all youth attending the school.  This could pose a problem where all parents are eligible for the Adult program regardless of actual income.  A true picture of income for the parents of these youth would not be obtained.

Agency’s Response, June 2016: The collection of this data element aligns with WIOA statutory requirements, and therefore, cannot be waived.

Action: No action.

19. Comment: PIRL 400 – Employment Status at Program Entry: Many migrant and seasonal farmworkers who enroll with NFJP grantees wish to pursue full-time employment in the agricultural industry, often with employers that they work for seasonally. In these cases, participants enter training and gain skills to perform higher paying full-time jobs with the same employer and are never laid off or unemployed. We suggest that a distinction be made for these individuals who remain employed and purse promotion with their current employer to be considered as “employed seeking agricultural upgrade” to ensure that these important and meaningful placements do not negatively impact entered employment outcome calculations.

Agency’s Response, June 2016: Under WIOA the “entered employment” measure has been replaced with the “percentage of program participants in unsubsidized employment during the second quarter after exit from the program,” and “the percentage of program participants who are in unsubsidized employment during the fourth quarter after exit from the program.” An individual’s employment status at program entry does not negatively affect the “employment during the second and fourth quarter after exit” measure if the individual continues to be employed in the second and fourth quarters after exit. Thus, if an individual is employed at the time of enrollment and remains employed in the second quarter after exit, and the fourth quarter after exit they will counted towards the “employment during the second and fourth quarter after exit” measures.

(Note: this response was previously provided on 30-day public comment period for this ICR.)

Action: No change.

20. Comment: PIRL 805 – Cultural Barriers: Collecting this data is burdensome for several reasons:

• Program participants may be unaware of cultural barriers that hinder their chance at employment; therefore reporting will be inaccurate.

• It will be very difficult to pose this question to program participants in a way that will not appear bigoted, judgmental or potentially racist. The question is an attempt to quantify participants’ perception of something that is inherently both deeply personal and entirely subjective.

• DOL may want to consult an anthropologist to assess the usefulness and scientific viability of this data for study.

Agency’s Response, June 2016: This measure is based on participants’ perception of themselves; it’s not to be determined by program staff or others. DOL will provide TA on collecting these data, and may use data to help design service delivery approaches and inform future guidance.

Action: No change.

21. Comment: PIRL 413 – MSFW Designation and PIRL 808 – Migrant and Seasonal Farmworker Status: Please explain the different between PIRL 413 (MSFW Designation) and PIRL 808 (Migrant and Seasonal Farmworker Status at Program Entry).

Agency’s Response, June 2016: DOL has revised the data element names for both PIRL 413 and PIRL 808. Data element number 413 is collecting information for the Monitor Advocate System under Wagner-Peyser. Data element number 808 is collecting information for NFJP under WIOA Section 167. The Monitor Advocate System and NFJP use two different definitions. One notable difference between the two is that NFJP includes “low-income individual” to determine eligibility.

Action: The Department has revised the name of PIRL 413 to “Migrant and Seasonal Farmworker Designation (Wagner Peyser)” and PIRL 808 “Migrant and Seasonal Farmworker Status at National Farmworker Jobs Program Entry (WIOA Section 167)”.

22. Comment: PIRL 600 – 603: Information regarding TANF, SSI, and SSDI participation: Collection of this data element will be burdensome for agencies and programs, which, like NFJP, do not have access to a state system for verification of status. Grantees will face additional costs in staff time and training as well as potential charges incurred for access to state systems that will reduce the amount of funding available for participant services.

Action: No change.

Rationale: Data collection of this element is not dependent on the grantees having access to a State system to independently verify receipt of TANF, SSI or SSDI. This data can be collected from a participant via self-attestation or source documentation provided by the participant.

(Note: this response previously provided on last public comment period)

20. Cultural Barriers/Grant Programs

1. Comment: A commenter requested that the Department collect information on all the barriers defined in Sec. 3(24) of WIOA. The commenter also request the collection of the following data elements; “Low Income” as defined in Sec 3(36) of WIOA, “Other Barrier” as defined by the grantee in 4-year plan, Low Levels of Literacy/Basic Skills Deficient, and Pregnant/Parenting Youth. The commenter indicated that this data is being collected in the current data system and that continuing to collect this data will assist the Department in its authority to set performance targets which factor in socio-economic characteristics of participants served.

Agency’s Response, June 2016: The Native American program’s current Management Information System (MIS) collects all the barriers to employment as defined in WIOA, Sec. 3(24) except that it does not collect Displaced Homemaker (WIOA, Sec.3(24)(A) and Older Individual (WIOA, Sec.3(24)(E). However, for Older Individual, the age range of WIOA participants is collected and therefore a count or calculation could be made for this data element.

Regarding the collection of “Low Income”, “Other Barrier”, “Low Level of Literacy/Basic Skills Deficiency” and “Pregnant/Parenting Youth” as data elements for the Native American program, the Department has determined that data element 806 (Single Parent at Program Entry) captures Pregnant/Parenting Youth data and data element 2303 (Public Assistance Recipient) captures Low Income data. Therefore, these two data elements do not need to be added. However, the Department concurs with the commenter that the collection of all relevant barriers is an important consideration in establishing performance targets. The Department will consider adding the barriers suggested by the commenter which are not currently in the ICR. These data elements are, “Displaced Homemaker”, “Other Barriers to Employment” and “Low Level of Literacy/Basic Skills Deficiency”.

Action: Add additional elements.

2. Comment: A commenter asserted that the data elements for the INA youth program increased by 85% however the Department indicated that the majority of data elements are already being collected by the INA program.

Agency’s Response, June 2016: When the Department made its assessment of the increase in data elements for the Native American program, it viewed the data collection as a combination of Adult and Youth data elements. However, there is a significant increase in new data elements for the Youth program when the programs are viewed separately but it should also be noted that some of the current data elements relating to the Youth performance indicators will no longer be needed which offsets some of the reporting burden of collecting new data elements.

3. Comment: A commenter expressed concern that the collection of individual records is new to the Youth program and will be costly to implement and the median funding amount for Native American youth grants for PY 2014 is $30,000. The commenter requests that the Department provide additional funding and support to ensure a successful transition to WIOA Youth reporting requirements.

Agency’s Response, June 2016: The Department recognizes that small grantees will not have the expertise and resources to develop a system that collects and transmits individual participant records to the Department. Therefore, the Department will provide, and is currently providing, resources and expertise to develop a management information system for the Native American program.

4. Comment: A commenter raised a concern about setting performance targets for the Native American program since there is limited, and in some cases, not statistical data for American Indians and various tribal areas. The commenter also hopes that there will be meaningful dialogue and strategic planning in setting INA performance goals.

Agency’s Response, June 2016: The Department recognizes that labor market information is limited on American Indian reservations and may not be as accurate as non-reservation areas. However, the statistical adjustment model will also factor in the characteristics of participants served by the grantee and therefore it is not totally dependent on labor market information. Nevertheless, the limitation of labor market information on American Indian reservations increases the importance of consulting with the Native American Employment and Training Council (NAETC) when establishing performance targets for grantees and it is the Department’s intent to establish fair and reasonably attainable levels of performance for all INA grantees.

5. Comment: A commenter indicated that it looks forward to working with the Department in consultation with NAETC and its expert workgroup members in addressing commitments made by the Department related to performance indicators, management information systems and reporting.

Agency’s Response, June 2016: The Department is committed to working with the INA grantee community and the NAETC in all of these areas.

6. Comment: Cultural barriers at program entry. The definition is as follows:   A participant who perceives that they possess attitudes, beliefs, customs or practices that influence a way of thinking, acting, or working that serves as a barrier to employment.   Question for DOL:   is this a subset of the “Individual who requires additional assistance to complete an educational program or to secure/hold employment” eligibility criteria?   Can the applicant self-certify to cultural barriers at program entry? 

Agency’s Response, June 2016: Cultural barrier is a separate data element and is not tied to Youth eligibility criteria. Yes. Participants will be able to self-identify this element.

Action: None.

7. Comment: Under WIA, NFJP grantees were able to extend the eligibility period for individuals who were unable to work due of circumstances such as incarceration beyond the most recent 24 months. The proposed wording of the PIRL would disqualify those individuals.

Action: No change related to this comment. However, see the response to the above comment related to PIRL 413 (MSFW Designation) and PIRL 808 (Migrant and Seasonal Farmworker Status at Program Entry).

Rationale: The data element reflects the WIOA statutory definition of seasonal farmworkers and migrant farmworkers (sec 167(i)(2) and 167(i)(3).

21. Burden

1. Comment: Beginning on page 27 of the 1205-0521 Supporting Statement ETA states “The Department estimates that it takes participants 15 minutes to provide their WIOA-related information to the program coordinators reflected in Table 5. Each program in the 52 states and 5 outlying areas has different requirements of the information it needs to collect for WIOA purposes and those are delineated below in Table 6. It should be noted that the participant burden recorded in this ICR is separate from the 15-minute burden associated with the PIRL in the Joint WIOA Performance ICR.”

a. Time estimates in table 5 for participants to provide the information is under-estimated by half.

b. Time estimate in table 5 for participants does not include the time required to gather required verification documents that validate responses to required data elements.

c. Time estimates in table 6 for staff to collect the information provided is under-estimated by half.

Agency’s Response, June 2016: For DOL programs, the overall burden is divided across the two performance-related ICRs.

Action: None.

2. Comment: Section 13, beginning on page 33, states “There will also be an associated start-up cost for each respondent to implement WIOA. These costs include: enhancements to data collection systems to adhere to WIOA requirements, training staff on data collection rules, and for ETA there is an associated IT cost for transitioning data intake systems to collect exit information in a new method (e.g., data systems must be able to re-open closed cases if a participant re-enters the program within a given program year). Additionally, State data reporting systems must be programmed to include all the necessary elements listed in the PIRL, as well as the capability to submit an individual record file in a specified format to ETA. It should also be noted that there is no start-up cost associated for the Job Openings report, as the current requirements for that report are not changing. These costs are listed below; it should be noted that the costs included here are one-time only. The estimated start-up cost is based on previous iterations of State grantees upgrading performance reporting systems. The one time start- up cost for the first year is $15,000 and $5,000 a year for maintenance for the remaining 2 years. Cost over 3 years is $25,000 /3 =$8,333:

a.    “The estimated start-up cost is based on previous iterations of State grantees upgrading performance reporting systems.” The first year estimate of $15,000 is under estimated and cannot be evaluated without estimated hours that were not provided. This ICR changes data collection processes that require major reprogramming of state systems to accommodate the new applicant/participant flow in one stops, reflected in the following:

i. Self-service reportable services that do not extend soft exit, and

ii. New data collection required for staff-assisted services at the basic career services level as noted above, and

iii. Determine eligibility and collect associated documentation for each program prior to program entry, and

iv. Collect 17 data elements, formerly only 3, at Program Entry – which is not clearly defined in an integrated business model.

b.   The maintenance cost estimate of $5,000 per year for two additional years is under estimated considering all of the “future” guidance noted throughout documents; therefore, the costs cannot be evaluated without estimated hours not provided.

Agency’s Response, June 2016: After revisiting these cost estimates, the Department believes these are an accurate representation of an average cost.

Action: None.

22. Title I Youth Program

1. Comment: Youth who live in a high-poverty area are counted as low income.   Youth between 14 and 24 at the time of enrollment who live in a “high poverty area” are automatically eligible for WIOA youth—regardless of any other factor(s). We expect the final WIOA regulations will clarify the definition of “high poverty area.”   For example, we expect that any youth between the ages of 14 and 24 living in the federal “Promise Zone” in North Minneapolis would automatically qualify for WIOA. More clarity would be welcomed.

Agency’s Response, June 2016: This level of specificity is not included in this ICR; the final rule section for youth will include a definition of a high poverty area.

Action: None.

2. Comment: Definition of homeless youth expanded, now including, in addition to previous definition, participants living in a hotel, motel, trailer park, or camping ground due to a lack of adequate housing; participants sharing the housing of other persons due to loss of housing, economic hardship, or similar reasons; participants abandoned in a hospital; participants awaiting foster care placement; or a migratory child who in the preceding 36 months was required to move from one school district to another due to changes in the parent’s or parents spouse’s seasonal employment in agricultural, dairy, or fishing work.  The expansion of the definition of a homeless youth is positive in terms of recognizing the significance of homelessness.    For those youth who were previously enrolled as NOT HOMELESS, but now meet the expanded definition, to what extent (if any) do local service providers have to go back and reclassify these youth accordingly?

Agency’s Response, June 2016: Local service providers are not required to go back and reclassify youth participants.

Action: None

23. Data Sharing Agreements

Comment: Furthermore, though we appreciate that the Departments are committed to renegotiations of the WRIS and WRIS2 data at the federal level, the State agencies still need significant lead time to set up their data sharing agreements and data exchange processes.  VR agencies currently not reporting on retention measures will need time to set up the agreements and information exchange processes with their respective UI agencies to include State, WRIS/WRIS2 and federal wage information to accommodate their case management systems. Our VR agency, which manages tens of thousands of exiters annually, will need to rely on automated data exchange to ensure timely and accurate reporting for retention.

Agency’s Response, June 2016: WRIS is not covered or discussed in this ICR. However, the Departments are working with States in the process to revise data sharing agreement.

Action: None.

24. Issues regarding Timeline/Transition/Compliance Date/Implementation Period

Comment: The Departments recognized that several of the data elements are new to VR programs and that agencies need time to implementation changes in data collection methodologies and systems.  The revised timeline for implementation of the RSA 911 will be published in the ICR.  It would be helpful for States to understand the expectations for report submission as soon as reasonably possible.

The Departments have indicated that States will not be required to retroactively collect data required by WIOA, specifically this was discussed for the skills gain attainment and Title IV only needing to report on participants from July 1, 2016 forward.  Would the Departments please clarify the first due date for reporting on measures for employment 2Q from exit, employment 4Q from exit, and earnings 2Q after exit?  In other words, will these retention and associated median earnings measures only be counted for Title IV active and exiting participants from July 1, 2016 and thereafter? Or will the 2^nd and 4^th quarter retention and median earnings need to reported on the first report (due November 15, 2016) for participants that exited Title IV prior to July 1, 2016?  

Agency’s Response, June 2016: These items will be key discussion points in the upcoming joint guidance to States.

Action: None.

1 Making an element previously required 20% of the time mandatory 100% of the time represents a significant expansion.

2 Commenter does not object to providing additional information that it already captures through other means, such as pre-participation wage records as there is no impact on the Participant and the delivery of services.

3 Though a case can be made that these should be updated because if a person did not have a barrier to employment at the Date of Participation but during their POP “gained” one, that should be reported under §116 which requires performance to be broken out by barrier to employment.

4 Although the worker had rumors of a layoff, they had not been notified that they were affected and thus the proactive Participant would not be coded as “Employed but Received Notice of Termination.”

30