Summary of Comments Submitted in Response to 30-day FRN and Agency Responses

DOL-Only Performance Accountability, Information, and Reporting System 30-Day FRN Public Comment and Agency Response

ICR REFERENCE # 201710-1205-004

OMB Control No. 1205-0521

Department of Labor

Information Collection Request

DOL-Only Performance Accountability, Information, and Reporting System

Summary of 30-day Federal Register Notice (FRN) Comments and Responses

Executive Summary (Overview of Document)

The Department of Labor (DOL) published a 30-day Notice in the Federal Register (FRN) seeking public comments concerning proposed extension for the authority to conduct the information collection request (ICR) titled, “DOL-Only Performance Accountability, Information and Reporting System” on October 30, 2017 (OMB ICR Reference Number 201710-1205-004).

This document provides a summary of the 11 sets of public comments received in response to the 30-day comment Notice on the DOL only ICR and the Department’s responses to those comments. The Department has organized this Summary of Comments and Responses by issues raised by the commenters, the particular forms of the ICR documents and by programs.

The comments are organized into 7 sections as outlined in the table of contents below. Each comment contains a comment number in the first column. The actual comment received appears in the second column. The third column provides the agency’s response.

TABLE OF CONTENTS (Hyperlinked to each section)

DOL-ONLY PIRL

Reportable Individual

Program Report Specs

TEGL 10-16 References

WIOA Youth

NFJP

Miscellaneous/No Action

AMENDED DOL-Only ICR – COMMENT RESPONSES
#	COMMENT	DEPARTMENT RESPONSE
DOL-ONLY PIRL
	The addition of the IWT program column Incumbent Worker Training may or may not produce a recognized credential, or a measurable skill gains. Why is this data required for an Incumbent Worker if they are not participants in the program? (Commenter) recommends removing these data collection requirements as it will place a burden on the employer.	Although the Department is not including individuals that only receive IWT in the performance accountability calculations, the Department is still interested in the outcomes. In addition, the employer does not have sole responsibility for collecting these requirements, as the state, locals, and training providers can all be a part of these outcomes.
	TEGL 10-16 indicates that certain information is required for IWT participants including demographic information. The IWT column in the PIRL does not include an “R” for most of the demographic data elements. It appears that the IWT requirements in the PIRL may be incomplete.	All of the demographic data the Department intends to collect is reflected in the IWT column of the PIRL.
	The performance data elements that are required for IWT rely on an exit date for calculations, but the PIRL data element 901 – Date of Exit is not required for IWT.	The Department has made PIRL 901 required for IWT in addition to other revisions to align the IWT column with published guidance. The Department will make any necessary changes to guidance upon approval of this ICR.
	PIRL 307 – TAP Workshop in 3 Prior Years This was removed from the LERS, but is now being brought back. To reduce the data collection for Wagner-Peyser staff, (Commenter) recommends that this data element be removed.	The Department is maintaining this requirement. The collection of this data element is important in assessing whether more recently separated veterans/transitioning service members have had access to the employment information provided by TAP workshops.
	PIRL 413 – Migrant and Seasonal Farmworker Designation (Wagner-Peyser) This element is now required for Incumbent Worker Training. This additional data collection places a burden on the employer providing the training. (Commenter) recommends removing this requirement for IWT only participants.	The Department has unchecked the requirement for IWT to collect for PIRL 413
	PIRL 601 – Exhausting TANF Within 2 Years The Wagner-Peyser program is not a case management program, and thus they enroll multiple people daily. Adding this additional data element places a burden on Wagner-Peyser staff to collect additional data prior to providing the client the services they need. Additionally, Wagner-Peyser staff do not have access to TANF records, so the answer to this question may be inconsistent and inaccurate as it would rely on jobseeker self-attestation. To ensure accurate data collection, and reduce the burden on the Wagner-Peyser program, (Commenter) recommends that this data element be removed as a requirement for Wagner-Peyser.	This element represents a WIOA defined barrier to employment that must be reported on for all of the WIOA core programs so that the data may be disaggregated in accordance with statute. TANF is a required one-stop partner program, as is Wagner-Peyser. The programs are encouraged to work together to share this information whenever possible to ensure consistent data.
	PIRL 907 – Recipient of Incumbent Worker Training Why is this data element required for Wagner-Peyser and JVSG? These programs do not send participants to Incumbent Worker Training, so it is not appropriate to include it for these programs. (Commenter) recommends that this data element be removed for Wagner-Peyser and JVSG.	Although Wagner-Peyser and JVSG cannot provide training, it is valuable to know which co-enrolled participants have received training. The Department will keep this requirement.
	PIRL 1101 – Most Recent Date Accessed Self-Service It appears that the data element name has been cut short. Recommend adding “activities” or some other identifier after “self-service.”	The Department has revised the name of data element 1101 to “Most Recent Date of Self-Service Activities”.
	PIRL 1300 – Received Training (WIOA) This data element is now required for Wagner-Peyser and JVSG, which do not place participants into training. Please provide clarification around the reporting requirements for these programs related to this data element; i.e. is this just for coenrolled participants?	Although Wagner-Peyser and JVSG cannot provide training, it is valuable to know which co-enrolled participants have received training. The Department will keep this requirement. For programs that cannot provide training, such as Wagner-Peyser and JVSG, this element may be left blank if unknown.
	PIRL data elements 2220, 2221, 2219, 2217, and 2218 related to number of hours of training provided. Comment: These data elements can only accept a number that is three integers in length; accordingly, the number of hours the grantees can report is restricted to 999. Participants may train more than 999 hours, and that data currently cannot be captured. We recommend adjusting these data elements to be able to accept at least four integers.	The Department concurs and has expanded the field length of PIRL 2217 – 2221 to accept at least four integers.
	In light of the removal of PIRL Elements 1100-1115, grantees are concerned that Elements 1004 and 1116 are essentially the same. By definition, Element 1004 includes basic and individualized career services and excludes self-services, information services or activities, and follow-up services, which are career services that do not require significant staff involvement. This leaves basic and individualized career services that require a significant expenditure of staff involvement. By definition, Element 1116 includes basic career services requiring a significant expenditure of staff involvement. Basic career services reported in Element 1116 cannot, by definition, be recorded in data elements 1102-1115 since Elements 1102-1115 are no longer reported. The data that would be included in Element 1116, therefore is technically not “additional” or “other.” The most recent date of the services will be reported in both Elements 1004 and 1116. Grantees request guidance on whether they report a value for 1004 and just leave 1116 blank. In addition, should grantees include all Basic Services in 1116 and both basic and individualized services in 1004?	1116 represents participants receiving basic services only. So, while there is overlap in counting basic services with 1004, the purpose of the cells requires that the dates be recorded separately. 1116 is providing the count of staff assisted basic career services. That element number is pulled into demographic analyses, as well as the aggregate performance measures. Many programs report using the PIRL. Other programs are using the 1102-1115 data elements. Therefore, the words “additional” and “other” apply to them. Do not leave either field blank because of perceived overlap. Grantees must include the most recent date of basic or individualized career services in 1004. Grantees should leave 1004 and/or 1116 blank only if the participant did not receive the applicable career services
	Regarding the PIRL, it would be beneficial to capture additional information regarding veterans - more specifically those transitioning out of active duty.  The PIRL should collect: - The date transitioning service members (TSMs) received the required "warm handover" from DOD. - TSMs evaluated as not having an adequate post-transition housing plan and who receive services from their local American Job Center. Adding these fields will be beneficial to the evaluation of the TAP program as well as DOL's homeless veteran program.	The Department will not be making the suggested revisions at this time.
	401 – IC Eligible Status (This is not feasible since INA grantees would not have access to state integrated data systems to capture this information.) 407 – Highest School Grade Completed at Program Entry (This description is confusing, would create burdens on case managers and clients, and would not necessarily be accurate data through “word of mouth” information. 408 – Highest Education Level Completed at Program Entry (This description is confusing, would create burdens on case managers and clients, and would not necessarily be accurate data through “word of mouth information.) 409 – School Status at Program Entry (This description is confusing, would create burdens on case managers and clients, and would not necessarily be accurate data through “word of mouth information.) RECOMMENDATION: It is therefore recommended that the WIOA Section 166 data elements be further reviewed and re-evaluated with the Native American Employment & Training Council (NAETC) as previously described and to meet the outcomes that OMB is interested in: ● Evaluated whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information will have practical utility; ● Enhance the quality, utility, and clarity of the information to be collected; and ● Minimize the burden of the collection of information on those who are to respond, etc. Recommend that consultation with NAETC be made to include the INA additional performance measures.	The Department considers the burden associated with collecting these elements to be relatively low as they can be mostly collected through self-attestation. The Department will consider these comments when developing guidance on data validation.
Reportable Individual
	PIRL 903 – Adult (WIOA) Seeking clarification on the appropriate circumstance to record “Reportable Individual” for this data element. For States with multiple programs utilizing the same MIS system, the requirement to report a funding stream for reportable individual poses a problem as program applications/eligibility are not required to provide reportable services. Please provide clarification on when to report an Adult as reportable only.	The determination of who is a reportable individual for each program will depend on the service delivery strategy being implemented. For example, some States may choose to use only title III funds to provide services to reportable individuals who are not participants. Other States may decide to serve this same group using funds from title I Adult, title I Dislocated Worker, and title III Wagner-Peyser. The program that reportable individuals are reported under should be reflective of this strategy.
	PIRL 904 – Dislocated Worker (WIOA) Seeking clarification on the appropriate circumstance to record “Reportable Individual” for this data element. For States with multiple programs utilizing the same MIS system, the requirement to report a funding stream for reportable individual poses a problem as program applications/eligibility are not required to provide reportable services. Please provide clarification on when to report a Dislocated Worker as reportable only.	The determination of who is a reportable individual for each program will depend on the service delivery strategy being implemented. For example, some States may choose to use only title III funds to provide services to reportable individuals who are not participants. Other States may decide to serve this same group using funds from title I Adult, title I Dislocated Worker, and title III Wagner-Peyser. The program that reportable individuals are reported under should be reflective of this strategy.
	PIRL 1007 – Date of Most Recent Reportable Individual Contact Please provide clarification on the necessity of this data element, and how it differs from data element number 1002.	PIRL1002 refers to delivery of specific services, while PIRL1007 refers to various levels of contact with an individual that would qualify them as a reportable individual and is used to aggregate counts of reportable individuals for all applicable DOL programs. For example, providing identifying information prior to enrollment in certain programs would make an individual a reportable individual. If there is a date in PIRL1002, then there should be a date in PIRL1007, but a date in PIRL1007 does not always mean there will be a date in PIRL1002.
Program Report Specs
	Field Label C.8: Eligible migrant and seasonal farmworkers Why are we utilizing the WIOA sec 167 definition for all programs? The NFJP program serves a small percentage of WIOA participants, so (Commenter) recommends that the calculation for this line item be changed to include PIRL data element 413 – Migrant Seasonal Farmworker Designation.	WIOA programs are reporting on PIRL 808 to track eligible migrant and seasonal farmworkers, as defined in section 167(i) which is listed as individuals with barriers to employment as defined under WIOA Section 3(24) j States have a responsibility under Wagner-Peyser Act regulations to provide employment services, benefits, and protections to migrant and seasonal farmworkers (MSFWs) on a basis that is qualitatively equivalent and quantitatively proportionate to services provided to non-MSFWs. Since Local Employment Services offices must determine whether participants are MSFWs as described at 20 CFR 651.10., PIRL 413 is used for this purpose. These two PIRL data elements are distinct from each other.  Other programs, besides Wagner-Peyser and NFJP, track what kind of services the workforce system is providing to MSFWs. The Monitor Advocate System is the method by which the Department, by monitoring established service level indicators, ensures the equitable provision of Wagner-Peyser Employment Services to these two populations (MSFWs and non-MSFWs) of participants in the WIOA title III ES program. These are not for performance accountability purposes.
	Field Label D.5: Measurable Skill Gains Why is Other Reason for Exit = 007 included in the Measurable Skill Gains calculation for WIOA Adult/Dislocated Worker? Per TEGL 10-16, Criminal Offender takes a WIOA Youth out of the MSG denominator. This should only be included for Title II.	This is consistent with the Measurable Skill Gains specification for the annual report. The commenter is correct, 07 is not an allowable code value for titles I and III, but the Department will not change the specification in order to keep them consistent across reports. Instead, code value 07 will be rejected by edit checks within the WIPS reporting system.
TEGL 10-16 References
		In accordance with the commenter’s suggestion, the Department has removed 22 elements (#413, #1902-1922) and added 3 elements (#900, 901, 908) from the IWT column to align the IWT requirements in the PIRL to those in TEGL 10-16 resulting in a net reduction in burden of 18 elements. The Department will make any necessary changes to guidance upon approval of this ICR.
WIOA Youth
	Field Label C.7: Eligible migrant and seasonal farmworkers Why are we utilizing the WIOA sec 167 definition for all programs? The NFJP serves a small percentage of WIOA participants, so (Commenter) recommends that the calculation for this line item be changed to include PIRL data element 413 – Migrant Seasonal Farmworker Designation.	WIOA programs are reporting on PIRL 808 to track eligible migrant and seasonal farmworkers, as defined in section 167(i) which is listed as individuals with barriers to employment as defined under WIOA Section 3(24) j States have a responsibility under Wagner-Peyser Act regulations to provide employment services, benefits, and protections to migrant and seasonal farmworkers (MSFWs) on a basis that is qualitatively equivalent and quantitatively proportionate to services provided to non-MSFWs. Since Local Employment Services offices must determine whether participants are MSFWs as described at 20 CFR 651.10., PIRL 413 is used for this purpose. These two PIRL data elements are distinct from each other.  Other programs, besides Wagner-Peyser and NFJP, track what kind of services the workforce system is providing to MSFWs. The Monitor Advocate System is the method by which the Department, by monitoring established service level indicators, ensures the equitable provision of Wagner-Peyser Employment Services to these two populations (MSFWs and non-MSFWs) of participants in the WIOA title III ES program. These are not for performance accountability purposes.
	Field Label D.5: Measurable Skill Gains Why is Other Reason for Exit = 007 included in the Measurable Skill Gains calculation for WIOA Youth? Per TEGL 10-16, Criminal Offender takes a WIOA Youth out of the MSG denominator. This should only be included for Title II.	This is consistent with the Measurable Skill Gains specification for the annual report. The commenter is correct, 07 is not an allowable code value for titles I and III, but the Department will not change the specification in order to keep them consistent across reports. Instead, code value 07 will be rejected by edit checks within the WIPS reporting system.
	The “youth participants who have received occupational skills training,” “total current period” and “total previous period” sections require the youth to have received an individualized career service to be included. If there is no sequence of service requirement under WIOA, the first youth element provided to the youth participant may be occupational skills training, which would not place them into the MSG indicator as it is written now. (Commenter) recommends that the language “date of first individualized career services is not null” is removed or rephrased.	The Department agrees with the commenter and has deleted the language “date of first individualized career services is not null” from the MSG indicator specifications.
	PIRL 1913 – Date of Most Recent Post-Test Score #2 The language indicates “record the date on which the post-test was administered to the participant during his/her first year of participation in the program.” This language appears to contradict the language in PIRL 1906, which indicates that if multiple post-tests were administered then record the most recent date. (Commenter recommends updating the language in 1913 to remove “first year of participation in the program” to ensure an EFL gain in subsequent PYs can be recorded.	The Department agrees with the commenter and has deleted the phrase “during his/her first year of participation in the program.”
	PIRL 1920 – Date of Most Recent Post-Test Score #3 The language indicates “record the date on which the post-test was administered to the participant during his/her first year of participation in the program.” This language appears to contradict the language in PIRL 1906, which indicates that if multiple post-tests were administered then record the most recent date. (Commenter) recommends updating the language in 1920 to remove “first year of participation in the program” to ensure an EFL gain in subsequent PYs can be recorded.	The Department agrees with the commenter and has deleted the phrase “during his/her first year of participation in the program.”
NFJP
	PIRL 808 – Migrant and Seasonal Farmworker Status at National Farmworker Jobs Program Entry (WIOA sec. 167) Why is this data element required for Title I, Title III, DWG and JVSG? These programs do not administer the NFJP program, so it is not appropriate to expect them to capture this information at NFJP program entry. Additionally, the definitions under this data element do not align with PIRL 413, so it would create confusion when trying to determine which MSFW definitions the client meets. (Commenter) strongly recommends that this data element be removed from Title III, Title I, DWG and JVSG.	PIRL808 uses the definition of MSFW barrier to employment established in WIOA sec. 167 as required. PIRL413 refers to MSFW as it relates to WP and Monitor Advocate only. PIRL808 is not specific to NFJP program entry and has been renamed to avoid further confusion.
	PIRL data element 1315, Type of Training Service Number 3 Comment: When the PIRL captures Type of Training Service Number 1 and 2, grantees provide “start” and “end” dates for each training service. Please correct this inconsistency. Additionally, NFJP grantees recommend that, rather than collecting multiple start and end dates for just two training services, the agency collect the start and end date of the most recent training because participants may participate in three or more trainings.	The Department has added the requirements for NFJP for PIRL 1314, 1317, 1318.
Miscellaneous/No Action
	This response is submitted on behalf of the Wintu Tribe of Northern California, By vice-Chairman Gary Rickard. The Wintu Tribe of Northern California is in full support of California Indian Manpower's response to comment request ICR 1205-0521.  We believe the additional cost of the proposed data being requested by DOL is excessive to many of the small grantees and will take away from their ability to provide needed services.  Additionally, DOL has underestimated the staff time required to collect and submit the required data, also the cost.  We request the Department hold Tribal consultation on this matter.	The Department acknowledges these comments, however either a response is not warranted, further action from the Department is not needed, or were non-substantive and/or unrelated to the contents of this information collection request.  Consequently, we will not respond to those comments.
	On the PIRL file, number 401, UC Eligible Status, is not something that would be easy to collect for participants during the intake process. Not sure who is expected to know the specifics to this criteria collection. Can there be some clarity on what type of performance 401 captures?
	There is little accuracy to the estimate provided, the time and cost suggested is too low especially since an additional quarter has been added to the follow up procedures. 12 months is a lot of time to devote to follow-up procedures.
	The idea that one data collection reporting system will work for all DINAP Grantees is not realistic. The reporting and collection of information should not try to be a “one size fit all” for DINAP Grantees. Example, there are grantees that service 50 participants compared to another grantee with a population of 400,000 to serve. Expecting participants to continue to keep in contact is an on going barrier. Capturing data on people who do not want to be contacted or plan to use services and leave is a burden when extended an additional three months.
	Grantees are concerned that the inclusion of both basic and individualized career services in 1004 overlaps with reporting Elements in the 1200s (Individualized Services).
	Grantees need further guidance on which code should be used in PIRL data element #1303 to reflect classroom training (community college, technical school, etc. leading to a certificate, credential or degree). In element #2220, grantees are instructed to record total hours of occupational skills training received, which “includes vocational education and classroom training.” This suggests that the correct choice for classroom training in #1303 might be “06 - Other Occupational Skills Training,” but the instructions say that 06 should only be used in rare instances when other codes are clearly not appropriate.
	Below are recommendations for regulatory language and implementation considerations relating to the DOL-Only Performance Accountability, Information and Reporting System (OMB CONTROL No. 1205-0521). This document was drafted by New York Association of Training & Employment Professionals (NYATEP) members and supporters, and includes feedback from a range of stakeholders in the fields of education, workforce, and economic development in New York State. We deeply appreciate your consideration of these recommendations. Workforce Innovation and Opportunity Act (WIOA) Title I WIOA Sec. 116 (29 USC 3141) Individuals with Barriers to Employment, page 4 -5 In addition to the “such other groups, as the Governor determines to have barriers to employment, there should be an additional bullet that indicates “individuals deemed in need of additional support” by the local workforce development board policy”. Pay-for-Performance, page 5-6 In cases where the local workforce area enters into a contract for services, the performance milestones that determines contractor effectiveness should be locally negotiated with the service provider. Additionally the local workforce board is responsible for setting the local standard and measuring continuous improvement and program effectiveness. WIOA Sec. 169 (29 USC 3224) Continuing Evaluation of Programs, page 6 The State should be required to document its policies and procedures related to continuous improvement and measurement of program effectiveness within the State WIOA Plan. The State should consult local workforce boards to understand the structure, design and delivery of locally operated services, and work collectively to develop regional goals to improve service delivery and collect appropriate data. Additionally, State’s should be required to include within the narrative of the Pay-for-Performance reports the local areas engaging in Pay-for-Performance, the value of the contract, and the WIOA outcomes achieved for each contract. Trade Adjustment Assistance Act Program (TAARA) TAARA sec. 239(j) (Agreements with States. [19 USC 2311Ij)] Accessibility of State Reports, page 10 We are in agreement that all state performance reports must be made available, through electronic means, to the public. The report should be posted within 90 days of the close of the program year. We recommend consulting with Bureau of Labor Statistics for best practices in designing and developing easy to read, meaningful reports. Wagner-Peyser Employment Service Wagner-Peyser Act sec. 3(c), 29 USC 49b(c) Continuous Improvement, page 12 The Secretary should consult State and local workforce development boards to provide feedback into the ‘continuous improvement models” for the national system and for the collection of customer ‘satisfaction’ within the system. There is a need to ensure that any continuous improvement models across WIOA align. Additionally, the WIOA funded system should also meet the demand of ‘career-seekers’ not solely those out of work and seeking employment. Electronic Data Collection and Burden Reductions, page 17 – 19 Local workforce areas are appreciative of the Departments efforts to continuously reduce the data collection and reporting burden. However, a critical aspect of data collection, that is time consuming and burdensome, is data validation. Data validation ranges from pay stub collection, to States requiring ‘proof’ of receiving various forms of ‘assistance’. We urge the Department to strongly encourage States to adopt policies and procedures to reduce the burden of data validation, in particular in instances where the state currently holds the data for other programmatic uses, like public assistance, disability services, housing, among others. The State should be validating this information using electronic means or providing local workforce areas with access to this information for purposes of validation. Data Security and Confidentiality, page 21 We appreciate the Department’s attention to the importance of data confidentiality to protect individual and employer participants. We encourage the Department to require the State to articulate in the State WIOA Plan the policies and procedures to increase data sharing across WIOA-funded programs (i.e. Title I, II, III, IV), mandated partners and other federally funded programs. As well as require the State to increase procedures to ensure data confidentiality and practices to protect local workforce areas around cybersecurity.

4