Responses to Public Comments

ATTACHMENT H

RESPONSE TO PUBLIC COMMENTS

This attachment contains responses to public comments on Apprenticeship Evidence-Building Portfolio. The 60-day comment period closed on February 16, 2020. The table below includes the comments and the contractor’s responses.

Response to Public Comments on OMB Control Number 1290-0NEW
Comment	Response
From AFL-CIO
DOL does not have the statutory authority to establish Industry Recognized Apprenticeship Programs (IRAPs) and they should not be included in this ICR.	IRAPs were mistakenly included in the ICR. References to them have been deleted. However, please note that DOL has provided additional guidance on IRAPs since the release of the FRN.
IRAPs must not be conflated with Registered Apprenticeship Programs.	IRAPs were mistakenly included in the ICR. References to them have been deleted. However, please note that DOL has provided additional guidance on IRAPs since the release of the FRN.
The proposed ICR fails to examine the critical protections and quality assurances afforded apprentices under Registered Apprenticeship and the long term outcomes that are the hallmark of RA programs.	This is an important qualification. Thank you for noting this. One component of the study is to compare different apprenticeship models and to understand the attributes of different apprenticeship programs. We will gather information on whether unregistered apprenticeships include some of the attributes of registered apprenticeship outlined in the comment letter, including apprenticeship contracts with employers, the role of journey worker mentors, and the offering of nationally recognized credentials. We have added these to the interview topic guides for program staff and partners.
The proposed Project Schedule is problematic.	The schedule reported in the PRA is the earliest dates for focus groups and a final report. The project allows for follow-up and final reporting through the end of September 2024. DOL may issue another contract to extend the follow-up period beyond the end of this contract.
Impact Evaluations as proposed are problematic.	The challenges to RCT (including incumbent workers) is understood by the study team. In addition to an RCT, the study is considering other design options—including quasi-experimental designs (QEDs). The design selected will depend on the study questions, grantee programs, and other relevant factors. The final design option(s) will be chosen upon consultation with DOL and experts from the Technical Working Group.
From North America’s Building Trades Unions (NABTU)
There are serious questions regarding the legal foundation of the Industry-Recognized Apprenticeship Programs (IRAPs), which ETA proposes to include in this study.	IRAPs were mistakenly included in the ICR. References to them have been deleted. However, please note that DOL has provided additional guidance on IRAPs since the release of the FRN.
The research design is seriously flawed.	The concerns brought up relate to evaluation of IRAPs. IRAPs were mistakenly included in the ICR. References to them have been deleted. However, please note that DOL has provided additional guidance on IRAPs since the release of the FRN.
The programs will have to be in operation long enough to actually have apprentices who have completed, or at least, be well advanced in, their training.	The follow-up period can continue through the end of the current contract, with final reporting to be completed by the end of September 2024. DOL will have the ability to evaluate a longer term follow-up period through a new contract.
From Professor Tingting Zhang
In the baseline survey, in addition to the question of the highest education, would it be possible to ask the sequence of their educational path? For example, have the survey respondents been working full time in between two levels of education?	This is an important question, but would require a number of complex response options, as this could vary considerably from person to person. It will not be possible to add such a series of questions at this time to the survey.
Question C9 on pg. 11 asked whether the employer referred to the survey respondents to the specific program for training. Is this question the beginning of the training module? Can we ask the question about the intention to participate? Such as are you willing to participate in a specific program if the program is available? Do you plan to enroll in the specific program for a career change or upskilling required by the employer?	We have added one question to the baseline survey to collect information on this issue (page A-3): What is the main reason you are seeking to participate in this program? Response options: 1) I want a career change, 2) I want to gain more skills in my current field, or 3) other: (specify).
Can we ask follow-up questions such as the length of the training, whether respondents receive release time to participate in training and the job tenure of the respondent?	To get at these issues would require a series of questions and would burden incumbent workers more than other types of respondents. Incumbent workers are also a small subset of participants.
I noticed the LinkedIn question D8 on pg. 16. Will we be able to connect the LinkedIn profile to the survey data?	We are collecting LinkedIn information in order to help locate the sample for future follow up, and it does link to the survey data as it is a question within the survey.
I am curious about the section E in which staff will assess the likelihood of participation of a specific program or obtaining an industry-recognized credential. What type of information will the staff use to make such an assessment? I want to get more information.	We are relying on staff experience to make these assessments. The purpose of the question is to try to determine if the assessment is predictive of actual participation and credential receipt, can use that to determine which control group members would have participated and gotten a credential, and will help the estimation of impacts.
The interview guides are very detailed; hence, the interview would be extremely time-consuming. How much of the information can be obtained outside the interviews? In addition, I would like to know additional key features of the apprenticeship programs including the apprentices/journey person ratio, type of certificates (Certificate/License/Attestation and other short program credentials/Documentation of Achievement or Completion) awarded, whether those programs qualify for career, technical or pre-college, whether those programs are post career, technical or professional training programs, what language is used during the training, and whether the training is recognized in other states.	The additional items you suggest are of interest. Since as you note the topic guides are already detailed, some of this additional information will be collected through a future planned grantee program survey data collection that will be covered in a future PRA package.
For program participants, I am interested in the apprenticeship experience. For the starter, information about the discontinued program (canceled, suspended, or terminated) or transferred to another program, and the job hours completed for duration-based programs.	The follow up survey will have a section on experience with workforce development and education and training services that will include these factors.
A detailed description of the subsequent surveys was not provided at the moment. As for subsequent labor market outcomes, I would like to know whether the respondents are employed or self-employed and whether they are working in their trained field. I am also interested in whether they have participated in any additional training programs since the last survey.	The follow up survey will have a section on labor market outcomes that includes these factors.