1205-0426 Supporting Statement A 102822 clean

Post Enrollment Data Collection for Job Corps Exiters

OMB Control No. 1205-0426

Expiration Date: 12/31/2022

SUPPORTING STATEMENT A

Post Enrollment Data Collection for

Job Corps Exiters

OMB Control No. 1205-0426

This information collections request (ICR) seeks OMB approval for an extension without change in the process of collecting information through surveys which is necessary to evaluate the Job Corps program and to meet WIOA reporting requirements.

1. Justification.

1. Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

Job Corps is the nation's largest and most comprehensive residential education and job training program for opportunity youth, ages 16 through 24. The Economic Opportunity Act of 1964 originally established Job Corps. The program was transferred from the Office of Economic Opportunity to the Department of Labor in 1969. Job Corps was later authorized as part of the Comprehensive Employment and Training Act (1978) under Title IV-B, and this authorization continued as part of the Job Training Partnership Act (JTPA) of 1982 and the Workforce Investment Act (WIA) of 1998. Now authorized by the Workforce Innovation and Opportunity Act (WIOA), Job Corps is administered federally through the Department of Labor, Employment and Training Administration, Office of Job Corps. The Job Corps program is comprised of 6 regional offices and 121 Job Corps centers nationwide.

The 2014 Workforce Innovation Opportunity Act (WIOA) required the Office of Job Corps to collect and report specific post enrollment outcomes for eligible Job Corps participants beginning in Program Year (PY) 2016. The WIOA performance reporting requirements, which replaced those of the 1998 Workforce Investment Act (WIA), are designed to provide a common set of metrics to be reported by similar programs. WIOA substantially changed many outcome metrics for Job Corps compared to those required under WIA. In order to collect the necessary information to meet the new WIOA reporting requirements, the Office of Job Corps revised its post enrollment data collection system (PEDC) in 2019, which primarily collects data through survey instruments, and obtained OMB approval under the Paperwork Reduction Act (PRA) for this modified system (OMB # 1205-0426, expiration date: 12/31/2022).

The regulatory citations that authorize this information collection are detailed below:

686.945 Provides the procedures for the management of Job Corps student records. These can be found in the Department of Labor’s (Department) Privacy Act System of Records Notice (SORN) DOL/GOVT-2 Job Corps Student records management of student records. The Department’s Secretary issues guidelines for maintaining records for each student during enrollment and for disposition of such records after separation.

The Department has a direct role in the operation of Job Corps and does not serve as a pass-through agency for this program. The Department establishes Job Corps centers, and it is the responsibility of the Department to select operators for and provide funding to each center. Of the 121 current centers, 24 are managed and operated by the U.S. Department of Agriculture - Forest Service (USDA) through an interagency agreement. Of the remaining centers, 95 are managed and operated by contractors selected by the Department and an additional two are being operated under demonstration grants. The 95 centers are operated by private organizations, including private for-profit companies, in most cases contracted through competitive procurements that are negotiated and conducted in accordance with WIOA, the Competition in Contracting Act, and the Federal Acquisition Regulations. Many of the current contractors operate more than one center. The two centers operated under demonstration grants are run by the State of Idaho and the National Guard Job Challenge program, respectively. Additionally, there are 26 public colleges and universities operating Job Corps Scholars Program demonstration grants.

686.565 Authorizes Job Corps to conduct experimental, research and demonstration projects related to the Job Corps program according to WIOA sec. 156(a), provided that such projects are developed, approved, and conducted in accordance with policies and procedures developed by the Secretary.

To manage the program, Job Corps must collect information from center operators and grantees regularly to exercise its federal oversight duties and ensure safe and smooth operation of the program. In accordance with 5 CFR 1320, the Department is seeking approval for forms connected with operation of the Job Corps centers and oversight of projects under 686.565.

To meet the new WIOA reporting requirements, the Office of Job Corps revised its post enrollment data collection system (PEDC), which primarily collects data through survey instruments, and obtained OMB approval under the Paperwork Reduction Act (PRA) for this modified system (OMB #1205-0426, expiration date 12/31/2022).

After reviewing initial results of this WIOA-based PEDC, Job Corps proposed revisions to improve effectiveness of the surveys during the last collection request in 2019. These revisions were approved and implemented successfully. Therefore, this request contains only changes in agency estimates.

The rationale for conducting the PEDC is explained in further detail in Section A1 of this Supporting Statement (Consequences of Less Frequent Data Collection). This collection is necessary to obtain the data required under Sections 159(c) and (d) of WIOA. Section 159(c) requires the Secretary of Labor to collect information on performance of Job Corps participants, centers, recruiters, and career transition service providers. It requires the information to be consistent with the WIOA performance indicators as amended by Section 116(b) (2) (A) (ii). It also includes additional metrics that Job Corps is required to track. In addition, WIOA Section 159(c) (1) includes reporting requirements for primary indicators of performance that are applicable to youth formula programs, which are found in Section 116(b) (2) (A) (ii).

686.955 Establishes procedures to ensure the timely and complete reporting of necessary financial and program information to maintain accountability. Center operators and operational support service providers are responsible for the accuracy and integrity of all reports and data they provide.

686.1000 Describes how performance of the Job Corps program as a whole, and performance of individual centers, outreach and admissions providers, and career transition service providers, is assessed in accordance with the regulations in this part and procedures and standards issued by the Secretary, through a national performance management system, including the Outcome Measurement System (OMS).

686.1010 Describes the primary indicators of performance for Job Corps centers and the Job Corps program as outlined in this submission.

686.1020 Describes the indicators of performance for Job Corps outreach and admissions providers.

686.1030 Describes the indicators of performance for Job Corps career transition service providers.

686.1040 Details the information that will be collected for use in the Annual Report.

The Job Corps program is responsible for collecting and reporting on the six WIOA-required primary performance measures. The information needed to report on one of those six measures, Measurable Skill Gains, is maintained within the Job Corps Center Information System. Information needed to report on the remaining five WIOA measures is addressed in this collection.

Those five measures are:

• the percentage of program participants who are in education or training activities, or in unsubsidized employment during the second quarter after exit from the program;

• the percentage of program participants who are in education or training activities, or in unsubsidized employment during the fourth quarter after exit from the program;

• the median earnings of program participants who are in unsubsidized employment during the second quarter after exit from the program;

• the percentage of program participants who obtain a recognized postsecondary credential, or a secondary school diploma or its recognized equivalent (and, for those who obtain a secondary school diploma or its recognized equivalent, have also obtained or retained employment or are in an education or training program leading to a recognized postsecondary credential), during participation in or within one year after exit from the program; and,

• the indicator(s) of effectiveness in serving employers.

The modules included in this extension without change collection instruments are similar to the approved data collection and are as follows: (1) Appendix A is an overview of the collection instruments; (2) Appendices B, C, and D are the scripts used in the primary and secondary collections.

2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.

Congress and the Secretary of Labor will use the data collected with these instruments to assess the effectiveness of the Job Corps program in meeting its objectives under WIOA. The data collected are used to fulfill the reporting requirements under Section 116(b) (2) (A) (i) and Section 159(c) (4) of WIOA. The Office of Job Corps has also incorporated these data into the Outcome Measurement System to evaluate the post-center outcomes of graduates and former enrollees. Job Corps will continue to use the information about student outcomes and satisfaction with specific services to develop or refine policies to improve delivery of educational and job training services to opportunity youth.

Upon separation from Job Corps, youth are prepared to pursue employment opportunities, additional educational and training experiences, or enter the Armed Forces. Job Corps career development specialists are responsible for assisting graduates and former enrollees with finding a job or enrolling in a school or training program. Career development specialists also help students create resumes and prepare for interviews. WIOA authorizes Job Corps to provide placement services to graduates for up to 12 months after they exit the program and to former enrollees for 3 months after exit.

The revised PEDC will continue to collect information on five of the six WIOA required primary performance measures. These are:

• the percentage of program participants who are in education or training activities, or in unsubsidized employment during the second quarter after exit from the program;

• the percentage of program participants who are in education or training activities, or in unsubsidized employment during the fourth quarter after exit from the program;

• the median earnings of program participants who are in unsubsidized employment during the second quarter after exit from the program;

• the percentage of program participants who obtain a recognized postsecondary credential, or a secondary school diploma or its recognized equivalent (and, for those who obtain a secondary school diploma or its recognized equivalent, have also obtained or retained employment or are in an education or training program leading to a recognized postsecondary credential), during participation in or within one year after exit from the program; and

• the indicator(s) of effectiveness in serving employers.

Job Corps may also use the information collected from the proposed PEDC to measure and report participant performance under the following metrics mandated by Section 159(d) of WIOA:

• the number of graduates who entered the Armed Forces;

• the number of graduates who received a regular secondary school diploma;

• the number of graduates who received a State recognized equivalent of a secondary school diploma;

• the percentage and number of graduates who enter postsecondary education;

• the average wage of graduates who enter unsubsidized employment

  1. on the first day of such employment; and

  2. on the day 6 months after such first day

The second and fourth quarter data collection effort also provides a means to verify work and school placements for all placed graduates and former enrollees. In addition, it provides information responsive to the WIOA measure on the effectiveness of serving employers.

The effort to verify work and school placements and obtain information on program effectiveness is consistent with recommendations by the Office of the Inspector General in their 1988 report that stated Job Corps should "monitor the services provided by placement contractors to ensure that contract requirements are being met and students are receiving quality placement services."

In addition to being key for compliance with the WIOA reporting requirement of effectiveness in serving employers, customer satisfaction data is important and useful to effect program improvement. Although this survey only serves as a backup source, results can be compared with internal outcome measurement programs to ensure they agree with actual customer sentiment. Customer feedback can help identify areas for improvement and lead to new initiatives. Data suggests that outbound phone interviews are easier to complete and provide the strongest response rates.

To fulfill these information collection objectives, Job Corps will use the two primary instruments identified in Appendices B and C and the secondary instruments identified in Appendix D to collect data. The Job Corps Data Center (JCDC) will provide information about Job Corps exiters who are eligible for all surveys. The information will include contact information in addition to the work or school placement status of graduates and former enrollees after exiting Job Corps.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also, describe any consideration of using information technology to reduce burden.

The data collection instruments are programmed into a Computer-Assisted Interviewing system (CAI). This system supports both interviewer-assisted and self-administered online survey modes. As is the case in the ongoing survey administration, the CAI will perform a variety of response checks and include programming to support complex skip patterns and checks for internal consistency. The system supports automated scheduling, callback, and online data storage, which makes it easier to control the sample, monitor the study, and reduce data entry costs. In addition to the functionality provided by the current system, the CAI can be enhanced to offer a secure self-administered web-based option for completing the surveys online. Automated online surveys reduce the respondent burden by allowing data collection to be completed at a time and pace that is most convenient for the respondent. Online surveys also have the potential to speed up the collection of data by reducing the number of surveys that need to be completed by an interviewer-assisted telephone mode. Students that participate via the self-administered online module would not need to be subsequently contacted by telephone to complete the survey with an interviewer.

Data obtained using the combined online and telephone survey modes is compared with historical data from the telephone-only survey to ensure the introduction of a new completion mode does not adversely impact survey results.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.

No other existing source of data can meet the specific data reporting requirements of WIOA. Section 116(i)(2) of WIOA notes that the Secretary of Labor may collect information on Job Corps exiters by using quarterly wage records and Job Corps uses this Unemployment Insurance (UI) wage match process through the Workforce Integrate Performance Systems (WIPS); however, Job Corps does not have access to individual-level results from the UI wage record data matching. UI wage matches have been incorporated into the Job Corps data file and used to calculate the aggregated outcomes for WIOA measures by a third-party entity, and these outcomes are returned to WIPS and reported for Job Corps. UI wage records however do not provide any information about enrollment in school or training programs or attainment of secondary or post-secondary credentials, which are key program outcomes, and are needed for accurately calculating several of the six primary WIOA measures. Despite the limitations of UI wage records, on July 1, 2018, Job Corps began participating in the Common Reporting Information System (CRIS) data sharing process through WIPS. Job Corps began to receive CRIS data matches in October 2021. The information captured in this collection will be considered supplemental data when reporting performance measures, as required by Section 159 of WIOA. Post-enrollment data which includes information on individual Job Corps participants remain necessary for program management and reporting of additional Job Corps specific WIOA metrics. Initial analysis of the CRIS results from UI wage matches indicates a correlation with the survey results. The information from the Employer/Educational Institution survey (Appendix D) are used to verify or corroborate results from the PEDC and CRIS matches. The instruments presented in this application are designed to collect this information.

5. If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.

This data collection will place minimal burden on small businesses. Businesses and educational institutions will be contacted through a short phone interview and/or by an online method. The brief employment/educational institution verification and satisfaction survey instrument is included in Appendix D and the post separation surveys for eligible Job Corps exiters are included in Appendix C Module 6.

6. Describe the consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

Since the information from eligible participants must be obtained according to WIOA requirements, less frequent data collection will not allow the Office of Job Corps to obtain eligible participant outcomes to meet reporting requirements under the performance measures specified in the law. WIOA requires the collection of information about outcomes in the second and fourth quarters after exit quarter from the program. The PEDC contact with eligible students during these two periods will collect information as required under the law.

7. Explain any special circumstances that would cause an information collection to be conducted in a manner inconsistent with OMB guidelines:

These data collection efforts do not involve any special circumstances.

8. If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the agency's notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments. Specifically address comments received on cost and hour burden.

Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and record keeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported.

Consultation with representatives of those from whom information is to be obtained or those

who must compile records should occur at least once every 3 years—even if the collection of information activity is the same as in prior periods? There may be circumstances that may preclude consultation in a specific situation. These circumstances should be explained.

In accordance with the Paperwork Reduction Act (PRA) of 1995, the public was given 60 days to review and comment on this information collection by way of a Federal Register notice published on February 24, 2022 (87 FR 10391). No public comments were received.

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

Consistent with WIOA, the Job Corps program provides ongoing services to graduates for 12 months and former enrollees for 90 days after exit. A key element of the provision of these services involves regular contact between the student and his or her career transition specialist after program separation. To encourage students to maintain contact with Job Corps, the program provides graduates and former enrollees with a transitional allowance. In addition, post-separation incentives are provided for taking the surveys. Studies found a dose-response relationship between monetary incentives and response rate by survey mode. These findings suggest that the size of the offered incentive will impact response rate and that tailoring estimates to specific survey modes and timing taking into account the target population is recommended (Mercer, et al. 2015).¹ Although higher incentive amounts would be potentially advantageous for increasing survey cooperation for these surveys, the availability of even these lower amounts are helpful in keeping Job Corps exiters engaged and in contact with the follow-up transition activities, including the survey.

Incentive approaches, rooted in the best evidence from the research literature, are incorporated into the survey methodology. For the online survey mode, we employ an “early bird” incentive to increase likelihood of respondents using the most efficient and cost-effective response mode. Research has shown that an early bird incentive model (in which an increased incentive is given for responding to a survey request by web before a specified deadline) is quite effective in improving completion rates (LeClere, F.S et al, 2012)². To further encourage students to complete the JC WIOA web survey within the two-week web push, the use of an “early-bird” incentive is only paid if the respondent completes the survey within the designated time period.

10. Describe any assurance of privacy provided to respondents and the basis for the assurance in statute, regulation, or agency policy.

This survey is conducted in accordance with all relevant regulations and requirements, including the Privacy Act of 1974 (5 USC 552a), the Privacy Act Regulations (34 CFR Part 5b), the Freedom of Information Act (5 CFR 552) and its related regulations (41 CFR Part 1-1, 45 CFR Part 5b, and 40 CFR 44502). In addition, the procedures for the management of Job Corps student records can be found in the Department of Labor’s Privacy Act System of Records Notice (SORN) DOL/GOVT-2 (Job Corps Student Records) 67 FR 16815. The Secretary issues guidelines for maintaining records for each student during enrollment and for disposition of such records after separation.

In May 2007, to support Job Corps’ commitment to securing Personal Identifiable Information (PII), all aspects of the data collection process were consolidated under the secure network environment provided and maintained by the JCDC. The application is accessible via Citrix, JCDC's preferred secure method.

JCDC takes several steps to ensure the safety and integrity of the student data housed at its facilities in Austin, Texas. All student data is housed on a central server, in a secured computer room in a locked building on a fenced and guarded facility. There are several layers of system security. The server network is a private network. The network is also protected from the Internet by a series of layered firewalls, access control lists, and intrusion detection systems.

All Job Corps employees and contractors enter data into the system through the private network.

All users must have a valid user ID and password (which is changed every 90 days) to enter the system. Additional security is obtained by using two-factor authentication, application-level user IDs, passwords, and specific permission applied at the database level. The integrity of the data is insured by running daily validation programs that use a set of pre-approved business rules established by the Office of Job Corps. To secure the integrity of data transmission, JCDC has established 256-bit encrypted Citrix sessions between Job Corps center operators who provide the data and the JCDC. All the survey contractor’s staff have completed security awareness training and have access to the Citrix environment and the CDSS suite of applications.

Additionally, the contractor survey sites are maintained in accordance with the Guide for Security and Privacy Controls for Federal Information Systems and Organizations (SP 800-53 Revision 4), at the moderate level, posted by The National Institute of Standards and Technology's Computer Security Resource Center. Internal access to individual records within the database at the Job Corps Data Center follow the principles of least privilege required and are controlled by all appropriate security measures, including controlled usernames, passwords, profile names, host names, firewall security IDs, and crypto cards. Input of data occurs only via a secure Citrix tunnel maintained by JCDC, ensuring a secure means of communication for the data collection contractors and hosting facility.

Additionally, personal computers (PCs) are accessible only after providing individual user passwords that must be changed on a regular basis. Screensavers are installed on all PCs, set to activate after ten minutes, and can only be released through the individual user's password. PC passwords are not physically documented; therefore, loss of access may be recouped only by the reassignment of a password by the information systems staff.

All Job Corps survey respondents are told that completing the interview is voluntary and that the information they provide will not be shared with anyone outside of the Job Corps community. Answers to some questions in the survey instrument will be provided to appropriate agency staff (for example, information that results in questioning the validity of the initial placement will be reported to Job Corps Regional Office staff for follow-up). Answers to other questions will be available at the individual level in reports for Job Corps service providers and at the aggregated level for reports about Job Corps’ performance (for example, the Job Corps Outcome Measurement System). Therefore, it is not possible to guarantee to respondents that their answers will not be released to anyone. However, respondents will be assured that Job Corps will use the collected data only for program evaluation purposes and that their answers will not be shared with anyone outside of Job Corps without their written approval.

Policies and procedures maintained at any facility operated by service providers under contract are designed to protect the privacy of data. All contract personnel are required to sign pledges to protect respondents' privacy. Respondent names and contact information are stored in a computer database that is separate from the database that stores survey responses. Additionally, privacy issues will be addressed in training sessions for all staff involved with the administration of the data collection instruments. Individual identifying information will be maintained separately from hardcopy collection forms and from computerized data files prepared for conducting the analysis.

To keep data private, it is important not to release data into public use files or to present results in reports so that individuals can be identified. Data collected through the survey instruments will not be made publicly available. Job Corps staff and contractors will analyze data for purposes of program management and quality improvement. Nonetheless, reports that use these data will be handled in a manner that eliminates the possibility of compromising privacy. Job Corps staff and contractors will follow commonly accepted guidelines (what the Federal Committee on Statistical Methodology calls the Threshold Rule) and display only aggregated data when there are more than three cases in a cell table. When there are three or fewer cases in a cell table, tabular data will be presented by combining categories or suppressing cells to ensure the elimination of possible individual identities. Given the size of Job Corps centers and the monthly year-to-date aggregation of data, the data presented in any cell table will most likely represent information from at least four observations.

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private. This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

These data collection instruments being submitted for approval do not include questions of a sensitive or personal nature. All respondents are assured that their responses will be kept private at the outset of the self-administered web survey or the interviewer-assisted telephone interview unless they specifically authorize its release in writing. In addition, respondents are informed that they do not have to answer questions with which they are uncomfortable.

All questions concerning wages and income have been successfully pretested, and similar questions have been used extensively in previous instruments with no evidence of adverse consequences. Additionally, the survey instrument is equipped to accommodate respondents who are uncomfortable providing a specific wage by allowing their answers to be coded in ranges. Specifically, similar questions about wages and income from employment were used in the "Evaluation of the Impact of the Job Corps Program on Participants' Post-Program Labor Market and Related Behavior" study (CMB 1205-0351, expiration date 10/96).

12. Provide estimates of the hour burden of the collection of information.

This data collection is an annual process that supports administration of approximately 46,700 initial engagement self-administered online surveys and interviewer-assisted telephone interviews, and 93,400 Job Corps post-separation self-administered online surveys and interviewer-assisted telephone interviews. This total is in line with the burden estimate of the currently approved version when Job Corps is at full enrollment. Presently, Job Corps is in the process of returning to full enrollment following the stand down due to COVID precautions. The self-administered Q2 and Q4 post-separation surveys require an average of 12 minutes to complete, and the interviewer-assisted telephone interviews require an average of 17 minutes. These administration times are based on previous experience with interviewer-assisted telephone instruments, the interviewer-assisted employer verification interviews, and the estimated time for a self-administered online instrument.

The initial engagement self-administered surveys require an average of 5 minutes to complete and the interviewer-assisted telephone interviews takes 6 minutes on average.

Table 1: Job Corps Exiters and Participants Survey and Employer Educational Institution Sample Sizes (Annual Estimated) *
Survey Status	Annual Totals
Population: Job Corps Exiters and Participants
Sample: Job Corps Exiters and Participants eligible for initial engagement, quarter 2, and quarter 4 surveys.	88,400
Population: Employers and Educational Institutions of Job Corps Exiters
Sample: Employers and educational institutions selected for survey attempts.	5,000

* Based on Program Year 2021 Data

The combined reporting burden for respondents associated with this data collection effort is estimated at approximately 21,700 hours, as shown in Table 2. The response rate for the second and fourth quarter surveys is about 40 percent over the past six months. However, we anticipate higher response rates with the survey, which is projected to take noticeably less time than the prior version and to be much more straightforward for respondents, which will help to make the interviewing experience more attractive to respondents. We will incorporate the use of an online survey approach, which the research and our survey contractor have found to be much more attractive to this younger age population, who can complete the survey at times convenient for them while using computers, tablets, or smartphones. The online survey is currently approved under this collection.

We are also testing several alternatives to our current incentive amounts and distribution schedules to more align with survey best practices. For example, evidence has shown that including prepaid monetary incentives in initial survey mailings has long proven to be successful in increasing respondents’ motivation to complete a survey (Church, 1993)³. As a result, the mailed invitation letter could be updated to include a modest pre-paid unconditional cash incentive of $1 - $2. Also, a refusal conversion incentive may prove effective with this population. Finally, we are investigating the prospect of using text messages for invitations and reminder messages—again a more popular option with this population. In other studies, conducted by our survey contractor with young and disadvantaged populations, they have achieved close to a 60 percent completion rate using online and phone administrations. We anticipate that successful implementation of these additional innovations will facilitate moving the response rate for this survey into the 60 percent range over time. As a result, a 60 percent response rate is estimated for the second quarter survey and a 40 percent response rate for the 4^th quarter survey for a combined 50 percent rate. The historical response rate for the Employer/Educational Institution Survey is 50 percent.

The combined reporting burden for respondents associated with this data collection effort is estimated at approximately 21,700 hours, as shown in Table 2. Costs to respondents are limited to the time they will spend either answering the self-administered online survey or the interviewer-assisted telephone survey.

Table 2: Estimates of Respondent Burden Average Time
Respondent Category	Total Responses	Hours per Respondent	Estimated Hours
Online survey of eligible Job Corps exiters after separation (initial engagement, and after the second and fourth quarters after exit quarter)	52,680	0.2	10,536
Telephone interview of eligible Job Corps exiters after separation (initial engagement, and after the second and fourth quarters after exit quarter)	35,720	0.28	10,002
Employer/Institution Verification Satisfaction Survey	5,000	0.2	1,000
Total	93,400		21,538

Table 3 shows the estimated costs to respondents, based on the Job Corps national average wage of placed graduates and former enrollees for Program Year 2021. A standard hourly rate of $25.00 per hour is used for estimating the costs of responses for employers and educational institutions. The total estimated cost of the burden for all respondents is approximately $269,683 per year. This total is calculated by multiplying—for each activity row in the table—the value in Column B, ‘Total Annual Responses’ by the corresponding value in Column C, ‘Monetized Value of Respondent Time’ and summing the products. For example, the respondent burden for the online initial engagement survey of Job Corps exiters is 44,680 *$ 2.91. This burden is offset by the incentive payments totaling approximately $835,480 (see Table 4 for the calculation) that will be provided to respondents for completing the follow-up surveys.

Table 3: Estimated Cost to Respondents for Data Collection - Number of Responses Based on Estimated Response Rates

A			B				C
Activity	Number of Respondents	Frequency	Total Annual Responses	Time Per Response (Hours)	Total Annual Burden (Hours)	Hourly Rate*	Monetized Value of Respondent Time
Online initial engagement survey of Job Corps exiters	22,340	2	44,680	0.2	8,936	$14.57	$130,197.52
Online survey of non-graduate Job Corps exiters during the second and fourth quarters after exit quarter	2,000	2	4,000	0.2	800	$14.57	$11,656.00
Online survey of Job Corps graduates during the second and fourth quarters after exit quarter	2,000	2	4,000	0.2	800	$14.57	$11,657.00
Telephone interviews with non-graduate Job Corps exiters during the second and fourth quarters after exit quarter.	3,118	2	6,236	0.28	1,746	$14.57	$25,440.39
Telephone interviews with Job Corps graduates after the second and fourth quarters after exit quarter	14,742	2	29,484	0.28	8,256	$14.57	$120,283.93
Employer/Institution Verification/Satisfaction survey**	5,000	1	5,000	0.2	1,000	$25.00	$25,000
Unduplicated Total	49,200	Varies	93,400	Varies	21,538	Varies
* Hourly Rate based on Monthly Center Summary Report (MPO35) for PY 2021; as of 6/30/22.
** Employer/Institution wage rate is based on BLS data (pay and benefits).

13. Provide an estimate for the total annual cost burden to respondents or record keepers resulting from the collection of information. (Do not include the cost of any hour burden already reflected on the burden worksheet).

There are no additional costs to the respondents for participating in this survey. All telephone or postage costs for contacting the respondents are borne by the Federal government through the data collection contractor.

14. Provide estimates of annualized costs to the Federal government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information.

The estimated cost of funding this data collection effort annually is $3,817,000 in 2021 dollars. This estimate includes the ongoing maintenance of the infrastructure needed to administer the CATI system, ongoing data communication to and from the Job Corps Data Center, labor costs for telephone interviews from trained staff who are not Job Corps employees, data processing including coding of occupational and industry information, and preparation of summary data tabulations. Supervision of this total system is also included here. This cost does not include one-time expenditures that will not recur in the future.

Approximately 87% ($3,320,790) of the cost is budgeted for labor. Approximately 8% ($305,360) is budgeted for respondent outreach and engagement; 3% ($114,510) of the cost is budgeted for software and equipment including the CATI facilities; and 2% ($76,340) of the cost is budgeted for long distance phone service.

Additionally, the incentive system encourages eligible Job Corps exiters to remain in

contact with Job Corps and, in turn, complete the follow-up surveys will cost approximately $835,480 annually (Table 4). The current incentive system allots $10 payments to participants completing the initial engagement and second quarter surveys and $20 payments for those completing the fourth quarter survey.

Table 4: Incentive Payments
Surveys	Respondents	Incentive	Amount
Job Corps exiters eligible for the initial engagement survey	22,340	$10	$223,400
Job Corps exiters eligible for the second quarter after exit quarter survey	26,232	$10	$262,320
Job Corps exiters eligible for the fourth quarter after exit quarter survey	17,488	$20	$349,760
Total	66,060		$835,480

The total combined annual cost to the Federal Government including the data collection effort and incentive payments is $4,430,000.

15. Explain the reasons for any program changes or adjustments reported on the burden worksheet.

Based on actual survey response data from the existing collection, Job Corps estimates it will have 44,680 Job Corps exiters eligible to take the initial engagement survey, 43,720 eligible respondents in the second and fourth quarter surveys, plus 5,000 employer/institution respondents, yielding 93,400 total responses. This results in a burden of 21,538 hours. This number differs from the previous version of this ICR due to a mathematical error in the past version of this ICR.

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

The data collected serves four primary purposes. First, the data is used to fulfill part of the performance measurement and reporting requirements for the Job Corps program specified under WIOA. Second, the data is used to determine long-term placement and wage results for the Outcome Measurement System (OMS), which is used to manage many aspects of the program including center-level performance. Third, the data is used for independent verification of contractor reported outcomes regarding initial placement and wages. Fourth, the initial engagement survey will be used to verify and update contact information. Having an alternate verification source is important, as these metrics are included in assessing performance of the very contractors who report the outcomes. Additionally, the metrics are part of the system used for determining incentive and award fees for performance-based service contracts. Finally, the information supports the continuous program improvement activities regularly conducted by the Office of Job Corps and program operators. Although the same data collection instruments support all these purposes, the analysis plan, reporting plan and time schedule for differ for each.

WIOA Performance Reporting

Job Corps is required to report WIOA outcomes quarterly and annually. As such, Job Corps is required to provide raw student-level data to the Workforce Integrated Performance System (WIPS), which will be used by ETA as a data repository for WIOA and related data from all ETA programs. These data will be used to produce aggregated quarterly and annual reports for each program. No individual-level results will be published.

• Six Primary WIOA Measures. The individual-level information obtained from survey results is used in calculating five of the six primary measures required under WIOA:

1. the percentage of program participants who are in education or training activities, or in unsubsidized employment during the second quarter after exit from the program;

2. the percentage of program participants who are in education or training activities, or in unsubsidized employment during the fourth quarter after exit from the program;

3. the median earnings of program participants who are in unsubsidized employment during the second quarter after exit from the program;

4. the percentage of program participants who obtain a recognized postsecondary credential, or a secondary school diploma or its recognized equivalent (and, for those who obtain a secondary school diploma or its recognized equivalent, have also obtained or retained employment or are in an education or training program leading to a recognized postsecondary credential), during participation in or within one year after exit from the program; and,

5. the indicator(s) of effectiveness in serving employers.

Individual-level variables will be calculated for each of these measures and included in the Participant Integrated Record Layout (PIRL) that is used to calculate the Quarterly Performance Report (QPR) for WIOA.

• WIOA Annual Reporting Requirements. WIOA also requires that Job Corps report on the wage graduates earned on the day six months after the first day of unsubsidized employment. Prior to the implementation of WIOA, Job Corps relied on surveys of initially placed graduates that were administered 6 and 12 months after initial placement. With the transition to WIOA, and with measures based on outcomes two and four calendar quarters after exit, Job Corps re-designed the surveys to collect data on a calendar quarter after exit timeline and no longer had a tool to directly report on wages on the day 6 months after initial placement. As a result, Job Corps developed a proxy for calculating this measure using the subset of participants with quarterly employment data that overlap the period five-to-seven months after initial placement. An average hourly wage is calculated for placed graduates that includes regular wages, overtime wages and tips (as applicable) and a mean value is calculated nationally and by center and CTS provider.

Job Corps Management and Performance Reporting. Job Corps utilizes data captured from the surveys for internal program management and data integrity monitoring, and performance reporting and evaluation of its program operators.

• Outcome Measurement System. The surveys also provide critical data to the Job Corps Outcome Measurement System (OMS), a key management tool used by Job Corps to assess overall program performance against goals. The PY 2019 OMS incorporates four key measures that align with those required under WIOA. Job Corps calculates the mean values of the placement rates in Quarter 2 and in Quarter 4, mean earnings in Quarter 2, and the employer retention rate based on the results of the surveys for inclusion in the OMS. These average values are calculated nationally and by centers and CTS providers to assess program performance. Performance measurement results are calculated and published by Job Corps monthly at the center, regional, and national levels.
  

• Verification of initial placement results and management/continuous improvement activities. The surveys are also the primary source for independent re-verification of initial placements reported by CTS providers to ensure the integrity of the underlying data. Answers to the survey questions about initial job or school placements are compared to values reported by CTS providers to identify situations where the reported placement may ‘potentially’ be invalid (e.g., different employer/school name, hours worked, wages, hours in school). These differences are submitted to Job Corps’ Regional Offices as questionable placements (QP) for potential follow-up actions with the CTS provider to determine if the initial reported placement is valid. In addition, average values of such differences by type are calculated nationally and by center operator and included in reports that show patterns of potential data integrity problems across a wide variety of performance metrics to identify patterns of potential data integrity issues. These QP reports are for internal contractor monitoring and management purposes only and are not published or made public.

As part of the assessment of each contractor's performance, Job Corps can use survey results to verify placement outcomes of former enrollees and graduates who were reported by service providers as having entered a job or school program. QP reports (i.e., where there is a discrepancy between the placement reported through the PEDC and the placement reported by the service provider) are generated based upon either results from the survey or the employer/school verification. QP reports are submitted to Job Corps’ Regional Offices for adjudication. These QP reports are for internal contractor monitoring and management purposes only and are not published or made public.

17. If seeking approval not to display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

The instruments are primarily administered as computer-assisted online surveys or computer-assisted telephone interviews. The OMB expiration date is displayed on the screen for online surveys. Since there are no hardcopy or online instruments for telephone interviewees to receive or view, a printed expiration date cannot be displayed for this method of data collection. Interviewers will provide the OMB number and expiration date. In conducting follow-up activities with employers and schools, it is anticipated that respondents will sometimes prefer to complete a hardcopy version of the instrument rather than an online survey or telephone interview. In such cases, the instrument will be mailed or sent via facsimile to the contact at the employing organization or school and it will display the OMB control number and expiration date.

18. Explain each exception to the topics of the certification statement identified in “Certification for Paperwork Reduction Act Submissions,”

No exceptions are requested in the "Certification of Paperwork Reduction Act Submissions."

1 Mercer, A., Caporaso, A., Cantor, D., and Townsend, R. (2015).  How much gets you how much? Monetary incentives and response rates in household surveys.  Public Opinion Quarterly, 79, 105-129.

2 LeClere, F., S. Plumme, J. Vanicek, A. Amaya, and K. Carris. 2012. “Household Early Bird Incentives: Leveraging Family Influence to Improve Household Response Rates.” In American Statistical Association Joint Statistical Meetings, Section on Survey Research.

3 Church, Allan H. 1993. Estimating the Effects of Incentive on Mail Survey Response rates: A Meta-Analysis. Public Opinion Quarterly 57: 62-79.

12