SUPPORTING STATEMENT
FOR THE PAPERWORK REDUCTION ACT OF 1995
INFORMATION COLLECTION REQUEST FOR THE
JOB CORPS PROCESS STUDY
The U.S. Department of Labor, Employment and Training Administration (ETA) is requesting clearance for an information collection to conduct site visits to Job Corps centers and a survey of center directors for a process study of the Job Corps program. The study seeks to explore and identify associations between centers’ practices and their performance on a range of relevant outcomes, including gains in foundational academic skills during students’ time on center; completion of relevant academic and career technical programs; attainment of credentials; and placement and earnings after students leave the center. ETA expects that the study’s results can be used for peer-to-peer learning, technical assistance and development of performance measurement systems. IMPAQ International and its subcontractors, Battelle Memorial Institute and Decision Information Resources (henceforth the IMPAQ team), are conducting the study.
The Job Corps program was established by the Economic Opportunity Act of 1964 and is currently authorized by Title I-C of the Workforce Investment Act of 1998 and administered by the Office of Job Corps (OJC) in ETA. A total of 125 primarily residential Job Corps centers operate in 48 states, the District of Columbia, and Puerto Rico, providing intensive education and training services to disadvantaged youth ages 16 to 24. Over the course of its 45 years of operation, the program has served almost 3 million youth; it welcomes more than 60,000 new students every year.
Job Corps offers academic instruction to improve students’ reading and math skills and assist them in obtaining General Educational Development (GED) certificates or high school diplomas, career technical training in over 100 industry-recognized programs, and development of social skills. Job Corps provides dormitory-style housing, meals, medical care, and counseling. It also assists graduates and students with career transition services and placement in jobs, higher education, and the military.
Private corporations enter into contracts to run most Job Corps Centers, Outreach and Admissions (OA), and Career Transition Services (CTS). Contracts are awarded through a competitive bidding process and include a two-year base period with three one-year options. About one-fourth of Job Corps centers are located on Federal lands and operated by the U.S. Forest Service. Individual centers are encouraged to form partnerships with schools, colleges, local businesses, community organizations, and other local agencies to provide additional resources to support and train students.
In the fall of 2010, ETA contracted with the IMPAQ team to conduct this study to address the following broad questions:
What center practices appear to be associated with center performance or particular dimensions of performance and how?
How do interactions among center practices and characteristics mediate these associations? Put differently, do some strategies or practices work especially well (or especially badly) for certain kinds of centers?
ETA requests clearance for the IMPAQ team to conduct three principal research activities: (1) in-depth interviews during 16 site visits to Job Corps centers, including interviews with senior center management, instructors, social and residential staff, on- and off-center partners, operator executives, and regional staff; (2) student focus groups to be conducted during the site visits; and (3) a Web-based survey of Job Corps center directors.
This information collection is authorized under Section 172 of the Workforce Investment Act of 1998. Please see Appendix A.
Centers will be selected for the site visits based in part on an analysis of Office of Job Corps center-level Outcome Measurement System data, student satisfaction survey results, and administrative employment information available from the Wage Record Interchange System (WRIS). The determination of aspects of performance to highlight and purposive identification of centers for visits will be done by the IMPAQ team jointly with ETA and the Department’s Chief Evaluation Office based on the results of the analysis. During the site visits, the researchers will interview center managers and staff, in addition to conducting observations and student focus groups. The purpose will be to identify practices that may be associated with performance and understand the program context in which these practices are implemented.
Candidate promising practices identified through the site visits and hypothesized to be associated with performance outcomes will form the foundation of the instrument for the survey, which will be administered to all Job Corps center directors.1 Survey responses will be used to determine systematic associations between practices or sets of practices and aspects of center performance.
The information collected during the site visits will be used to inform the Center Director survey. We will use the information we collect during the site visits both to select the questions to be asked in the survey and the response options. Sixteen centers were selected in order to have a variety of centers representing centers that are rated high, low and rising on factors related to center success. These factors are: (1) Career and Education Training, (2) Academic Skills Improvement, (3) Wages/ Earnings, and (4) Placement Success. Within each factor we selected two centers that are high performing, one center that is low performing, and one center that has improved. By visiting centers along the spectrum of success within each factor, we expect to be able to pinpoint center practices related to success.
During the site visits, we need to talk to a variety of stakeholders since different stakeholders may bring different perspectives to the questions and the responses we intend to include in the survey. For example, an instructor at a center may help us identify specific response categories to a question that would not be gleaned from a conversation with the Center Director. Thus, speaking to different stakeholders will help us to make the questions and answer categories more specific.
The site visit and survey information will be used to conduct an analysis and develop an understanding of center-level practices that are associated with performance. Job Corps and ETA will be able to use the study’s findings to assess Job Corps center practices and possibly to recommend wider dissemination of certain successful practices. Findings may also be useful to other youth training programs.
Electronic technology (e.g., Web-based materials) will be used whenever possible to reduce the time burden on respondents. The Job Corps center survey will be administered via a Web-based instrument developed using KeySurvey. The software provides 128-bit Secure Sockets Layer encryption capabilities for ensuring private responses and for exporting responses for future analyses. The software also allows for the identification of responders and non-responders without compromising the privacy of responses.
Web-based surveys are cost-effective and allow the project team to monitor the response rate in real time and send customized reminder e-mails, in addition to enabling respondents to complete the survey at a time of their choosing.
In the 1990s, the Department sponsored the National Job Corps Study, an experimental impact evaluation, which included a process component informed by site visits.2 However, those visits took place over 15 years ago and cannot be used for our purposes due to substantial changes in practices, center management, and other relevant considerations.
Regional OJC staff periodically visit Job Corps centers to monitor their compliance with program rules and regulations; these visits include significant walkthrough observations and meetings with key staff. However, the visits, performed by Federal staff, are not aimed at identifying innovative or distinctive practices, which this study seeks to do. Rather, typical purposes for the monitoring visits are provision of technical assistance, facility surveys, and periodic assessments.
Therefore, the proposed information collection is in no way duplicative of prior efforts.
The instruments and procedures for conducting the in-depth interviews, focus groups, and surveys have been designed to minimize the burden on all respondents and will not have a significant impact on small businesses or other small entities.
6. Consequences of not Collecting the Data
Without the proposed site visits and survey there would not be adequate information to identify promising practices, let alone associate them with center performance or student outcomes. The information collected during the site visits will largely be used to inform the survey instrument design (i.e., which questions to include in the survey and the response options). A secondary purpose of the site visits is to help us interpret the quantitative responses. During the site visits we expect to gain insights that will help in the interpretation of the analytic results. The information collected during the site visits will allow us to include more close-ended, as opposed to open-ended, questions in the survey, which will limit the burdensome and expensive analysis of open-ended questions.
This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d)(2). No special circumstances apply.
As required by 5 CFR 1320.8(d), a notice was published in the Federal Register on January 18, 2012 (Vol.77, page 2567) for 60 days’ public comment. No comments were received.
No payments or gifts will be provided to participants of the information collection activities.
The center managers, staff members, and students interviewed by research team members will be assured that their responses will be kept private within the limits of the law. All findings in any published reports or briefings will be presented at the aggregate level, so that it is not possible to link comments to particular individuals. Similarly, interview notes or recordings will not be shared with ETA staff or anyone else outside the study team. Paper copies of interview notes and audio recordings will be secured in a locked file cabinet. If any notes are recorded on laptop computers, such notes will be stored in a SQL Server database located in an access-controlled server room at the evaluation contractor.
The surveyed center directors will be assured that their responses will be kept private within the limits of the law. Survey data will be stored on an evaluation contractor’s server that is protected by a firewall that monitors and evaluates all attempted connections from the Internet. Personal information (name, telephone number, and e-mail address) on each survey response will be maintained in a separate data file apart from the survey data so that individuals outside of the project team cannot link particular responses to individual respondents. Once the survey is completed, all sensitive data on each respondent will be destroyed. The entire survey database will be encrypted so that any data stored will be further protected. Finally, access to any data with identifying information will be limited only to contractor staff directly working on the survey. Survey findings will be presented at a level of aggregation such that it will not be possible to link specific responses to individual respondents.
Participants will not be asked questions of a sensitive nature.
Exhibit 1 shows the estimated annualized burden hours for the respondents to participate in this study. Most in-depth interviews with Job Corps center staff and the student focus groups will last about one hour each. However, the center director interview will last two hours, and the academic instruction manager, career technical training manager, and counseling manager interviews will last 90 minutes.
Each of the 125 center directors will be asked to complete a Web-based survey. The survey will take approximately 180 minutes to complete. This time includes the time for completing the survey and reading our introductory letter, reminder e-mails, etc.
The total burden hours are estimated at 759.
Exhibit 1: Estimated Annualized Burden Hours
Data Collection Activity |
Number of Respondents |
Frequency of Response |
Average Time per Respondent |
Burden Hours |
In-Depth Interviews |
||||
Center Director |
16 |
Once |
120 minutes |
32 |
|
|
|
|
|
Academic Instruction Manager |
16 |
Once |
90 minutes |
24 |
Career Technical Training (CTT) Manager |
16 |
Once |
90 minutes |
24 |
Work-Based Learning (WBL) Coordinator |
16 |
Once |
60 minutes |
16 |
Career Preparation Period (CPP) Manager |
16 |
Once |
60 minutes |
16 |
Counseling Manager |
16 |
Once |
90 minutes |
24 |
Peer Leadership Coordinator |
16 |
Once |
30 minutes |
8 |
Social Development Manager |
16 |
Once |
60 minutes |
16 |
|
|
|
|
|
Academic Instructor |
48 |
Once |
60 minutes |
48 |
CTT Instructor |
48 |
Once |
60 minutes |
48 |
|
|
|
|
|
|
|
|
|
|
Business and Community Liaison |
16 |
Once |
60 minutes |
16 |
Organizational Partners: OA, CTS |
32 |
Once |
60 minutes |
32 |
|
|
|
|
|
|
|
|
|
|
Regional Office Project Manager |
16 |
Once |
60 minutes |
16 |
Subtotal Interviews |
320 |
|||
Focus Groups with Students |
||||
Students |
64 |
Once |
60 minutes |
64 |
Subtotal Focus Groups |
64 |
|||
Survey |
||||
Center Director |
125 |
Once |
180 minutes3 |
375 |
Subtotal Survey |
375 |
|||
TOTAL |
653 |
n/a |
n/a |
759 |
Exhibit 2 shows the estimated annualized cost burden based on the respondents' time to participate in the study. The total cost burden is estimated to $27,207. The time spent by respondents to this collection will be billed to ETA contracts, so the government bears the associated cost.
Exhibit 2: Estimated Cost of the Hour Burden
Data Collection Activity |
Burden Hours |
Average Hourly Wage Rate |
Cost of the Hour Burden |
In-Depth Interviews |
|||
Center Director |
32 |
$47.96* |
$1,534.72 |
|
|
|
|
Academic Instruction Manager |
24 |
$34.45* |
$826.80 |
CTT Manager |
24 |
$34.45* |
$826.80 |
WBL Coordinator |
16 |
$27.99 |
$447.84 |
CPP Manager |
16 |
$27.99 |
$447.84 |
Counseling Manager |
24 |
$27.99 |
$671.76 |
Peer Leadership Coordinator |
8 |
$27.99 |
$223.92 |
Social Development Manager |
16 |
$27.99 |
$447.84 |
|
|
|
|
Academic Instructor |
48 |
$20.98 |
$1,007.04 |
CTT Instructor |
48 |
$20.34 |
$976.32 |
|
|
|
|
|
|
|
|
Business and Community Liaison |
16 |
$27.99 |
$447.84 |
Organizational Partners: OA, CTS |
32 |
$27.99 |
$895.68 |
|
|
|
|
|
|
|
|
Regional Office Project Manager |
16 |
$27.99 |
$447.84 |
Subtotal Interviews |
$9,202.24 |
||
Survey |
|||
Center Director Survey |
375 |
47.96 |
$17,985.00 |
Subtotal Survey |
$17,985.00 |
||
TOTAL |
927 |
n/a |
$27,207.72 |
Average hourly wage rates marked with an asterisk indicate that the data is the average of the recommended salary range from the OJC. All wage rates were estimated based on the mean of the average wages for that occupation from the National Compensation Survey: Occupational wages in the United States December 2009 – January 2011, U.S. Department of Labor, Bureau of Labor Statistics. |
|||
There are no direct costs to respondents other than that of their time of participation. There will be no start-up or ongoing financial costs incurred by respondents. There are no record keepers.
The cost to the Federal government for conducting the site visits and designing, testing, and programming the survey is $133,464, which is the total contractor cost of conducting the site visits and the center director survey. The total survey cost is $23,028, and the total site visit cost is $110,436. The annualized cost is therefore $133,464 divided by 3, or $44,488.
This is a new, one time data collection effort counting as 759 hours towards ETA’s Information Collection Budget.
The project will utilize the following secondary data sources to develop center-level outcome measures: OJC administrative data, student follow-up survey data, and WRIS data. Analysis of these extant datasets will not impose any burden on parties outside the research team. Frequency tabulations and other descriptive statistics summarizing these datasets will be prepared with the goal of obtaining a better understanding of Job Corps centers’ performance on student outcomes, satisfaction with the program, and job placement and earnings.
After conducting the proposed collection, the research team will also conduct multivariate regression analyses to identify associations of practices with performance.
Visits to selected Job Corps centers will involve in-person, in-depth interviews yielding information that is largely qualitative in nature, and analysis will extract major themes or commonalities and key quotations.
The project began in October 2010 and will end in March 2013. The project design and interview instruments were prepared in early 2011. The survey instrument was prepared during the summer of 2011 and will be revised after the completion of the site visits, which will likely involve a request for a non-substantive change from OMB. Upon OMB clearance the project team will begin scheduling the site visits, conducting site visit training, and conducting the site visits. We anticipate that these activities will take place between May and July of 2012. Upon completion of the site visits, the project team will revise the survey and submit it for OMB approval. We anticipate that the survey will be conducted in September and October of 2012.
Various tabulations and analyses will be published in a final report delivered to ETA at the end of the project. The report will include analysis of the site visit data collected from 16 site visits during the course of the study, as well as the center director survey.
Information collected during the site visits and the results of the Center Director survey will not be shared directly with Center Directors. Once the final report has been approved by DOL, DOL may publish the report, at which time the report will be shared with the Center Directors.
The OMB approval number and expiration date will be displayed or cited on all information collection instruments. For the center director survey, the OMB number and expiration date will be displayed on each page of the online instrument as well as in e-mail communications with the survey respondents.
There are none.
1 Although the survey instrument is to a large extent developed, based on the experience of the research team and consultations with ETA staff, it will evolve as a result of site visits. We expect to submit a non-substantive change request to the Office of Management and Budget (OMB) to make those modifications.
2 See, for example, the summary in J. Burghardt, et al., Does Job Corps Work? Summary of the National Job Corps Study, Princeton: Mathematica Policy Research, 2001, http://wdr.doleta.gov/research/keyword.cfm?fuseaction=dsp_puListingDetails&pub_id=2259&mp=y&start=141&sort=7.
3 After the site visit data has been analyzed, the survey will be revised and shortened to approximately one hour. Center Directors that are interviewed during a site visit and take a survey have will have an estimated burden of three hours. Center Directors that just take the survey will have an estimated burden of one hour.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | SUPPORTING STATEMENT FOR PAPERWORK REDUCTION ACT 1995 |
| Author | Administrator |
| File Modified | 0000-00-00 |
| File Created | 2021-01-31 |