December 9, 2016
NOTE TO THE REVIEWER OF: |
OMB CLEARANCE 1220-0141 “Cognitive and Psychological Research” |
FROM: |
Jennifer Edgar Office of Survey Methods Research |
SUBJECT: |
Submission of Materials for Pretesting of the Survey of Employer Provided Training: Phase 1 |
Please accept the enclosed materials for approval under the OMB clearance package 1220-0141 “Cognitive and Psychological Research.” In accordance with our agreement with OMB, we are submitting a brief description of the study.
The total estimated respondent burden for this study is 48 hours.
If there are any questions regarding this project, please contact Jennifer Edgar at 202-691-7528.
Introduction
There
is significant interest in understanding how employers are able to
reach the goal of having a workforce with the skills that are needed
to do their jobs. To what extent do workers already have the
training that is needed to do their jobs when hired by firms? To
what extent are employers finding it necessary to sponsor or directly
provide training to their workers? How does the need for employer
provided or sponsored training vary by occupation and industry? What
are some of the key characteristics of firms and jobs that increase
the likelihood that employers are providing or sponsoring training
for their employees?
In 1993 and 1995, the Bureau of Labor Statistics fielded a Survey of Employer Provided Training (SEPT), which provided some insight into these important questions. To address a refreshed interest in these same questions, BLS is revisiting this topic and has taken steps to identify what data would be of highest value to data users. Environmental scanning and an academic literature review were conducted to identify current trends and practices, and findings used to develop a set of measurement objectives. To explore how feasible it is to measure the intended data, BLS is intending to conduct exploratory research. The research would identify the kinds of information firms readily have available on training, and, given the nature of such information, what is the best method of data collection? Additionally, we will conduct informal interviews with stakeholders to ensure that the measurement objectives are in line with their data needs.
Methodology
This research includes two components, the second of which will build on the findings of the first.
Stakeholder Consultations. BLS will reach out to up to 20 potential data users and others who have a vested interested in development of a new SEPT, to get their feedback on the draft measurement objectives (Attachment A). Stakeholders have been identified by the program office from lists of known users and interest groups. An email invitation will be sent and the sessions will be scheduled at the participant’s convenience.
A combination of group meetings and individual interviews will be conducted, in-person or telephone, and no scripted protocol will be used for the conversations, though an informal set of questions will be followed (Attachment B). Stakeholders will be given a copy of the measurement objectives in advance, and asked for their feedback. Notes will be taken by the BLS staff participating and summarized in a memo. Results will be used to refine the measurement objectives.
Exploratory Interviews. Up to 20 exploratory interviews will be conducted with establishments from a variety of industries and size classes. See Attachment B for recruitment screener. The goal of these interviews will be to identify what information each respondent has about the desired information, and at what level of detail, what records they keep on training and how easy or difficult it is to access. Information about the types of trainings conducted and supported by the establishment will also be explored. See Attachment C for draft interview protocol. It will be updated if any changes to the measurement objectives are made as a result of the stakeholder consultations, and an updated version will be submitted prior to interviewing.
Burden Hours
The stakeholder consultation participants will be recruited from OSMR’s database of general population study participants via an email solicitation for participation (Attachment F). No burden is associated with recruitment, as only those interested in participating will contact the BLS to arrange scheduling. We expect stakeholder consultations to take no more than one hour per person, with up to 20 stakeholders consulted, for a total of 20 hours. .
We expect to contact up to 50 establishments to recruit 20 to participate in the exploratory interviews. Recruitment and screening is expected to take 10 minutes. Each of the 20 interviews is expected to take no more than one hour, for a total of 28 burden hours.
Total expected burden for this study is 48 hours.
Payment to Respondents
There will be no payments to respondents.
Data Confidentiality
Participants interviewed over the phone will be informed of the confidentiality provisions and will be asked for their verbal consent. Participants interviewed in person will be asked to sign a consent form (Attachment E) which contains The following confidentiality statement:
In accordance with the Privacy Act of 1974, as amended (5 U.S.C. 552a), you are hereby notified that this study is sponsored by the U.S. Department of Labor, Bureau of Labor Statistics (BLS), under authority of 29 U.S.C. 2. Your voluntary participation is important to the success of this study and will enable the BLS to better understand the behavioral and psychological processes of individuals, as they reflect on the accuracy of BLS information collections. The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.
Attachments
Attachment A: SEPT Measurement Objectives
Attachment B: Stakeholder Questions
Attachment C: Exploratory Interviews Recruitment Screener
Attachment D: Exploratory Interview Protocol
Attachment E: Consent Form
Attachment F: Email Solicitation Template
Attachment A: SEPT Measurement Objectives
The primary objectives of the survey are to measure the percentage of workers receiving training and the hours of training they received in order to create annual estimates.
Note: The classification scheme to differentiate the informal from formal training has yet to be determined (i.e., within formal blended training there can be informal training components that should be measured separately), as are the different sources and types of training. Stakeholder consultations and exploratory interviews will inform these decisions.
Primary Estimation Objectives
Incidence of Training
Percentage of establishments providing or financing training over 12 months.
By mode of training (in person, online, blended, on the job, etc.)
By source of training (establishment, private vendor, trade association, etc.)
By type of training (basic skills, safety, job skills, etc.)
Percentage of workers receiving training from or financed by employer over 12 months.
By mode of training (in person, online, blended, on the job, etc.)
By source of training (establishment, private vendor, trade association, etc.)
By type of training (basic skills, safety, job skills, etc.)
Intensity of training
Hours per worker of training from or financed by employer over 12 months
By mode of training (in person, online, blended, on the job, etc.)
By source of training (establishment, private vendor, trade association, etc.)
By type of training (basic skills, safety, job skills, etc.)
Dollars per employee of expenditures on training over 12 months
By mode of training (in person, online, blended, on the job, etc.)
By source of training (establishment, private vendor, trade association, etc.)
By type of training (basic skills, safety, job skills, etc.)
Secondary Estimation Objectives
Identifying trends over time is a lower priority measurement objective, and will depend on the feasibility of collecting this information.
Trends over time
Change in the percentage of employees receiving training over the past X years.
Change in expenditure on training over the past X years.
Sampling Strata and publication cross tabulations
The choice of sampling strata to support publication of the primary estimation objectives depend on the availability of information on the sampling frame to create stratified probability samples and the costs of data collection for the chosen level of strata detail.
Two sampling strata that are most likely are size of establishment and NAICS industry. The choice of size and industry strata is scalable (such as the number of employment size classes and industry detail at the 1-, 2-, 3-, 4-, 6-digit level).
Covariates
The sample size and composition will determine what covariates can be used. These will be estimated in conjunction with preparing the sampling and survey methodology to lead to cost estimates for different options. When budget won’t allow for certain analysis, post-collection stratification could be an option to produce some estimates.
These lists assume that there will be both an employer and employee survey, a decision that will be based on exploratory research and early design decisions. If SEPT is not implemented as two, independent, surveys, the covariates will be updated.
Establishment Level Covariates
Possibly available from the sampling frame:
Industry
Multi-establishment firm
Average establishment wage
Choice of sampling frame could be partially determined by the availability of covariates in the frame.
High priority:
Establishment size
Number or percent of part-time employment
Number or percent of employees covered by a collective bargaining agreement
Number or percent of temporary help workers
Number or percent of contract workers
Establishment contracts out for services (transportation, security, food service, janitorial, payroll, etc.)
Whether establishment supplies contract/temporary workers to other firms
Turnover over a given period
Medium priority:
New technology [new products or new production practices over last X years].
Distribution of employment by major occupation
Change in contracting for any of above services over X years
Workplace practices (pay for knowledge, employee involvement in technical decisions, job redesign or reengineering, job rotation, co-worker review of performance, quality circles, Total Quality Management, self-directed work teams, etc.)
Low priority:
Age of establishment
Presence of employee benefits (Pension, Profit sharing, Vacation, Health care, Employee Assistance Program, Employee Wellness Program, Flextime, Flexiplace, Child care, Parental/family leave, etc.)
Capital intensity
Hiring of workers with experience in occupation vs. hiring of entry level workers.
Employee Level Covariates
Age
Sex
Education
Race
Ethnicity
Tenure at current employer
Occupation
Tenure in occupation
Earnings (CPS-style question)
Weeks worked per year
Hours worked per week
Covered by a collective bargaining agreement
Attachment B: Stakeholder Outline
Session Outline
Introductions
BLS review of process thus far and next steps
Exploratory work
Lit reviews
This list is the ideal set of measures, not likely we can get everything – R burden and information availability
High level walk through of measurement objectives document
Questions
Do you think there are missing estimation objectives in the measurement objectives documentation?
Do you think there are unnecessary estimation objectives included in the measurement objectives documentation?
Realizing that we will be limited in what we can get, which of these estimation objectives would be your top priority?
Are there other determinants-of-training covariates that you would suggest be included?
We will be conducting research with companies, but from your experience:
How training is delivered, in what modes?
Who is delivering trainings?
What types of training are provided?
For private sector group: What records are kept on these topics?
Would you be able to connect us with anyone in your companies, or other contacts, who could tell us about how training operates, and is tracked? This would be invaluable as we move forward with our next stage of research.
Attachment C: Exploratory Interviews Recruitment Screener
RECRUITER: If you reach a generic gatekeeper, request to talk to someone in the human resources or training office.
I’m calling on behalf of the Bureau of Labor Statistics, which is part of the Department of Labor. The Bureau is conducting a study to learn how businesses train their employees.
This voluntary study is being collected by the Bureau of Labor Statistics under OMB No. 1220-0141. We will use the information you provide for statistical purposes only.
As part of this study, I would like to ask a few general questions about your company to see if you’d be eligible for an interview.
First, what is the industry that your company works in? (If more than one, then ask for the industry that would include their main business activity.)
___________________________________________________________________________
Does this company have multiple worksites or locations, or is this location the only one?
Single-unit
Multi-unit
Approximately what is the total number of employees that work at this location?
|__|__|__|__|__| number
[screen based on quotas provided]
IF SCREENED OUT (Based on industry, firm composition and size quotas)
Thank you for taking the time to answer my questions. Those are all the questions I have at this time.
IF SCREENED IN
We are interested in learning more about how your company trains your employees. We’d like to set up a telephone interview, which should last less than an hour. The information provided will be kept completely confidential, used only for research purposes.
Would you, or someone from your company, be able to help us with this important research? It would allow us to determine how to best collect information about employee training, to be able to measure the resources and efforts companies like yours are putting in.
Arrange date and time for interview
Date: ____/____/________
Time: ____:______ AM / PM PAC / CST / MTN / EST
We will send an email with some additional information and confirmation of our meeting time. After review you may feel that other staff would be helpful in our discussion. Please feel free to invite additional staff members.
If you have any questions about what staff you should include, or need to reschedule, please feel free to contact me at XXX .
COLLECT CONTACT INFORMATION FOR RESPONDENT TO CONFIRM APPOINTMENT AHEAD OF SCHEDULED TIME.
Respondent name:
Respondent title:
Company name:
Physical address:
City/State/Zip:
Telephone number:
Email address:
Attachment D: SEPT Exploratory Interviews Protocol
SEPT Exploratory Interviews Protocol
Introduction
Thanks for agreeing to participate in our study. We are looking for information about how companies train their employees and what information they keep about training. We’ll ask you a series of questions, please answer to the best of your ability – but if we ask something you do not know the answer to, please tell us that as it’s likely other people wouldn’t know either.
The information you provide will be used for research purposes only, and only seen by the researchers involved in the project. I expect the interview to take about an hour, and really appreciate your time. Any questions before we begin?
This voluntary study is being collected by the Bureau of Labor Statistics under OMB No. 1220-0141. We will use the information you provide for statistical purposes only. Your participation is voluntary, and you have the right to stop at any time.
Employee Background
I’d like to start by getting some information about yourself:
What is your job title?
Can you tell me a little about your role in the company?
What department do you work for?
How long have you been with this company?
How long have you been doing this type of work?
Company background
Now, I’d like to get a little background on the company:
What does (Company Name) do/make exactly?
About how many years has (Company Name) been in business?
How many people work at this location?
How many locations do you have in (State; US)? Is there a separate headquarters?
What type of work do your employees do?
Here’s a list of main occupation categories that we use at BLS. Which of these do your employees belong to? [Show Card A – occupations]
Would you say your average worker has been with the company less than one year, between 1 and 3 years, or over 3 years?
Do any of your employees belong to a union?
Training Information
As I mentioned earlier, we’re exploring training that companies provide to their employees. I’d like to talk about training at your company, the more we can learn about how training works, and what information is kept, the better our survey will be.
Can you tell me about how employees are trained at this company?
Who determines what training an employee will receive? The employee, the supervisor, combination, other?
Generally, what kind of training does your company either pay for other otherwise sponsor? [Probe for specific methods below after open ended and]
Formal, classroom type trainings
Online trainings
Informal, on-the job training
Allow employees to take free training on work time
Other types of training?
Where do you get training you use? What is the source? [Probe for specific vendors below after open ended and]
Your company’s internal staff resources
Private vendors
Academia—colleges, universities
Trade associations
Online sources – specify
Other
What information is covered in the trainings you provide? What topics are covered? Probe for specific topics below after open ended and]
Management
Professional or technical skills
Computer programming or software
Clerical or administrative
Sales or customer service
Production or construction
Basic skills
Occupational safety
Employee health or wellness
Orientation
Awareness
Communications
Anything else?
Does your company have a general training policy or philosophy?
Does your company have a policy on how new employees are trained?
In general, when an employee starts at this company, do they have the skills they need to do their job?
Are there some skills that employees don’t commonly have when they come in that you provide training for?
Is there any training that all employees get when they start a new position?
Does your company have a policy on what training an employee has to complete to maintain their job? Is this the same for all employees, or does it vary by occupational group?
Is training provided when your company acquires new technology? [probe: employee technology (e.g. copy machines), production technology (e.g. manufacturing)
Are there other events that automatically result in employee training?
What are the major types of training that your company provides or funds?
Within your company, who are the most knowledgeable persons about employee training? [probe: Department and job title]
Training Records
Does anyone keep track of what training an employee receives? The employee? The supervisor? HR?
Does your company keep records on employee training? {probe for specifics}
How are these records stored?
Electronically? What software?
hat type of information is available?
Training mode? (online, classroom, etc.)
Training topic/type? (admin, technical, mgmt.)
Training source (academic, private vendor, etc.)
Training cost (direct costs, associated with tuition)
Which employees are trained
How are the records structured?
By employee
Is there employee information available as well? (job title, tenure, demographics)
By department
By training event
By training type
Other
Is there any training information that is not included on the records?
Informal training
Internal training
External training
Free training
Other?
Who maintains the training records?
How often are they updated?
Who has access to the records?
How easy or difficult is it to access them?
Can custom queries be run – for example to get an inventory of all training provided in last 30 days?
Who in your company knows the most about training records?
We are also interested in the amount of money that employers spend on providing training. Would you be able to find the following information:
How much the company spent directly on providing training, such as tuition
How much the company spent subsidizing training?
Are there records available for this information? How hard would it be to obtain this information?
{for multi-units}. Are records centralized for the entire company, or does each location have their own records?
Survey Questions
We have some questions that we think we’d like to include on a training survey. I’m going to ask you the question, if you can answer it please do. Otherwise, let me know how you’d go about getting the information to answer the question. {for multi-units: please answer for your individual location, not for the company as a whole}
{Probes, for each questions
How would you/did you go about answering the question?
Is that an easy or difficult question to answer?
Is that information on a record?
Who in the company would have access to that information?
How much time would it take to get that information?}
Formal Training
In the last 3 months:
Did your establishment provide or finance any formal training for employees? If no, skip to X
How many employees received formal training?
What topics were covered in the formal training provided, for example management training, computer training, or manufacturing-related training)?
Specifically, was formal training offered on these topics?
Management
Professional or technical skills
Computer programming or software
Clerical or administrative
Sales or customer service
Production or construction
Basic skills
Occupational safety
Employee health or wellness
Orientation
Awareness
Communications
Were any other topics covered by formal training?
Who conducted the training (e.g., this establishment, a college, private company, other)
What method was used to deliver the training provided?
In-person, on site, classroom
In-person, at another site, classroom
Online only
Blended, a combination of online and in-person
Did employees receive formal training by any other modes??
Intensity
We are also interested in the resources companies spend on providing training to their employees.
Thinking about just the last 3 months, please specify how many total employee hours were spent in formal training and how much money the establishment spent on the formal training.
{Probes, for each section (only go through each individually if P is especially cooperative. Otherwise talk about the categories (e.g. topic x hours) as a whole and go through the response process and probes
How would you/did you go about answering the question?
Is that an easy or difficult question to answer?
Is that information on a record?
Who in the company would have access to that information?
How much time would it take to get that information?}
|
Employee Hours in last 3 months |
Total Cost in last 3 months |
Training Topic |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Training Provider |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Training Mode |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Change Over Time (Secondary Estimation Objective)
We are also interested in understanding how training changes over time.
In the last 3 years, how has the percentage of employees who receive formal training changed? Has it increased, decreased or stayed the same?
{Probe: What are you basing that answer on? Do you have information about the number of employees trained each year? Does your company monitor how these numbers change?}
In the last 3 years, how has the total number of hours employees spent in receive formal training changed? Has it increased, decreased or stayed the same?
{Probe: What are you basing that answer on? Do you have information about the hours employees spend in formal training each year? Does your company monitor how these numbers change?}
In the last 3 years, how has the amount of money spent by your establishment on formal training changed? Has it increased, decreased or stayed the same?
{Probe: What are you basing that answer on? Do you have information about the amount spent on formal training each year? Does your company monitor how these numbers change?}
Informal Training
We know that much of employee training is not formal, information is shared through other informal sources such as on the job training, mentoring, or observations.
Would you say that the employees at this company receive informal on-the-job training? Please tell me about it
Does your company create opportunities for informal training, such a mentoring programs, or are do they arise organically or ad hoc?
Are any records kept on the informal training employees receive?
Employee Level Information
Beyond understanding what individual companies do, we are also interested in understanding training at an individual employee level. To do this, ideally we’d ask you for information about the employees who have been trained.
Is this information that you would have?
Demographics? (race, age, gender, education)
Job information? (title, full-part time, tenure)
Employee pay rate or earnings
Do you have this for all your employees?
Is this information available in records?
Would you have any concern about providing this information?
Show Card A – Occupations
Business and Financial Operations Occupations
Computer and Mathematical Occupations
Architecture and Engineering Occupations
Life, Physical, and Social Science Occupations
Community and Social Service Occupations
Education, Training, and Library Occupations
Arts, Design, Entertainment, Sports, and Media Occupations
Healthcare Practitioners and Technical Occupations
Healthcare Support Occupations
Protective Service Occupations
Food Preparation and Serving Related Occupations
Building and Grounds Cleaning and Maintenance Occupations
Personal Care and Service Occupations
Office and Administrative Support Occupations
Farming, Fishing, and Forestry Occupations
Construction and Extraction Occupations
Installation, Maintenance, and Repair Occupations
Transportation and Material Moving Occupations
Attachment E: Consent Form
OMB Control Number: 1220-0141
Expiration Date: April 30, 2018
The Bureau of Labor Statistics (BLS) is conducting research to increase the quality of BLS surveys. This study is intended to suggest ways to improve the procedures the BLS uses to collect survey data.
The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent. The Privacy Act notice on the back of this form describes the conditions under which information related to this study will be used by BLS employees and agents.
During this research you may be audio and/or videotaped, or you may be observed. If you do not wish to be taped, you still may participate in this research.
We estimate it will take you an average of 60 minutes to participate in this research (ranging from 45 minutes to 60 minutes).
Your participation in this research project is voluntary, and you have the right to stop at any time. If you agree to participate, please sign below.
Persons are not required to respond to the collection of information unless it displays a currently valid OMB control number. The OMB control number is 1220-0141 and expires April 30, 2018.
------------------------------------------------------------------------------------------------------------
I have read and understand the statements above. I consent to participate in this study.
___________________________________ ___________________________
Participant's signature Date
___________________________________
Participant's printed name
___________________________________
Researcher's signature
In accordance with the Privacy Act of 1974, as amended (5 U.S.C. 552a), you are hereby notified that this study is sponsored by the U.S. Department of Labor, Bureau of Labor Statistics (BLS), under authority of 29 U.S.C. 2. Your voluntary participation is important to the success of this study and will enable the BLS to better understand the behavioral and psychological processes of individuals, as they reflect on the accuracy of BLS information collections. The BLS, its employees, agents, and partner statistical agencies, will use the information you provide for statistical purposes only and will hold the information in confidence to the full extent permitted by law. In accordance with the Confidential Information Protection and Statistical Efficiency Act of 2002 (Title 5 of Public Law 107-347) and other applicable Federal laws, your responses will not be disclosed in identifiable form without your informed consent.
Attachment F: Email Solicitation Template
XXXX,
I am writing to you to be part of a stakeholder consultation connected a new establishment-based survey of employer provided or sponsored training. With support from DOL-ASP, this year we are conducting exploratory research on how we might in the future, with dedicated funding, conduct a survey of employers on the kinds and amounts of formal training that they provide or finance for their employees. We conducted similar surveys, with support from the Department of Labor’s Employment and Training Administration, back in 1993 and 1995. We would like to meet to share our thinking with you and get your feedback on the choices we have made.
We have spent a fair amount of time doing a scan of the academic and business literatures to identify how the training environment has changed since 1995 and the kinds of information that analysts and policymakers would find useful from a survey of employer provided training. Based on this scan, our prior experience with conducting the survey, and our own views, we have defined our estimation objectives. From there we compiled a list of desired measures, both direct measures of training and covariates that help explain the decision to train or not and how much.
The next phase of our research includes conversations with a broad array of user groups, including private sector establishments, academic researchers, non-profits, think tanks, and various government agencies, among others. The goal of the consultation is to ask participants to share their views on our estimation objectives and intended measures. In particular, we will ask your feedback on the objectives (to be provided in advance), what concepts are missing, and some specific questions about the types of training, sources of training, and methods of training delivery that you are familiar with.
Please let me know if you would be interested in participating in such a consultation. For efficiency purposes, we will try to coordinate with a small number of other stakeholders to meet with at the same time. I will follow this email up with a calendar request. I am assuming that a one-hour meeting should be sufficient.
Thanks in advance for your interest.
Mike
Michael W. Horrigan, Ph.D.
Associate Commissioner
Office of Employment and Unemployment Statistics
Bureau of Labor Statistics
(202) 691-5735
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Edgar, Jennifer - BLS |
| File Modified | 0000-00-00 |
| File Created | 2021-01-22 |