Download:
pdf |
pdfU.S. BUREAU OF LABOR STATISTICS
Occupational Requirements Survey
The Occupational Requirements Survey (ORS) is conducted by the U.S.
Bureau of Labor Statistics (BLS) to collect data about the requirements of
work. This establishment survey collects information about the physical
demands, environmental conditions, education and training, and mental
requirements for jobs in the U.S. economy. Current ORS data products and
additional information can be found at www.bls.gov/ors.
Quick Facts: Occupational Requirements Survey
Table of Contents
Subject areas
Job requirements
Key measures
Cognitive and mental requirements, Education and training,
Environmental conditions, Physical demands of job
How the data are
obtained
Classification
system
Periodicity of data
availability
Survey of businesses
Industry, Occupation
Annual
Geographic detail
National
Scope
Civilian noninstitutional population, Private sector, State and
local government
News releases
Key products
Occupational group profiles
Database query tool
Program webpage
www.bls.gov/ors
1
CONCEPTS .....................2
DATA SOURCES..............5
DESIGN............................7
CALCULATION...............12
PRESENTATION ............22
HISTORY........................24
MORE INFORMATION...27
U.S. BUREAU OF LABOR STATISTICS
HANDBOOK OF METHODS
Concepts
The purpose of the Occupational Requirement Survey (ORS) is to collect the various physical demands,
environmental conditions, education and training, and mental requirements for occupations within the national
economy. The information in ORS is unique, compared with other job requirement documentation (such as the
Dictionary of Occupational Titles or the Occupational Information Network (O*NET)), as the data give a better
understanding of some of the cognitive and mental requirements for a job. In addition, ORS provides insight into
the duration of specific physical demands and environmental exposures, as well as the amount of education,
training, and experience needed to perform in the occupation.
The ORS data elements are grouped into four main categories:
• Physical demands
• Environmental conditions under which the work is typically performed
• Education and training, known as specific vocational preparation or SVP
• Mental and cognitive demands
It is important to note that the ORS is designed to capture information regarding what is required to perform a
job and is not focused on the specific capabilities or experience of the worker. For example, a job may require a
bachelor’s degree, but some workers performing the job may have a doctoral degree (Ph.D.). In this case, the
ORS would capture the requirement of this particular occupation as being a bachelor’s degree. The distinction is
significant because the desired outcome of the survey is to portray the requirements of a job, not necessarily the
characteristics of the worker performing that job. See the Data sources section for more detail on what ORS
data elements are collected.
The ORS is an establishment based survey and includes establishments in the 50 states and the District of
Columbia in the private sector and state and local governments. Major exclusions from the survey are workers
in federal and quasi-federal agencies, military personnel, agriculture workers, workers in private households, the
self-employed, volunteers, unpaid workers, individuals receiving long-term disability compensation, and those
working overseas. Individuals who set their own pay, such as business owners, and family members--paid token
wages--are also excluded.
The following sections provide definitions of key concepts and further explanation regarding occupational
selection and estimation processes used for this survey. For more detailed definitions of survey terminology,
please refer to the ORS Collection Manual.
2
U.S. BUREAU OF LABOR STATISTICS
HANDBOOK OF METHODS
Key concepts and definitions
Physical demands. Refer to the level and/or duration of physical exertion generally required to perform
occupational tasks. For more information on individual demands, see the ORS Visual Overview for Physical
Demands Data Elements.
Environmental conditions. Refer to the various tangible or concrete hazards or difficulties that are in the vicinity
of which a job is performed. For more information about individual environmental elements, see the ORS Visual
Overview for Environmental Conditions Data Elements.
Education, training, and experience. In ORS, this is known as Specific Vocational Preparation (SVP) and refers
to the amount of preparation time required by a typical worker to learn the techniques, acquire the information,
and develop the facility needed for average performance in a specific job/worker situation.
Cognitive and mental demands. The requirements related to a worker’s need to use judgment, make decisions,
and adapt to changes on the job.
Task. A distinct activity assigned to or performed by workers in an occupation that results in a meaningful
outcome. A list of tasks aids field economists in understanding the relationship of ORS elements to an
occupation.
Work as generally performed. Refers to the way in which most workers normally complete the duties, tasks, and
responsibilities as assigned. Field economists collect occupational information representative of the typical
duties performed in that occupation.
Accommodation. A modification or adjustment to a job or change in the work environment that enables a person
with a disability or other constraints to compete equally or carry out the occupational tasks as generally
performed. The ORS only collects how work is performed without accommodations, as not all employers can
offer the same accommodations.
Job. A group of workers in an establishment that have the same position. The term job refers to a single position
in a single company, whereas occupation refers to a profession or trade. Example: “waiters at Smith’s
Restaurant” is a job, whereas “waiters” is an occupation.
Duration Scale. Scale used to categorize duration of a physical demand being performed or exposure to an
environmental condition. The scale is as follows:
•
•
•
•
Seldom (up to 2 percent of the workday)
Occasionally (2 percent up to 1/3 of the workday)
Frequently (1/3 up to 2/3 of the workday)
Constantly (2/3 or more of the workday)
3
U.S. BUREAU OF LABOR STATISTICS
HANDBOOK OF METHODS
Percentage of workers. Measurement of the portion of workers in a given domain (civilian workers, private
industry workers, state and local workers, or of a given occupation or industry) that have a certain requirement.
For example, a percentage of teachers that are required to reach overhead.
Average percent of the day. Duration measurement of the average portion of the work day in which workers
spend doing a physical activity or are exposed to an environmental condition.
Average time spent (in hours or days). Duration measurement of the average time in hours or days in which
workers spend doing a physical activity, are exposed to an environmental condition, or spend obtaining
education, experience, or training.
Strength. The capacity for exertion or endurance determined by the amount of weight lifted or carried, the
duration of lifting or carrying that weight, and how long a worker sits or stands/walks per day.
Full-time or part-time status. For the ORS, full-time or part-time status is not determined by the number of hours
worked, but is based instead on the establishment’s definition of those terms.
Time-based or incentive-based pay. Time-based workers are those whose wages are based solely on an hourly
rate or salary. Incentive workers are those whose wages are based at least partially on productivity payments,
such as piece rates, commissions, and production bonuses.
Union or nonunion workers. The ORS defines a union worker as any employee in a union occupation who
satisfies all of the following conditions: a labor organization is recognized as the bargaining agent for all workers
in the occupation; wage and salary rates are determined through collective bargaining or negotiations; and
settlement terms, which must include earnings provisions and may include benefit provisions, are embodied in a
signed, mutually binding collective bargaining agreement. A nonunion worker is an employee in an occupation
not meeting all of the ORS-defined conditions for union coverage.
Last Modified Date: March 29, 2017
4
U.S. BUREAU OF LABOR STATISTICS
HANDBOOK OF METHODS
Data Sources
Bureau of Labor Statistics (BLS) field economists are extensively trained and given detailed instructions on data
collection techniques. They employ a variety of methods, including personal visits, mail, telephone, and email, to
obtain data from Occupational Requirements Survey (ORS) respondents. Field economists do not use paper or
an online questionnaire to collect these data; instead, they rely on a conversational interview and descriptive
documents, such as task lists, to collect occupational requirements from respondents. Respondents are typically
human resources managers or specialists, occupational safety managers, or supervisors. Field economists
attempt to gather the following information from the respondent:
• The primary business activity of the establishment. The field economist determines the correct North
American Industry Classification System (NAICS) industry code for the establishment.
• A list of employees or a list of job titles with employee counts. With an employee list, the field economist
uses equal probability sampling to select a sampled job, for which each entry on the list has an equal
chance of selection. With job titles and employee counts the field economist uses probability proportional
to size sampling, that is the greater the number of employees in an occupation within the establishment
the greater chance the occupation will be selected.
• The tasks, knowledge required, controls and complexity, contacts, and environmental conditions of the
job. The field economist determines the correct O*NET-SOC 2010 occupation code and work level for
each sampled job based on the job description and type of work performed. (For more information on pay
factors and work levels, see National Compensation Survey: Guide for Evaluating Your Firm’s Jobs and
Pay).
• The amount of employees in each sampled job that is matched to an occupational description. The field
economist determines how many employees in the establishment can be defined by the occupational
code for the sampled job.
• Work attributes for the workers in the matched occupation in terms of whether they work full or part time,
classified as union or nonunion workers, and paid on a time or incentive basis. The field economist
determines these three work attributes of the employee in the matched occupation.
• The various occupational requirements that the employee must meet to successfully perform the selected
occupation. The field economist collects elements that pertain to the matched occupation’s physical
demands, environmental conditions, education and training, and mental requirements.
Collection period
A BLS field economist contacts the sampled establishment for the collection of data. From each establishment,
the ORS collects data on physical demands, environmental conditions, education and training (or specific
vocational preparation (SVP)), and mental requirements for the selected job(s). These data are then used to
produce the ORS estimates discussed in the Calculation section.
The collection for ORS data is done on an annual basis. Collection of the first ORS sample began in September
2015 and was collected for approximately 9 months. Subsequent sample group collections will be done annually
from May to July (of the following year).
5
U.S. BUREAU OF LABOR STATISTICS
HANDBOOK OF METHODS
Confidentiality
All data collected in the ORS are subject to the BLS confidentiality requirements that prevent the disclosure of
identifying information. Data collected from the ORS are used solely for statistical purposes. BLS has a strict
confidentiality policy which ensures that the survey sample composition, lists of reporters, and names of
respondents will be kept confidential. In addition, the policy assures respondents that published figures will not
reveal the identity of any specific respondent and will not allow the data of any specific respondent to be
identified. Each published estimate is screened to ensure that it meets these confidentiality requirements.
Last Modified Date: March 29, 2017
6
U.S. BUREAU OF LABOR STATISTICS
HANDBOOK OF METHODS
Design
Occupational Requirements Survey (ORS) data are collected from a national probability sample selected in two
stages: (1) a probability sample of establishments and (2) a probability sample of occupations within sampled
establishments. Probability samples are subject to sampling and nonsampling errors, which are discussed in the
Calculation section.
Selecting sample establishments (stage 1)
In stage 1, the ORS uses a probability proportional to size (PPS) technique to select a sample of private
industry and state and local government establishments from across the nation. The larger the establishment,
the greater its chance of being selected. Establishments from all 50 states and the District of Columbia are
eligible for selection. ORS stratifies by 23 major industry groups and ownership (private industry and state and
local government). More detailed information on ORS sample design can be found in Occupational
Requirements Survey Sample Design.
With some minor exceptions, an establishment is a single economic unit that engages in one, or predominantly
one, type of economic activity. For private industries in the survey, the establishment usually is at a single
physical location, such as a mine, a factory, an office, or a store, that produces goods or provides services. For
private industry, if a sampled establishment is owned by a larger entity with many locations, only the
employment and characteristics of the establishment selected for the sample are considered for the survey. For
state and local governments, an establishment can include more than one physical location, such as a school
district or a police department. Each establishment is assigned a six-digit industry code using the North
American Industry Classification System (NAICS). When a single physical location encompasses two or more
distinct economic activities, the industry code assigned is based on the establishment's principal product, or
group of products, produced or distributed, or services rendered. When determining the principle product or
service rendered, revenue generated or employment are used to determine the primary business activity and
assign an industry code.
The sampling frame, or universe, is the list of establishments from which the survey sample is selected. The
ORS establishment sample is drawn from the Quarterly Census of Employment and Wages (QCEW). Because
the sample of establishments used to collect ORS data is chosen ahead of time, establishment weights reflect
employment at the time of sampling, not collection.
Industry classification of establishments. All federal statistical agencies currently use NAICS to classify survey
establishments by industry. NAICS revises its industry classifications every 5 years to stay current with industrial
organization in North America. In selecting new establishment samples, the ORS uses the most recent version
of NAICS (2012) as one of the stratification variables.
Sample groups. The 2016 annual estimates are from a single sample of collected data. The ORS is an
establishment-based survey, using a national sample design. To maximize the amount of publishable
information, BLS combines data across three annual ORS samples. The number of publishable occupations
7
U.S. BUREAU OF LABOR STATISTICS
HANDBOOK OF METHODS
and the level of occupational detail is expected to increase with the addition of each subsequent sample group
until the full ORS sample size of up to 30,000 sampled establishments is reached. Because the ORS combines
data across sample groups, there is a possibility that an establishment will be reselected in a subsequent
sample. However, ORS data are not collected from the same establishment in more than one sample group. In
this case, the data from the original interview with the establishment would be used.
Probability sampling of occupations within sampled establishments
(stage 2)
The ORS collects data on workers who are employed by the sampled establishment. People working onsite at a
surveyed establishment, but paid by a contractor, are not included in data collection from the establishment,
unless the contractor is part of the sample. In that case, the ORS collects data on employees of the contractor
who are working offsite at other establishments, as well as those working onsite. To be included in the ORS,
employees in sampled occupations must receive payments (cash, check, or direct deposit payments) from the
establishment for services performed and the establishment must pay the employer’s portion of Medicare taxes
on those individuals’ wages.
The number of workers in an establishment includes workers on paid vacation or other types of leave; salaried
officers, executives, and staff members of incorporated firms; employees temporarily assigned to other units;
and noncontract employees for whom the reporting unit is the permanent duty station, regardless of whether
that unit issues their paychecks.
In stage 2, field economists use a four step process to select and classify jobs for which data are to be collected
during the initial contact with the sampled establishment.
Step 1: Field economist receives the establishment’s complete list of employees and their job titles and performs
the probability selection of occupations (PSO) technique. The field economist uses the PSO technique to
randomly select the jobs for which data are to be collected. This process ensures that the probability of selecting
a given job is proportional to the number of workers in the job at the establishment. The number of jobs selected
for data collection is based on the establishment’s employment size, according to the following criteria:
PSO Technique
Number of employees
Number of jobs selected
1–49
50–249
Up to 4
6
250 or more
8
Exceptions include state and local government units, for which up to 20 jobs may be selected.
Step 2: Field economists match employees working in the sampled jobs with an occupation. The sampled jobs
are classified into occupations based on the workers’ actual job duties and responsibilities, not on their job titles
or specific education. For example, an employee trained as an engineer, but working as a drafter, is reported as
a drafter. An employee who performs the duties of two or more occupations is reported as working in the
occupation that requires the highest level of skill or in the occupation in which the employee spends the most
8
U.S. BUREAU OF LABOR STATISTICS
HANDBOOK OF METHODS
time if there is no measurable difference in skill requirements. Each occupation is designated by an eight-digit
code in the Occupational Information Network’s (O*NET) detailed occupational taxonomy, referred to as O*NETSOC 2010 occupations. This code is part of a hierarchical structure as shown in the following exhibit.
Level of detail
O*NET-SOC 2010 code
Occupation title
2 digit
17-0000.00
Engineering occupations
3 digit
17-3000.00
Drafters, engineering technicians, and mapping technicians
5 digit
17-3010.00
Drafter
6 digit
17-3011.00
Architectural and civil drafters
17-3011.01
Architectural drafters
17-3011.02
Civil drafters
8 digit
O*NET-SOC 2010 occupations are grouped under and include the Standard Occupational Classification (SOC)
detailed occupations. SOC detailed occupations are grouped under broad occupations, broad occupations are
part of a minor group, and minor groups are part of a major group. The example above shows the hierarchy of
‘architectural drafters’ and ‘civil drafters’ O*NET-SOC 2010 occupations.
The SOC designates 23 major groups and there are 1,110 O*NET-SOC 2010 occupations within these 23
groups. For the purposes of the ORS, occupations can fall into 22 major groups and 1,090 occupations; only the
major group 23 (code 55-0000.00), military-specific occupations, is excluded.
Step 3: Identification of occupational attributes of the worker, such as full-time or part-time status, union or
nonunion status, and whether the work is paid on a time or incentive basis. The field economist records specific
attributes of the worker in the sampled job, for each selected occupation. Each such occupation must include
9
U.S. BUREAU OF LABOR STATISTICS
HANDBOOK OF METHODS
only workers with the same attributes; for example, the occupation cannot include both full-time and part-time
workers. For definitions of occupational attributes see the Concepts section.
Step 4: Field economists evaluate the job to determine the work level of its duties and responsibilities using a
point-factor system of points based on the following factors:
•
•
•
•
Knowledge
Job controls and complexity
Contacts
Physical environment
Each factor consists of several points and a description. The duties and responsibilities of the job, along with
consideration given to work performed and the skills, education, and training required for the job are evaluated.
Points are then totaled to determine the overall work level for the job. Generally, the greater the impact,
complexity, or difficulty of the factor, the higher the number of points assigned, and the higher the work level. As
the following exhibit shows, there are some occupations that cannot be “leveled,” because for the following jobs
points cannot be determined for all four factors, thus points are not assigned and a level cannot be determined.
Jobs that cannot be leveled
O*NET-SOC 2010 code
Occupation title
11-1031.00
Legislators
27-1013.00
Fine artists, including painters, sculptors, and illustrators
23-1021.00
Administrative law judges, adjudicators, and hearing officers
23-1022.00
Arbitrators, mediators, and conciliators
23-1023.00
Judges, magistrate judges, and magistrates
27-2011.00
Actors
27-2012.00
Producers and directors
27-2012.01
Producers
27-2012.02
Directors-stage, motion pictures, television, and radio
27-2012.03
Program directors
27-2012.04
Talent directors
27-2012.05
Technical directors/managers
27-2021.00
Athletes and sports competitors
27-2022.00
Coaches and scouts
27-2023.00
Umpires, referees, and other sports officials
27-2031.00
Dancers
27-2032.00
Choreographers
27-2041.00
Music directors and composers
27-2041.01
Music directors
27-2041.04
Music composers and arrangers
27-2042.00
Musicians and singers
27-2042.01
Singers
27-2042.02
Musicians, instrumental
27-2099.00
Entertainers and performers, sports and related worker, all other
27-3011.00
Radio and television announcers
27-3012.00
Public address systems and other announcers
10
U.S. BUREAU OF LABOR STATISTICS
HANDBOOK OF METHODS
Jobs that cannot be leveled
41-9012.00
Models
Last Modified Date: March 29, 2017
11
U.S. BUREAU OF LABOR STATISTICS
HANDBOOK OF METHODS
Calculation
The ORS estimates provide data about the physical demands, environmental conditions, education and training
(or specific vocational preparation (SVP)), and mental requirements for how a job is typically performed. Many
of the ORS data elements have the percentage of workers, mean (in hours, days, or percentage of a day),
percentiles, and mode estimates for each occupational definition. For example, one ORS data element
measures the amount of time during a typical workday that a nurse, spends stooping. Physical demands, such
as stooping, are captured in hours, so mean and percentile estimates (10th, 25th, 50th, 75th, and 90th
percentiles) are calculated. Also, the number of hours spent stooping are grouped within duration categories,
and a percentage-of-workers estimate is calculated for each category.
Duration Scale
Seldom
Up to 2 percent of the workday
Occasional
2 percent and up to 1/3 of the workday
Frequent
1/3 up to 2/3 of the workday
Constant
2/3 or more of the workday
Finally, the mode of the categories is identified, that is, which duration category with the largest weighted
number of workers. The formulas used to calculate these estimates are shown below. The type of estimator
used depends on the type of data element. For any categorical data element, a percentage of workers is
calculated, and a mode identified for these percentages. For continuous data elements (such as duration and
maximum weight lifted/carried elements), mean and percentile estimates is calculated.
Field economists collect 74 ORS data elements; however, as shown by the example given, many estimates can
be calculated from one element. This results in many more calculated ORS estimates per occupation (or
occupational group). For a full list of calculated elements, please see Appendix A at the end of this section.
Percentage. The formula for the percentage of employees with a given characteristic out of all employees in the
domain is
where:
i is the establishment,
g is the occupation within establishment i,
I is the total number of establishments,
12
U.S. BUREAU OF LABOR STATISTICS
HANDBOOK OF METHODS
Gi is the total number of quotes in establishment i,
Xig is 1 if worker ig meets the condition set in the domain (denominator) condition and 0 otherwise.
Zig is 1 if worker ig meets the condition set in the characteristic condition and 0 otherwise.
OccFWig is the final quote weight for occupation g in establishment i.
Average (mean). The formula for the average value of a quantity for a characteristic is
where:
i is the establishment,
g is the occupation within establishment i,
I is the total number of establishments,
Gi is the total number of quotes in establishment i,
Xig is 1 if worker ig meets the condition set in the domain (denominator) condition and 0 otherwise.
Zig is 1 if worker ig meets the condition set in the characteristic condition and 0 otherwise.
OccFWig is the final quote weight for occupation g in establishment i.
Qig is the value of a quantity for a specific characteristic for occupation g in establishment i.
Percentiles. The grouping of various categories are used to describe the distribution of a numeric value. The
following percentiles p are calculated: 10th, 25th, 50th (median), 75th, and 90th. The pth percentile is the value
Qig, where the value of a quantity is for a specific category, such that
·
the sum of final quote weights (OccFWig) across quotes with a value less than Qig is less than p percent
of all final quote weights and
·
the sum of final quote weights (OccFWig) across quotes with a value more than Qig is less than (100 – p)
percent of all final quote weights.
13
U.S. BUREAU OF LABOR STATISTICS
HANDBOOK OF METHODS
It is possible that there is no specific quote ig for which both of these properties hold. This occurs when there
exists a quote for which the OccFWig of records whose value is less than Qig equals p percent of the total
weighted quote employment. In that situation, the pth percentile is the average of Qig and the value on the quote
with the next-lowest value.
Mode. The mode is the category of a characteristic with the largest weighted number of workers within the
associated category group. Refer to Appendix A at the end of this section for a list of elements that have mode
estimates.
Education and training
Although most of the estimates for these requirements are simply based on establishment responses about the
selected occupations’ various tasks, there are some that require an additional level of calculation. One such
type of estimation is SVP level which is determined by the amount of preparation time required by the worker in
order to develop the skills needed to perform the given occupation. The elements that make up this preparation
are the minimum education, pre-employment training, previous work experience, and post-employment training
for the job. These elements are then aggregated and used to determine the SVP level needed for the job shown
in the table below:
Specific vocational preparation level
Preparation time
1
Short demonstration only (4 hours or less)
2
Anything beyond short demonstration up to and including 1 month
3
Over 1 month up to and including 3 months
4
Over 3 months up to and including 6 months
5
Over 6 months up to and including 1 year
6
Over 1 year up to and including 2 years
7
Over 2 years up to and including 4 years
8
Over 4 years up to and including 10 years
9
Over 10 years
Strength
Another element that is based off of other elements’ estimates is strength. It is measured in five levels; from
sedentary to very heavy work. The levels are determined by how much weight a worker is required to lift or carry
seldom, occasionally, frequently, and constantly, as well as standing/walking in some special cases. The
strength level is determined by satisfying at least one of the lifting/carrying conditions shown in the table below,
or as defined by the “strength special cases” table. For example, if a job requires a worker to lift or carry 11–20
pounds occasionally, then it would be classified as light work. However, if that same job were to require lifting or
carrying that same weight frequently, then it would be classified as medium work.
Strength level
Lifting/carrying
Seldom
Light work
11-20pounds
Medium work
21-50 pounds
14
Heavy work
51-100 pounds
Very heavy work
>100 pounds
U.S. BUREAU OF LABOR STATISTICS
HANDBOOK OF METHODS
Strength level
Occasionally
11-20 pounds
21-50 pounds
51-100 pounds
>100 pounds
Frequently
≤10 pounds
11-25 pounds
26-50 pounds
>50 pounds
Constantly
Negligible weight
≤10 pounds
11-20 pounds
>20 pounds
The following table outlines the special cases for strength. In instances where field economists are unable to
determine from the respondent certain job requirements, they code these data as unknown. See the section
“Weighting, Nonresponse Adjustment, Imputation, and Benchmarking” for more information.
Strength Level—Special Cases
Unknown
If lifting occasionally, frequently, or constantly is unknown. Or if none of the conditions in the strength chart is met
and stand/walking or sitting is unknown.
Sedentary
If none of the conditions in the strength chart is met and standing/walking is less than or equal to 1/3 of the work
schedule
Light work
If none of the conditions in the strength chart is met and does not meet the special conditions for unknown or
sedentary.
Reliability of ORS estimates
ORS estimates are derived from a sample of occupations selected from responding establishments. Two types
of errors are possible in an estimate based on a sample survey: sampling and nonsampling errors. Sampling
errors occur because the sample makes up only a part of the population it represents. The sample used for the
survey is one of a number of possible samples that could have been selected under the sample design, each
producing its own estimate. A measure of the variation among sample estimates is the standard error.
Nonsampling errors are data errors that stem from any source other than sampling error, such as data collection
errors and data-processing errors.
Standard errors can be used to measure the precision with which an estimate from a particular sample
approximates the expected result of all possible samples. The chances are about 68 out of 100 that an estimate
from the survey differs from a complete population figure by less than the standard error. The chances are about
90 out of 100 that this difference is less than 1.6 times the standard error. Statements of comparison appearing
in ORS publications are significant at a level of 1.6 standard errors or better. This means that, for differences
cited, the estimated difference is more than 1.6 times the standard error of the difference.
The ORS uses balanced repeated replication (BRR) to estimate the standard error. The procedure for BRR
entails first partitioning the sample into variance strata composed of a single sampling stratum or clusters of
sampling strata, and then splitting the sample units in each variance stratum evenly into two variance primary
sampling units (PSUs). Next, half-samples are chosen, so that each contains exactly one variance PSU from
each variance stratum. Choices are not random but are designed to yield a “balanced” collection of halfsamples. By using half-samples, we can compute a “replicate” estimate with the same formula for the regular, or
“full-sample,” estimate, except that the final weights are adjusted. If a unit is in the half-sample, its weight is
multiplied by (2 – k); if not, its weight is multiplied by k. For all ORS publications, k = 0.5, so the multipliers are
1.5 and 0.5.
The BRR estimate of standard error with R half samples is
15
U.S. BUREAU OF LABOR STATISTICS
HANDBOOK OF METHODS
,
where:
the summation is over all replicates of half-samples r = 1,...,R,
is the rth replicate estimate, and
is the full-sample estimate.
Data collection and processing errors are mitigated primarily through quality assurance programs that include
the use of data collection reinterviews, observed interviews, computer edits of the data and systematic
professional review of the data. The programs also serve as a training device to provide feedback to field
economists, or data collectors, on errors and the sources of errors that can be remedied by improved collection
instructions or computer-processing edits. Field economists receive extensive training to maintain high
standards in data collection.
Once estimates of occupational requirements are produced, the estimates are verified, or validated. The focus
of the verification at this stage is to compare the estimates with their expected values. These values are based
on comparisons with leveling information, the Dictionary of Occupational Titles (DOT), the Occupational
Information Network (O*NET), testing estimates and environmental scanning for related data or research on
occupational requirements. In addition, we compare data between similar occupations.
The data are designated as “fit for use” in publication, once the estimates are evaluated for confidentiality and
reliability. Not all calculated series meet the criteria for publication. Before any series is published, it is reviewed
to make sure that it meets specified statistical reliability and confidentiality requirements. The review prevents
the publication of a series that could reveal information about a specific establishment or that has a large
sampling error. For additional information on data review and estimate validation, see Validation of Estimates in
the Occupational Requirements Survey: Analysis of Approaches and Occupational Requirements Survey (ORS)
Data Review Process.
To assist users in ascertaining the reliability of ORS estimates, we publish standard errors. Standard errors
provide users with a measure of the precision of an estimate to ensure that it is within an acceptable range for
their intended purpose. The standard errors are calculated from collected and imputed data. BLS is researching
methods for estimating the variance excluding imputed values. For additional information, see https://
www.bls.gov/ncs/ors/or/se.htm.
16
U.S. BUREAU OF LABOR STATISTICS
HANDBOOK OF METHODS
Weighting, nonresponse adjustment, imputation, and benchmarking
Participation in the survey is voluntary; therefore, a company official may refuse to participate in the survey. In
addition, some establishments selected from the sampling frame may be out of the scope of the survey or may
have gone out of business. To address the problems of nonresponse and missing data, the ORS program
adjusts the weights of the remaining establishments and imputes missing values, to ensure that occupational
requirement estimates are representative of requirements for civilian workers during the estimation process.
This section describes the current weight adjustments, imputation, and benchmarking methods.
Weight adjustments and imputation are made in accordance with the following steps:
1.
An establishment is considered responding if it provided information on at least one usable occupation.
An occupation is classified as usable if the following data are present: occupational attributes (full-time or
part-time schedule, union or nonunion status, and time or incentive type of pay), work schedule, and
occupational requirements data for any of the data elements. An establishment is considered
nonresponding if it provided neither occupational requirements, occupational classification, worker
attributes, nor work schedule data for any occupation. Establishment nonresponse during the initial
interview is treated with adjustments that redistribute the weights of nonrespondents to similar
respondents by characteristics such as the industry, size class, and geographic area of the
establishment. For example, if the nonresponding establishment was in the manufacturing industry and
had an employment of 350 workers, the ORS program would adjust the weights of responding
manufacturing establishments with 250–499 workers by a nonresponse factor during estimation. This
nonresponse adjustment factor (NRAF) at the establishment level is calculated using the following
formula:
,
where:
= weighted employment of all usable establishments in the nonresponse cell
= weighted employment of all viable but not usable establishments in the nonresponse cell
If there are no responding establishments to reweight within the industry/employment group, then
additional responding units from similar geographic areas are considered.
Establishments no longer in operation or out of the scope of the survey, and establishments with no
workers within the scope of the survey, are excluded from the survey estimates.
17
U.S. BUREAU OF LABOR STATISTICS
2.
HANDBOOK OF METHODS
Other response and nonresponse adjustment factors may be included for any special situations that
may have occurred during data collection. For example, an establishment weight adjustment factor is
applied when a sample unit is one of two establishments owned by a given company and the respondent
provides data for both locations combined instead of data for the sampled unit. In this example, the
weight of the sampled unit is adjusted to reflect the employment data collected.
3.
Item nonresponse is a situation in which an establishment responds to the survey but is unable or
unwilling to provide some of the occupational requirements data or worker attributes for a given sampled
occupation. Item nonresponse is addressed through item imputation in certain situations. Item imputation
replaces missing values for an item or items with values derived from establishments with similar
establishment and worker characteristics that have a value for the item. For ORS estimates, items with
missing values are imputed within groups of ORS characteristics that are related. For example, one ORS
group refers to categorical variables only and includes such characteristics as hearing, vision, and
driving. Within the group, values are imputed using occupational information from similar occupations in
similar establishments. Imputation of one group of ORS characteristics does not affect the imputation for
any other group.
4.
Poststratification, or benchmarking, is the process of adjusting the weight of each establishment in the
survey to match the most current distribution of employment by industry. The ORS establishment sample
is drawn from the Quarterly Census of Employment and Wages (QCEW). Because the sample of
establishments used to collect ORS data are chosen ahead of time, establishment weights reflect
employment at the time of sampling, not collection. The benchmark process updates those weights by
current employment. Benchmarking ensures that survey estimates reflect the most current industry–
government (hereafter, ownership) employment counts in proportions consistent with the private industry,
state government, and local government sectors. For example, let’s say 40 private industry, 10 local
government, and 5 state government units in the service sector were selected from the ORS sampling
frame. These units consist of establishments employing 200,000 private workers, 30,000 local
government workers, and 10,000 state government workers. If, by the time of survey processing, the
private service sector experienced an employment increase of 10,000 workers (or 5 percent) and there
is no increase in employment in the service sectors of state and local government, then the sample
would underrepresent current employment in the private industry service sector in the absence of
benchmarking. In this example, the ORS would adjust the sample weights of the 40 service sector
firms in private industry to ensure that the number of workers in establishments in the sampling frame
rises to 210,000. The ownership employment counts for the private industry service sector would then
reflect the current proportions of 84 percent for private industry, 12 percent for local government, and 4
percent for state government employment.
Employment information is derived from the Quarterly Census of Employment and Wages (QCEW)
Longitudinal Database, a file of railroad employment, and the Current Employment Survey (CES). The
QCEW and the railroad information provide employment data, but since these sources do not have
current employment data, the CES is used to make an adjustment to the employment.
18
U.S. BUREAU OF LABOR STATISTICS
HANDBOOK OF METHODS
For more information, please see Estimation Processes Used in the Occupational Requirements Survey and
Imputation Methodology for the Occupational Requirements Survey (ORS).
Appendix A. List of calculated occupational requirements by category and estimate type
Occupational requirement
Categorical
Continuous
Percentage
Mode
Mean
Percentile (1)
Standing and walking
…
…
√
√
Sitting
…
…
√
√
Sitting vs standing at will
√
√
…
…
One on one
√
√
…
…
Group
√
√
…
…
Telephone
√
√
…
…
Other sounds
√
√
…
…
Pass a hearing test
√
√
…
…
Near visual acuity
√
√
…
…
Far visual acuity
√
√
…
…
Peripheral vision
√
√
…
…
√
√
√
√
Ramps/stairs: structural only
√
√
…
…
Ramps/stairs: work-related
√
√
√
√
Ladders/ropes/scaffolds
√
√
√
√
Weight (range) lifted/carried- seldom
√
√
…
…
Weight (range) lifted/carried - occasionally
√
√
…
…
Weight (range) lifted/carried - frequently
√
√
…
…
Weight (range) lifted/carried - constantly
√
√
…
…
Most weight ever lifted/carried (pounds)
…
…
√
√
√
√
√
√
√
√
…
…
√
√
√
√
√
√
…
…
√
√
√
√
√
√
…
…
Crawling
√
√
√
√
Crouching
√
√
√
√
Stooping
√
√
√
√
Kneeling
√
√
√
√
√
√
√
√
Physical demands
Sitting or standing/walking
Hearing
Vision
Communication
Verbal
Climbing
Lifting/Carrying
Manipulation
Foot/leg controls
One or both
Gross manipulation
One or both
Fine manipulation
One or both
Postural
Pushing/Pulling
With hand/arm
19
U.S. BUREAU OF LABOR STATISTICS
HANDBOOK OF METHODS
Appendix A. List of calculated occupational requirements by category and estimate type
Occupational requirement
Categorical
One or both
Continuous
√
√
…
…
√
√
√
√
√
√
…
…
√
√
√
√
√
√
…
…
√
√
√
√
√
√
…
…
√
√
√
√
√
√
…
…
Keyboarding: traditional
√
√
√
√
Keyboarding: touch screen
√
√
√
√
Keyboarding: 10-key
√
√
√
√
Keyboarding: other
√
√
√
√
Keyboarding: any keyboarding
√
√
…
…
Driving, type of vehicle
√
√
…
…
√
√
…
…
Extreme Cold (non-weather related)
√
√
√
√
Extreme Heat (non-weather related)
√
√
√
√
Wetness (non-weather related)
√
√
√
√
Humidity
√
√
√
√
Heavy vibration
√
√
√
√
High, exposed places
√
√
√
√
Proximity to moving mechanical parts
√
√
√
√
Outdoors
√
√
√
√
Hazardous contaminants
√
√
√
√
Noise Intensity Level
√
√
…
…
Degree by type
√
√
…
…
Associates degree time (days)
…
…
√
√
Vocational associates degree time (days)
…
…
√
√
High school vocational time (days)
…
…
√
√
Literacy (if no high school required)
√
√
…
…
Pre-employment training (license, certification, other)
√
√
√
√
Prior work experience
√
√
√
√
Post-employment training
√
√
√
√
√
√
…
…
√
√
…
…
With foot/leg
One or both
With feet only
One or both
Reaching
Overhead
One or both
At or below the shoulder
One or both
Tasks
Strength
Strength
Environmental conditions
Education, training, and experience
Minimum formal education or literacy required
Other training & experience
Requirements calculated for SSA
Specific vocational preparation (SVP)
Cognitive and mental demands
Adaptability: work location
20
U.S. BUREAU OF LABOR STATISTICS
HANDBOOK OF METHODS
Appendix A. List of calculated occupational requirements by category and estimate type
Occupational requirement
Categorical
Continuous
Adaptability: work schedule
√
√
…
…
Adaptability: work tasks
√
√
…
…
Regular contacts: type of interaction
√
√
…
…
Regular contacts: frequency of contact
√
√
…
…
Other contacts: type of interaction
√
√
…
…
Other contacts: frequency of contact
√
√
…
…
Decision-making
√
√
…
…
Supervision
√
√
…
…
Pace of work
√
√
…
…
√
√
…
…
Control of work pace
√ = Potential estimate for occupational requirement
… = No estimate for this occupational requirement
(1) Percentile estimates are calculated at the 10th, 25th, 50th (median), 75th, and 90th
Last Modified Date: March 29, 2017
21
U.S. BUREAU OF LABOR STATISTICS
HANDBOOK OF METHODS
Presentation
Occupational Requirement Survey (ORS) news releases, data, and other information can be found at
www.bls.gov/ors. The primary purpose of collecting ORS data is to provide a comprehensive dataset on the
physical demands, environmental conditions, education and training (or specific vocational preparation (SVP)),
and mental requirements for jobs in the U.S. economy by detailed occupations. Users may include:
•
•
•
•
•
•
•
•
•
•
•
•
Jobseekers
Researchers
Insurance companies
Advocacy organizations
Data users within nonprofits
Employment agencies
State and federal agencies
Disability community
Vocational experts
Human resource professionals
Medical professionals
Actuaries
ORS data are used for a variety of purposes. Uses may include:
•
•
•
•
•
•
Assisting the Social Security Administration in its disability adjudication process
Using data for new opportunities in research, such as in academia or government
Tracking the nature of work
Benchmarking job descriptions or developing targeted recruiting plans
Helping insurance companies assess risk management
Assisting temporary help firms properly match an employee to job openings
Accessing data
The complete set of 2016 ORS data can be found at www.bls.gov/ors/#data. Both a multi-screen query tool and
flat files are available for data users. Flat files can be downloaded at https://download.bls.gov/pub/time.series/
or/, which also includes a description of these files and the structure of ORS series. In addition, selected
occupational information about 21 major occupational groups can be found in occupational profiles.
The 2016 annual estimates released for the ORS are from a single sample of collected data. The ORS is an
establishment-based survey and uses a national sample design. To maximize the amount of publishable
information, the BLS is combining data across three annual ORS samples. The number of publishable
occupations and the level of occupational detail is expected to increase with the addition of each subsequent
year’s sample until the full ORS sample size of up to 30,000 sample establishments is reached.
Although the occupational requirements data collected may have many uses, their limitations must be kept in
mind. The data are subject to sampling error, which may cause deviations from the results that would be
22
U.S. BUREAU OF LABOR STATISTICS
HANDBOOK OF METHODS
obtained if the actual requirements for jobs in all establishments could be used. Nonsampling error is present in
surveys as well. (See the Calculation section for more information.) Also, the current imputation process used by
ORS remains under development and may be refined in the future.
Last Modified Date: March 29, 2017
23
U.S. BUREAU OF LABOR STATISTICS
HANDBOOK OF METHODS
History
Timeline Events:
October 2012: Occupational Requirements Survey established as a test survey
November 2012: Phase 1 test: Initial proof of concept
January 2013: Phase 2 test: Collection protocol testing
April 2013: Phase 3 test: Broad scale testing
November 2013: Observations Test conducted concurrently with other FY 2014 tests
November 2013: ORS-Only Efficiency Innovations Test
December 2013: NCS/ORS Joint Collection Test
December 2013: New Data Element Tests
November 2013: Central Office Elements Test
February 2014: Alternative Modes Test
October 2014 – September 2015: Pre-production testing
June 2015: Job Observations Pilot Test
September 2015 – December 2016: First year of production data collection and estimation
December 2016: Published estimates from first production sample
The Social Security Administration (SSA) contracted with BLS to produce occupational data that would describe
the requirements of an occupation. These data would aid SSA in determining eligibility for Social Security
Disability Insurance (SSDI) and Supplemental Security Income (SSI) disability benefits for applicants. During the
developmental stages of the Occupational Requirements Survey (ORS), BLS identified its existing infrastructure
already available to coordinate with the ORS. That framework had the capability to manage and implement a
new survey to meet data needs as well as systems and processes to support all the steps of the survey. In
addition, field economists who work on the National Compensation Survey (NCS) were already familiar with
collecting data elements similar to those ORS captures. For example, the NCS classifies each job selected
using the Standard Occupational Classification System (SOC), collects worker characteristics (such as
bargaining status and part-time or full-time workers), and determines industry classification using the North
24
U.S. BUREAU OF LABOR STATISTICS
HANDBOOK OF METHODS
American Industry Classification System (NAICS) for sampled establishments. BLS has experience collecting
and reviewing information on the knowledge required to perform the job, job controls provided, the complexity of
tasks, the contacts made by workers, and the physical environment where the work is performed—all similar to
the types of data ORS would be designed to collect. This initial determination eventually led to formalized testing
that would determine if the existing infrastructure can be used to collect data on occupational requirements.
Testing
BLS established ORS as a test survey in FY 2013 (that is, during October 1, 2012– September 30, 2013). In FY
2013 and 2014, several feasibility tests were performed to assess the viability of collecting data on occupational
requirements using the platform currently used by the NCS.
In FY 2013, testing was conducted in three phases: The main objective of phase 1 was to ensure that BLS field
economists knew how to describe the survey and find respondents for the ORS data elements. BLS also
created and tested an initial set of data collection protocols and collection aides. In phase 2, BLS expanded the
number of field economists that could describe and collect ORS data while obtaining additional information not
included in phase 1. That test also evaluated the effectiveness of collection tools. The primary goal of phase 3
was to test whether field economists could collect ORS data elements and relevant information across the
country in a uniform and efficient manner. Also during phase 3, BLS tested the feasibility of collecting both ORS
and NCS elements; adding more ways to conduct ORS interviews; including new data capture systems and
review procedures; and establishing the Central Office Collection (COC). Some companies have special
arrangements with BLS, regarding the manner in which data should be collected for their individual
establishments and a COC may require permission and coordination from headquarters in order to proceed with
collecting data. Test objectives were successfully met in these phases, and the findings from these tests
suggested that the collection of the ORS data was viable.
As a result of FY 2013 testing, areas were identified where further testing was needed before moving to fullscale production. In FY 2014, six feasibility tests were completed to refine ORS methodology tested in previous
phases:
1. ORS Only Efficiency Innovations Test – refined the methods to develop more efficient approaches for
data collection as identified during FY 2013 testing
2. NCS/ORS Joint Collection Test – determined how best to collect occupational requirements data
elements and NCS data elements from the same establishment
3. New Data Element Tests – determined the new mental and cognitive demands of work data elements and
evaluated the use of occupational task lists as developed by the Department of Labor, Employment and
Training Administration (ETA), Occupational Information Network (O*NET) program during data collection
4. Central Office Collection (COC) Test – determined how best to collect occupational requirements data
elements from large firms and state governments
5. Alternative Modes Test – determined how to collect occupational requirements data elements efficiently
when a personal visit is not optimal via phone, email, or fax
6. Observations – observed incumbents performing work in selected jobs and evaluated differences
between data collected through observation and those collected through establishment interview
25
U.S. BUREAU OF LABOR STATISTICS
HANDBOOK OF METHODS
These tests provided evidence that the NCS platform could be adapted to ORS data collection and
demonstrated the effectiveness of the revised materials and procedures.
Testing activities in FY 2013 and 2014 laid the foundation for the preproduction test conducted in FY 2015.
Unlike the earlier tests, which were small-scale, testing a subset of data elements or the viability of different
collection methods, the preproduction test was designed as a relatively large-scale, nationally representative
test of ORS data collection. ORS preproduction data collection began in October 2014 and continued until May
2015. The sampling, data collection, procedures, and review were designed to mimic what will occur during
ORS production. The results from the ORS preproduction test demonstrated that data on occupational
requirements could be collected using the processes established by BLS. As a result of the preproduction test,
some changes and refinements to several of the elements were made before the implementation of a move to
production. Detailed information on these tests and other testing activities can be found at www.bls.gov/ncs/ors/
pre-production.htm.
Archives
Last Modified Date: March 29, 2017
26
U.S. BUREAU OF LABOR STATISTICS
HANDBOOK OF METHODS
More Information
Additional information on the Occupational Requirements Survey (ORS) is available on the ORS
website: www.bls.gov/ors/.
ORS estimates are available on the BLS website:
• Time series data from the BLS LABSTAT database: www.bls.gov/ors/#data
• Latest news release: www.bls.gov/news.release/ors.nr0.htm
For a listing of frequently asked questions, see www.bls.gov/ors/#faq.
The Office of Compensation Levels and Trends, Branch of Survey Information and Publications, will be glad to
assist you with questions about any of the components of the Occupational Requirements Survey.
Email: Contact us
Telephone: (202) 691-6199 (Monday–Friday, 8:30 a.m. - 4:30 p.m. EST)
TDD
Information voice phone: (202) 691-5200 The Federal Relay Service: 1-800-877-8339
Write or personal visit
Bureau of Labor Statistics
OCLT/SI&P
2 Massachusetts Ave., N.E. - Suite 4160
Washington, D.C. 20212-0001
Last Modified Date: March 29, 2017
27
| File Type | application/pdf |
| File Modified | 2018-04-25 |
| File Created | 2018-02-13 |